People
RNDr. Júlia Škovierová, Ph.D.

2025, 2025 IEEE International Conference on Development and Learning (ICDL), Piscataway, IEEE Conference Publications), p. 1-6), ISBN 979-8-3315-4343-3

Human communication consists of multimodal signals, such as speech and gestures. These signals are not always aligned, making it difficult for artificial systems (e.g. robots) to find the proper mapping between a particular gesture and the corresponding part of a spoken instruction. The goal of our study is to identify whether and how declarative gestures during human-robot interaction are temporally synchronized with specific segments of language instructions. We conducted an experiment focused on this phenomenon, in which 26 participants taught a humanoid robot using declarative gestures and verbal instructions. The experiment was carried out in a virtual reality (VR) environment that allowed a precise capture of human movements. Gesture trajectories were annotated for the onset, peak and offset events, and statistically compared with the onset, matching language part, and offset of the instruction. The results indicate that there are significant differences between the speech and gesture onset times (W=348,495, p<0.001), with an average temporal difference of 0.56 ± 1.30 seconds, as well as between the gesture peaks and the matching language peaks (W=287,006,p<0.001), with an average difference of 0.66±1.25 seconds. Furthermore, the total duration of both signals differs significantly (W=48,672,p<0.001, with gestures lasting longer than speech. The analysis of the data distributions further revealed that even though both signals differ in absolute timing, there exists a correlation between specific key points: onset: 0.644 (p<0.001), peak: 0.646(p<0.001). These findings suggest that humans synchronize gestures with language instructions at a relational level rather than an absolute level. The findings can be applied to the design of multimodal interfaces for humanoid robots, which should help them understand better human instructions.

2019, 2019 IEEE 15th International Conference on Intelligent Computer Communication and Processing (ICCP), Piscataway, NJ, IEEE), p. 19-25), ISBN 978-1-7281-4914-1

In autonomous driving systems, the intention estimation of traffic participants is one of the most crucial aspects. In this paper, several machine learning methods are used to train classifiers capable of estimating the intention of a pedestrian to cross a zebra crossing. Their results are compared to a Bayesian network – an approach commonly used in autonomous driving. The data used for the estimation contain only position and heading of the pedestrians. The best performing method achieved the F2 score of 92.37%. Index Terms—Intention estimation, Machine learning, Automated driving.

2018, Proceedings of the 4th International Conference on Vehicle Technology and Intelligent Transport Systems, Porto, SciTePress - Science and Technology Publications), p. 341-348), ISBN 978-989-758-293-6

The reported work contributes to the self-driving car efforts, more specifically to scenario understanding from the ego-car point of view. We focus on estimating the intentions of pedestrians near a zebra crossing. First, we predict the future motion of detected pedestrians in a three seconds time horizon. Second, we estimate the intention of each pedestrian to cross the street using a Bayesian network. Results indicate, that the dependence between the error rate of motion prediction and the intention estimation is sub-linear. Thus, despite the lower performance of motion prediction for the time scope larger than one second, the intention estimation remains relatively stable.

2018, Neurocomputing, 307, p. 172-183), ISSN 0925-2312

The essential goal of this paper is to extend the functionality of the bio-inspired intelligent HTM (Hierarchical Temporal Memory) network towards two capabilities: (i) object recognition in color images, (ii) detection of multiple objects located in clutter color images. The former extension is based on the development of a novel scheme for the application of three parallel HTM networks that separately process color, texture, and shape information in color images. For the latter HTM extension, we propose a novel system in which HTM is combined with a modified model of computational visual attention. We adopt the results of Bi et al. (2010), Hu et al. (2005), and Kučerová (2011) and add new elements for the calculation of image saliency maps. The proposed algorithm enables one to automatically locate individual objects in clutter images. For computer experiments, a special image database is created to simulate ideal single object images and cluttered images with multiple objects on an inhomogeneous background. The recognition performance of HTM alone and in combination with the salient-region detection method is evaluated. We show that the attention subsystem is able to satisfactorily locate multiple objects in clutter color images with an inhomogeneous background. We also perform benchmark calculations for two selected computer vision methods used for object detection in color clutter images. Namely, the cascade detector and template matching methods are used. Our study confirms that the proposed attention system can improve the capabilities of HTM for object classification in cluttered images. The compound system of visual attention and HTM outperforms the compared methods in both criteria (recall and correct detection rate). However, as expected, the system cannot match the recognition accuracy achieved by HTM for single object images, and thus, further research is needed.

2018, ISBN 978-80-270-3395-9