Effectiveness of verbal communication with conjunction of gestures and facial expression in human-robot interaction

We conducted a user study to evaluate the optimal level of verbal and non-verbal communication and the use of facial emotion recognition which the robot should use during a human-robot interaction for a given specific task. This research question was chosen based on previous studies done by psychologists on multi-modal communication in human-human interaction and to see if it applies the same in human-robot interaction. It was also our aim to explore if participants rate the robot interacting with the human only with verbal communication different from a robot interacting with both verbal and non-verbal communication and which reactions people show during the interaction in both cases. Our results show that efficiency of the task improved when the robot uses more relevant social signals during the interaction. We also found that use of facial emotion recognition by the robot was not helpful during the task but the participants found it to be more enjoyable and friendly.