Paper

Addressing User Experience and Societal Impact in a User Study with a Humanoid Robot

Using a preliminary exploratory case study the presented work investigates the feasibility of methods for the evaluation of user experience factors in human-humanoid interaction as well as measurements of societal impact in user studies. The case study is based on two tasks participants had to perform with the HRP-2 robot. The robot was controlled by the participants via speech commands to pick up (task1) and put down (task2) an object onto another place. Main goal of the case study was to explore the methodological concept on how to measure novice users’ experiences during the collaboration with the humanoid robot HRP-2 via speech commands and if the general attitude towards robotics changes because of the interaction with the robot. To address users’ experiences during the user study, participants were asked after each task to state their thoughts and feelings they had during the interaction (retrospective think aloud). Furthermore, participants were interviewed by means of two validated standardized questionnaires (NARS and AttrakDiff) and an especially developed questionnaire. The preliminary results show that retrospective think aloud is a good way to gather qualitative data on users’ experiences. Furthermore a final interview on societal impact of humanoid robots gave insights into how novice users imagine a future society with robots.123

The USUS Evaluation Framework for Human-Robot Interaction

To improve the way humans are interacting with robots various factors have to be taken into account. An evaluation framework for Human-Robot Collaboration with humanoid robots addressing usability, social acceptance, user experience, and societal impact (abb. USUS) as evaluation factors is proposed (see figure 1). The theoretical framework USUS is based on a multi-level indicator model to operationalize the evaluation factors. Evaluation factors are described and split up into several indicators, which are extracted and justified by literature review. The theoretical factor indicator framework is then combined with a methodological framework consisting of a mix of methods derived and borrowed from various disciplines (HRI, HCI, psychology, and sociology). The proposed method mix allows addressing all factors within the USUS framework and lays a basis for understanding the interrelationship of the USUS Factors.

Development of a Teleoperator Interface for Humanoid Robots by Means of Heuristic Evaluation Technique

This paper describes the development of a new interaction paradigm for remotely controlled humanoid robots. In order to develop a remote control interface which is intuitive and easy to use even for novice users, an iterative user-centered design approach was applied from the early stages of the interface development. Therefore, an expert evaluation with five specialists in the field of user interface design was conducted. The expert evaluation consisted of a heuristic evaluation of the given interface, followed by individual expert interviews. The results of the evaluation lead to a redesign of the interface. The adopted approach turned out to be very useful for identifying usability problems and design recommendations early in the development process, and therefore supports fast and costeffective interface development.

Autonomous vs. Tele-operated: How People Perceive Human-Robot Collaboration with HRP-2

Effective collaboration between robots and humans is not only a question of interface design and usability, but also of user experience and social acceptance. To investigate these aspects for Human-Robot Collaboration with the HRP-2 robot, two videobased focus groups enhanced with “creative stimuli” were conducted. The following research question was addressed: “Is the HRP-2 robot perceived differently in an autonomous collaboration condition compared to a tele-operated collaboration condition, in terms of social acceptance and user experience?” The results show that participants in general are open to a humanoid robot as working partner as long as there is a clear distinction between a human and a robot, in terms of tasks and working procedures. Furthermore, participants stated a positive attitude toward the remotely-controlled HRP-2 robot.

“I would choose the other card” – Humanoid Robot gives an Advice

This article reports on a user study conducted to asses the credibility of a humanoid robot. The study set-up was based on the “Monty Hall Problem”. Overall 13 people between the ages of 19 and 84 took part in the study (7 male and 6 female). The experiment was set up as a card-game where the participant had to guess which of the three cards shows a price. At one point of the experiment the robot advised the participant to change his/her mind and choose another card. During the user study the participants had to fill in a questionnaire on their level of certainty about their choice and the credibility of the robot. The results showed a significant correlation between the believability of the robot and the certainty in the decision made. Furthermore, the outcomes showed differences between participants who followed the robot’s advise and participants who did not, regarding credibility, certainty of the decision made and the estimation whether the robot was helpful or not.

Evaluating the ICRA 2008 HRI Challenge

This paper reports on the evaluation of the ICRA 2008 Human-Robot Interaction (HRI) Challenge. Five research groups demonstrated state-of-the-art work on HRI with a special focus on social and learning abilities. The demon- strations were rated by expert evaluators, in charge of award- ing the prize, and 269 participants, i.e. 20 percent of the con- ference attendees through a standardized questionnaire (se- mantic differential). The data was analyzed with respect to six independent variables: expert evaluators vs. attendees, nationality of participants, origin region of the demo, age, gender and knowledge level of the attendees. Conference at- tendees tended to give higher scores for Social Skills, Gen- eral Impression, and Overall Score than the expert evalua- tors. Irrespectively of the level of knowledge, age, and gen- der, conference attendees rated all demos relatively homoge- neously. However, a comparative analysis of the conference attendees's ratings nationality-wise showed that demonstra- tions were rated di®erently depending on the region of ori- gin. Conference attendees for the USA and Asian countries tended to rate demos from the same country of origin more frequently and more positively.

Looking forward to a ”robotic society”? - Imaginations of future human-robot relationships

This article reports on an explorative investigation comparing the imaginations of future humanrobot relationships of user study participants who interacted with a humanoid robot for the first time on the one hand and those of experts from the industry on the other hand. By means of indepth interviews, data on the following topics was gathered from 58 user study participants and six experts: (1) quality of life, health, and security, (2) working conditions and employment, (3) education, (4) cultural context. A content analysis of the interview material derived five repeated key aspects of the future ”robotic society”: (1) replacement, (2) competition, (3) safety and supervision, (4) increasing productivity, (5) cost and benefit assessment. In addition, a description of what makes a robot different from a machine or a human could be obtained. The article highlights the difference regarding viewpoints and understandings between novice users and experts.

A Methodological Variation for Acceptance Evaluation of Human- Robot Interaction in Public Places

Several variations of methodological approaches are used to study the social acceptance in Human-Robot Interaction. Due to the introduction of robots in the home, working practice and usage typically informing the design of new forms of technology are missing. Studying social acceptance in Human-Robot Interaction thus needs new methodological concepts. We propose a so called breaching experiment with additional ethnographic observation to close this gap. To investigate the methodological concept we have been conducting a field trial on a public place. We gathered feedback using questionnaires, in order to estimate whether this method can be beneficially to evaluate social acceptance. We could show that breaching experiments can be a useful method to investigate social acceptance in the field.

User Experience Evaluation with a Wizard of Oz Approach: Technical and Methodological Considerations

User experience evaluation in human-robot interaction is most often an expensive and difficult task. To allow the evaluation of various factors and aspects of user experience, a fully functional (humanoid) robot is recommended. This work presents technical and methodological considerations on the applicability of the Wizard of Oz (WOz) approach to enable user experience evaluation in the field of Human-Robot Interaction. We briefly describe the technical aspects of the setup, the applicability of the method, and a first case study using this methodological approach to gain an early understanding of the user experience factors that are important for the development of a human-humanoid interaction scenario.