Question answering with the RoboTutor Nao

Question answering with the RoboTutor Nao

Ercan, Selman (TU Delft Electrical Engineering, Mathematics and Computer Science)

Hindriks, Koen (mentor)
Neerincx, Mark (graduation committee)
Broekens, Joost (graduation committee)
Lofi, Christoph (graduation committee)

Delft University of Technology

RoboTutor

2017-08-17

This thesis project worked towards extending the interactive capabilities of the RoboTutor Nao, by enabling it to answers natural language questions about topics such as computers, robots and the Nao itself. The main questions we focused on were 1) what a system for answering questions in Dutch and intended for elementary school age children should look like, 2) how the interaction with users should be structured for a smooth interaction and 3) what type of experimental studies and metrics are best for evaluating the quality of the QA system and assess its impact on users. We chose an existing English-language QA system and adapted it to work with Dutch content. An interaction manager was developed for handling the interaction between the system and users. Finally, the system was configured so that it could be interacted with via the Nao robot. The system was then evaluated via two experimental studies. First a quantitative evaluation without users was carried out, evaluating the performance of the system using a collection of test questions. Secondly, a field study with users was done. We took the robot to a day care center where children could ask it questions and assessed the effects of these interactive Q\&A sessions. Results from the quantitative evaluation highlighted slow answering times as the main shortcoming, while recall (70%) and precision (50%) scores were more competitive. The field study showed that even though the occasional repetitiveness and lack of speed were picked up on by users, this did little to curb their enthusiasm. Interaction was smooth, users were interested in many topics the system supported and they continued interacting with it despite the occasional mishap. Overall, the results indicate that, with some speedup, the delivered QA system would be a suitable choice for answering questions about the mentioned topics.

Question Answering
Natural Language Processing
Information Retrieval
Robots
Human-automation interaction

http://resolver.tudelft.nl/uuid:c034919a-0b1c-442a-8001-eeb95d93694d

Student theses

master thesis

© 2017 Selman Ercan