Title
Just the right mood for HIT!: Analyzing the role of worker moods in conversational microtask crowdsourcing
Author
Qiu, S. (TU Delft Web Information Systems) 
Gadiraju, Ujwal (TU Delft Web Information Systems)
Bozzon, A. (TU Delft Web Information Systems; TU Delft Human-Centred Artificial Intelligence)
Contributor
Bielikova, Maria (editor)
Mikkonen, Tommi (editor)
Pautasso, Cesare (editor)
Date
2020
Abstract
Conversational agents are playing an increasingly important role in providing users with natural communication environments, improving outcomes in a variety of domains in human-computer interaction. Crowdsourcing marketplaces are simultaneously flourishing, and it has never been easier to acquire large-scale human input from online workers. Recent works have revealed the potential of conversational interfaces in improving worker engagement and satisfaction. At the same time, worker moods have been shown to have significant effects on quality related outcomes. Little is known about the role of worker moods in shaping work in conversational microtask crowdsourcing. In this paper, we conducted a crowdsourcing study addressing 600 unique online workers, to investigate the role that worker moods play in conversational microtask crowdsourcing. We also explore whether suitable conversational styles of the agent can affect the performance of workers in different moods. Our results show that workers in a pleasant mood tend to produce significantly higher quality results (over 20%), exhibit greater engagement (an increase by around 19%) and report a lower cognitive load (by over 12%), and a suitable conversational style can have a significant impact on workers in different moods. Our findings advance the current understanding of conversational microtask crowdsourcing and have important implications on designing future conversational crowdsourcing systems.
Subject
Conversational agent
Conversational style
Crowdsourcing
Moods
Worker moods
Worker performance
To reference this document use:
http://resolver.tudelft.nl/uuid:4a12dc16-42e6-4cfb-97ca-47bbe30f130e
DOI
https://doi.org/10.1007/978-3-030-50578-3_26
Publisher
Springer, Cham
Embargo date
2021-11-08
ISBN
978-3-030-50577-6
Source
Web Engineering - 20th International Conference, ICWE 2020, Proceedings
Event
20th International Conference on Web Engineering, ICWE 2020, 2020-06-09 → 2020-06-12, Helsinki, Finland
Series
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 0302-9743, 12128
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2020 S. Qiu, Ujwal Gadiraju, A. Bozzon