Improving Reactions to Rejection in Crowdsourcing through Self-Reflection

Improving Reactions to Rejection in Crowdsourcing through Self-Reflection

Edixhoven, Tom (Student TU Delft)
Qiu, S. (TU Delft Web Information Systems)
Kuiper, Lucie (Student TU Delft)
Dikken, Olivier (Student TU Delft)
Smitskamp, Gwennan (Student TU Delft)
Gadiraju, Ujwal (TU Delft Web Information Systems)

2021

In popular crowdsourcing marketplaces like Amazon Mechanical Turk, crowd workers complete tasks posted by requesters in return for monetary rewards. Task requesters are solely responsible for deciding whether to accept or reject submitted work. Rejecting work can directly affect the monetary reward of corresponding workers, and indirectly influence worker qualifications and their future work opportunities in the marketplace. Unexpected or unwarranted rejections therefore result in negative emotions and reactions among workers. Despite the high prevalence of rejections in crowdsourcing marketplaces, little research has explored ways to mitigate the negative emotional repercussions of rejections on crowd workers. Addressing this important research gap, we investigate whether introducing self-reflection at different stages after task execution can alleviate the emotional toll of rejection decisions on crowd workers. Our work is inspired by prior studies in psychology that have shown that self-reflection on negative personal experiences can positively affect one's emotion. To this end, we carried out an experimental study investigating the impact of explicit self-reflection on the emotions of rejected crowd workers. Results show that allowing workers to self-reflect on their delivered work, especially before receiving a rejection, has a significantly positive impact on their self-reported emotions in terms of valence and dominance. Our findings reveal that introducing a self-reflection stage before workers receive acceptance or rejection decisions on submitted work, can help in positively influencing the emotions of a worker. These findings have important design implications towards fostering a healthier requester-worker relationship and contributing towards the sustainability of the crowdsourcing marketplace.

crowdsourcing
emotion
microtask
mood
rejection
self-reflection

http://resolver.tudelft.nl/uuid:7bf66225-4576-441c-b471-e48b3d190bf1

https://doi.org/10.1145/3447535.3462482

Association for Computing Machinery (ACM)

2022-01-01

9781450383301

WebSci 2021 - Proceedings of the 13th ACM Web Science Conference

13th ACM Web Science Conference, WebSci 2021, 2021-06-21 → 2021-06-25, Virtual, Online, United Kingdom

ACM International Conference Proceeding Series

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2021 Tom Edixhoven, S. Qiu, Lucie Kuiper, Olivier Dikken, Gwennan Smitskamp, Ujwal Gadiraju