Increasing cheat robustness of crowdsourcing tasks

Increasing cheat robustness of crowdsourcing tasks

Eickhoff, C.
De Vries, A.P.

Electrical Engineering, Mathematics and Computer Science

Intelligent Systems

2012-02-01

Crowdsourcing successfully strives to become a widely used means of collecting large-scale scientific corpora. Many research fields, including Information Retrieval, rely on this novel way of data acquisition. However, it seems to be undermined by a significant share of workers that are primarily interested in producing quick generic answers rather than correct ones in order to optimise their time-efficiency and, in turn, earn more money. Recently, we have seen numerous sophisticated schemes of identifying such workers. Those, however, often require additional resources or introduce artificial limitations to the task. In this work, we take a different approach by investigating means of a priori making crowdsourced tasks more resistant against cheaters.

crowdsourcing
user experiments
stability
human factors

http://resolver.tudelft.nl/uuid:92497c0d-d4ed-444d-97f7-b3ef6cdbdff6

https://doi.org/10.1007/s10791-011-9181-9

Springer-Verlag

1573-7659

http://link.springer.com/article/10.1007/s10791-011-9181-9

Information Retrieval, 2012 Febr.

Institutional Repository

journal article

(c)2012 Eickhoff, C., De Vries, A.P. and Springer