Print Email Facebook Twitter Measuring Task Complexity in Human Computation Systems Title Measuring Task Complexity in Human Computation Systems Author Abcouwer, F.C.A. Contributor Houben, G.J.P.M. (mentor) Bozzon, A. (mentor) Larson, M.A. (mentor) Faculty Electrical Engineering, Mathematics and Computer Science Department Web Information Systems Programme Information Architecture Date 2015-11-05 Abstract Human computation platforms offer requesters the possibility to outsource human intelligence tasks or HITs to large amounts of workers around the world. Researching the various aspects of HIT design has the potential to improve the human computation experience for both requesters and workers. This thesis researches the relationship between quantitative aspects of HITs and their perceived complexity. Specifically, we consider three categories of task attributes: Metadata, Content and Visual. In order to measure these attributes, we constructed a real-time human computation market crawler that we used to gather HIT data and reinstantiate HITs from the largest human computation market, Amazon Mechanical Turk, and a task attribute analysis pipeline to obtain task attribute measurements from the market data. We present an analysis of the various task attributes, and perform an experiment predicting HIT throughput based on task attribute and market data. Our analysis shows that market statistics, task batch size and semantic content features can be used to predict throughput. We also conduct an initial exploration into measuring subjective HIT complexity as perceived by workers. Based on our analysis and experimental results, we identify three key points of advice for human computation task designers. Subject human computationcrowdsourcingmechanical turktask complexitytask throughput To reference this document use: http://resolver.tudelft.nl/uuid:543100c9-6712-4fd3-a8fd-34e03a714734 Part of collection Student theses Document type master thesis Rights (c) 2015 Abcouwer, F.C.A. Files PDF MSc_Thesis_FCA_Abcouwer.pdf 5.62 MB Close viewer /islandora/object/uuid:543100c9-6712-4fd3-a8fd-34e03a714734/datastream/OBJ/view