Print Email Facebook Twitter Building a workbench to improve sharing and reproducibility of MOOC experiments Title Building a workbench to improve sharing and reproducibility of MOOC experiments Author de Goede, Jochem (TU Delft Electrical Engineering, Mathematics and Computer Science) Contributor Hauff, Claudia (mentor) Lofi, Christoph (mentor) Erdweg, Sebastian (graduation committee) Houben, Geert-Jan (graduation committee) Degree granting institution Delft University of Technology Date 2017-08-30 Abstract This thesis improves sharing of code and reproducibility (S&R) in research for massive open online courses (MOOCs). Reproducibility is recreating an experiment by a different researcher. Science in general struggles with repro- ducibility. MOOC experiments often contain useful code that could be used by other researchers, but that code is oftentimes not shared with others. To improve S&R in MOOC research, this thesis first identifies the challenges MOOC sci- entists encounter when trying to share code and when making their experiments reproducible. Then, user interviews are performed to further understand MOOC research and to better understand the challenges identified earlier. A conceptual experimental workflow is designed and implemented in the form of a workbench. The workbench is then evaluated. The identified challenges based on literature with regards to S&R are: (1) Difficulty in selecting the right tools, (2) lack of a standardized workflow that en- ables reprodubility and (3) manual work required to enable reproducibility with- out a clear incentive to perform that work. From the user interviews, researchers indicate experiencing these same challenges. Based on these challenges, I pro- pose an experimental workflow that focuses on making research reproducible and on sharing code. This workflow is implemented in the form of a work- bench, where researchers can create and manage their MOOC experiments. This workbench allows for sharing code between researchers and is evaluated using a real-world data science task by three actual large-scale learning analytics re- searchers and a visiting volunteer researcher. The evaluation finds that the work- bench needs more work to be suitable for actual MOOC research use and that more researcher education is needed to improve sharing and reproducibility in MOOC research. Subject workbenchMOOCreproducibility To reference this document use: http://resolver.tudelft.nl/uuid:b7f50550-c80f-44bb-92ec-71aba0a4e24d Part of collection Student theses Document type master thesis Rights © 2017 Jochem de Goede Files PDF thesis_Building_a_workben ... egoede.pdf 3.21 MB Close viewer /islandora/object/uuid:b7f50550-c80f-44bb-92ec-71aba0a4e24d/datastream/OBJ/view