Print Email Facebook Twitter Data Driven Decisions Title Data Driven Decisions: Validating and Supporting a Continuous Experimentation Development Environment Author Mulders, Ernst (TU Delft Electrical Engineering, Mathematics and Computer Science) Contributor Gousios, Giorgos (mentor) van Deursen, A. (graduation committee) Katsifodimos, A (graduation committee) Anderson, Kevin (mentor) Degree granting institution Delft University of Technology Date 2019-12-04 Abstract The number of conducted A/B tests is growing throughout companies in software development. Many of these companies develop their own in-house Experimentation Platform to support these experiments. In this thesis we identify factors that influence the trustworthiness and soundness of A/B tests by conducting a literature review. We discuss nineteen influential factors categorised as essentials and pitfalls. Using the data of 268 experiments from ING we verify the trustworthiness of ING’s own Experiment Platform, and conclude that there is room for improvement. Finally, we provide a method for developers and engineers to consider these factors during the experimentation phase by modelling them into a questionnaire containing 67 questions. These questions are grouped into three categories, which are referred to as the three A’s of A/B Test validation: Availability, Analysability and Accuracy. To help administer this questionnaire we introduce the first Open-Source toolkit for this matter: ABvalidator. Subject Continuous ExperimentationA/B TestingExperimentation To reference this document use: http://resolver.tudelft.nl/uuid:08f2c0b4-2aa8-4e12-9b58-073dcdfb4553 Part of collection Student theses Document type master thesis Rights © 2019 Ernst Mulders Files PDF Final_Thesis_Data_Driven_ ... isions.pdf 2.63 MB Close viewer /islandora/object/uuid:08f2c0b4-2aa8-4e12-9b58-073dcdfb4553/datastream/OBJ/view