TestAxis: Save Time Fixing Broken CI Builds Without Leaving Your IDE

Title

TestAxis: Save Time Fixing Broken CI Builds Without Leaving Your IDE

 Author

Boone, Casper (TU Delft Electrical Engineering, Mathematics and Computer Science)

Contributor

Zaidman, A.E. (mentor) ORCID 0000-0003-2413-3935
Katsifodimos, A (graduation committee)
Brandt, C.E. (graduation committee)

Degree granting institution

Delft University of Technology

 Programme

Computer Science

 Date

2021-06-23

 Abstract

The most common reason for Continuous Integration (CI) build failures is failing tests. When a build fails, a developer often has to scroll through hundreds to thousands of log lines to find which test is failing and why. Finding the issue is a tedious process that relies on a developer's experience and increases the cost of software testing. Providing CI build test results with additional context in the developer's local development environment could help solve failing tests more quickly. We propose TestAxis, a test result inspection tool that brings CI test results to the Integrated Development Environment (IDE) offering an experience similar to running a test locally. Moreover, it surfaces additional information that is too expensive to collect in local development, for example, a unique view of the code under test that was changed leading up to the build failure. We implement TestAxis as a plugin for IntelliJ and conduct a user study to evaluate its usefulness and performance benefits. The participants solve programming assignments evaluating the three main features: the test results overview, the test code editor, and the changed code under test display. We show that TestAxis helps developers fix failing tests 13.4% to 30.4% faster. The participants found the features of TestAxis useful and would incorporate it in their development workflow to save time. With TestAxis we set an important step towards removing the need to manually inspect build logs and bringing CI build results to the IDE, ultimately saving developers time.

 Subject

Continuous Integration
Software Testing
IDE
Software Engineering

 To reference this document use:

http://resolver.tudelft.nl/uuid:f8375d5f-3bbd-4559-863b-6951e9d6bab0

 Part of collection

Student theses

 Document type

master thesis

 Rights

© 2021 Casper Boone
Files
PDF testaxis_casper_boone_june_2021.pdf 9.84 MB
Close viewer