An Empirical Analysis of InCoder on the Statement Prediction Task

An Empirical Analysis of InCoder on the Statement Prediction Task

van der Heijden, Frank (TU Delft Electrical Engineering, Mathematics and Computer Science)

Izadi, M. (mentor)
van Deursen, A. (mentor)
Lukina, A. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2022-06-24

Automatic code completions are a widely used feature when programming code efficiently. These completions can be made by various code language models, and these can be differentiated in three categories: single token completion, statement (line) completion and block completions. These completions, and in particular statement predictions are usually created using only the left context, missing key information and context on the other side. InCoder, a novel state of the art model is capable of using both contexts. In this study we aim to show the impact of using both contexts in statement completions. The results show that on average, an improvement of 9.9% exact match and similar results for Edit Similarity, BLEU-4, ROUGE-L F1, and METEOR when using both contexts instead of only the left context.

code completion
incoder
statement completion
empirical analysis

http://resolver.tudelft.nl/uuid:69bb797f-4a62-4fef-afe8-1d9492b8d28e

Student theses

bachelor thesis

© 2022 Frank van der Heijden