Empirical Investigation of Learning Curves

Empirical Investigation of Learning Curves: Assessing Convexity Characteristics

Gogora, Kristián (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Pattern Recognition and Bioinformatics)

Viering, T.J. (mentor)
Krijthe, J.H. (mentor)
Yue, Z. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2023-06-28

Nonconvexity in learning curves is almost always undesirable. A machine learning model with a non-convex learning curve either requires a larger quantity of data to observe progress in its accuracy or experiences an exponential decrease of accuracy at low sample sizes, with no improvement in accuracy even when more data points are added. This paper proposes a novel approach to determine the convexity of a learning curve, which relies on calculating the second derivative of the learning curve to estimate its convexity. Along the way, we have confirmed the correctness of the proposed method from multiple perspectives, such as testing it with baselines or establishing confidence intervals for the convexity of the learning curve. Lastly, we compare our method to an alternative method and highlight some of its shortcomings.

Learning curve
Convex functions
Machine learning

http://resolver.tudelft.nl/uuid:7faaef46-5c18-46fa-b259-11170e890cd1

Student theses

bachelor thesis

© 2023 Kristián Gogora