Explain Strange Learning Curves in Machine Learning

Explain Strange Learning Curves in Machine Learning

Chen, Zhiyi (TU Delft Electrical Engineering, Mathematics and Computer Science)

Viering, T.J. (mentor)
Loog, M. (mentor)
Smaragdakis, G. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2022-06-20

The learning curve illustrates how the generalization performance of the learner evolves with more training data. It can predict the amount of data needed for decent accuracy and the highest achievable accuracy. However, the behavior of learning curves is not well understood. Many assume that the more training data provided, the better the learner performs. However, many counter-examples exist for both classical machine learning algorithms and deep neural networks. As presented in previous works, even the learning curves for simple problems using classical machine learning algorithms have unexpected behaviors. In this paper, we will explain what caused the odd learning curves generated while using ERM to solve two regression problems. Loog et al. [1] first proposed these two problems. As a result of our study, we conclude that the unexpected behaviors of the learning curves under these two problem settings are caused by incorrect modeling or the correlation between the expected risk and the output of the learner.

Machine Learning
Learning Curve
Empirical Risk Minimizer

http://resolver.tudelft.nl/uuid:66470285-2449-450a-876d-4bc7443de4a9

Student theses

bachelor thesis

© 2022 Zhiyi Chen