Evaluating catastrophic forgetting of state-of-the-art NLP models for predicting moral values

Evaluating catastrophic forgetting of state-of-the-art NLP models for predicting moral values

Arsene, Florentin (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Intelligent Systems)

Murukannaiah, P.K. (mentor)
Liscio, E. (mentor)
Marroquim, Ricardo (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2021-07-02

Personal moral values represent the motivation behind individuals' actions and opinions. Understanding these values is helpful both in predicting individuals' actions, such as violent protests, and building AI that can better collaborate with humans. Predicting moral values is a challenging problem due to the abstract and subjective essence of moral values. With the help of seven Twitter datasets corresponding to different domains, we train state-of-the-art Natural Language Processing models in predicting moral values. An interesting limitation of the models is that they all suffer from catastrophic forgetting. Catastrophic forgetting is the degree to which models worsen their performance on older data after being trained on new data. We conclude that catastrophic forgetting occurs irrespective of the models being trained and can be mitigated by not only training on new data but by training on a combination of old and new data. This is all possible under one assumption: old data is available. We deliver an evaluation of catastrophic forgetting for each model, explain the differences between the models, and suggest possible future work that can be built upon this research.

Natural Language Processing
Moral Values
Moral Foundation Theory
Catastrophic Forgetting

http://resolver.tudelft.nl/uuid:583dc2d7-90b0-4437-89c7-1b3da8de8992

2022-12-31

Student theses

bachelor thesis

© 2021 Florentin Arsene