Group Distributionally Robust Optimization for Solving Out-Of-Domain Generalization and Finding Causal Invariant Relationships

Group Distributionally Robust Optimization for Solving Out-Of-Domain Generalization and Finding Causal Invariant Relationships

Guan, Zenan (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Pattern Recognition and Bioinformatics)

Krijthe, J.H. (mentor)
Karlsson, R.K.A. (mentor)
Bongers, S.R. (mentor)
Höllt, T. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2022-06-24

Out-of-Domain (OOD) generalization is a challenging problem in machine learning about learning a model from one or more domains and making the model perform well on an unseen domain. Empirical Risk Minimization (ERM), the standard machine learning method, suffers from learning spurious correlation in the training domain, therefore may perform badly when the unseen domain has different distribution from the training domain. Group Distributionally Robust Optimization (group DRO) is a method proposed to handle the OOD generalization problem. In this paper, the goals are to 1) measure if group DRO has a better OOD generalization performance than ERM. 2) evaluate if group DRO finds causally invariant relationships between the input and output. Semi-synthetic bird images with different backgrounds are used to form our data sets to construct a binary image classification problem for experiments. Results show that group DRO improves OOD generalization performance over ERM, and group DRO can find invariant relationships. However, the ability of group DRO to find invariant relationships is limited when the spurious correlation in the training domain is strong.

OOD generalization
group DRO
invariant relationships
spurious correlation

http://resolver.tudelft.nl/uuid:13b55f31-48d8-404a-98ee-36391f8db866

Student theses

bachelor thesis

© 2022 Zenan Guan