Using Personalized Federated Learning to train Diffusion Models

Using Personalized Federated Learning to train Diffusion Models

HADJIGEORGIOU, MARIOS (TU Delft Electrical Engineering, Mathematics and Computer Science)

Cox, B.A. (mentor)
Decouchant, Jérémie (graduation committee)
Wang, Q. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2023-06-30

Federated Learning (FL) is widely favoured in the training of machine learning models due to its privacy-preserving and data diversity benefits. In this research paper, we investigate an extension of FL referred to as Personalized Federated Learning (PFL) for the purpose of training diffusion models. We explore the personalization technique of Trans- fer Learning (TL) and analyse evaluation metrics to capture personalization scores. Transfer Learning has been proven to produce good personalization results under IID and non-IID data distributions. We explore the impact of specific hyperparame- ters and data distribution techniques and examine how the personalization results can be improved even further. We demonstrate that the learning rate and the number of base layers of the convolutional neural network(CNN) form a normal distribution in terms of per-user improvement results. Increasing the number of users creates unstable converges re- sults with the per-user personalization score experi- encing an overall improvement over the pre-trained model independent of the number of users. Our evaluations show that tuning to the optimal hyper- parameter values for specific non-IID data distri- butions produces better personalization scores than other PFL methods.

Federated learning GANs
Personalized Federated Learning
Personalisation
transfer learning
Hyperparameter tuning
Personalization Metrics

http://resolver.tudelft.nl/uuid:4c7e22a0-53dd-4fd6-ae34-29b4691d1b8c

Student theses

bachelor thesis

© 2023 MARIOS HADJIGEORGIOU