Training diffusion models with federated learning

Training diffusion models with federated learning: A communication-efficient model for cross-silo federated image generation

de Goede, Matthijs (TU Delft Electrical Engineering, Mathematics and Computer Science)

Cox, B.A. (mentor)
Decouchant, Jérémie (mentor)
Wang, Q. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2023-06-30

The training of diffusion-based models for image generation is predominantly controlled by a select few Big Tech companies, raising concerns about privacy, copyright, and data authority due to the lack of transparency regarding training data. Hence, we propose a federated diffusion model scheme that enables the independent and collaborative training of diffusion models without exposing local data. Our approach adapts the Federated Averaging (FedAvg) algorithm to train a Denoising Diffusion Probabilistic Model (DDPM). Through a novel utilization of the underlying UNet backbone, we achieve a significant reduction of up to 74% in the number of parameters exchanged during training, compared to the naive FedAvg approach, whilst simultaneously maintaining image quality comparable to the centralized setting, as evaluated by the FID score.

Federated Learning
Diffusion Models
AI
Image Generation
Generative Models

http://resolver.tudelft.nl/uuid:49e11cf3-5a0a-40bc-9a62-1d7fe05fbe4d

Student theses

bachelor thesis

© 2023 Matthijs de Goede