Application · Meta-learning through unrolled training
Federated Learning via Synthetic Data
Jack Goetz, Ambuj Tewari
arXiv 2020 · first public 2020-08-11 · arXiv 2008.04489
In one paragraph
Proposes transmitting a small amount of synthetic data from client to server in place of a gradient update, avoiding the multi-million-parameter cost of standard federated learning updates. Reports more than an order-of-magnitude reduction in communication cost with minimal model degradation in early experiments.
Where it sits
- Meta-learning through unrolled training (Bi-level performance matching)
- Setting: Image classification
Abstract (verbatim from arXiv)
Federated learning allows for the training of a model using data on multiple clients without the clients transmitting that raw data. However the standard method is to transmit model parameters (or updates), which for modern neural networks can be on the scale of millions of parameters, inflicting significant computational costs on the clients. We propose a method for federated learning where instead of transmitting a gradient update back to the server, we instead transmit a small amount of synthetic `data'. We describe the procedure and show some experimental results suggesting this procedure has potential, providing more than an order of magnitude reduction in communication costs with minimal model degradation.
BibTeX (generated; prefer the venue's official entry)
@article{goetz2020federated,
title = {Federated Learning via Synthetic Data},
author = {Jack Goetz and Ambuj Tewari},
journal = {arXiv preprint arXiv:2008.04489},
year = {2020}
}Nearby in Meta-learning through unrolled training
Shiye Lei, Sen Zhang, Dacheng Tao · NeurIPS 2024notableOther datapaper ↗code ↗