A Survey on Dataset Distillation: Approaches, Applications and Future Directions

Jiahui Geng, Zongxiong Chen, Yuandou Wang, Herbert Woisetschlaeger, Sonja Schimmler, Ruben Mayer, Zhiming Zhao, Chunming Rong

IJCAI 2023 · first public 2023-05-03 · arXiv 2305.01975

paper ↗code ↗core✓ abstract read

In one paragraph

This survey proposes a taxonomy of dataset distillation approaches and systematically reviews the methods, data modalities, and applications (continual learning, neural architecture search, privacy protection) covered by the field to date, and summarizes open challenges and future directions.

Where it sits

Setting: Image classification

Abstract (verbatim from arXiv)

Dataset distillation is attracting more attention in machine learning as training sets continue to grow and the cost of training state-of-the-art models becomes increasingly high. By synthesizing datasets with high information density, dataset distillation offers a range of potential applications, including support for continual learning, neural architecture search, and privacy protection. Despite recent advances, we lack a holistic understanding of the approaches and applications. Our survey aims to bridge this gap by first proposing a taxonomy of dataset distillation, characterizing existing approaches, and then systematically reviewing the data modalities, and related applications. In addition, we summarize the challenges and discuss future directions for this field of research.

BibTeX (generated; prefer the venue's official entry)

@article{geng2023survey,
  title   = {A Survey on Dataset Distillation: Approaches, Applications and Future Directions},
  author  = {Jiahui Geng and Zongxiong Chen and Yuandou Wang and Herbert Woisetschlaeger and Sonja Schimmler and Ruben Mayer and Zhiming Zhao and Chunming Rong},
  journal = {IJCAI 2023},
  year    = {2023}
}