Dataset Distillation in the Era of Large-Scale Data: Methods, Analysis, and Future Directions

In one paragraph

This survey identifies four shifts since 2023 that earlier surveys predate: scaling distillation to large datasets, moving from optimizing distilled samples alone to improving the whole distillation pipeline, developing theoretical and empirical understanding of core mechanisms, and expanding from images to diverse modalities. It analyzes methods for each stage, benchmarks contemporary methods on large-scale datasets, summarizes distillation across unimodal and multimodal data, and lays out open challenges and directions.