Large Scale Dataset Distillation with Domain Shift

Noel Loo, Alaa Maalouf, Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus

ICML 2024 · first public 2024-07

paper ↗code ↗catalogued✓ abstract read

In one paragraph

Reframes large-scale dataset distillation as a domain-shift problem between the synthetic and real data distributions, derives a universal upper bound on the distillation loss under this framing and optimizes it efficiently, reporting state-of-the-art results and improved cross-architecture generalization on Tiny-ImageNet, ImageNet-1K and ImageNet-21K.

Where it sits

Distribution and feature matching (Surrogate matching)
Setting: Image classification

BibTeX (generated; prefer the venue's official entry)

@article{loo2024large,
  title   = {Large Scale Dataset Distillation with Domain Shift},
  author  = {Noel Loo and Alaa Maalouf and Ramin Hasani and Mathias Lechner and Alexander Amini and Daniela Rus},
  journal = {ICML 2024},
  year    = {2024}
}

Nearby in Distribution and feature matching

2026-06

RAHA — Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation

Jongoh Jeong, Sun-Kyung Lee, Kuk-Jin Yoon · ECCV 2026notableVision–language paper ↗code ↗

2026-05

MDM — Multimodal Distribution Matching for Vision-Language Dataset Distillation

Jongoh Jeong, Hoyong Kwon, Minseok Kim et al. · CVPR 2026notableVision–language paper ↗code ↗

2026-03

Sneakdoor — SNEAKDOOR: Stealthy Backdoor Attacks against Distribution Matching-based Dataset Condensation

He Yang, Dongyi Lv, Song Ma et al. · NeurIPS 2025notablepaper ↗code ↗

2026-03

Harmonic Dataset Distillation for Time Series Forecasting

Seungha Hong, Sanghwan Jang, Wonbin Kweon et al. · AAAI 2026notableTime series paper ↗

2025-11

Algorithmic Guarantees for Distilling Supervised and Offline RL Datasets

Aaryan Gupta, Rishi Saket, Aravindan Raghuveer · ICLR 2026notableOther data paper ↗