Method · Kernel and closed-form inner solvers
MMDD
Efficient Multi-modal Dataset Distillation via Analytic Parameter Matching
Deyu Bo, Xinchao Wang
ICML 2026 · first public 2026-01
In one paragraph
MMDD replaces the bi-trajectory distillation used by prior multimodal dataset distillation methods (aligning expert and student trajectories per modality, which needs many stored checkpoints and double backpropagation) with analytic parameter matching, directly matching the closed-form analytic parameters of the modal projectors, which the paper connects theoretically to matrix whitening. It caches only two matrices instead of trajectory checkpoints and computes the match in a single forward pass, reporting up to 65x storage reduction and 9.6x distillation speedup while scaling to 1000 synthetic pairs on Flickr30k and MS-COCO.
Where it sits
- Kernel and closed-form inner solvers (Bi-level performance matching)
- Setting: Image–text pairs (vision–language)
Design choices
| Labels | hard |
| Prior / networks used | pretrained-classifier |
| Optimization regime | single-level |
| What is stored | pixels, embedding-features |
| Largest scale evaluated | flickr30k-coco |
Builds on
BibTeX (generated; prefer the venue's official entry)
@article{bo2026efficient,
title = {Efficient Multi-modal Dataset Distillation via Analytic Parameter Matching},
author = {Deyu Bo and Xinchao Wang},
journal = {ICML 2026},
year = {2026}
}Nearby in Kernel and closed-form inner solvers
Provable KRR DD — Provable and Efficient Dataset Distillation for Kernel Ridge Regression
Yilan Chen, Wei Huang, Tsui-Wei Weng · NeurIPS 2024notablepaper ↗
Rethinking Backdoor Attacks on Dataset Distillation: A Kernel Method Perspective
Ming-Yu Chung, Sheng-Yen Chou, Chia-Mu Yu et al. · ICLR 2024notablepaper ↗
KRR-ST — Self-Supervised Dataset Distillation for Transfer Learning
Dong Bok Lee, Seanie Lee, Joonho Ko et al. · ICLR 2024corePre-training & transferpaper ↗code ↗