Method · Decoupled teacher-driven synthesis
OGM
Beyond Soft Label: Dataset Distillation via Orthogonal Gradient Matching
Deyu Bo, Xinchao Wang
CVPR 2026 · first public 2026-01
In one paragraph
Shows theoretically that batch-norm statistic matching in decoupled synthesis mainly aligns the scale of real and synthetic gradients but overlooks their direction, though experiments show direction, not scale, is what drives training; Orthogonal Gradient Matching (OGM) orthogonalizes real and synthetic gradients by fixing all singular values to one and matches their singular vectors with a forward-pass-only least-squares loss, reporting 47.0% top-1 with soft labels and 16.7% with hard labels at IPC=10 on ImageNet-1K, ahead of RDED.
Where it sits
- Decoupled teacher-driven synthesis (Decoupled synthesis)
- Setting: Image classification
Design choices
| Labels | mixed |
Builds on
BibTeX (generated; prefer the venue's official entry)
@article{bo2026beyond,
title = {Beyond Soft Label: Dataset Distillation via Orthogonal Gradient Matching},
author = {Deyu Bo and Xinchao Wang},
journal = {CVPR 2026},
year = {2026}
}Nearby in Decoupled teacher-driven synthesis
Fixed Anchors Are Not Enough: Dynamic Retrieval and Persistent Homology for Dataset Distillation
Muquan Li, Hang Gou, Yingyi Ma et al. · CVPR 2026notablepaper ↗
Grounding and Enhancing Informativeness and Utility in Dataset Distillation
Shaobo Wang, Yantai Yang, Guo Chen et al. · ICLR 2026notablepaper ↗