Method · Optimization and training recipes
Parameter pruning DD
Dataset Distillation Using Parameter Pruning
Guang Li, Ren Togo, Takahiro Ogawa, Miki Haseyama
IEICE Transactions on Fundamentals 2023 · first public 2022-09-29 · arXiv 2209.14609
In one paragraph
Prunes parameters that are difficult to match during the distillation process, on the premise that this yields synthetic datasets that are more robust to which parameters are matched. Reports improved distillation performance over baselines on two benchmark datasets.
Where it sits
- Optimization and training recipes (Orthogonal design choices)
- Setting: Image classification
Abstract (verbatim from arXiv)
In this study, we propose a novel dataset distillation method based on parameter pruning. The proposed method can synthesize more robust distilled datasets and improve distillation performance by pruning difficult-to-match parameters during the distillation process. Experimental results on two benchmark datasets show the superiority of the proposed method.
BibTeX (generated; prefer the venue's official entry)
@article{li2022dataset,
title = {Dataset Distillation Using Parameter Pruning},
author = {Guang Li and Ren Togo and Takahiro Ogawa and Miki Haseyama},
journal = {IEICE Transactions on Fundamentals 2023},
year = {2022}
}Nearby in Optimization and training recipes
PTM-ST — Multimodal Dataset Distillation via Phased Teacher Models
Shengbin Guo, Hang Zhao, Senqiao Yang et al. · ICLR 2026notableVision–languagepaper ↗code ↗