Method · Dataset quantization
Adaptive Dataset Quantization
Muquan Li, Dongyang Zhang, Qiang Dong, Xiurui Xie, Ke Qin
AAAI 2025 · first public 2024-12-22 · arXiv 2412.16895
In one paragraph
Replaces Dataset Quantization's uniform bin sampling with an adaptive strategy that scores each generated bin's representativeness (texture level), diversity (contrastive-learning based) and importance, then samples accordingly; reports an average 3% accuracy gain over DQ across benchmarks with better cross-architecture generalization.
Where it sits
- Dataset quantization (Selection)
- Setting: Image classification
Builds on
Abstract (verbatim from arXiv)
Contemporary deep learning, characterized by the training of cumbersome neural networks on massive datasets, confronts substantial computational hurdles. To alleviate heavy data storage burdens on limited hardware resources, numerous dataset compression methods such as dataset distillation (DD) and coreset selection have emerged to obtain a compact but informative dataset through synthesis or selection for efficient training. However, DD involves an expensive optimization procedure and exhibits limited generalization across unseen architectures, while coreset selection is limited by its low data keep ratio and reliance on heuristics, hindering its practicality and feasibility. To address these limitations, we introduce a newly versatile framework for dataset compression, namely Adaptive Dataset Quantization (ADQ). Specifically, we first identify the sub-optimal performance of naive Dataset Quantization (DQ), which relies on uniform sampling and overlooks the varying importance of each generated bin. Subsequently, we propose a novel adaptive sampling strategy through the evaluation of generated bins' representativeness score, diversity score and importance score, where the former two scores are quantified by the texture level and contrastive learning-based techniques, respectively. Extensive experiments demonstrate that our method not only exhibits superior generalization capability across different architectures, but also attains state-of-the-art results, surpassing DQ by average 3\% on various datasets.
BibTeX (generated; prefer the venue's official entry)
@article{li2024adaptive,
title = {Adaptive Dataset Quantization},
author = {Muquan Li and Dongyang Zhang and Qiang Dong and Xiurui Xie and Ke Qin},
journal = {AAAI 2025},
year = {2024}
}