Coreset selection

Herding · ICML 2009

Herding greedily picks samples so that the running mean of selected features tracks the class mean, a first-moment criterion. It later became the standard selection baseline in condensation papers because it optimizes the same statistic distribution matching does, restricted to real images.

k-Center coreset · ICLR 2018

Casts selection as covering the data in feature space and chooses points that minimize the largest distance from any sample to its nearest selected one, derived from a generalization bound. The geometric baseline that recurs throughout condensation benchmarks.

Forgetting · ICLR 2019

Ranks samples by training dynamics: examples that are learned and never forgotten are redundant. On CIFAR-10, 30% of the training set can be removed this way with accuracy comparable to full-data training, far better than removing a random 30%.

EL2N / GraNd · NeurIPS 2021

Scores sample importance early in training (gradient and error norms), making training-dynamics selection much cheaper than tracking forgetting over a full run.

SelMatch · ICML 2024

Brings selection into synthesis: initializes a trajectory-matched set from selected real images and updates only part of it, because pure synthesis falls below random selection as images per class grow.

D2C — Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspective

Rui Huang, Shitong Shao, Zikai Zhou et al. · CVPR 2026notablePre-training & transferpaper ↗

Applies a condensation-based perspective to training diffusion models efficiently rather than to classification: a Select phase scores real images with a diffusion difficulty score combined with interval sampling to pick a compact, informative subset, and an Attach phase enriches each selected image with additional semantic and visual conditioning signals. Reports an FID of 4.3 after 40k steps on ImageNet 256x256 with SiT-XL/2 using only 0.8% of the training images, about 233x and 100x faster than training vanilla SiT-XL/2 and SiT-XL/2 + REPA respectively.

OD3 — OD3: Optimization-free Dataset Distillation for Object Detection

Salwa K. Al Khatib, Ahmed ElHagry, Shitong Shao et al. · ICLR 2026notableDense predictionpaper ↗code ↗

OD3 distills an object-detection dataset without any bi-level optimization, in two stages it iteratively places real object instances at suitable locations in synthesized images (candidate selection), then screens out low-confidence placements with a pretrained observer model (candidate screening). On MS COCO and PASCAL VOC at compression ratios from 0.25% to 5%, the paper reports new state-of-the-art results, surpassing the prior detection-distillation method by more than 14 points of mAP50 at a 1.0% compression ratio.

GADC — Geometry-Aware Dataset Condensation for Diffusion Model Training

Xiao Cui, Yulei Qin, Mo Zhu et al. · ICML 2026paper ↗code ↗

Reformulates real-subset selection for training diffusion models as a geometry-aware distribution-alignment problem via one-sided partial optimal transport, allowing unmatched mass in low-density regions so the selected subset preserves the distributional geometry diffusion likelihood objectives need, complemented by feature-statistics and semantic-consistency regularizers and a two-stage discrete optimization; reports superior fidelity and distributional coverage for training diffusion models across variants, subset sizes, resolutions and training rounds.

UniRain — UniRain: Unified Image Deraining with RAG-based Dataset Distillation and Multi-objective Reweighted Optimization

Qianfeng Yang, Qiyuan Guan, Xiang Chen et al. · CVPR 2026Dense predictionpaper ↗code ↗

UniRain is a unified image-deraining framework that trains one asymmetric mixture-of-experts model across rain streak/raindrop and day/night conditions; to build its mixed training set it uses a retrieval-augmented-generation-based dataset distillation pipeline that selects high-quality samples from all public deraining datasets, alongside a multi-objective reweighted optimization strategy. The paper reports favorable performance against state-of-the-art deraining models on its proposed benchmarks and multiple public datasets.

BPS — Balanced Dataset Distillation via Modeling Multiple Visual Pattern Distribution

Guanghui Shi, Xuefeng Liang, Qixiang Wen · CVPR 2026paper ↗code ↗

Models each class as multiple visual-pattern clusters via a hierarchical semantic structure, instead of assuming one cluster per class, and selects a balanced coreset of complementary class-general (center) and marginal (boundary) real samples from every cluster; proves the selected coreset aligns with the original dataset in information coverage and reports state-of-the-art results across four benchmarks with model-agnostic, one-shot-for-all-IPCs selection.

EEG-DLite — EEG-DLite: Dataset Distillation for Efficient Large EEG Model Training

Yuting Tang, Weibang Jiang, Shanglin Li et al. · AAAI 2026Other datapaper ↗code ↗

Introduces EEG-DLite, a data-selection framework for EEG foundation-model pre-training that encodes EEG segments into compact latent representations with a self-supervised autoencoder and then filters outliers and redundant samples in that latent space; reports that training on only 5% of a 2,500-hour EEG corpus curated this way matches or exceeds training on the full dataset across several downstream tasks.

Video DC study — A Large-Scale Study on Video Action Dataset Condensation

Yang Chen, Sheng Guo, Bo Zheng et al. · arXiv 2024Videopaper ↗code ↗

This large-scale study systematically examines video dataset condensation along three axes: temporal processing of video data, the evaluation protocol, and how condensation algorithms adapt to the space-time domain. It finds that labeling method matters a great deal for condensation performance, that simple sliding-window sampling suffices for temporal processing, and that dataset-distillation methods win in harder scenarios while sample-selection methods win in easier ones; it also proposes a unified evaluation protocol and reports state-of-the-art results on HMDB51, UCF101, SSv2 and K400 under it.

LCMat — Loss-Curvature Matching for Dataset Selection and Condensation

Seungjae Shin, Heesun Bae, Donghyeok Shin et al. · AISTATS 2023paper ↗code ↗

Matches the loss curvature — not just the loss value or a first-order gradient — of the original and reduced datasets over a neighborhood of the model-parameter space, deriving an implementable upper bound on the worst-case curvature gap, and applies the same objective to both coreset selection and dataset condensation, improving generalization over existing baselines in both settings.

DeepCore — DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning

Chengcheng Guo, Bo Zhao, Yanbing Bai · DEXA 2022paper ↗

Contributes a code library unifying coreset-selection methods for deep learning and an empirical study of them on CIFAR-10 and ImageNet under matched settings. Reports that although individual methods have advantages in specific settings, random selection remains a strong baseline overall.