Image classification

LCMat — Loss-Curvature Matching for Dataset Selection and Condensation

Seungjae Shin, Heesun Bae, Donghyeok Shin et al. · AISTATS 2023paper ↗code ↗

Matches the loss curvature — not just the loss value or a first-order gradient — of the original and reduced datasets over a neighborhood of the model-parameter space, deriving an implementable upper bound on the worst-case curvature gap, and applies the same objective to both coreset selection and dataset condensation, improving generalization over existing baselines in both settings.

SelMatch — SelMatch: Effectively Scaling Up Dataset Distillation via Selection-Based Initialization and Partial Updates by Trajectory Matching

Yongmin Lee, Hye Won Chung · ICML 2024notablepaper ↗code ↗

Finds that trajectory-matching distillation fails to close the coverage gap between easy and hard test samples as images-per-class grows, and addresses this by initializing the synthetic set with selection-based (coreset) samples and only partially updating them through trajectory matching, tuning the selection-to-update ratio to the target IPC; consistently outperforms leading selection-only and distillation-only methods on CIFAR-10/100 and Tiny-ImageNet from 5% to 30% subset ratios.

GADC — Geometry-Aware Dataset Condensation for Diffusion Model Training

Xiao Cui, Yulei Qin, Mo Zhu et al. · ICML 2026paper ↗code ↗

Reformulates real-subset selection for training diffusion models as a geometry-aware distribution-alignment problem via one-sided partial optimal transport, allowing unmatched mass in low-density regions so the selected subset preserves the distributional geometry diffusion likelihood objectives need, complemented by feature-statistics and semantic-consistency regularizers and a two-stage discrete optimization; reports superior fidelity and distributional coverage for training diffusion models across variants, subset sizes, resolutions and training rounds.

DQ — Dataset Quantization

Daquan Zhou, Kai Wang, Jianyang Gu et al. · ICCV 2023corepaper ↗code ↗

Partitions a dataset into non-overlapping bins by a diversity-driven criterion and samples from every bin, decoupling the compressed subset from any specific matching architecture; reports the first successful ImageNet-1K-scale, distillation-free compression at state-of-the-art ratios, and shows that 60% of ImageNet plus 20% of Alpaca instruction data trains vision and language models with negligible accuracy loss across classification, segmentation, detection and instruction tuning.

Dataset Quantization with Active Learning based Adaptive Sampling

Zhenghao Zhao, Yuzhang Shang, Junyi Wu et al. · ECCV 2024paper ↗code ↗

Shows that uniform per-class sample counts are unnecessary for dataset quantization -- some classes tolerate large reductions with little performance loss -- and proposes an active-learning-based adaptive sampling strategy (DQAS) that reallocates samples from stable to sensitive classes, plus a revised pipeline that uses final-stage quantization features to build more precise bins; outperforms prior dataset compression methods across multiple datasets.

Adaptive Dataset Quantization

Muquan Li, Dongyang Zhang, Qiang Dong et al. · AAAI 2025notablepaper ↗

Replaces Dataset Quantization's uniform bin sampling with an adaptive strategy that scores each generated bin's representativeness (texture level), diversity (contrastive-learning based) and importance, then samples accordingly; reports an average 3% accuracy gain over DQ across benchmarks with better cross-architecture generalization.

Dataset Color Quantization: A Training-Oriented Framework for Dataset-Level Compression

Chenyue Yu, Lingao Xiao, Jinhong Deng et al. · ICLR 2026paper ↗

Reduces color-space redundancy across a dataset by enforcing consistent palettes across similar images, selectively retaining semantically important colors under model-perception guidance, and preserving structural detail needed for feature learning; reports improved training performance under aggressive compression on CIFAR-10/100, Tiny-ImageNet and ImageNet-1K.

DD — Dataset Distillation

Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba et al. · arXiv 2018landmarkpaper ↗code ↗

Introduces dataset distillation: a small set of synthetic training images, not necessarily on the real-data manifold, optimized by backpropagating through several steps of gradient descent so that training a fixed-initialization network on them approximates training on the full dataset, compressing 60,000 MNIST images into 10 synthetic images with close to the original accuracy.

RaT-BPTT — Embarassingly Simple Dataset Distillation

Yunzhen Feng, Ramakrishna Vedantam, Julia Kempe · ICLR 2024notablepaper ↗code ↗

Revisits backpropagation-through-time dataset distillation directly as a bi-level optimization problem and introduces Random Truncated BPTT, combining gradient truncation with a random window over the unrolled steps to stabilize gradients and cover long-term dependencies, setting new state-of-the-art results across standard benchmarks and revealing pronounced intercorrelation among the resulting distilled points that a boosting mechanism can exploit to build near-optimal subsets at multiple budgets.

Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation

Muquan Li, Hang Gou, Dongyang Zhang et al. · NeurIPS 2025paper ↗

Replaces the random truncation used in backpropagation-through-time dataset distillation with Automatic Truncated BPTT (AT-BPTT), which chooses truncation windows and sizes from stage-aware gradient statistics and a low-rank Hessian approximation, reporting a 6.16-point average accuracy gain over baselines on CIFAR-10/100, Tiny-ImageNet and ImageNet-1K along with a 3.9x speedup and 63% less memory.

InfMatch — Dataset Distillation by Influence Matching

Haoru Tan, Wang Wang, Sitong Wu et al. · CVPR 2026Vision–languagepaper ↗code ↗

Replaces process-level surrogates (per-step gradients or training trajectories) with a differentiable, sample-level estimator of each synthetic point's influence on the converged model parameters, computed in linear time by unrolling the optimization dynamics with a first-order Taylor approximation, then learns the synthetic set by matching this influence to that of the real data; reports 31.5% accuracy on Tiny-ImageNet at IPC-10 (a 4.7-point gain over NCFM) and extends to vision-language distillation on Flickr30K.

KIP — Dataset Meta-Learning from Kernel Ridge-Regression

Timothy Nguyen, Zhourong Chen, Jaehoon Lee · ICLR 2021landmarkpaper ↗code ↗

Introduces Kernel Inducing Points: treats dataset distillation as meta-learning support points for kernel ridge regression under the infinite-width neural-tangent-kernel correspondence, giving a closed-form inner solution and compressing MNIST and CIFAR-10 by one to two orders of magnitude while the resulting synthetic data remains usable to train finite-width networks.

KIP-ConvNet — Dataset Distillation with Infinitely Wide Convolutional Networks

Timothy Nguyen, Roman Novak, Lechao Xiao et al. · NeurIPS 2021notablepaper ↗code ↗

Extends KIP's kernel-ridge-regression meta-learning to infinitely wide convolutional networks via a distributed NTK computation, reporting over 65% CIFAR-10 test accuracy from just 10 datapoints — up from a prior best of 40% — with matching state-of-the-art results across MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100 and SVHN.

FRePo — Dataset Distillation using Neural Feature Regression

Yongchao Zhou, Ehsan Nezhadarya, Jimmy Ba · NeurIPS 2022corepaper ↗code ↗

Replaces the closed-form kernel-ridge-regression solver of prior kernel-based distillation with regression onto the features of a pool of finite-width networks (neural Feature Regression with Pooling), reducing memory by an order of magnitude and training time by two orders of magnitude while improving accuracy on CIFAR-100, Tiny-ImageNet and ImageNet-1K.

RFAD — Efficient Dataset Distillation Using Random Feature Approximation

Noel Loo, Ramin Hasani, Alexander Amini et al. · NeurIPS 2022notablepaper ↗code ↗

Replaces KIP's exact O(|S|^2) neural tangent kernel computation with a random feature approximation of the Neural Network Gaussian Process kernel that scales as O(|S|), giving at least a 100x speedup that runs on a single GPU while matching KIP's accuracy across kernel-regression and finite-width training, including on model-interpretability and privacy-preservation tasks.

RCIG — Dataset Distillation with Convexified Implicit Gradients

Noel Loo, Ramin Hasani, Mathias Lechner et al. · ICML 2023notablepaper ↗code ↗

Formulates dataset distillation as a bi-level problem solved with implicit gradients, convexifies the meta-gradient computation by learning on top of a frozen finite-width neural tangent kernel, and reduces implicit-gradient bias by analytically computing final-layer parameters from the body parameters; reports a 108% average improvement over prior distillation algorithms at one image per class on resized ImageNet, 66% on Tiny-ImageNet and 37% on CIFAR-100.

Provable KRR DD — Provable and Efficient Dataset Distillation for Kernel Ridge Regression

Yilan Chen, Wei Huang, Tsui-Wei Weng · NeurIPS 2024notablepaper ↗

Proves that for kernel ridge regression one data point per class is necessary and sufficient to exactly recover the original model in many settings, derives necessary and sufficient conditions for exact recovery with linear and surjective-feature-map kernels, and shows k+1 points suffice for deep linear networks with k classes; the resulting closed-form construction outperforms KIP while reported up to 15,840x faster on CIFAR-100.

DNTK — Efficient Analysis of the Distilled Neural Tangent Kernel

Jamie Mahowald, Brian Bell, Alex Ho et al. · arXiv 2026paper ↗

Shows that the neural tangent kernel's dependence on per-point Jacobians can be reduced by compressing the input data itself with NTK-tuned dataset distillation, cutting required Jacobian evaluations by 20-100x, and that per-class NTK matrices keep a low effective rank under this reduction. Combines NTK-tuned distillation with existing projection and sketching methods as the distilled neural tangent kernel (DNTK), reporting up to five further orders of magnitude reduction in NTK computation while preserving kernel structure and predictive performance.

DC — Dataset Condensation with Gradient Matching

Bo Zhao, Konda Reddy Mopuri, Hakan Bilen · ICLR 2021landmarkpaper ↗code ↗

Introduces dataset condensation as gradient matching: at every training step, match the gradients a network computes on a small synthetic set to the gradients it computes on the corresponding class of real data, learning the synthetic images by backpropagating through this per-step matching loss rather than unrolling full training.

DSA — Dataset Condensation with Differentiable Siamese Augmentation

Bo Zhao, Hakan Bilen · ICML 2021landmarkpaper ↗code ↗

Adds differentiable Siamese augmentation — applying the same random augmentation to a real and a synthetic batch before computing the gradient-matching loss — reporting roughly 7-point accuracy gains on CIFAR-10 and CIFAR-100 over the unaugmented gradient-matching baseline.

DCC — Dataset Condensation with Contrastive Signals

Saehyung Lee, Sanghyuk Chun, Sangwon Jung et al. · ICML 2022paper ↗code ↗

Shows that class-wise gradient matching can perform worse than random selection when the dataset contains a large share of task-irrelevant information, traces this to the loss function's lack of a contrastive signal between classes, and adds a modified loss plus a bi-level warm-up that recovers effective synthesis for fine-grained classification and improves over gradient-matching baselines on SVHN, CIFAR-10 and CIFAR-100.

Learning to Generate Synthetic Training Data using Gradient Matching and Implicit Differentiation

Dmitry Medvedev, Alexander D'yakonov · AIST 2021paper ↗code ↗

Combines ideas from Generative Teaching Networks, gradient matching and the Implicit Function Theorem into new data distillation techniques aimed at reducing training-data requirements. Reports the new methods are more computationally efficient than the prior techniques they combine and improve the performance of models trained on the distilled MNIST data.

Data-efficient Neural Network Training with Dataset Condensation

Bo Zhao · The University of Edinburgh 2023paper ↗

PhD thesis presenting the author's own gradient-matching (DC), differentiable-Siamese-augmentation (DSA) and distribution-matching (DM) approaches to dataset condensation as one body of work on data-efficient neural network training.

LCMat — Loss-Curvature Matching for Dataset Selection and Condensation

Seungjae Shin, Heesun Bae, Donghyeok Shin et al. · AISTATS 2023paper ↗code ↗

Matches the loss curvature — not just the loss value or a first-order gradient — of the original and reduced datasets over a neighborhood of the model-parameter space, deriving an implementable upper bound on the worst-case curvature gap, and applies the same objective to both coreset selection and dataset condensation, improving generalization over existing baselines in both settings.

IADD — Importance-Aware Adaptive Dataset Distillation

Guang Li, Ren Togo, Takahiro Ogawa et al. · NN 2024paper ↗

Observes that parameter-/gradient-matching distillation treats every network parameter as equally important and proposes IADD, which automatically assigns importance weights to different parameters during distillation to synthesize more robust distilled sets. Reports outperforming other parameter-matching SOTA methods on multiple benchmarks and in cross-architecture generalization, and validates the method on real-world COVID-19 detection.

MTT — Dataset Distillation by Matching Training Trajectories

George Cazenavette, Tongzhou Wang, Antonio Torralba et al. · CVPR 2022landmarkpaper ↗code ↗

Introduces trajectory matching: instead of matching single-step gradients, optimize synthetic data so that training a network on it for several steps lands close to where a network trained on real data lands after many more steps, using precomputed, cached expert training trajectories to avoid recomputing them, and shows this outperforms prior methods and scales to higher-resolution images.

TESLA — Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory

Justin Cui, Ruochen Wang, Si Si et al. · ICML 2023corepaper ↗code ↗

Derives a way to compute MTT's unrolled trajectory-matching gradient exactly with constant memory (about 6x less), letting trajectory matching scale to ImageNet-1K, and introduces a soft-label assignment that improves convergence on datasets with many classes; scales to 50 images per class on ImageNet-1K on a single GPU (versus 2 IPC for prior methods) with only a 5.9-point accuracy drop from full-data training using 4.2% of the data.

FTD — Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation

Jiawei Du, Yidi Jiang, Vincent Y. F. Tan et al. · CVPR 2023notablepaper ↗code ↗

Identifies the accumulated trajectory error of trajectory-matching distillation — the mismatch between the short synthetic trajectory optimized during distillation and the long trajectory used at evaluation — and regularizes expert trajectories toward flatter minima so that weights trained on the synthetic data are more robust to this error, improving accuracy by up to 4.7 points on a higher-resolution ImageNet subset.

DATM — Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching

Ziyao Guo, Kai Wang, George Cazenavette et al. · ICLR 2024corepaper ↗code ↗

Aligns the training-stage difficulty of the expert-trajectory segments matched during trajectory-matching distillation with the size of the synthetic set — early, easy-pattern segments for small budgets and late, hard-pattern segments for larger ones — letting trajectory matching keep improving as the synthetic set grows and reporting the first near-lossless dataset-distillation results.

SeqMatch — Sequential Subset Matching for Dataset Distillation

Jiawei Du, Qin Shi, Joey Tianyi Zhou · NeurIPS 2023paper ↗code ↗

Argues that optimizing an entire synthetic dataset as one static, uniformly-updated entity couples its instances together and prevents later-training-epoch high-level features from being captured, and instead generates the synthetic instances sequentially so each new subset acquires knowledge the earlier subsets have not, improving over prior matching-based methods on SVHN, CIFAR-10/100 and Tiny-ImageNet.

SelMatch — SelMatch: Effectively Scaling Up Dataset Distillation via Selection-Based Initialization and Partial Updates by Trajectory Matching

Yongmin Lee, Hye Won Chung · ICML 2024notablepaper ↗code ↗

Finds that trajectory-matching distillation fails to close the coverage gap between easy and hard test samples as images-per-class grows, and addresses this by initializing the synthetic set with selection-based (coreset) samples and only partially updating them through trajectory matching, tuning the selection-to-update ratio to the target IPC; consistently outperforms leading selection-only and distillation-only methods on CIFAR-10/100 and Tiny-ImageNet from 5% to 30% subset ratios.

MCT — Towards Stable and Storage-efficient Dataset Distillation: Matching Convexified Trajectory

Wenliang Zhong, Haoyu Tang, Qinghai Zheng et al. · CVPR 2025paper ↗code ↗

Diagnoses three problems with matching training trajectories (MTT) — instability of SGD-generated expert trajectories, slow convergence, and high storage cost for stored trajectories — and replaces the target trajectory with a convex combination of expert trajectories derived from linearized neural-tangent-kernel dynamics, giving a smaller, continuously samplable, more stable target for the student to match.

ATT — Dataset Distillation by Automatic Training Trajectories

Dai Liu, Jindong Gu, Hu Cao et al. · ECCV 2024paper ↗code ↗

Replaces MTT's fixed synthetic-step count with an automatically and adaptively adjusted trajectory length to counter the 'Accumulated Mismatching Problem' caused by forcing the synthetic set to conform to one fixed-length segment of every expert trajectory, improving cross-architecture generalization and stability over fixed-length trajectory matching.

PAD — Prioritize Alignment in Dataset Distillation

Zekai Li, Ziyao Guo, Wangbo Zhao et al. · arXiv 2024notablepaper ↗code ↗

Identifies that trajectory-matching distillation extracts and embeds 'misaligned' information from its agent (expert) model, and fixes this by pruning the target dataset to match the compression ratio before computing expert trajectories and by using only the agent model's deep layers for the matching loss, avoiding excess low-level information; reports state-of-the-art results built on trajectory matching.

LTDD — Distilling Long-tailed Datasets

Zhenghao Zhao, Haoxuan Wang, Yuzhang Shang et al. · CVPR 2025notablepaper ↗code ↗

Identifies why trajectory-matching distillation fails on long-tailed data — biased expert trajectories from imbalanced training propagate into biased synthetic sets, and experts trained on tail classes give poor guidance and soft labels — then proposes Distribution-agnostic Matching, which keeps the student trajectory away from the biased expert path, and Expert Decoupling, which matches backbone and classifier separately to improve tail-class guidance and soft-label quality. Reported as the first effective method for long-tailed dataset distillation.

NSD — Neural Spectral Decomposition for Dataset Distillation

Shaolei Yang, Shen Cheng, Mingbo Hong et al. · ECCV 2024paper ↗code ↗

Represents the whole distilled dataset as a shared set of spectrum tensors combined pairwise with per-image transformation matrices, rather than as independent images, so information is shared across the synthetic set through simple matrix multiplication, and optimizes this low-rank representation with a trajectory-matching objective guided by the real distribution, reporting state-of-the-art results on CIFAR-10/100, Tiny-ImageNet and an ImageNet subset.

Teddy — Teddy: Efficient Large-Scale Dataset Distillation via Taylor-Approximated Matching

Ruonan Yu, Songhua Liu, Jingwen Ye et al. · ECCV 2024paper ↗

Replaces the multi-step-gradient dependence of bi-level, trajectory-style dataset distillation with a memory-efficient first-order Taylor approximation, and replaces repeated per-iteration model training with a pre-cached pool of weak models generated from a single base model, reporting up to 12.8-point gains and 46.6% less runtime over prior methods on Tiny-ImageNet and full-size ImageNet-1K.

TGDD — TGDD: Trajectory Guided Dataset Distillation with Balanced Distribution

Fengli Ran, Xiao Pu, Bo Liu et al. · AAAI 2026paper ↗code ↗

Reformulates distribution matching as dynamic alignment along the model's training trajectory rather than a single static feature space, capturing how feature representations evolve during training and adding a distribution-constraint regularizer to reduce class overlap, reporting a 5.0-point accuracy gain on high-resolution benchmarks with no added optimization overhead across ten datasets.

DM — Dataset Condensation with Distribution Matching

Bo Zhao, Hakan Bilen · WACV 2023landmarkpaper ↗code ↗

Introduces distribution matching: instead of matching gradients or unrolling training, synthesize images so their class-wise feature means match those of real images across many randomly sampled embedding networks, removing the bi-level optimization entirely and making condensation cheap enough to scale to larger, more realistic datasets.

CAFE — CAFE: Learning to Condense Dataset by Aligning Features

Kai Wang, Bo Zhao, Xiangyu Peng et al. · CVPR 2022corepaper ↗code ↗

Aligns multi-scale feature statistics between real and synthetic batches instead of matching gradients directly, while preserving synthetic-set discriminability with a classification loss and a dynamic bi-level optimization schedule that adaptively prevents over- and under-fitting, reporting up to an 11-point accuracy gain over prior methods on SVHN.

Data-efficient Neural Network Training with Dataset Condensation

Bo Zhao · The University of Edinburgh 2023paper ↗

PhD thesis presenting the author's own gradient-matching (DC), differentiable-Siamese-augmentation (DSA) and distribution-matching (DM) approaches to dataset condensation as one body of work on data-efficient neural network training.

IDM — Improved Distribution Matching for Dataset Condensation

Ganlong Zhao, Guanbin Li, Yipeng Qin et al. · CVPR 2023corepaper ↗code ↗

Identifies two weaknesses of plain distribution matching — an imbalanced number of features per matching term and embeddings not validated for the distance computation — and addresses them with partitioning-and-expansion augmentation, a richer pool of partially trained sampled models, and class-aware distribution regularization, outperforming most optimization-oriented distillation methods at a fraction of their compute.

DataDAM — DataDAM: Efficient Dataset Distillation with Attention Matching

Ahmad Sajedi, Samir Khaki, Ehsan Amjadian et al. · ICCV 2023notablepaper ↗code ↗

Matches spatial attention maps computed at multiple layers of randomly initialized networks between real and synthetic batches, instead of matching gradients or single-layer features, reporting up to 6.5-point gains on CIFAR-100 and 4.1-point gains on ImageNet-1K over prior methods.

WMDD — Dataset Distillation via the Wasserstein Metric

Haoyang Liu, Yijiang Li, Tiancheng Xing et al. · ICCV 2025notablepaper ↗code ↗

Computes the Wasserstein barycenter of pretrained-classifier features from the real data and optimizes synthetic images to align with this barycenter in feature space, combined with per-class batch-norm statistics to preserve intra-class variation, reporting state-of-the-art results across several high-resolution dataset-distillation benchmarks while keeping the efficiency of distribution-matching approaches.

Dataset Distillation via Adversarial Prediction Matching

Mingyang Chen, Bo Huang, Junda Lu et al. · arXiv 2023paper ↗

Minimizes the prediction discrepancy on real data between a model trained on the full dataset and one trained on the distilled set, using a single-level adversarial framework in place of nested bi-level optimization or long trajectory unrolling. Reports distilled sets at 10% of the original size reaching 94% of full-data accuracy on average across datasets including ImageNet-1K, at 2.5x less memory and 5x less runtime than the prior state of the art, with ImageNet-1K distillation possible in 6.5GB of GPU memory.

M3D — M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancy

Hansong Zhang, Shikun Li, Pengju Wang et al. · AAAI 2024notablepaper ↗code ↗

Argues that prior distribution-matching methods only align first-moment (mean) statistics and so under-match the real and synthetic feature distributions, and proposes minimizing the maximum mean discrepancy in a reproducing kernel Hilbert space to align all orders of moments, surpassing the optimization-oriented method IDC on high-resolution ImageNet.

IID — Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation

Wenxiao Deng, Wenbin Li, Tianyu Ding et al. · CVPR 2024paper ↗code ↗

Adds a class-centralization constraint that pulls same-class synthetic features together and a covariance-matching constraint that aligns local feature covariance matrices, addressing dispersed within-class features and mean-only matching in prior distribution-matching methods, reporting up to 6.6 points on CIFAR-10 and 2.5-2.9 points on SVHN/CIFAR-100/Tiny-ImageNet over prior distribution-matching baselines.

LQM — Dataset Condensation with Latent Quantile Matching

Wei Wei, Tom De Schepper, Kevin Mets · CVPR 2024 WorkshopGraphspaper ↗

Shows that matching only the mean of latent feature embeddings, as in standard distribution matching, lets very different distributions appear identical, and proposes Latent Quantile Matching, which instead matches the quantiles of the embedding distributions to minimize a goodness-of-fit statistic; matches or beats prior distribution-matching methods on image and graph-structured datasets and improves continual graph learning.

DANCE — DANCE: Dual-View Distribution Alignment for Dataset Condensation

Hansong Zhang, Shikun Li, Fanzhao Lin et al. · IJCAI 2024paper ↗code ↗

Improves distribution matching with an inner-class 'pseudo long-term' alignment through several intermediate pretrained encoders and an inter-class distribution-calibration term from expert models, addressing the persistent-training and distribution-shift weaknesses of plain distribution matching while keeping its efficiency.

D3S — Large Scale Dataset Distillation with Domain Shift

Noel Loo, Alaa Maalouf, Ramin Hasani et al. · ICML 2024paper ↗code ↗

Reframes large-scale dataset distillation as a domain-shift problem between the synthetic and real data distributions, derives a universal upper bound on the distillation loss under this framing and optimizes it efficiently, reporting state-of-the-art results and improved cross-architecture generalization on Tiny-ImageNet, ImageNet-1K and ImageNet-21K.

DSDM — Diversified Semantic Distribution Matching for Dataset Distillation

Hongcheng Li, Yucan Zhou, Xiaoyan Gu et al. · MM 2024Other datapaper ↗code ↗

Matches both class-wise Gaussian prototypes and covariance matrices of pretrained-model features between real and synthetic data, rather than means alone, to increase within-class diversity of the distilled set, reporting state-of-the-art distribution-matching results on both image and speech datasets.

DDM — Decomposed Distribution Matching in Dataset Condensation

Sahar Rahimi Malakshan, Mohammad Saeed Ebrahimi Saadabadi, Ali Dabouei et al. · WACV 2025notablepaper ↗

Decomposes the dataset distribution that distribution matching targets into content and style, and identifies two shortcomings of the plain distribution-matching objective: a style mismatch between real and condensed data, and limited intra-class diversity in the condensed set. Matches per-layer feature-map statistical moments as a style term and maximizes intra-class KL divergence among synthetic samples as a content-diversity term, reporting accuracy gains of up to 4.1% on CIFAR-10, 4.2% on CIFAR-100, 4.3% on Tiny-ImageNet, 2.0% on ImageNet-1K, 3.3% on ImageWoof, 2.5% on ImageNette, and 5.5% in continual-learning accuracy over the distribution-matching baseline.

NCFM — Dataset Distillation with Neural Characteristic Function: A Minmax Perspective

Shaobo Wang, Yicun Yang, Zhiyuan Liu et al. · CVPR 2025notablepaper ↗code ↗

Reformulates distribution matching as a minmax problem using a Neural Characteristic Function Discrepancy, where a network optimizes the sampling of characteristic-function frequency arguments to maximize the measured discrepancy while the synthetic data is optimized to minimize it, aligning phase and amplitude of real and synthetic features in the complex plane; reports a 20.5-point gain on ImageSquawk, over 300x less GPU memory, and the first lossless CIFAR-100 compression on a single 2080 Ti.

HDD — Hyperbolic Dataset Distillation

Wenyuan Li, Guang Li, Keisuke Maeda et al. · NeurIPS 2025notablepaper ↗code ↗

Embeds pretrained-network features into Lorentz hyperbolic space and matches the hyperbolic (geodesic) distance between synthetic and real class centroids instead of Euclidean distribution matching, explicitly encoding the data's hierarchical structure into the distilled set; pruning in hyperbolic space is shown to retain model performance with only 20% of the distilled set.

OPTICAL — OPTICAL: Leveraging Optimal Transport for Contribution Allocation in Dataset Distillation

Xiao Cui, Yulei Qin, Wengang Zhou et al. · CVPR 2025paper ↗

Argues that sample-generation-based distillation methods give every real instance equal, uniform contribution when shaping each synthetic sample and so ignore instance-level real-synthetic relationships, and reformulates the matching objective as a bi-level matching-and-approximating problem where an optimal-transport matrix allocates contributions from real instances before the synthetic samples are refined against that allocation; reports gains across seven datasets and three architectures as a plug-in compatible with multiple distillation frameworks.

DEDA — Diversity-Enhanced Distribution Alignment for Dataset Distillation

Hongcheng Li, Yucan Zhou, Xiaoyan Gu et al. · ICCV 2025paper ↗

Matches both class-wise means and covariance matrices of pretrained-model features between real and synthetic data in a Gaussian-distribution-alignment scheme, then adds a regularizer that maximizes diagonal and minimizes off-diagonal covariance terms in the last feature layer specifically, countering the low diversity and gradient starvation that mean/BN-statistics-only alignment produces; reports state-of-the-art results on CIFAR-10/100, Tiny-ImageNet and ImageNet-1K with no added compute.

TGDD — TGDD: Trajectory Guided Dataset Distillation with Balanced Distribution

Fengli Ran, Xiao Pu, Bo Liu et al. · AAAI 2026paper ↗code ↗

Reformulates distribution matching as dynamic alignment along the model's training trajectory rather than a single static feature space, capturing how feature representations evolve during training and adding a distribution-constraint regularizer to reduce class overlap, reporting a 5.0-point accuracy gain on high-resolution benchmarks with no added optimization overhead across ten datasets.

GeoDM — GeoDM: Geometry-aware Distribution Matching for Dataset Distillation

Xuhui Li, Zhengquan Luo, Zihui Cui et al. · ICML 2026paper ↗

Extends distribution matching to a product of Euclidean, hyperbolic and spherical manifolds with learnable curvature per geometry, so the synthetic-data manifold can capture the flat, hierarchical and cyclical structure of the real data rather than only Euclidean statistics, adding an optimal-transport loss; reports a smaller generalization-error bound than Euclidean distribution matching and gains over state-of-the-art distillation methods on standard benchmarks.

SRe2L — Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective

Zeyuan Yin, Eric Xing, Zhiqiang Shen · NeurIPS 2023landmarkpaper ↗code ↗

Introduces Squeeze, Recover and Relabel (SRe2L), decoupling the bilevel optimization between model and synthetic data: a teacher is trained once on the real data (squeeze), synthetic images are then optimized by matching the frozen teacher's batch-norm statistics and predictions (recover), and student networks are trained against the teacher's soft labels (relabel); the pipeline that first made ImageNet-1K-scale condensation practical, reaching 60.8% top-1 accuracy on ImageNet-1K and 42.5% on Tiny-ImageNet at IPC=50, more than 30 points above prior methods, and 16-52x faster than MTT with far less memory.

G-VBSM — Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching

Shitong Shao, Zeyuan Yin, Muxin Zhou et al. · CVPR 2024notablepaper ↗code ↗

Argues that SRe2L's single-backbone, single-layer 'local-match-global' matching limits generalization, and generalizes it to match densities across many backbones, layers and statistics at once; the first algorithm to perform well at both small and large scale, reporting 38.7% on CIFAR-100 (IPC 10), 47.6% on Tiny-ImageNet (IPC 50), and 31.4% on full 224x224 ImageNet-1K (IPC 10), beating prior state of the art by 3.9-10.1 points.

CDA — Dataset Distillation via Curriculum Data Synthesis in Large Data Era

Zeyuan Yin, Zhiqiang Shen · TMLR 2024notablepaper ↗code ↗

Extends SRe2L's decoupled synthesis with curriculum data augmentation that refines the gradient update from global to local over the synthesis iterations, since the first few update steps determine the final image outline; reports 63.2% top-1 on ImageNet-1K at IPC 50 and 36.1% on ImageNet-21K at IPC 20, the first reported ImageNet-21K distillation at 224x224 resolution, beating SRe2L, TESLA and MTT by more than 4 points.

RDED — On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm

Peng Sun, Bei Shi, Daiwei Yu et al. · CVPR 2024corepaper ↗code ↗

Identifies realism, diversity and efficiency as the properties large-scale dataset distillation needs and shows existing methods trade off against them; RDED selects and stitches realistic real-image crops scored by a teacher rather than optimizing synthetic pixels, distilling all of ImageNet-1K to 10 images per class in 7 minutes and reaching 42% top-1 with ResNet-18 on a single RTX-4090, versus 21% in 6 hours for the prior state of the art.

Information Compensation: A Fix for Any-scale Dataset Distillation

Peng Sun · ICLR 2024 Workshoppaper ↗

Addresses information loss that arises when decoupled ('squeeze-recover-relabel' style) dataset distillation is applied across different compression scales or budgets. Method details could not be independently verified: OpenReview returned only a browser-verification challenge, and no alternative fetchable copy of the abstract was found.

EDC — Elucidating the Design Space of Dataset Condensation

Shitong Shao, Zikai Zhou, Huanran Chen et al. · NeurIPS 2024corepaper ↗code ↗

Systematically explores the design space of decoupled dataset condensation across scales, contributing soft category-aware matching and a tuned learning-rate schedule grounded in empirical and theoretical analysis; establishes a benchmark for small- and large-scale condensation, reporting 48.6% top-1 on ImageNet-1K with ResNet-18 at IPC 10 (0.78% compression ratio), ahead of SRe2L, G-VBSM and RDED by 27.3, 17.2 and 6.6 points respectively.

Curriculum Dataset Distillation

Zhiheng Ma, Anjia Cao, Funing Yang et al. · TIP 2025paper ↗code ↗

Synthesizes decoupled ImageNet-scale images along a simple-to-complex curriculum with curriculum evaluation, plus an adversarial term against the distilling network, to counter the homogeneity and simplicity of prior batch-to-global synthesis; reports gains of 11.1% on Tiny-ImageNet, 9.0% on ImageNet-1K and 7.3% on ImageNet-21K over prior large-scale distillation, with improved cross-architecture generalization and robustness to noise.

INFER — Breaking Class Barriers: Efficient Dataset Distillation via Inter-Class Feature Compensator

Xin Zhang, Jiawei Du, Ping Liu et al. · ICLR 2025notablepaper ↗code ↗

Breaks the one-hot, per-class synthesis barrier of prior distillation with a Universal Feature Compensator (UFC) that enhances feature integration across classes and can generate multiple additional synthetic instances from a single UFC input, plus label interpolation across classes that shrinks stored soft-label size toward zero; reports outperforming SRe2L by 34.5% at IPC=50 on ImageNet-1K with ResNet-18.

DWA — Diversity-Driven Synthesis: Enhancing Dataset Distillation through Directed Weight Adjustment

Jiawei Du, Xin Zhang, Juncheng Hu et al. · NeurIPS 2024notablepaper ↗code ↗

Provides a theoretical and empirical analysis of diversity within synthesized datasets and argues that the parallelizable but isolated synthesis common to decoupled methods needs an explicit diversity mechanism; introduces dynamic, directed weight adjustment during synthesis so each synthetic instance mirrors a large, varying subset of the real data, improving performance on CIFAR, Tiny-ImageNet and ImageNet-1K at minimal extra computational cost.

DELT — DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillation

Zhiqiang Shen, Ammar Sherif, Zeyuan Yin et al. · CVPR 2025notablepaper ↗code ↗

Diagnoses a diversity problem in batch-to-global (decoupled) matching methods such as SRe2L, G-VBSM, WMDD and CDA, where images optimized independently under the same reused global supervision end up homogeneous; DELT partitions the images-per-class budget into subtasks distilled at distinct early and late optimization phases, then generalizes back to the full task, reporting 2-5% average accuracy gains, 5%+ higher per-class diversity, and up to 39.3% less synthesis time on CIFAR, Tiny-ImageNet, ImageNet-1K and its sub-datasets.

FocusDD — FocusDD: Real-World Scene Infusion for Robust Dataset Distillation

Youbing Hu, Yun Cheng, Olga Saukh et al. · arXiv 2025paper ↗

Uses a pretrained Vision Transformer to locate key information patches in real images and stitches several such multi-object patches into one resolution-independent distilled image, augmented with a downsampled view of the original for generalization; the first method to apply distilled datasets to object detection as well as classification, reporting 71.0%/62.6% ImageNet-1K validation accuracy for ResNet50/MobileNet-v2 at IPC 100, and 24.4%/32.1% mAP for YOLOv11n/s on COCO2017 at IPC 50.

CV-DD — Dataset Distillation via Committee Voting

Jiacheng Cui, Zhaoyi Li, Xiaochen Ma et al. · arXiv 2025paper ↗code ↗

Aggregates distributions and predictions from a committee of multiple pretrained models to produce higher-quality soft labels and reduce model-specific bias in decoupled synthesis; reports consistent gains over single- and multi-model distillation baselines and improved generalization to non-training-based frameworks and synthetic-to-real transfer tasks.

NRR-DD — Enhancing Dataset Distillation via Non-Critical Region Refinement

Minh-Tuan Tran, Trung Le, Xuan-May Le et al. · CVPR 2025paper ↗code ↗

Enriches synthetic images' non-critical regions with class-general information while preserving instance-specific fine-grained detail elsewhere, instead of the common all-or-nothing choice between the two feature types; also introduces Distance-Based Representative knowledge transfer, which trains without soft labels by using the distance between synthetic-data predictions and one-hot labels, storing only two distances per instance while remaining competitive across small- and large-scale settings.

FADRM — FADRM: Fast and Accurate Data Residual Matching for Dataset Distillation

Jiacheng Cui, Xinyue Bi, Yaxin Luo et al. · NeurIPS 2025notablepaper ↗code ↗

Introduces data-level residual (skip) connections -- Data Residual Matching -- to counter information vanishing during pixel-space synthesis, balancing newly optimized signal against core information already present in the raw data, while optimization-level refinements cut training time and peak GPU memory by half; reports 47.7% (single-model) and 50.0% (multi-model) top-1 on ImageNet-1K with ResNet-18 at 0.8% compression, ahead of RDED by 5.7 points and of EDC/CV-DD by 1.4/4.0 points.

PRISM — PRISM: Diversifying Dataset Distillation by Decoupling Architectural Priors

Brian B. Moser, Shalini Sarode, Federico Raue et al. · TMLR 2026paper ↗code ↗

Argues that decoupled distillation inherits the inductive bias of whichever teacher supervises it, over-smoothing intra-class diversity as dataset size grows; PRISM decouples logit-matching from batch-normalization-alignment supervision, assigning them to different teacher architectures (a primary model for logits, a stochastic subset for BN alignment), with a scalable cross-class batch-formation scheme for fast parallel synthesis; on ImageNet-1K it reproducibly outperforms single-teacher SRe2L and multi-teacher G-VBSM at low and mid IPC with measurably richer intra-class diversity.

RLDD — Rethinking Long-tailed Dataset Distillation: A Uni-Level Framework with Unbiased Recovery and Relabeling

Xiao Cui, Yulei Qin, Xinyue Li et al. · AAAI 2026notablepaper ↗code ↗

Moves long-tailed dataset distillation away from trajectory matching to a statistical-alignment perspective, recalibrating Batch Normalization statistics via a full forward pass with dynamically adjusted momentum, using separate observer and teacher expert models for reliable statistics and soft-label generation, and initializing synthetic images by incrementally selecting high-confidence, diverse augmentations. Reports 15.6% top-1 accuracy improvement on CIFAR-100-LT and 11.8% on Tiny-ImageNet-LT at IPC=10, IF=10, over prior state-of-the-art long-tailed distillation methods.

HALD — Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local Semantic Drift

Jiacheng Cui, Bingkui Tong, Xinyue Bi et al. · ICML 2026notablepaper ↗code ↗

Shows theoretically and empirically that limiting soft-label crops per image to save storage causes local semantic drift -- visually ambiguous crops pull soft supervision away from the image's true label -- and that hybridizing hard labels back in as a content-invariant semantic anchor corrects it; HALD uses hard labels as an intermediate corrective signal alongside sparse soft labels, reaching 42.7% top-1 on ImageNet-1K with only 285MB of soft-label storage (a 100x reduction) and a 9.0-point gain over LPLD.

OGM — Beyond Soft Label: Dataset Distillation via Orthogonal Gradient Matching

Deyu Bo, Xinchao Wang · CVPR 2026notablepaper ↗

Shows theoretically that batch-norm statistic matching in decoupled synthesis mainly aligns the scale of real and synthetic gradients but overlooks their direction, though experiments show direction, not scale, is what drives training; Orthogonal Gradient Matching (OGM) orthogonalizes real and synthetic gradients by fixing all singular values to one and matches their singular vectors with a forward-pass-only least-squares loss, reporting 47.0% top-1 with soft labels and 16.7% with hard labels at IPC=10 on ImageNet-1K, ahead of RDED.

Grounding and Enhancing Informativeness and Utility in Dataset Distillation

Shaobo Wang, Yantai Yang, Guo Chen et al. · ICLR 2026notablepaper ↗

Defines Informativeness (information within one sample) and Utility (how essential a sample is to the training set) to mathematically characterize optimal dataset distillation, replacing the field's heuristic balancing of efficiency and quality; InfoUtil instantiates both with game-theoretic Shapley-value attribution to extract key per-sample information and gradient-norm-based selection of globally influential samples, reporting a 6.1% accuracy improvement over the prior state of the art on ImageNet-1K with ResNet-18.

E2D — Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization

Muhammad J. Alahmadi, Peng Gao, Feiyi Wang et al. · arXiv 2026paper ↗code ↗

Speeds up decoupled dataset distillation with a two-phase Exploration-Exploitation Distillation (E2D) schedule: initialization from full real images preserves semantic content, an exploration phase applies uniform updates to locate high-loss regions, and an exploitation phase concentrates further updates there. Reports surpassing prior decoupled methods on ImageNet-1K while running about 18x faster, and improving accuracy on ImageNet-21K at roughly 4.3x the speed.

Fixed Anchors Are Not Enough: Dynamic Retrieval and Persistent Homology for Dataset Distillation

Muquan Li, Hang Gou, Yingyi Ma et al. · CVPR 2026notablepaper ↗

Identifies a 'pull-to-anchor' effect in residual-matching decoupled pipelines that rely on static real patches, which reduces intra-class diversity; RETA's Dynamic Retrieval Connection selects a real patch from a prebuilt pool by minimizing a fit-complexity score in teacher feature space before injecting it via a residual connection, and Persistent Topology Alignment penalizes topological discrepancies between real and synthetic sets via persistence images over a mutual k-NN feature graph; reports 64.3% top-1 on ImageNet-1K with ResNet-18 at 50 images per class, +3.1 points over the prior best.

FD2 — FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation

Hongxu Ma, Guang Li, Shijie Wang et al. · ECCV 2026notablepaper ↗

Improves decoupled dataset distillation for fine-grained image datasets, where coarse class-label supervision otherwise yields distilled samples with large intra-class variation and limited inter-class separation; FD2 uses counterfactual attention learning during pretraining to build discriminative class prototypes, then applies a fine-grained characteristic constraint that aligns each sample with its prototype while repelling others, plus a similarity constraint that diversifies attention across same-class samples; reports improvements when combined with existing decoupled methods on fine-grained and general benchmarks.

CIM — Condensing Large-Scale Datasets Directly with Minimal Information Loss

Xinyi Shang, Peng Sun, Bei Shi et al. · ECCV 2026notablepaper ↗code ↗

Argues that the squeeze-recover-relabel pipeline's implicit dual compression (data to model, model back to images) causes a distribution shift that makes the pretrained model an unreliable relabeler; CIM instead directly quantifies and minimizes a metric-driven information gap between real and synthetic distributions, reporting 48.7% top-1 on ImageNet-1K at IPC=10 in 80 minutes on one RTX-4090, ahead of NRR-DD and DELT by 2.6 and 2.9 points respectively.

IT-GAN — Synthesizing Informative Training Samples with GAN

Bo Zhao, Hakan Bilen · NeurIPS 2022 Workshopcorepaper ↗code ↗

Freezes a pretrained GAN and optimizes the latent vectors that generate informative training images, rather than optimizing for visual realism, so synthesized samples are chosen to accelerate and improve downstream network training; verified to let networks train faster and reach better performance than with plain GAN samples, offering a path to dataset condensation without directly optimizing pixels.

Generative Dataset Distillation

Jovan Cicvarić · University of Tübingen 2023Other datapaper ↗

Master's thesis on generative dataset distillation by optimizing latent codes of pretrained GANs (StyleGAN2 and StyleGAN-XL) instead of pixels directly, evaluated on ImageNet-1K, CIFAR-10/100 and an imitation-learning driving task; the resulting approach placed second and won a best-paper award in the Generative Track of the 2024 Dataset Distillation Challenge.

GLaD — Generalizing Dataset Distillation via Deep Generative Prior

George Cazenavette, Tongzhou Wang, Antonio Torralba et al. · CVPR 2023landmarkpaper ↗code ↗

Distills into a small number of intermediate feature vectors in a pretrained deep generative model's latent space instead of pixels, augmenting existing matching-based distillation objectives with this generative prior; significantly improves cross-architecture generalization across settings and is among the first to combine a generative prior with dataset-distillation matching objectives at this scale.

Dataset Distillation in Latent Space

Yuxuan Duan, Jianfu Zhang, Liqing Zhang · arXiv 2023paper ↗

Moves three mainstream dataset-distillation algorithms from pixel space into the latent space of a pretrained autoencoder, cutting time and memory cost and allowing more latent codes to be stored than pixel images at the same budget, which extends distillation to higher-resolution datasets and larger data ratios than the pixel-space versions reach.

LD3M — Unlocking Dataset Distillation with Diffusion Models

Brian B. Moser, Federico Raue, Sebastian Palacio et al. · NeurIPS 2025notablepaper ↗code ↗

The first method to learn gradient-based distilled latents and class embeddings end-to-end through a pretrained latent diffusion model, rather than only sampling from a fixed prior or relying on GANs/autoencoders; a linearly decaying skip connection injected at every reverse step preserves gradient signal across dozens of timesteps without fine-tuning the diffusion weights, improving downstream accuracy by up to 4.8 points (1 IPC) and 4.2 points (10 IPC) over the prior state of the art on ImageNet subsets at 128x128 and 256x256.

H-GLaD — Hierarchical Features Matter: A Deep Exploration of Progressive Parameterization Method for Dataset Distillation

Xinhao Zhong, Hao Fang, Bin Chen et al. · CVPR 2025notablepaper ↗code ↗

Extends GLaD-style generative-prior distillation by systematically exploring hierarchical feature layers within a pretrained GAN, rather than a single fixed latent space, since different layers offer different informative guidance for distillation; also introduces a class-relevant feature-distance metric that lowers the cost of evaluating synthetic sets. Matches or surpasses diffusion-based generative distillation at extreme compression (IPC=1, IPC=10) at comparable time cost.

MGDD — MGDD: A Meta Generator for Fast Dataset Distillation

Songhua Liu, Xinchao Wang · NeurIPS 2023paper ↗

Produces synthetic images from a generator network conditioned on a dataset-distillation initialization, with synthetic labels solved in closed form via least squares in feature space, and meta-trains the generator across many datasets so that adapting to a new target needs only a few steps; reports parity with state-of-the-art distillation baselines at 22x less computation and strong generalization to synthetic-set sizes unseen during adaptation.

DiM — DiM: Distilling Dataset into Generative Model

Kai Wang, Jianyang Gu, Daquan Zhou et al. · arXiv 2023corepaper ↗code ↗

Trains a generative model to store the target dataset by minimizing the difference in logits, from a pool of models, between real and generated images, so the same trained generator produces samples for any distillation ratio and large architectures without re-optimizing; reports state-of-the-art results on four datasets, including higher accuracy on complex architectures (75.1% ResNet-18) than simple ones (72.6% ConvNet-3) at 10 images per class on CIFAR-10.

Dataset Condensation via Generative Model

David Junhao Zhang, Heng Wang, Chuhui Xue et al. · arXiv 2023paper ↗

Condenses a dataset into a trained generative model rather than pixels, so storage stays roughly constant as classes or resolution grow, with intra-class and inter-class losses that push same-class samples apart and separate class centers to increase diversity and discriminability; reports the first successful ImageNet-1K condensation using a generative-model artifact.

D2M — Data-to-Model Distillation: Data-Efficient Learning Framework

Ahmad Sajedi, Samir Khaki, Lucy Z. Liu et al. · ECCV 2024paper ↗code ↗

Transfers knowledge into the parameters of a pretrained generative model by matching real and synthetic representations, rather than storing pixels, so the same trained generator serves any distillation ratio without retraining; scales to ImageNet-1K at 128x128, validated on 15 datasets with an application to neural architecture search.

Generative Dataset Distillation: Balancing Global Structure and Local Details

Longzhen Li, Guang Li, Ren Togo et al. · CVPR 2024 Workshoppaper ↗

Trains a conditional GAN to serve as the distilled artifact while explicitly balancing global structural attributes against local details such as texture and shape, which prior generative distillation methods weighted toward high-level semantics alone; continuously optimizes the generator toward a more information-dense synthetic dataset.

Generative Dataset Distillation Based on Self-knowledge Distillation

Longzhen Li, Guang Li, Ren Togo et al. · ICASSP 2025paper ↗

Improves generative dataset distillation (a follow-up to the same authors' global/local-balancing GAN method) by integrating self-knowledge distillation for more precise logit-based distribution matching between synthetic and original data, adding a standardization step on logits before matching to keep their range consistent; reports gains over existing generative distillation baselines.

Minimax Diffusion — Efficient Dataset Distillation via Minimax Diffusion

Jianyang Gu, Saeed Vahidian, Vyacheslav Kungurtsev et al. · CVPR 2024corepaper ↗code ↗

Fine-tunes a diffusion model with additional minimax criteria that push generated images toward representativeness and diversity, modeled theoretically as hierarchical diffusion control, replacing the sample-wise iterative optimization of prior distillation methods; under 100 IPC on ImageNet-Woof, needs under one-twentieth the distillation time of prior methods while achieving better performance.

D4M — D$^4$M: Dataset Distillation via Disentangled Diffusion Model

Duo Su, Junjie Hou, Weizhi Gao et al. · CVPR 2024landmarkpaper ↗code ↗

Argues that architecture-dependent bi-level/uni-level matching objectives are the reason cross-architecture generalization degrades, and instead condenses the dataset into a latent diffusion model's space, incorporating label information into per-category prototypes so the same distilled representation serves any evaluation architecture without regenerating separate datasets; reports superior performance and robust generalization across most benchmarks compared with prior state-of-the-art methods.

Generative Dataset Distillation Based on Diffusion Model

Duo Su, Junjie Hou, Guang Li et al. · ECCV 2024 Workshoppaper ↗code ↗

Entry to the generative track of the ECCV 2024 Dataset Distillation Challenge: samples from SDXL-Turbo conditioned on class-name text prompts with post-generation augmentation, exploiting its speed to reach IPC=10 for Tiny-ImageNet and IPC=20 for CIFAR-100 within the challenge's 10-minute generation budget, versus IPC=1 for other diffusion entries; placed third in the track.

UniDD — Towards Universal Dataset Distillation via Task-Driven Diffusion

Ding Qi, Jian Li, Junyao Gao et al. · CVPR 2025notableDense predictionpaper ↗

UniDD extends dataset distillation beyond classification to detection and segmentation with a task-driven diffusion model, first mining task-relevant information by training task-specific proxy models (Universal Task Knowledge Mining), then using those proxies to guide a diffusion process that generates task-specific synthetic images with localized, region-level control rather than the global, single-class updates of prior generation methods (Universal Task-Driven Diffusion). The paper reports consistently outperforming state-of-the-art methods on ImageNet-1K, Pascal VOC and MS COCO across classification, detection and segmentation.

IGD — Influence-Guided Diffusion for Dataset Distillation

Mingyang Chen, Jiawei Du, Bo Huang et al. · ICLR 2025notablepaper ↗code ↗

Frames dataset distillation as controlled diffusion generation guided by a trajectory influence function that links generated samples to downstream training effectiveness, requiring no diffusion retraining; reports 60.3% top-1 at IPC=50 on ImageNet-1K, state-of-the-art among diffusion-based methods.

Pushforward quantization — Dataset Distillation as Pushforward Optimal Quantization

Hong Ye Tan, Emma Slade · ICLR 2026notablepaper ↗

Shows that decoupled/disentangled dataset-distillation methods with an encoder-decoder structure are instances of classical optimal quantization and Wasserstein-barycenter problems, and uses this connection to propose clustering in a diffusion model's latent space to select prototype points, reporting better performance and cross-model generalization than D4M on ImageNet-1K, plus state-of-the-art results using the distilled noise initializations with a stronger diffusion transformer.

D3HR — Taming Diffusion for Dataset Distillation with High Representativeness

Lin Zhao, Yushu Wu, Xinru Jiang et al. · ICML 2025notablepaper ↗code ↗

Identifies inaccurate distribution matching, deviation from the random-noise domain, and separate per-image sampling as flaws in diffusion-based distillation, and fixes them with DDIM inversion that maps the full dataset's latents into a high-normality Gaussian domain, plus an efficient sampling scheme that aligns representative latents to that domain; reports higher cross-architecture accuracy than state-of-the-art diffusion-based baselines.

CONCORD — CONCORD: Concept-Informed Diffusion for Dataset Distillation

Jianyang Gu, Haonan Wang, Ruoxi Jia et al. · arXiv 2025paper ↗code ↗

Retrieves fine-grained, distinguishable concepts from a large language model for each class label and feeds them into the diffusion denoising process to correct missing or incorrect object detail at the instance level, addressing a gap left by prior generative distillation methods that match only dataset-level statistics. Reports state-of-the-art results on ImageNet-1K and its subsets without relying on a pretrained classifier.

MGD³ — MGD$^3$: Mode-Guided Dataset Distillation using Diffusion Models

Jeffrey A. Chan-Santiago, Praveen Tirupattur, Gaurav Kumar Nayak et al. · ICML 2025notablepaper ↗code ↗

Uses a pretrained diffusion model with no distillation-loss fine-tuning, addressing diversity in three stages: Mode Discovery finds distinct data modes, Mode Guidance steers sampling toward intra-class diversity, and Stop Guidance halts guidance early enough to avoid artifacts; reports accuracy gains of 4.4%, 2.9%, 1.6% and 1.6% on ImageNette, ImageIDC, ImageNet-100 and ImageNet-1K over the state of the art, at much lower computational cost than fine-tuning-based methods.

Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memory

Mingzhuo Li, Guang Li, Jiafeng Mao et al. · ICIP 2025paper ↗

Uses a self-adaptive memory to measure how closely a diffusion-generated distilled set's distribution aligns with the real dataset, and feeds that alignment signal back into sampling to steer the diffusion model toward generating more diverse images during distillation. Reports outperforming existing generative dataset-distillation methods in most settings.

CaO2 — CaO$_2$: Rectifying Inconsistencies in Diffusion-Based Dataset Distillation

Haoxuan Wang, Zhenghao Zhao, Junyi Wu et al. · ICCV 2025notablepaper ↗code ↗

Identifies objective inconsistency (the distillation process diverges from the evaluation objective) and condition inconsistency (mismatches between generated images and their conditions) in diffusion-based distillation, and fixes both with a two-stage Condition-aware Optimization with Objective-guided Sampling: probability-informed sample selection, then latent refinement for higher conditional likelihood; reports a 2.3% average accuracy gain over the best baselines on ImageNet and its subsets.

VLCP — Dataset Distillation via Vision-Language Category Prototype

Yawen Zou, Guang Li, Duo Su et al. · ICCV 2025notablepaper ↗code ↗

Adds text prototypes -- derived from descriptions an open-source large language model generates for each class -- alongside image prototypes to guide diffusion-based synthesis, so distilled images incorporate semantic and contextual information image-only distillation overlooks; works even on datasets with no pre-existing text descriptions, and reports state-of-the-art validation performance with logically coherent images that better preserve target objects.

Task-Specific Generative Dataset Distillation with Difficulty-Guided Sampling

Mingzhuo Li, Guang Li, Jiafeng Mao et al. · ICCV 2025 Workshoppaper ↗code ↗

Argues that prior generative distillation aligns the distilled set with the original dataset's overall distribution but ignores what the downstream classifier specifically needs, and proposes sampling from a larger generated image pool with a distribution matched to the original dataset's per-sample difficulty, applying a logarithmic transform to correct distributional bias, rather than matching representativeness alone.

Information-Guided Diffusion Sampling for Dataset Distillation

Linfeng Ye, Shayan Mohajer Hamidi, Guang Li et al. · NeurIPS 2025 Workshoppaper ↗

Steers diffusion-model sampling for dataset distillation by maximizing a combination of prototype information I(X;Y) and IPC-scaled contextual information H(X|Y), estimated with tractable variational lower bounds, to counter the low intra-class diversity of diffusion-sampled synthetic sets at small images-per-class; reports gains over existing diffusion-based distillation methods on Tiny-ImageNet and ImageNet subsets, especially at low IPC.

Label-Consistent Dataset Distillation with Detector-Guided Refinement

Yawen Zou, Guang Li, Zi Wang et al. · arXiv 2025paper ↗

Uses a detector trained on the original dataset to flag diffusion-generated synthetic images with label mismatches or low classification confidence, generates several diffusion candidates conditioned on the flagged image's prototype and label, and replaces it with the candidate that jointly maximizes detector confidence and dissimilarity to already-accepted synthetic samples. Reports state-of-the-art validation performance from the resulting gains in label consistency and image detail.

Enhancing Diffusion-based Dataset Distillation via Adversary-Guided Curriculum Sampling

Lexiao Zou, Gongwei Chen, Yanda Chen et al. · ICME 2025paper ↗

Argues that independently diffusion-sampled images lack diversity and create redundancy when aggregated into a distilled set, and proposes Adversary-guided Curriculum Sampling: partitions the distilled set into curricula, guiding each via an adversarial loss against a discriminator trained on already-sampled images to reduce overlap, with the discriminator's difficulty increasing across curricula for simple-to-complex coverage; reports a 4.1% gain on ImageWoof and 2.1% on ImageNet-1K over prior diffusion-based state of the art.

Dataset Condensation with Color Compensation

Huyu Wu, Duo Su, Junjie Hou et al. · TMLR 2025paper ↗code ↗

After a calibrated real-image selection, fine-tunes a pretrained latent diffusion model on the condensed set to enhance the color diversity of the selected images rather than generating new ones from scratch, arguing color is both an information carrier and a semantic unit that prior condensation methods overlook; FID/IS results show the enhanced images avoid model collapse, and DC3 outperforms state-of-the-art methods across benchmarks.

DAP — Diffusion Models as Dataset Distillation Priors

Duo Su, Huyu Wu, Huanran Chen et al. · ICLR 2026notablepaper ↗

Formalizes the representativeness already implicit in a pretrained diffusion model by quantifying feature-space similarity between synthetic and real data with a Mercer kernel, then uses it as training-free guidance for the reverse diffusion process instead of adding external constraints; reports state-of-the-art fidelity and cross-architecture generalization on ImageNet-1K and its subsets.

Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation

Xiao Cui, Yulei Qin, Wengang Zhou et al. · NeurIPS 2025notablepaper ↗

Reformulates generative dataset distillation as Optimal Transport distance minimization to capture instance-level and intra-class geometry that global mean/variance matching misses, with three components: OT-guided diffusion sampling that aligns latent distributions, label-image-aligned soft relabeling that adapts label distributions to the complexity of the distilled images, and OT-based logit matching between student outputs and soft-label distributions; reports at least a 4% accuracy gain over the state of the art at IPC=10 on ImageNet-1K across architectures.

CoDA — CoDA: From Text-to-Image Diffusion Models to Training-Free Dataset Distillation

Letian Zhou, Songhua Liu, Xinchao Wang · ICLR 2026notablepaper ↗code ↗

Uses an off-the-shelf text-to-image diffusion model, with no target-dataset fine-tuning, by first discovering the target dataset's 'intrinsic core distribution' with a density-based mechanism and then steering sampling to align with it; reports 60.4% top-1 at IPC 50 on ImageNet-1K, matching or beating methods that require a dataset-specific diffusion model.

SCG — Set-Coupled Guidance: Set-Level Coordination in Diffusion-Based Dataset Distillation

Ziang Gan, Qi Zhu, Libao Zhang · ICML 2026paper ↗code ↗

Proposes a plug-and-play auxiliary controller that coordinates diffusion sampling across an entire per-class image budget at once ('IPC-at-once') rather than image by image, injecting set-symmetric feedback at each denoising step so the generated set is coordinated as a group instead of independently sampled.

Distribution shift in diffusion DD — Mitigating the Distribution Shift of Diffusion-based Dataset Distillation

Yue Xu, Chenyu Hu, Pengyu An et al. · CVPR 2026paper ↗

Identifies two distribution shifts in diffusion-based distillation: an optimal synthetic distribution should simplify, not replicate, the real data distribution given the synthetic set's small capacity, and the sampling process itself introduces a further deviation from the learned distribution; fixes both with an L1 sparsity regularizer during diffusion training and, at sampling time, a synchronous (non-sequential) denoising of the whole synthetic dataset with distribution regularizers.

DGS — Difficulty-guided Sampling: Bridging the Target Gap between Dataset Distillation and Downstream Tasks

Mingzhuo Li, Guang Li, Linfeng Ye et al. · arXiv 2026paper ↗

Argues that existing distillation objectives match dataset-level statistics but ignore task-specific information needed for the downstream classifier, creating a target gap; introduces difficulty-guided sampling (DGS), a plug-in post-processing module that resamples an existing method's pool of distilled images to match a target difficulty distribution, and difficulty-aware guidance (DAG), which steers a diffusion generation process by the same difficulty signal. Reports consistent improvements when added to multiple existing distillation settings.

ManifoldGD — ManifoldGD: Training-Free Hierarchical Manifold Guidance for Diffusion-Based Dataset Distillation

Ayush Roy, Wei-Yang Alex Lee, Rudrasis Chakraborty et al. · CVPR 2026notablepaper ↗code ↗

Argues that existing training-free diffusion guidance for distillation relies on simple mode-based centroids that are suboptimal, and instead computes a multi-scale coreset of prototypes via hierarchical divisive clustering of VAE latent features, then at every denoising step projects the mode-alignment vector onto the local tangent space of a manifold built from nearby prototypes, keeping generation manifold-faithful while staying semantically consistent; the first geometry-aware training-free distillation framework, improving FID, real-synthetic embedding distance and classification accuracy over prior training-free and training-based baselines.

EVLF — EVLF: Early Vision-Language Fusion for Generative Dataset Distillation

Wenqi Cai, Yawen Zou, Guang Li et al. · CVPR 2026notablepaper ↗code ↗

Observes that late-stage cross-attention fusion in diffusion-based distillation lets text prompts dominate generation, over-correcting samples toward prompt patterns at the expense of visual detail; Early Vision-Language Fusion instead aligns text and visual embeddings with a lightweight cross-attention module at the encoder-to-backbone transition, and plugs into any diffusion-based distillation pipeline with an encoder regardless of denoiser architecture or sampling schedule.

IMS3 — IMS3: Breaking Distributional Aggregation in Diffusion-Based Dataset Distillation

Chenru Wang, Yunyi Chen, Zijun Yang et al. · CVPR 2026notablepaper ↗

Identifies a goal misalignment in diffusion-based distillation -- diffusion models optimize generative likelihood, over-concentrating on high-density regions and under-covering the boundary samples classification needs -- and addresses it with Inversion-Matching, an inversion-guided fine-tuning process that aligns denoising trajectories with their inversion counterparts to broaden distributional coverage, plus Selective Subgroup Sampling, a training-free step that picks representative yet distinctive synthetic subsets for better inter-class separability.

Learnability-guided diffusion — Learnability-Guided Diffusion for Dataset Distillation

Jeffrey A. Chan-Santiago, Mubarak Shah · CVPR 2026notablepaper ↗

Finds that prior diffusion-based distillation methods produce redundant samples -- disjoint subsets of a distilled set share 80-90% overlapping signal -- because they optimize visual diversity or average training dynamics without accounting for sample-to-sample similarity; Learnability-Guided Diffusion instead builds the synthetic set incrementally, training a model on what exists so far and generating new samples guided by learnability scores balanced against validity under a reference model, cutting redundancy by 39.1% and reaching 60.1% on ImageNet-1K, 87.2% on ImageNette and 72.9% on ImageWoof.

DMGD — DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Models

Qichao Wang, Yunhong Lu, Hengyuan Cao et al. · CVPR 2026notablepaper ↗

Proposes training-free Dual Matching Guided Diffusion: Semantic Matching via conditional-likelihood optimization removes the need for an auxiliary classifier, a dynamic guidance mechanism balances diversity against semantic alignment, and an optimal-transport-based Distribution Matching term aligns generated samples with the target distribution's structure, with two efficiency strategies (Distribution Approximate Matching, Greedy Progressive Matching); reports average accuracy gains of 2.1%, 5.4% and 2.4% on ImageNet-Woof, -Nette and -1K over fine-tuning-based diffusion methods.

DIVER — DIVER:Diving Deeper into Distilled Data via Expressive Semantic Recovery

Qianxin Xia, Zhiyong Shu, Wenbo Jiang et al. · ICML 2026paper ↗code ↗

Takes distilled images from any prior single-stage method and 'dives deeper' with a pretrained diffusion model in three steps -- semantic inheritance projects the abstract distilled images' high-level semantics into the diffusion latent space to filter out architecture-specific noise, semantic guidance directs the reverse process to preserve that semantics, and semantic fusion applies the guidance only during the concrete phase to avoid artifacts -- improving cross-architecture generalization with runtime comparable to a raw DiT pass on ImageNet at 4GB GPU memory.

ProtoVAR — ProtoVAR: Efficient Dataset Distillation via Prototype-Guided Visual Autoregressive Modeling

Mingyu Wang, Wei Jiang · ICML 2026paper ↗

Replaces diffusion sampling with a visual autoregressive (VAR) model guided by multi-scale class prototypes, adding a fast filtering stage that keeps only the most informative generated samples; reported to run up to 70x faster than diffusion-based generative distillation while remaining competitive on ImageNet-scale benchmarks.

Path-guided flow matching — Path-Guided Flow Matching for Dataset Distillation

Xuhui Li, Zhengquan Luo, Xiwei Liu et al. · arXiv 2026paper ↗

The first flow-matching-based generative distillation framework: conducts class-conditional flow matching in a frozen VAE's latent space so synthesis solves an ODE in a few deterministic steps instead of the long stochastic chain of diffusion, with a continuous path-to-prototype guidance algorithm that keeps trajectories reliably on-path to their assigned prototype while preserving diversity; matches or beats prior diffusion-based distillation with about 7.6x higher efficiency and 78% mode coverage.

HIERAMP — HIERAMP: Coarse-to-Fine Autoregressive Amplification for Generative Dataset Distillation

Lin Zhao, Xinru Jiang, Xi Xiao et al. · CVPR 2026paper ↗code ↗

Argues that global semantic proximity alone misses the hierarchical structure of object semantics (e.g. an eye's position is constrained by the head's outline), and uses a visual autoregressive model's coarse-to-fine scales to inject class-token-guided amplification of salient regions at each scale, concentrating diversity in coarse layout choices and detail in fine object structure; consistently improves validation performance over global-proximity objectives across dataset distillation benchmarks.

IDC — Dataset Condensation via Efficient Synthetic-Data Parameterization

Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh et al. · ICML 2022corepaper ↗code ↗

Proposes an efficient synthetic-data parameterization that packs multiple down-sampled or augmented image formations into the storage of one, exploiting spatial data regularity, together with an improved optimization for gradient-matching-based condensation; substantially improves condensed-data quality over the prior state of the art on CIFAR-10, ImageNet subsets, and Speech Commands.

Addressable memories — Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks

Zhiwei Deng, Olga Russakovsky · NeurIPS 2022notablepaper ↗code ↗

Compresses a dataset into compact addressable memories -- a set of bases ('memories') shared across classes and combined through learned addressing functions to generate diverse training examples -- so compressed size need not grow linearly with the number of classes, and generalized queries beyond the original classes become possible; reports up to 16.5% and 9.7% retained-accuracy gains on CIFAR-10/100 respectively and, applied to continual learning, a 23.2% accuracy improvement on the MANY benchmark.

PRANC — PRANC: Pseudo RAndom Networks for Compacting deep models

Parsa Nooralinejad, Ali Abbasi, Soroush Abbasi Koohpayegani et al. · ICCV 2023paper ↗code ↗

Reparametrizes a deep model as a linear combination of many frozen, pseudo-random 'basis' networks generated from a single seed, so the model is stored as one scalar seed plus learned mixture coefficients; applied to compress image-classification models nearly 100x and, in one experiment, to compact individual images via their associated implicit neural representations, with memory-efficient on-the-fly weight generation for inference.

KFS — Dataset Condensation with Latent Space Knowledge Factorization and Sharing

Hae Beom Lee, Dong Bok Lee, Sung Ju Hwang · arXiv 2022paper ↗

Assumes a generative process for the distilled data -- a set of learnable latent codes in a compact space combined with several tiny decoders -- so combining codes and decoders interchangeably multiplies the number of distinct synthetic examples at nearly the same parameter count; reports new state-of-the-art results by significant margins on SVHN, CIFAR-10, CIFAR-100 and Tiny-ImageNet.

HaBa — Dataset Distillation via Factorization

Songhua Liu, Kai Wang, Xingyi Yang et al. · NeurIPS 2022corepaper ↗code ↗

Factorizes a distilled dataset into a small set of shared 'bases' and per-image 'hallucination networks' that recombine them into training images, a plug-and-play representation portable to any distillation baseline, with adversarial contrastive constraints on the bases and hallucinators to keep generated images diverse and discriminative; reports improved downstream accuracy while cutting compressed parameters by up to 65%, and about 10 points higher cross-architecture accuracy than baselines.

Translative pre-training — Few-Shot Dataset Distillation via Translative Pre-Training

Songhua Liu, Xinchao Wang · ICCV 2023paper ↗

Learns a 'distillation space' via a translator network, pretrained on large datasets with image-to-image translation, that maps synthetic images optimized cheaply in an arbitrary fixed network's space into the space of the target few-shot distillation network, needing only a few adaptation steps per new target dataset; reports about 15x faster distillation with performance comparable to iterative bi-level baselines, and satisfactory generalization across datasets, budgets and class counts.

SPEED — Sparse Parameterization for Epitomic Dataset Distillation

Xing Wei, Anjia Cao, Funing Yang et al. · NeurIPS 2023paper ↗code ↗

Applies dictionary learning and sparse coding to dataset distillation: Spatial-Agnostic Epitomic Tokens and Sparse Coding Matrices represent and select the most significant features, decoded by a Feature-Recurrent Network into hierarchical, high-compression synthetic images; reports state-of-the-art results on high-resolution benchmarks and downstream applications, compatible with a variety of matching objectives.

Slimmable DC — Slimmable Dataset Condensation

Songhua Liu, Jingwen Ye, Runpeng Yu et al. · CVPR 2023paper ↗

Introduces slimmable dataset condensation: a significance-aware parameterization whose components can be truncated to shrink an already-condensed dataset to a smaller storage budget without re-accessing the original data, addressing the inconsistency of matching networks over time and the underdetermined solution space that make naive successive compression fail; a theoretical bound shows discarding minor components is safe, and using them as initialization for further training gives fast convergence.

An Efficient Dataset Condensation Plugin and Its Application to Continual Learning

Enneng Yang, Li Shen, Zhenyi Wang et al. · NeurIPS 2023notablepaper ↗code ↗

Observes that dataset-condensation matching objectives operate in high-dimensional pixel space even though natural images are locally connected and have lower intrinsic dimension, and proposes a plugin that instead condenses images into two low-rank matrices, matching raw and synthetic data in this low-dimensional manifold for higher condensation efficiency. Demonstrated as a continual-learning replay buffer, presented at NeurIPS 2023.

HMN — Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensation

Haizhong Zheng, Jiachen Sun, Shutong Wu et al. · ECCV 2024paper ↗

Stores condensed data in a three-tier Hierarchical Memory Network representing dataset-, class- and instance-level features, matching the hierarchical way real images share features within a classification system; the hierarchy also keeps images independent enough to allow instance-level pruning of redundant information, and it outperforms prior parameterization baselines on five public datasets.

FreD — Frequency Domain-based Dataset Distillation

Donghyeok Shin, Seungjae Shin, Il-Chul Moon · NeurIPS 2023notablepaper ↗code ↗

Parameterizes each synthetic instance in the frequency domain instead of pixel space, selecting a subset of frequency dimensions by explained variance so that a limited budget is spent only on the most informative components; theoretically and empirically more budget-efficient than prior parameterizations, and composes with existing distillation methods for consistent gains across benchmarks.

H-GLaD — Hierarchical Features Matter: A Deep Exploration of Progressive Parameterization Method for Dataset Distillation

Xinhao Zhong, Hao Fang, Bin Chen et al. · CVPR 2025notablepaper ↗code ↗

Extends GLaD-style generative-prior distillation by systematically exploring hierarchical feature layers within a pretrained GAN, rather than a single fixed latent space, since different layers offer different informative guidance for distillation; also introduces a class-relevant feature-distance metric that lowers the cost of evaluating synthetic sets. Matches or surpasses diffusion-based generative distillation at extreme compression (IPC=1, IPC=10) at comparable time cost.

UDD — UDD: Dataset Distillation via Mining Underutilized Regions

Shiguang Wang, Zhongyu Zhang, Jian Cheng · PRCV 2024paper ↗

Identifies underutilized regions inside synthetic images as wasted capacity in multi-formation-style distillation, and introduces two utilization-sensitive search policies (response-based and data-jittering-based) that dynamically locate and fill these regions during optimization, together with a category-wise feature contrastive loss for inter-class separability. Reports 4.0% and 3.7% accuracy gains over the next-best method on CIFAR-10 and CIFAR-100 at IPC=1.

NSD — Neural Spectral Decomposition for Dataset Distillation

Shaolei Yang, Shen Cheng, Mingbo Hong et al. · ECCV 2024paper ↗code ↗

Represents the whole distilled dataset as a shared set of spectrum tensors combined pairwise with per-image transformation matrices, rather than as independent images, so information is shared across the synthetic set through simple matrix multiplication, and optimizes this low-rank representation with a trajectory-matching objective guided by the real distribution, reporting state-of-the-art results on CIFAR-10/100, Tiny-ImageNet and an ImageNet subset.

AutoPalette — Color-Oriented Redundancy Reduction in Dataset Distillation

Bowen Yuan, Zijian Wang, Mahsa Baktashmotlagh et al. · NeurIPS 2024paper ↗code ↗

Reduces color redundancy in distilled images with a palette network that dynamically allocates colors from a reduced palette to each pixel based on the region's importance for training, plus a color-guided initialization strategy that selects real images with the least redundant color patterns across the dataset.

DDiF — Distilling Dataset into Neural Field

Donghyeok Shin, HeeSun Bae, Gyuwon Sim et al. · ICLR 2025corepaper ↗code ↗

Parameterizes each synthetic instance as a neural field mapping coordinates to output quantities instead of storing pixels directly, theoretically shown to be more expressive than prior parameterizations at the same per-instance budget; reports superior performance on several benchmarks and extends beyond images to video, audio and 3D voxel data.

Rate-utility DD — Dataset Distillation as Data Compression: A Rate-Utility Perspective

Youneng Bao, Yiping Liu, Zhuo Chen et al. · ICCV 2025notablepaper ↗code ↗

Frames dataset distillation as a joint rate-utility optimization: synthetic samples are optimizable latent codes decoded by extremely lightweight networks, the Shannon entropy of the quantized latents is the rate term, and any existing distillation loss serves as the utility term, traded off via a Lagrange multiplier; introduces bits-per-class (bpc), a storage metric accounting for sample, label and decoder-parameter costs, and reports up to 170x greater compression than standard distillation at comparable accuracy on CIFAR-10, CIFAR-100 and ImageNet-128.

GSDD — Parameterizing Dataset Distillation via Gaussian Splatting

Chenyang Jiang, Zhengcen Li, Hang Zhao et al. · arXiv 2025paper ↗code ↗

Parameterizes each synthetic image as a set of 2D Gaussian splats rendered with CUDA-based batched splatting operators, giving a sparse yet expressive per-image representation that increases dataset diversity under a fixed storage budget; reports gains on standard benchmarks, ImageNet-1K, and video distillation.

Dataset Color Quantization: A Training-Oriented Framework for Dataset-Level Compression

Chenyue Yu, Lingao Xiao, Jinhong Deng et al. · ICLR 2026paper ↗

Reduces color-space redundancy across a dataset by enforcing consistent palettes across similar images, selectively retaining semantically important colors under model-perception guidance, and preserving structural detail needed for feature learning; reports improved training performance under aggressive compression on CIFAR-10/100, Tiny-ImageNet and ImageNet-1K.

Post Training Quantization for Efficient Dataset Condensation

Linh-Tam Tran, Sung-Ho Bae · AAAI 2026notablepaper ↗

Applies post-training quantization to already-condensed images to cut storage further: a patch-based scheme keeps quantization localized to avoid the severe degradation plain quantization causes at extreme bit-widths (e.g. 2-bit), quantization-aware clustering groups similar patches to reduce parameter overhead, and a refinement module aligns dequantized images back toward the originals; plugs into any dataset-condensation method's output, nearly doubling test accuracy at 2-bit compression on some methods (e.g. 26.0% to 54.1% for DM at IPC=1).

Soft-label DD — Soft-Label Dataset Distillation and Text Dataset Distillation

Ilia Sucholutsky, Matthias Schonlau · IJCNN 2021Textpaper ↗code ↗

Extends dataset distillation with soft (distributional) labels learned jointly with the synthetic images instead of one hard label per sample, letting a single synthetic sample encode information for multiple classes; reports 2-4% accuracy gains over hard-label distillation, over 96% MNIST accuracy from 10 distilled images and about 92% from just 5, and extends the same approach to distilling sequential text data, retaining near-original accuracy on IMDB sentiment analysis from 20 distilled sentences.

Learned labels — Flexible Dataset Distillation: Learn Labels Instead of Images

Ondrej Bohdal, Yongxin Yang, Timothy Hospedales · NeurIPS 2020 WorkshopTextpaper ↗code ↗

Studies label distillation -- learning synthetic soft labels for a small, fixed set of real images -- as an alternative to distilling images, using a more robust meta-learning algorithm and a first-order strategy based on convex optimization layers; shows label distillation improves on prior image-based distillation, generalizes across optimizers and architectures more readily, and can even transfer across datasets, e.g. learning Japanese character recognition from synthetically labeled English letters.

GIFT — GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Cost

Xinyi Shang, Peng Sun, Tao Lin · ICLR 2025notablepaper ↗code ↗

Shows that models trained on distilled data are highly sensitive to the loss function used for soft labels, and introduces GIFT, a plug-and-play soft-label refinement plus cosine-similarity loss that extracts more of the label information already present; reports consistent gains across distillation methods and scales, including a 30.8% cross-optimizer generalization improvement over RDED on ImageNet-1K at IPC=10, at no extra compute cost.

INFER — Breaking Class Barriers: Efficient Dataset Distillation via Inter-Class Feature Compensator

Xin Zhang, Jiawei Du, Ping Liu et al. · ICLR 2025notablepaper ↗code ↗

Breaks the one-hot, per-class synthesis barrier of prior distillation with a Universal Feature Compensator (UFC) that enhances feature integration across classes and can generate multiple additional synthetic instances from a single UFC input, plus label interpolation across classes that shrinks stored soft-label size toward zero; reports outperforming SRe2L by 34.5% at IPC=50 on ImageNet-1K with ResNet-18.

HeLlO — Heavy Labels Out! Dataset Distillation with Label Space Lightening

Ruonan Yu, Songhua Liu, Zigeng Chen et al. · ICCV 2025notablepaper ↗

Replaces stored per-crop soft labels with an image-to-label projector distilled from a foundation model (e.g. CLIP) via LoRA-style fine-tuning, so synthetic labels are generated online from synthetic images instead of stored, plus an image-optimization step that narrows the gap between original and distilled label generators; reports comparable accuracy to state-of-the-art distillation with only about 0.003% of the storage a full soft-label set would need on large-scale datasets.

Label-Augmented Dataset Distillation

Seoungyoon Kang, Youngsun Lim, Hyunjung Shim · WACV 2025paper ↗

Sub-samples each synthetic image to generate additional dense labels that capture richer semantics than one label per image, at only about 2.5% extra storage on ImageNet subsets; the label-generation strategy is complementary to existing distillation methods, improving three high-performance algorithms by an average of 14.9% accuracy and improving cross-architecture robustness.

DRUPI — DRUPI: Dataset Reduction Using Privileged Information

Shaobo Wang, Youxin Jiang, Tianle Niu et al. · arXiv 2024paper ↗

Introduces Dataset Condensation using Privileged Information (DRUPI/DCPI): alongside condensed images and labels, synthesizes auxiliary feature or attention labels as an additional training target; finds that moderately (not maximally) discriminative and diverse feature labels work best, and shows the technique plugs into existing condensation methods for consistent gains on ImageNet-1K, CIFAR-10/100 and Tiny-ImageNet.

LPLD — Are Large-scale Soft Labels Necessary for Large-scale Dataset Distillation?

Lingao Xiao, Yang He · NeurIPS 2024notablepaper ↗code ↗

Traces the need for large stored soft-label sets in ImageNet-scale distillation to high within-class similarity, itself caused by batch-normalization matching that mixes samples from different classes in one batch; batching within classes during synthesis instead raises within-class diversity, letting soft labels be compressed with simple random pruning; compresses required soft labels from 113GB to 2.8GB (40x) at IPC=200 on ImageNet-1K with a 2.6-point accuracy gain.

ADSA — Rectifying Soft-Label Entangled Bias in Long-Tailed Dataset Distillation

Chenyang Jiang, Hang Zhao, Xinyu Zhang et al. · NeurIPS 2025notablepaper ↗code ↗

Derives an imbalance-aware generalization bound for models trained on distilled data and traces long-tailed dataset distillation's performance loss to soft-label bias entangled from two sources, the distillation model and the distilled images themselves, identified through systematic perturbation of imbalance levels; proposes ADSA, a lightweight Adaptive Soft-label Alignment module that calibrates this bias and plugs into existing distillation pipelines. Reports up to 11.8% tail-class accuracy improvement and 41.4% overall accuracy on ImageNet-1k-LT with EDC at IPC=50.

RLDD — Rethinking Long-tailed Dataset Distillation: A Uni-Level Framework with Unbiased Recovery and Relabeling

Xiao Cui, Yulei Qin, Xinyue Li et al. · AAAI 2026notablepaper ↗code ↗

Moves long-tailed dataset distillation away from trajectory matching to a statistical-alignment perspective, recalibrating Batch Normalization statistics via a full forward pass with dynamically adjusted momentum, using separate observer and teacher expert models for reliable statistics and soft-label generation, and initializing synthetic images by incrementally selecting high-confidence, diverse augmentations. Reports 15.6% top-1 accuracy improvement on CIFAR-100-LT and 11.8% on Tiny-ImageNet-LT at IPC=10, IF=10, over prior state-of-the-art long-tailed distillation methods.

Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation

Xiao Cui, Yulei Qin, Wengang Zhou et al. · NeurIPS 2025notablepaper ↗

Reformulates generative dataset distillation as Optimal Transport distance minimization to capture instance-level and intra-class geometry that global mean/variance matching misses, with three components: OT-guided diffusion sampling that aligns latent distributions, label-image-aligned soft relabeling that adapts label distributions to the complexity of the distilled images, and OT-based logit matching between student outputs and soft-label distributions; reports at least a 4% accuracy gain over the state of the art at IPC=10 on ImageNet-1K across architectures.

HALD — Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local Semantic Drift

Jiacheng Cui, Bingkui Tong, Xinyue Bi et al. · ICML 2026notablepaper ↗code ↗

Shows theoretically and empirically that limiting soft-label crops per image to save storage causes local semantic drift -- visually ambiguous crops pull soft supervision away from the image's true label -- and that hybridizing hard labels back in as a content-invariant semantic anchor corrects it; HALD uses hard labels as an intermediate corrective signal alongside sparse soft labels, reaching 42.7% top-1 on ImageNet-1K with only 285MB of soft-label storage (a 100x reduction) and a 9.0-point gain over LPLD.

VQAE — Vector-Quantized Soft Label Compression for Dataset Distillation

Ali Abbasi, Ashkan Shahbazi, Hamed Pirsiavash et al. · arXiv 2026Textpaper ↗

Quantifies the bit cost of soft teacher labels across augmentations in dataset distillation, showing they dominate total storage in large-class settings such as ImageNet-1K, and introduces a vector-quantized autoencoder (VQAE) to compress them. Reports 30-40x additional compression over RDED, LPLD, SRe2L and CDA on ImageNet-1K while retaining over 90% of their original accuracy, and validates the approach on both vision and language distillation benchmarks.

Soft Label Pruning and Quantization for Large-Scale Dataset Distillation

Xiao Lingao, Yang He · TPAMI 2026notablepaper ↗code ↗

Extends the within-class-batching idea for reducing soft-label storage with two further mechanisms: Label Pruning with Dynamic Knowledge Reuse improves label diversity per augmentation, and Label Quantization with Calibrated Student-Teacher Alignment improves diversity per image; reduces soft-label storage by 78x on ImageNet-1K and 500x on ImageNet-21K while improving accuracy by up to 7.2 and 2.8 points respectively.

Parameter pruning DD — Dataset Distillation Using Parameter Pruning

Guang Li, Ren Togo, Takahiro Ogawa et al. · IEICE Transactions on Fundamentals 2023paper ↗

Prunes parameters that are difficult to match during the distillation process, on the premise that this yields synthetic datasets that are more robust to which parameters are matched. Reports improved distillation performance over baselines on two benchmark datasets.

Acc-DD — Accelerating Dataset Distillation via Model Augmentation

Lei Zhang, Jie Zhang, Bowen Lei et al. · CVPR 2023paper ↗code ↗

Speeds up gradient-matching dataset distillation with two model-augmentation strategies — sampling early-stage (partially trained) models and perturbing model parameters — instead of the thousands of randomly initialized models gradient matching otherwise needs, reporting up to 20x faster synthesis with accuracy comparable to prior gradient-matching methods.

Translative pre-training — Few-Shot Dataset Distillation via Translative Pre-Training

Songhua Liu, Xinchao Wang · ICCV 2023paper ↗

Learns a 'distillation space' via a translator network, pretrained on large datasets with image-to-image translation, that maps synthetic images optimized cheaply in an arbitrary fixed network's space into the space of the target few-shot distillation network, needing only a few adaptation steps per new target dataset; reports about 15x faster distillation with performance comparable to iterative bi-level baselines, and satisfactory generalization across datasets, budgets and class counts.

DREAM — DREAM: Efficient Dataset Distillation by Representative Matching

Yanqing Liu, Jianyang Gu, Kai Wang et al. · ICCV 2023paper ↗code ↗

Replaces the random sampling of real images used as matching targets in gradient-, distribution- and trajectory-matching distillation with a representative-sample selection strategy chosen for even coverage and diversity, reducing the number of distillation iterations needed by more than 8x and improving final accuracy when trained longer.

Gold from Ores — Distill Gold from Massive Ores: Bi-level Data Pruning towards Efficient Dataset Distillation

Yue Xu, Yong-Lu Li, Kaitong Cui et al. · ECCV 2024paper ↗code ↗

Studies which real training samples are essential to the distillation process itself, proposing a data-pruning criterion based on empirical loss value refined by each sample's causal effect on the distillation outcome, which improves several distillation algorithms' results, including on full ImageNet-1K and Kinetics-400.

Progressive DD — Data Distillation Can Be Like Vodka: Distilling More Times For Better Quality

Xuxi Chen, Yu Yang, Zhangyang Wang et al. · ICLR 2024paper ↗code ↗

Argues a single synthetic subset cannot represent the drastically different training dynamics of early versus late training, and instead synthesizes multiple small synthetic subsets in sequence, each conditioned on the ones before it, training on their cumulative union without extra training time; improves existing distillation methods by up to 4.3 points and is the first to scale distillation to considerably larger synthetic datasets.

YOCO — You Only Condense Once: Two Rules for Pruning Condensed Datasets

Yang He, Lingao Xiao, Joey Tianyi Zhou · NeurIPS 2023paper ↗code ↗

Introduces two pruning rules — a Low LBPE (Loss-Based Prediction Error) Score and Balanced Construction — that shrink an already-condensed dataset to smaller sizes on demand without any extra condensation runs, for on-device settings with varying and limited compute; on CIFAR-10 at IPC-10, reports 6.98-8.89 point gains over condensation baselines and 6.31-23.92 point gains over pruning baselines.

DREAM+ — DREAM+: Efficient Dataset Distillation by Bidirectional Representative Matching

Yanqing Liu, Jianyang Gu, Kai Wang et al. · arXiv 2023paper ↗code ↗

Extends DREAM's representative-sample selection to bidirectional matching across multiple dimensions of the distillation objective, plugging into gradient-, distribution- and trajectory-matching frameworks alike and cutting the number of distillation iterations needed by more than 15x without hurting accuracy.

SeqMatch — Sequential Subset Matching for Dataset Distillation

Jiawei Du, Qin Shi, Joey Tianyi Zhou · NeurIPS 2023paper ↗code ↗

Argues that optimizing an entire synthetic dataset as one static, uniformly-updated entity couples its instances together and prevents later-training-epoch high-level features from being captured, and instead generates the synthetic instances sequentially so each new subset acquires knowledge the earlier subsets have not, improving over prior matching-based methods on SVHN, CIFAR-10/100 and Tiny-ImageNet.

RaT-BPTT — Embarassingly Simple Dataset Distillation

Yunzhen Feng, Ramakrishna Vedantam, Julia Kempe · ICLR 2024notablepaper ↗code ↗

Revisits backpropagation-through-time dataset distillation directly as a bi-level optimization problem and introduces Random Truncated BPTT, combining gradient truncation with a random window over the unrolled steps to stabilize gradients and cover long-term dependencies, setting new state-of-the-art results across standard benchmarks and revealing pronounced intercorrelation among the resulting distilled points that a boosting mechanism can exploit to build near-optimal subsets at multiple budgets.

MIM4DD — MIM4DD: Mutual Information Maximization for Dataset Distillation

Yuzhang Shang, Zhihang Yuan, Yan Yan · NeurIPS 2023paper ↗

Adds a mutual-information objective, estimated via an InfoNCE-style contrastive loss between same-label (positive) and different-label (negative) real/synthetic sample pairs, as a plug-in regularizer for existing gradient- and trajectory-matching distillation methods, on the argument that maximizing shared information is a more principled compression criterion than the heuristic indicators these methods already match.

IADD — Importance-Aware Adaptive Dataset Distillation

Guang Li, Ren Togo, Takahiro Ogawa et al. · NN 2024paper ↗

Observes that parameter-/gradient-matching distillation treats every network parameter as equally important and proposes IADD, which automatically assigns importance weights to different parameters during distillation to synthesize more robust distilled sets. Reports outperforming other parameter-matching SOTA methods on multiple benchmarks and in cross-architecture generalization, and validates the method on real-world COVID-19 detection.

Model Pool — Improve Cross-Architecture Generalization on Dataset Distillation

Binglin Zhou, Linhao Zhong, Wentao Chen · arXiv 2024paper ↗

Selects distillation models from a diverse pool according to a probability distribution during synthesis, rather than optimizing against one fixed network, so the distilled images are less biased toward a single architecture, and applies knowledge distillation when evaluating the distilled set. Reports improved test accuracy across a range of evaluation architectures compared to single-model distillation baselines.

MDC — Multisize Dataset Condensation

Yang He, Lingao Xiao, Joey Tianyi Zhou et al. · ICLR 2024paper ↗code ↗

Compresses N separate condensation runs at N target sizes into a single condensation process by adding an adaptive subset loss that keeps any prefix subset of the condensed images representative on its own, addressing the 'subset degradation problem' where a subset of a larger condensed set underperforms a set condensed directly at that smaller size; reports 5.22-6.40 point average accuracy gains condensing CIFAR-10 to 10 images per class.

BACON — BACON: Bayesian Optimal Condensation Framework for Dataset Distillation

Zheng Zhou, Hongbo Zhao, Guangliang Cheng et al. · arXiv 2024paper ↗code ↗

Casts dataset distillation as minimizing an expected risk functional over the joint data distribution under a Bayesian framework and derives a numerically tractable lower bound as an approximate solution, reporting a 3.46-point accuracy gain over IDM at IPC-10 on CIFAR-10 and 3.10 points on Tiny-ImageNet when combined with existing methods.

FYI — FYI: Flip Your Images for Dataset Distillation

Byunggwan Son, Youngmin Oh, Donghyeon Baek et al. · ECCV 2024paper ↗code ↗

Identifies bilateral equivalence -- object parts appearing on one side of a real image tend to recur, mirrored, on other images in the dataset -- which pushes synthesis to duplicate discriminative parts on both sides of synthetic images and limits fine-grained discrimination; FYI embeds horizontal flipping into the distillation process to counter this without changing training objectives or architectures, and integrates into several state-of-the-art methods for consistent gains on CIFAR-10/100, Tiny-ImageNet and ImageNet.

Not All Samples Should Be Utilized Equally: Towards Understanding and Improving Dataset Distillation

Shaobo Wang, Yantai Yang, Qilong Wang et al. · CVPR 2025 Workshoppaper ↗

Measures sample difficulty by gradient norm and finds that different matching-based distillation methods implicitly favor samples of different difficulty, then extends neural scaling laws for data pruning to dataset distillation to argue that prioritizing easier real samples during synthesis improves the distilled set, especially at low IPC, packaging this as a Sample Difficulty Correction plug-in that improves 7 distillation methods across 6 datasets.

UDD — UDD: Dataset Distillation via Mining Underutilized Regions

Shiguang Wang, Zhongyu Zhang, Jian Cheng · PRCV 2024paper ↗

Identifies underutilized regions inside synthetic images as wasted capacity in multi-formation-style distillation, and introduces two utilization-sensitive search policies (response-based and data-jittering-based) that dynamically locate and fill these regions during optimization, together with a category-wise feature contrastive loss for inter-class separability. Reports 4.0% and 3.7% accuracy gains over the next-best method on CIFAR-10 and CIFAR-100 at IPC=1.

Towards Model-Agnostic Dataset Condensation by Heterogeneous Models

Jun-Yeong Moon, Jung Uk Kim, Gyeong-Moon Park · ECCV 2024paper ↗code ↗

Addresses the tendency of condensed images to overfit to the particular network used during distillation by condensing with multiple heterogeneous models simultaneously, using a Gradient Balance Module to equalize each model's contribution and a Mutual Distillation with spatial-semantic decomposition to keep their semantics aligned, improving the cross-architecture usability of condensed images built on distribution-alignment and trajectory-tracking methods.

Teddy — Teddy: Efficient Large-Scale Dataset Distillation via Taylor-Approximated Matching

Ruonan Yu, Songhua Liu, Jingwen Ye et al. · ECCV 2024paper ↗

Replaces the multi-step-gradient dependence of bi-level, trajectory-style dataset distillation with a memory-efficient first-order Taylor approximation, and replaces repeated per-iteration model training with a pre-cached pool of weak models generated from a single base model, reporting up to 12.8-point gains and 46.6% less runtime over prior methods on Tiny-ImageNet and full-size ImageNet-1K.

EDF — Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios

Kai Wang, Zekai Li, Zhi-Qi Cheng et al. · CVPR 2025paper ↗code ↗

Uses Grad-CAM activation maps to up-weight discriminative image regions and down-weights low-loss (easy, redundant) supervision signals during synthesis, targeting the small, high-activation regions typical of complex datasets, and introduces the Comp-DD benchmark of sixteen ImageNet-1K subsets (eight easy, eight hard) to measure this, reporting consistent gains over prior methods on the hard subsets.

CMI — Going Beyond Feature Similarity: Effective Dataset Distillation based on Class-Aware Conditional Mutual Information

Xinhao Zhong, Bin Chen, Hao Fang et al. · ICLR 2025paper ↗code ↗

Introduces conditional mutual information, estimated in the feature space of pretrained networks, as a class-aware complexity regularizer added to an existing dataset-distillation loss to counter synthetic sets that are needlessly hard to learn from, reporting improved accuracy and training efficiency as a general add-on to distribution-matching-based methods.

CCFS — Curriculum Coarse-to-Fine Selection for High-IPC Dataset Distillation

Yanda Chen, Gongwei Chen, Miao Zhang et al. · CVPR 2025paper ↗code ↗

Addresses the accuracy decay of dataset distillation at high images-per-class by curriculum-selecting real images to combine with the synthetic set at each stage, coarse-to-fine and conditioned on the current synthetic data rather than choosing real data once and independently, reporting gains of up to 6.6 points on CIFAR-10 and 5.8 on CIFAR-100 over prior real+synthetic combination methods at high IPC.

Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation

Muquan Li, Hang Gou, Dongyang Zhang et al. · NeurIPS 2025paper ↗

Replaces the random truncation used in backpropagation-through-time dataset distillation with Automatic Truncated BPTT (AT-BPTT), which chooses truncation windows and sizes from stage-aware gradient statistics and a low-rank Hessian approximation, reporting a 6.16-point average accuracy gain over baselines on CIFAR-10/100, Tiny-ImageNet and ImageNet-1K along with a 3.9x speedup and 63% less memory.

DiRe — DiRe: Diversity-promoting Regularization for Dataset Condensation

Saumyaranjan Mohanty, Aravind Reddy, Konda Reddy Mopuri · WACV 2026paper ↗code ↗

Proposes DiRe, a diversity regularizer combining cosine similarity and Euclidean distance terms that plugs into existing condensation methods off the shelf to reduce redundancy among synthesized samples; reports consistent generalization and diversity-metric improvements when added to state-of-the-art condensation methods from CIFAR-10 to ImageNet-1K.

DGS — Difficulty-guided Sampling: Bridging the Target Gap between Dataset Distillation and Downstream Tasks

Mingzhuo Li, Guang Li, Linfeng Ye et al. · arXiv 2026paper ↗

Argues that existing distillation objectives match dataset-level statistics but ignore task-specific information needed for the downstream classifier, creating a target gap; introduces difficulty-guided sampling (DGS), a plug-in post-processing module that resamples an existing method's pool of distilled images to match a target difficulty distribution, and difficulty-aware guidance (DAG), which steers a diffusion generation process by the same difficulty signal. Reports consistent improvements when added to multiple existing distillation settings.

FD2 — FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation

Hongxu Ma, Guang Li, Shijie Wang et al. · ECCV 2026notablepaper ↗

Improves decoupled dataset distillation for fine-grained image datasets, where coarse class-label supervision otherwise yields distilled samples with large intra-class variation and limited inter-class separation; FD2 uses counterfactual attention learning during pretraining to build discriminative class prototypes, then applies a fine-grained characteristic constraint that aligns each sample with its prototype while repelling others, plus a similarity constraint that diversifies attention across same-class samples; reports improvements when combined with existing decoupled methods on fine-grained and general benchmarks.

InfMatch — Dataset Distillation by Influence Matching

Haoru Tan, Wang Wang, Sitong Wu et al. · CVPR 2026Vision–languagepaper ↗code ↗

Replaces process-level surrogates (per-step gradients or training trajectories) with a differentiable, sample-level estimator of each synthetic point's influence on the converged model parameters, computed in linear time by unrolling the optimization dynamics with a first-order Taylor approximation, then learns the synthetic set by matching this influence to that of the real data; reports 31.5% accuracy on Tiny-ImageNet at IPC-10 (a 4.7-point gain over NCFM) and extends to vision-language distillation on Flickr30K.