Settings · other modalities
Other data types
Point clouds, tabular data, recommender interactions, speech, EEG, event streams, reinforcement learning and more, each with a handful of papers adapting condensation to its structure.
One synthetic sample
Domain-specific (a point set, a table row, a user–item matrix, a spectrogram, a state–action set, …).
Standard evaluation
Domain-specific; always training a model on the synthetic data and testing on real data.
Budget
domain-specific
Benchmarks
ModelNet / ScanObjectNN · recommendation datasets · speech emotion corpora · EEG corpora · RL environments
What is different here
- Each domain has an invariance that images lack (permutation for point sets, sparsity for interaction matrices, temporal dynamics for RL) that the synthesis must respect.
Comparison pitfalls
- Most domains have one to three papers; treat claims as early evidence.
Start with
Methods by family 16
A paper appears under every family it belongs to. Oldest first within a family.
Selection
Coreset selection 1
EEG-DLite — EEG-DLite: Dataset Distillation for Efficient Large EEG Model Training
Yuting Tang, Weibang Jiang, Shanglin Li et al. · AAAI 2026Other datapaper ↗code ↗
Introduces EEG-DLite, a data-selection framework for EEG foundation-model pre-training that encodes EEG segments into compact latent representations with a self-supervised autoencoder and then filters outliers and redundant samples in that latent space; reports that training on only 5% of a 2,500-hour EEG corpus curated this way matches or exceeds training on the full dataset across several downstream tasks.
Selection
Dataset quantization 1
C2TC — C$^{2}$TC: A Training-Free Framework for Efficient Tabular Data Condensation
Sijia Xu, Fan Li, Xiaoyang Wang et al. · arXiv 2026Other datapaper ↗
Condenses tabular data without any gradient-based training by reformulating condensation as class-adaptive cluster allocation: a heuristic local search alternates between soft cluster assignment and class-wise clustering, and a hybrid categorical encoding keeps the clustering meaningful over heterogeneous discrete features. Reports at least two orders of magnitude faster condensation than prior gradient-based tabular baselines across 10 real-world datasets, with better downstream accuracy.
Bi-level performance matching
Meta-learning through unrolled training 1
TD3 — TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation
Jiaqing Zhang, Mingjia Yin, Hao Wang et al. · WWW 2025Other datapaper ↗code ↗
TD3 distills sequential-recommendation interaction data within a meta-learning bi-level framework, using Tucker decomposition to store the synthetic sequence summary as four compact factors (synthetic user latent factor, temporal dynamics factor, shared item latent factor, and a relation core) instead of a full interaction tensor, and adds a surrogate feature-space alignment objective on top of naive performance matching, optimized with RaT-BPTT. The paper reports superior and cross-architecture generalizable performance versus prior baselines on multiple public recommendation datasets.
Bi-level performance matching
Kernel and closed-form inner solvers 1
Distill-CF — Infinite Recommendation Networks: A Data-Centric Approach
Noveen Sachdeva, Mehak Preet Dhaliwal, Carole-Jean Wu et al. · NeurIPS 2022notableOther datapaper ↗code ↗
Uses the Neural Tangent Kernel equivalence of an infinitely-wide autoencoder (infinity-AE) to give recommendation-model training a closed-form solution, then builds Distill-CF on top of it with differentiable Gumbel-sampling to synthesize tiny, high-fidelity summaries of large, sparse user-item interaction matrices; reports 96-105% of infinity-AE's full-data performance using as little as 0.1% of the original interactions.
Surrogate matching
Gradient matching 1
CGM — Gradient Matching for Categorical Data Distillation in CTR Prediction
Cheng Wang, Jiacheng Sun, Zhenhua Dong et al. · RecSys 2023Other datapaper ↗
Proposes CGM (Categorical data distillation with Gradient Matching), which extends gradient-matching dataset distillation to the high-dimensional, sparse categorical features of click-through-rate prediction data, addressing the blocked gradient flow through categorical embeddings and the cost of the resulting bi-level optimization; distills a small synthetic dataset that trains CTR models from scratch toward performance close to training on the full data.
Surrogate matching
Trajectory matching 1
DDFAD — DDFAD: Dataset Distillation Framework for Audio Data
Wenbo Jiang, Rui Zhang, Hongwei Li et al. · arXiv 2024Other datapaper ↗
Applies dataset distillation to audio for the first time: extracts a Fused Differential MFCC feature (FD-MFCC) from each clip, distills it with the matching-training-trajectory method, and reconstructs playable audio from the distilled features with a Griffin-Lim-based algorithm. Reports effectiveness across several audio datasets.
Surrogate matching
Distribution and feature matching 4
DSDM — Diversified Semantic Distribution Matching for Dataset Distillation
Hongcheng Li, Yucan Zhou, Xiaoyan Gu et al. · MM 2024Other datapaper ↗code ↗
Matches both class-wise Gaussian prototypes and covariance matrices of pretrained-model features between real and synthetic data, rather than means alone, to increase within-class diversity of the distilled set, reporting state-of-the-art distribution-matching results on both image and speech datasets.
SADM — Dataset Distillation of 3D Point Clouds via Distribution Matching
Jae-Young Yim, Dongwook Kim, Jae-Young Sim · NeurIPS 2025notableOther datapaper ↗code ↗
Extends distribution matching to 3D point-cloud dataset distillation, jointly optimizing synthetic point geometry and orientation; introduces a Semantically Aligned Distribution Matching loss computed on per-channel sorted features to correct the semantic misalignment caused by unordered point indexing, and jointly learns rotation angles to correct orientation variation, reporting consistent gains over prior distillation methods with strong cross-architecture generalization.
PACE — Learning from Dense Events: Towards Fast Spiking Neural Networks Training via Event Dataset Distillation
Shuhan Ye, Yi Yu, Qixin Zhang et al. · arXiv 2025Other datapaper ↗
PACE (Phase-Aligned Condensation for Events) is the first dataset distillation framework for spiking neural networks and event-based vision; its ST-DSM module uses residual membrane potentials to densify spike-based features and match amplitude and phase between synthetic and real event streams fine-grained in space and time, while a plug-and-play quantizer (PEQ-N) keeps the output compatible with standard event-frame pipelines. On N-MNIST the paper reports 84.4% accuracy (about 85% of full-data training) with more than 50x faster training and 6000x less storage, and stronger gains than coreset selection and other distillation baselines on DVS-Gesture and CIFAR10-DVS.
Algorithmic Guarantees for Distilling Supervised and Offline RL Datasets
Aaryan Gupta, Rishi Saket, Aravindan Raghuveer · ICLR 2026notableOther datapaper ↗
Gives a dataset-distillation algorithm for linear regression that matches training and synthetic losses with respect to a fixed set of randomly sampled regressors, with no model training in the loop, proving it needs only O(d^2) sampled regressors for any bounded linear model's synthetic-data loss to track its real-data loss (with a matching Omega(d^2) lower bound), then extends the algorithm to offline reinforcement learning by matching Bellman loss instead of behavioral cloning, using both rewards and next-state information.
Generative priors
Optimization in a generative latent space 1
Generative Dataset Distillation
Jovan Cicvarić · University of Tübingen 2023Other datapaper ↗
Master's thesis on generative dataset distillation by optimizing latent codes of pretrained GANs (StyleGAN2 and StyleGAN-XL) instead of pixels directly, evaluated on ImageNet-1K, CIFAR-10/100 and an imitation-learning driving task; the resulting approach placed second and won a best-paper award in the Generative Track of the 2024 Dataset Distillation Challenge.
Generative priors
The generator as the distilled artifact 2
Dataset-Distillation Generative Model for Speech Emotion Recognition
Fabian Ritter-Gutierrez, Kuan-Po Huang, Jeremy H. M Wong et al. · Interspeech 2024Other datapaper ↗
Introduces the first dataset-distillation approach for speech, applied to Speech Emotion Recognition on IEMOCAP: a GAN is trained not to reconstruct real utterances but to distill discriminative information useful for downstream training, then replaces the dataset as a sampler of any desired synthetic size; reports comparable performance under the original class imbalance and a 0.3-point absolute UAR gain under balanced classes, with about 95% less storage and faster downstream training.
Point Cloud DD — Point Cloud Dataset Distillation
Deyu Bo, Xinchao Wang · ICML 2025notableOther datapaper ↗
This paper adapts dataset distillation to unstructured 3D point clouds by theoretically showing that matching rotation-invariant features between real and synthetic data matters for 3D distillation, introducing a plug-and-play point cloud rotator that aligns point clouds to a canonical orientation, and replacing fixed-size synthetic data with a point-wise generator that produces point clouds at multiple resolutions instead of optimizing one fixed set directly.
Orthogonal design choices
Synthetic-set parameterization 2
TD3 — TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation
Jiaqing Zhang, Mingjia Yin, Hao Wang et al. · WWW 2025Other datapaper ↗code ↗
TD3 distills sequential-recommendation interaction data within a meta-learning bi-level framework, using Tucker decomposition to store the synthetic sequence summary as four compact factors (synthetic user latent factor, temporal dynamics factor, shared item latent factor, and a relation core) instead of a full interaction tensor, and adds a surrogate feature-space alignment objective on top of naive performance matching, optimized with RaT-BPTT. The paper reports superior and cross-architecture generalizable performance versus prior baselines on multiple public recommendation datasets.
3DDP — Parameterization-Based Dataset Distillation of 3D Point Clouds through Learnable Shape Morphing
Dongwook Kim, Jae-Young Yim · ICLR 2026notableOther datapaper ↗code ↗
This method extends parameterization-based dataset distillation to 3D point clouds by storing a small set of coarser-resolution anchor point clouds plus learnable per-sample morphing weights, so that morphing the anchors' shapes generates a more diverse synthetic set than the same memory budget would allow if samples were stored directly; a uniformity-aware matching loss keeps the morphed samples structurally consistent with the originals. The paper reports outperforming existing dataset distillation methods on ModelNet10, ModelNet40, ShapeNet, ScanObjectNN and OmniObject3D.
Setting-specific methods
FOSTER — FOSTER: First-order Dataset Distillation for Text-based Sequential Recommendation
Hung Vinh Tran, Tong Chen, Xinyi Gao et al. · arXiv 2026Other datapaper ↗
Proposes FOSTER for text-based sequential-recommendation dataset distillation, replacing the bi-level gradient computation that language-model-based item encoding makes prohibitively expensive with a first-order optimization scheme using trajectory-anchored parameter resets, stochastic item-subset sampling in place of full-corpus embedding extraction, and a regularizer that encourages co-occurrence of semantically similar items in the synthetic sequences; reports it approximates full-dataset performance with as few as 20 synthetic interaction sequences across three benchmarks.
TDColER — On Learning Representations for Tabular Data Distillation
Inwon Kang, Parikshit Ram, Yi Zhou et al. · arXiv 2025Other datapaper ↗
Studies representation learning for tabular dataset distillation, where feature heterogeneity and non-differentiable learners (tree ensembles, nearest-neighbor predictors) break image-style pipelines, and proposes TDColER, a column-embedding representation-learning framework layered on top of existing distillation schemes, evaluated on TDBench, a new benchmark of 226,890 distilled tabular datasets and 548,880 downstream models. Reports TDColER improving distilled-data quality by 0.5-143% across seven tabular learning models.
Analyses, benchmarks, surveys and applications
BTM — Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation
Pafue Christy Nganjimi, Andrew Soltan, Danielle Belgrave et al. · arXiv 2026Other datapaper ↗
Gives a geometric account of why trajectory matching is hard to supervise with a small fixed synthetic set: such a set can only reproduce a limited, low-rank span of the parameter changes real SGD training induces, so a spectrally broad supervision signal creates a representability bottleneck. Proposes Bezier Trajectory Matching (BTM), which replaces SGD expert trajectories with quadratic Bezier surrogates between initial and final model states, reducing trajectory storage and better matching what a fixed synthetic set can represent. Reports BTM matching or improving on standard trajectory matching across five clinical tabular-EHR and time-series datasets (three NHS emergency-department cohorts, eICU and MIMIC-III), with the largest gains at low prevalence and low synthetic-data budgets.
DIET — DIET: Learning to Distill Dataset Continually for Recommender Systems
Jiaqing Zhang, Hao Wang, Mingjia Yin et al. · arXiv 2026Other datapaper ↗
Formulates streaming dataset distillation for recommender systems and proposes DIET, which maintains an evolving distilled memory updated stage-wise via a bi-level optimization framework with influence-guided initialization from influential samples and selective, influence-aware memory updates, so the distilled set stays aligned with long-term training dynamics without reprocessing the full streaming data; reports compressing training data to 1-2% of the original size while preserving full-data performance trends and cutting model-iteration cost by up to 60x, with generalization across model architectures.
Secure and Explainable Fraud Detection in Finance via Hierarchical Multi-source Dataset Distillation
Yiming Qian, Thorsten Neumann, Xueyining Huang et al. · ICAIFW 2025Other datapaper ↗
Converts a trained random forest into transparent axis-aligned rule regions (leaf hyperrectangles) and generates synthetic transactions by uniformly sampling within each region, producing a compact, auditable surrogate dataset with rule-based explanations and calibrated uncertainty for multi-institution fraud detection. On the IEEE-CIS fraud dataset, reports 85-93% data volume reduction with competitive precision/micro-F1 and only a modest AUC drop, chance-level membership-inference attack success, and improved cross-cluster metrics when institutions share the distilled data.
Distilling Reinforcement Learning into Single-Batch Datasets
Connor Wilhelm, Dan Ventura · ECAI 2025Other datapaper ↗
Extends dataset distillation to reinforcement learning by distilling RL environments into a single batch of supervised-learning examples, using a novel extension of proximal policy optimization for meta-learning to fit the bi-level objective; demonstrates the approach on an extended cart-pole task, MuJoCo environments and Atari games, compressing complex RL environments into one-step supervised training and examining generalization across learner architectures.
Shiye Lei, Sen Zhang, Dacheng Tao · NeurIPS 2024notableOther datapaper ↗code ↗
This paper formulates offline behavior distillation (OBD), synthesizing a small set of expert-like behavioral data from suboptimal offline RL data so that a policy can be learned quickly, and shows that naive decision-difference objectives are hard to optimize through the intractable bi-level problem and give only quadratic-discount-complexity guarantees. It introduces action-value weighted PBC (Av-PBC), proving an equivalence between policy performance and the action-value weighted decision difference, and reports faster convergence, better distillation performance and robust cross-architecture/optimizer generalization than the naive objectives on multiple D4RL datasets.
Dataset Distillation for Offline Reinforcement Learning
Jonathan Light, Yuanzhe Liu, Ziniu Hu · ICML 2024 WorkshopOther datapaper ↗code ↗
Applies dataset distillation to offline reinforcement learning, distilling a smaller synthetic dataset (DDRL) from offline trajectories so that a policy trained on it matches one trained on the full offline dataset or with percentile behavioral cloning.
Andrei Lupu, Chris Lu, Jarek Liesen et al. · ICLR 2024Other datapaper ↗code ↗
Formalizes behaviour distillation: discovering and condensing the information needed to train an expert policy into a synthetic set of state-action pairs without access to expert data. Introduces HaDES, which searches such datasets with evolution strategies rather than backpropagation, and reports that as few as four state-action pairs can train continuous-control agents to competitive performance, generalizing out of distribution to other architectures and hyperparameters, and to zero-shot multi-task training.
Sachdeva & McAuley survey — Data Distillation: A Survey
Noveen Sachdeva, Julian McAuley · TMLR 2023coreGraphsOther datapaper ↗
This survey presents a formal framework for data distillation with a detailed taxonomy of existing approaches, and covers the method across three data modalities: images, graphs, and user-item interactions (recommender systems), identifying current challenges and future research directions for each.
BIB — Bidirectional Learning for Offline Model-based Biological Sequence Design
Can Chen, Yingxue Zhang, Xue Liu et al. · ICML 2023Other datapaper ↗code ↗
Extends the bidirectional (forward/backward) offline model-based optimization idea to biological sequence design by replacing the NTK proxy with a linearized pretrained language model, giving a closed-form loss that keeps the biophysical information the NTK could not represent. A bi-level auxiliary model learns how to weight the forward and backward mappings, and a derived learning-rate-adaptation module is reported to improve DNA/protein sequence design over the NTK-only predecessor.
BDI — Bidirectional Learning for Offline Infinite-width Model-based Optimization
Can Chen, Yingxue Zhang, Jie Fu et al. · NeurIPS 2022Other datapaper ↗code ↗
BDI adds a backward mapping to offline model-based optimization that distills information from the static design dataset into the high-scoring candidate designs it proposes, alongside the usual forward proxy. Using an infinite-width network and its neural tangent kernel makes the backward loss closed-form, which the paper reports mitigates the out-of-distribution failure of proxy-only gradient ascent across several design tasks.
New Properties of the Data Distillation Method When Working With Tabular Data
Dmitry Medvedev, Alexander D'yakonov · AIST 2020Other datapaper ↗code ↗
This paper applies the original backpropagation-through-time dataset distillation method to tabular data and finds that a model trained on the distilled samples can outperform one trained on the original dataset, but that the distilled data generalizes poorly to models with different hyperparameters than the one used during distillation. It shows that distilling with multiple architectures simultaneously mitigates this poor cross-hyperparameter generalization.