Synthetic-set parameterization

IDC · ICML 2022

Generates several synthetic images from a limited storage budget by exploiting data regularity, and improves the gradient-matching optimization it is paired with.

Addressable memories · NeurIPS 2022

Learns bases shared across classes and combined by learned addressing functions, so storage need not grow linearly with the number of classes.

HaBa · NeurIPS 2022

Factorizes a dataset into bases and hallucination networks whose combinations reconstruct images; plug-and-play with existing objectives, with higher cross-architecture accuracy reported.

FreD · NeurIPS 2023

Optimizes a selected subset of frequency coefficients per image, chosen by explained variance, on top of existing distillation methods.

DDiF · ICLR 2025

Stores each instance as a neural field over coordinates, which also extends the parameterization to video, audio and 3D voxels.

Static-dynamic video DD — Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement

Ziyu Wang, Yue Xu, Cewu Lu et al. · CVPR 2024coreVideopaper ↗code ↗

This paper gives the first systematic study of video distillation, finding that the temporal dimension of synthetic clips is usually not well learned and contributes little, and responds by disentangling static and dynamic information: a static learning stage distills videos into still images via gradient matching, and a dynamic fine-tuning stage compensates motion with a separately stored, learnable dynamic memory block rather than distilling every frame. The paper reports state-of-the-art results on video datasets at multiple scales with a notably smaller memory storage budget than prior methods.

Post Training Quantization for Efficient Dataset Condensation

Linh-Tam Tran, Sung-Ho Bae · AAAI 2026notablepaper ↗

Applies post-training quantization to already-condensed images to cut storage further: a patch-based scheme keeps quantization localized to avoid the severe degradation plain quantization causes at extreme bit-widths (e.g. 2-bit), quantization-aware clustering groups similar patches to reduce parameter overhead, and a refinement module aligns dequantized images back toward the originals; plugs into any dataset-condensation method's output, nearly doubling test accuracy at 2-bit compression on some methods (e.g. 26.0% to 54.1% for DM at IPC=1).

3DDP — Parameterization-Based Dataset Distillation of 3D Point Clouds through Learnable Shape Morphing

Dongwook Kim, Jae-Young Yim · ICLR 2026notableOther datapaper ↗code ↗

This method extends parameterization-based dataset distillation to 3D point clouds by storing a small set of coarser-resolution anchor point clouds plus learnable per-sample morphing weights, so that morphing the anchors' shapes generates a more diverse synthetic set than the same memory budget would allow if samples were stored directly; a uniformity-aware matching loss keeps the morphed samples structurally consistent with the originals. The paper reports outperforming existing dataset distillation methods on ModelNet10, ModelNet40, ShapeNet, ScanObjectNN and OmniObject3D.

Rate-utility DD — Dataset Distillation as Data Compression: A Rate-Utility Perspective

Youneng Bao, Yiping Liu, Zhuo Chen et al. · ICCV 2025notablepaper ↗code ↗

Frames dataset distillation as a joint rate-utility optimization: synthetic samples are optimizable latent codes decoded by extremely lightweight networks, the Shannon entropy of the quantized latents is the rate term, and any existing distillation loss serves as the utility term, traded off via a Lagrange multiplier; introduces bits-per-class (bpc), a storage metric accounting for sample, label and decoder-parameter costs, and reports up to 170x greater compression than standard distillation at comparable accuracy on CIFAR-10, CIFAR-100 and ImageNet-128.

H-GLaD — Hierarchical Features Matter: A Deep Exploration of Progressive Parameterization Method for Dataset Distillation

Xinhao Zhong, Hao Fang, Bin Chen et al. · CVPR 2025notablepaper ↗code ↗

Extends GLaD-style generative-prior distillation by systematically exploring hierarchical feature layers within a pretrained GAN, rather than a single fixed latent space, since different layers offer different informative guidance for distillation; also introduces a class-relevant feature-distance metric that lowers the cost of evaluating synthetic sets. Matches or surpasses diffusion-based generative distillation at extreme compression (IPC=1, IPC=10) at comparable time cost.

An Efficient Dataset Condensation Plugin and Its Application to Continual Learning

Enneng Yang, Li Shen, Zhenyi Wang et al. · NeurIPS 2023notablepaper ↗code ↗

Observes that dataset-condensation matching objectives operate in high-dimensional pixel space even though natural images are locally connected and have lower intrinsic dimension, and proposes a plugin that instead condenses images into two low-rank matrices, matching raw and synthetic data in this low-dimensional manifold for higher condensation efficiency. Demonstrated as a continual-learning replay buffer, presented at NeurIPS 2023.

Dataset Color Quantization: A Training-Oriented Framework for Dataset-Level Compression

Chenyue Yu, Lingao Xiao, Jinhong Deng et al. · ICLR 2026paper ↗

Reduces color-space redundancy across a dataset by enforcing consistent palettes across similar images, selectively retaining semantically important colors under model-perception guidance, and preserving structural detail needed for feature learning; reports improved training performance under aggressive compression on CIFAR-10/100, Tiny-ImageNet and ImageNet-1K.

GSDD — Parameterizing Dataset Distillation via Gaussian Splatting

Chenyang Jiang, Zhengcen Li, Hang Zhao et al. · arXiv 2025paper ↗code ↗

Parameterizes each synthetic image as a set of 2D Gaussian splats rendered with CUDA-based batched splatting operators, giving a sparse yet expressive per-image representation that increases dataset diversity under a fixed storage budget; reports gains on standard benchmarks, ImageNet-1K, and video distillation.

Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation

Sheng-Feng Yu, Jia-Jiun Yao, Wei-Chen Chiu · ICLR 2025Pre-training & transferpaper ↗

Improves self-supervised dataset distillation with a low-dimensional basis parameterization shared by images and their self-supervised representation targets, fixed (non-random) augmentations to stabilize training against a source of instability specific to self-supervised objectives, and a lightweight network that links the representations of augmented views from the same image for more compact pairs; reports gains in distillation efficiency, cross-architecture generalization and downstream transfer performance.

Latent Video Dataset Distillation

Ning Li, Antai Andy Liu, Jingran Zhang et al. · CVPR 2025 WorkshopVideopaper ↗code ↗

This method stores condensed video data as codes in the latent space of a state-of-the-art variational encoder, rather than in pixel space, and picks which latents to keep with a diversity-aware selection strategy plus a training-free compression step on top. The paper reports new state-of-the-art results on all tested video-distillation benchmarks, including a 2.6-point gain on HMDB51 at IPC 1 and a 7.8-point gain on MiniUCF at IPC 5.

GPS — GPS: Distilling Compact Memories via Grid-based Patch Sampling for Efficient Online Class-Incremental Learning

Mingchuan Ma, Yuhao Zhou, Jindi Lv et al. · arXiv 2025paper ↗

For online class-incremental replay, generates memory samples by sampling a grid of pixels from each real image into a compact low-resolution representation, with no trainable model or optimization, then reassembles the grid at replay time — avoiding the bi-level optimization cost of prior distilled-memory replay methods. Reports 3-4 percentage-point average end-accuracy gains when substituted into existing replay frameworks under memory-constrained settings, with limited computational overhead.

TD3 — TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation

Jiaqing Zhang, Mingjia Yin, Hao Wang et al. · WWW 2025Other datapaper ↗code ↗

TD3 distills sequential-recommendation interaction data within a meta-learning bi-level framework, using Tucker decomposition to store the synthetic sequence summary as four compact factors (synthetic user latent factor, temporal dynamics factor, shared item latent factor, and a relation core) instead of a full interaction tensor, and adds a surrogate feature-space alignment objective on top of naive performance matching, optimized with RaT-BPTT. The paper reports superior and cross-architecture generalizable performance versus prior baselines on multiple public recommendation datasets.

IDTD — Video Set Distillation: Information Diversification and Temporal Densification

Yinjie Zhao, Heng Zhao, Bihan Wen et al. · arXiv 2024Videopaper ↗

This paper introduces video set distillation, jointly reducing within-sample redundancy (across frames of one video) and inter-sample redundancy (across videos in a set), which prior key-frame selection, dataset pruning or single-video distillation methods each address only one of. Its IDTD method uses a Feature Pool and Feature Selectors to preserve diversity across samples and a Temporal Fusor to keep temporal information dense within each synthesized video, reporting state-of-the-art results in video dataset distillation.

AutoPalette — Color-Oriented Redundancy Reduction in Dataset Distillation

Bowen Yuan, Zijian Wang, Mahsa Baktashmotlagh et al. · NeurIPS 2024paper ↗code ↗

Reduces color redundancy in distilled images with a palette network that dynamically allocates colors from a reduced palette to each pixel based on the region's importance for training, plus a color-guided initialization strategy that selects real images with the least redundant color patterns across the dataset.

UDD — UDD: Dataset Distillation via Mining Underutilized Regions

Shiguang Wang, Zhongyu Zhang, Jian Cheng · PRCV 2024paper ↗

Identifies underutilized regions inside synthetic images as wasted capacity in multi-formation-style distillation, and introduces two utilization-sensitive search policies (response-based and data-jittering-based) that dynamically locate and fill these regions during optimization, together with a category-wise feature contrastive loss for inter-class separability. Reports 4.0% and 3.7% accuracy gains over the next-best method on CIFAR-10 and CIFAR-100 at IPC=1.

NSD — Neural Spectral Decomposition for Dataset Distillation

Shaolei Yang, Shen Cheng, Mingbo Hong et al. · ECCV 2024paper ↗code ↗

Represents the whole distilled dataset as a shared set of spectrum tensors combined pairwise with per-image transformation matrices, rather than as independent images, so information is shared across the synthetic set through simple matrix multiplication, and optimizes this low-rank representation with a trajectory-matching objective guided by the real distribution, reporting state-of-the-art results on CIFAR-10/100, Tiny-ImageNet and an ImageNet subset.

HMN — Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensation

Haizhong Zheng, Jiachen Sun, Shutong Wu et al. · ECCV 2024paper ↗

Stores condensed data in a three-tier Hierarchical Memory Network representing dataset-, class- and instance-level features, matching the hierarchical way real images share features within a classification system; the hierarchy also keeps images independent enough to allow instance-level pruning of redundant information, and it outperforms prior parameterization baselines on five public datasets.

Translative pre-training — Few-Shot Dataset Distillation via Translative Pre-Training

Songhua Liu, Xinchao Wang · ICCV 2023paper ↗

Learns a 'distillation space' via a translator network, pretrained on large datasets with image-to-image translation, that maps synthetic images optimized cheaply in an arbitrary fixed network's space into the space of the target few-shot distillation network, needing only a few adaptation steps per new target dataset; reports about 15x faster distillation with performance comparable to iterative bi-level baselines, and satisfactory generalization across datasets, budgets and class counts.

SPEED — Sparse Parameterization for Epitomic Dataset Distillation

Xing Wei, Anjia Cao, Funing Yang et al. · NeurIPS 2023paper ↗code ↗

Applies dictionary learning and sparse coding to dataset distillation: Spatial-Agnostic Epitomic Tokens and Sparse Coding Matrices represent and select the most significant features, decoded by a Feature-Recurrent Network into hierarchical, high-compression synthetic images; reports state-of-the-art results on high-resolution benchmarks and downstream applications, compatible with a variety of matching objectives.

Slimmable DC — Slimmable Dataset Condensation

Songhua Liu, Jingwen Ye, Runpeng Yu et al. · CVPR 2023paper ↗

Introduces slimmable dataset condensation: a significance-aware parameterization whose components can be truncated to shrink an already-condensed dataset to a smaller storage budget without re-accessing the original data, addressing the inconsistency of matching networks over time and the underdetermined solution space that make naive successive compression fail; a theoretical bound shows discarding minor components is safe, and using them as initialization for further training gives fast convergence.

KFS — Dataset Condensation with Latent Space Knowledge Factorization and Sharing

Hae Beom Lee, Dong Bok Lee, Sung Ju Hwang · arXiv 2022paper ↗

Assumes a generative process for the distilled data -- a set of learnable latent codes in a compact space combined with several tiny decoders -- so combining codes and decoders interchangeably multiplies the number of distinct synthetic examples at nearly the same parameter count; reports new state-of-the-art results by significant margins on SVHN, CIFAR-10, CIFAR-100 and Tiny-ImageNet.

PRANC — PRANC: Pseudo RAndom Networks for Compacting deep models

Parsa Nooralinejad, Ali Abbasi, Soroush Abbasi Koohpayegani et al. · ICCV 2023paper ↗code ↗

Reparametrizes a deep model as a linear combination of many frozen, pseudo-random 'basis' networks generated from a single seed, so the model is stored as one scalar seed plus learned mixture coefficients; applied to compress image-classification models nearly 100x and, in one experiment, to compact individual images via their associated implicit neural representations, with memory-efficient on-the-fly weight generation for inference.

CCMCL — Condensed Composite Memory Continual Learning

Felix Wiewel, Bin Yang · IJCNN 2021paper ↗code ↗

For rehearsal-based continual learning, learns a small set of shared components combined into per-class synthetic exemplars rather than learning each replay image directly, increasing memory efficiency of the stored rehearsal buffer. Reports improved performance over prior rehearsal baselines, especially at very small buffer sizes.