Distribution and feature matching

CAFE · CVPR 2022

Argues that gradient matching overfits to samples with dominant gradients and lacks global supervision of the data distribution, and aligns multi-scale features instead, still inside a bi-level scheme.

DM · WACV 2023

Removes bi-level optimization. Synthetic images match the feature distributions of real images in many sampled embedding spaces, which cuts synthesis cost enough to use larger datasets and architectures.

IDM · CVPR 2023

Identifies imbalanced feature numbers and unvalidated embeddings as weaknesses of naive distribution matching, and adds partition-and-expand augmentation, richer model sampling and class-aware regularization.

DataDAM · ICCV 2023

Matches spatial attention maps from several layers of randomly initialized networks rather than pooled features only.

M3D · AAAI 2024

Argues that mean matching ignores higher-order alignment, and minimizes maximum mean discrepancy in a reproducing kernel Hilbert space so that all moments are aligned.

DANCE · IJCAI 2024

Uses a few pretrained models to address distribution matching's limits within classes (a proxy for long-term training) and between classes (distribution shift out of the class region).

WMDD · ICCV 2025

Matches the Wasserstein barycenter of features from a pretrained classifier and keeps per-class BatchNorm statistics to preserve intra-class variation.

NCFM · CVPR 2025

Recasts matching as a min-max problem over a neural characteristic-function discrepancy that captures the full distribution, and reports large memory and speed savings.

AVDD — Audio-Visual Dataset Distillation

Saksham Singh Kushwaha, Siva Sai Nagender Vasireddy, Kai Wang et al. · TMLR 2024coreAudio–visual & omnipaper ↗code ↗

Extends distribution matching to audio-visual data, first matching visual-only and audio-only feature distributions with randomly initialized networks separately, then adding a joint matching loss that implicitly cross-matches real audio against synthetic visual features (and vice versa) and a modality-gap matching loss that aligns the audio-visual gap between real and synthetic data, plus herding-based initialization; on VGGS-10K at 10 images-per-class it reports 54.99% recognition accuracy versus 43.85% for plain distribution matching and 68.24% for the full dataset (Table 3), and raises audio-to-visual retrieval Recall@1 from 8.66% (DM) to 19.33% (Table 5).

RAHA — Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation

Jongoh Jeong, Sun-Kyung Lee, Kuk-Jin Yoon · ECCV 2026notableVision–languagepaper ↗code ↗

RAHA argues that full-dimensional Euclidean alignment is too restrictive for the rank-deficient image-text correlation typical of vision-language distillation (shared semantics concentrate in a low-dimensional range, with the remainder in a weakly correlated residual subspace), and instead lifts representations to hyperbolic space, optimizing distilled pairs with asymmetric objectives that enforce geodesic alignment on the shared range while regularizing the residual subspace to keep modality-private diversity. The paper reports competitive cross-modal retrieval and improved transfer robustness under fixed budgets versus prior similarity-level low-rank alignment (LoRS).

MDM — Multimodal Distribution Matching for Vision-Language Dataset Distillation

Jongoh Jeong, Hoyong Kwon, Minseok Kim et al. · CVPR 2026notableVision–languagepaper ↗code ↗

MDM distills vision-language pairs by initializing synthetic image-text pairs from clusters in the joint embedding space, forming a mixed teacher by interpolating independently fine-tuned models in weight space, and matching real and synthetic joint distributions on the unit hypersphere with a geometry-aware objective that exploits cross-modal agreement and discrepancy directions alongside symmetric contrastive learning. The paper reports compact synthetic sets that preserve multimodal semantics and remain robust under cross-architecture evaluation on image-text retrieval benchmarks.

Sneakdoor — SNEAKDOOR: Stealthy Backdoor Attacks against Distribution Matching-based Dataset Condensation

He Yang, Dongyi Lv, Song Ma et al. · NeurIPS 2025notablepaper ↗code ↗

Targets distribution-matching-based dataset condensation with a backdoor attack designed for stealth, exploiting the vulnerability of class decision boundaries and using a generative module that builds input-aware triggers aligned to local feature geometry to hide artifacts in both the condensed data and the triggered inference samples. Reports a better balance of attack success rate, clean accuracy and stealthiness than prior condensation backdoor attacks, evading both human inspection and statistical detection.

Harmonic Dataset Distillation for Time Series Forecasting

Seungha Hong, Sanghwan Jang, Wonbin Kweon et al. · AAAI 2026notableTime seriespaper ↗

HDT (Harmonic Dataset Distillation) decomposes each time series into its sinusoidal basis via the FFT and condenses by Harmonic Matching, aligning the core periodic structure of the synthetic and real series in the frequency domain so that updates never disturb temporal dependencies. The paper reports strong cross-architecture generalization and scalability for time series forecasting compared to conventional, image-centric condensation methods adapted to time series.

Algorithmic Guarantees for Distilling Supervised and Offline RL Datasets

Aaryan Gupta, Rishi Saket, Aravindan Raghuveer · ICLR 2026notableOther datapaper ↗

Gives a dataset-distillation algorithm for linear regression that matches training and synthetic losses with respect to a fixed set of randomly sampled regressors, with no model training in the loop, proving it needs only O(d^2) sampled regressors for any bounded linear model's synthetic-data loss to track its real-data loss (with a matching Omega(d^2) lower bound), then extends the algorithm to offline reinforcement learning by matching Bellman loss instead of behavioral cloning, using both rewards and next-state information.

DP-GENG — DP-GENG: Differentially Private Dataset Distillation Guided by DP-Generated Data

Shuo Shi, Jinghuai Zhang, Shijie Jiang et al. · AAAI 2026notablepaper ↗

Initializes the distilled dataset with DP-generated data for realism, then refines it under a small privacy budget with a DP-feature-matching (distribution-matching) objective and aligns examples to their class distribution with an expert model, alongside a privacy-budget allocation strategy with a theoretical DP guarantee. Reports outperforming prior differentially-private dataset-distillation methods in both utility and robustness to membership-inference attacks.

ImageBindDC — ImagebindDC: Compressing Multi-modal Data with Imagebind-based Condensation

Yue Min, Shaobo Wang, Jiaze Li et al. · AAAI 2026notableAudio–visual & omnipaper ↗

ImageBindDC condenses omnimodal data inside ImageBind's shared embedding space using a Characteristic Function loss that performs exact infinite-moment matching in the Fourier domain, enforcing uni-modal, cross-modal (real-synthetic pair) and joint-modal distributional consistency simultaneously rather than matching each modality separately. On NYU-v2 the paper reports lossless performance versus full-data training at 5 condensed datapoints per class, an 8.2 percentage-point absolute improvement over the previous best method, and over 4x less condensation time.

CovMatch — CovMatch: Cross-Covariance Guided Multimodal Dataset Distillation with Trainable Text Encoder

Yongmin Lee, Hye Won Chung · NeurIPS 2025notableVision–languagepaper ↗code ↗

Proposes CovMatch for image-text dataset distillation, which fixes the encoders at each distillation step and matches the cross-covariance between real and synthetic image and text feature matrices, together with per-modality feature-distribution regularization, instead of unrolling trajectories; unlike prior methods that freeze the text tower, CovMatch keeps a BERT-base text encoder trainable end to end (only its embedding layer is frozen) alongside a pretrained NFNet image encoder. On Flickr30K at 500 pairs it reports 38.4% mean recall (Table 2), a 6.8-point absolute gain over the LoRS baseline (31.6%), and 19.6% mean recall on COCO at 500 pairs, a 6.1-point gain over LoRS.

Dosser — Improving Noise Efficiency in Privacy-preserving Dataset Distillation

Runkai Zheng, Vishnu Asutosh Dasu, Yinong Oliver Wang et al. · ICCV 2025notablepaper ↗code ↗

Identifies that prior differentially-private dataset distillation wastes privacy budget because sampling and optimization are synchronized and training signals come from noisy, randomly-initialized networks; decouples sampling from optimization for better convergence and matches in an informative subspace to reduce the impact of DP noise. Reports a 10.0% accuracy improvement at 50 images per class on CIFAR-10 and an 8.3% gain using only one-fifth the distilled-set size of prior state-of-the-art private DD methods.

HDD — Hyperbolic Dataset Distillation

Wenyuan Li, Guang Li, Keisuke Maeda et al. · NeurIPS 2025notablepaper ↗code ↗

Embeds pretrained-network features into Lorentz hyperbolic space and matches the hyperbolic (geodesic) distance between synthetic and real class centroids instead of Euclidean distribution matching, explicitly encoding the data's hierarchical structure into the distilled set; pruning in hyperbolic space is shown to retain model performance with only 20% of the distilled set.

SADM — Dataset Distillation of 3D Point Clouds via Distribution Matching

Jae-Young Yim, Dongwook Kim, Jae-Young Sim · NeurIPS 2025notableOther datapaper ↗code ↗

Extends distribution matching to 3D point-cloud dataset distillation, jointly optimizing synthetic point geometry and orientation; introduces a Semantically Aligned Distribution Matching loss computed on per-channel sorted features to correct the semantic misalignment caused by unordered point indexing, and jointly learns rotation angles to correct orientation variation, reporting consistent gains over prior distillation methods with strong cross-architecture generalization.

FedVCK — FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysis

Guochen Yan, Luyuan Xie, Xinyi Gao et al. · AAAI 2025notablepaper ↗code ↗

FedVCK condenses each client's data with latent distribution constraints to improve knowledge quality, and each round selectively condenses only knowledge the current global model has not yet assimilated, cutting redundant communication; on the server, relational supervised contrastive learning gives extra supervision for the global update. Reports outperforming state-of-the-art federated learning methods on medical tasks under non-IID data and limited communication budgets.

DDM — Decomposed Distribution Matching in Dataset Condensation

Sahar Rahimi Malakshan, Mohammad Saeed Ebrahimi Saadabadi, Ali Dabouei et al. · WACV 2025notablepaper ↗

Decomposes the dataset distribution that distribution matching targets into content and style, and identifies two shortcomings of the plain distribution-matching objective: a style mismatch between real and condensed data, and limited intra-class diversity in the condensed set. Matches per-layer feature-map statistical moments as a style term and maximizes intra-class KL divergence among synthetic samples as a content-diversity term, reporting accuracy gains of up to 4.1% on CIFAR-10, 4.2% on CIFAR-100, 4.3% on Tiny-ImageNet, 2.0% on ImageNet-1K, 3.3% on ImageWoof, 2.5% on ImageNette, and 5.5% in continual-learning accuracy over the distribution-matching baseline.

DeSA — Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchors

Chun-Yin Huang, Kartik Srinivas, Xin Zhang et al. · ICML 2024notablepaper ↗code ↗

In serverless decentralized federated learning, DeSA synthesizes global anchor data from each client's raw data distribution and uses it for two local regularizers: aligning each client's latent embeddings to the anchors, and a knowledge-distillation loss that lets clients learn from one another, grounded in domain-adaptation and KD theory. Reports improved inter- and intra-domain accuracy across clients with diverse data and model heterogeneity.

FedAF — An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

Yuan Wang, Huazhu Fu, Renuga Kanagavelu et al. · CVPR 2024notablepaper ↗

FedAF replaces iterative model aggregation with a collaborative distribution-matching condensation step, where clients jointly learn condensed data and soft labels from peer knowledge that the server then trains the global model on directly. It reports faster convergence and higher global-model accuracy than prior one-shot and iterative FL baselines under label- and feature-skew heterogeneity.

GDEM — Graph Distillation with Eigenbasis Matching

Yang Liu, Deyu Bo, Chuan Shi · ICML 2024notableGraphspaper ↗code ↗

Proposes GDEM for graph distillation, which aligns the eigenbasis and node features of the synthetic and real graphs directly and replicates the real graph's spectrum, instead of relying on GNN-derived gradients, representations or trajectories that bias the synthetic spectrum and tie the result to one GNN architecture; adds a discrimination constraint to balance effectiveness and generalization, reporting state-of-the-art results with strong cross-architecture generalization and a theoretical guarantee that the synthetic graphs are restricted spectral approximations of the real ones.

FedDM — FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning

Yuanhao Xiong, Ruochen Wang, Minhao Cheng et al. · CVPR 2023notablepaper ↗code ↗

Each client builds a local surrogate loss landscape by matching the distribution of a small synthetic set to its own real data, and only this synthetic data (not model weights) is transmitted so the server assembles a global training objective from multiple clients' surrogate landscapes. Reports fewer communication rounds and better model quality than other federated learning methods, and shows the synthetic-data transmission can be made differentially private under a Gaussian mechanism at the same privacy budget.

ShapeCond — ShapeCond: Fast Shapelet-Guided Dataset Condensation for Time Series Classification

Sijia Peng, Yun Xiong, Xi Chen et al. · arXiv 2026Time seriespaper ↗code ↗

ShapeCond condenses time series for classification by extracting shapelets, discriminative local motifs that image-centric condensation methods miss, and using a shapelet-guided optimization strategy to preserve them explicitly, with a synthesis cost that is independent of sequence length. The paper reports large speedups over the prior state of the art (CondTSC), e.g. 29x faster and up to 10,000x faster than naively using shapelets on a 3,000-timestep dataset, while improving downstream accuracy over prior time-series condensation methods.

UniTSC — One Batch Is Enough: A Unified Dataset Condensation Framework for General Time Series Analysis

Wei Shao, Ziquan Fang, Zheqi Lu et al. · ICML 2026Time seriespaper ↗code ↗

UniTSC argues that prior time-series condensation methods are task-specific (optimized for one downstream task such as forecasting and performing poorly on others such as imputation), and proposes a task-invariant condensation framework that jointly captures temporal, spectral and topological properties of the data so the same condensed set generalizes across multiple time-series analysis tasks.

GeoDM — GeoDM: Geometry-aware Distribution Matching for Dataset Distillation

Xuhui Li, Zhengquan Luo, Zihui Cui et al. · ICML 2026paper ↗

Extends distribution matching to a product of Euclidean, hyperbolic and spherical manifolds with learnable curvature per geometry, so the synthetic-data manifold can capture the flat, hierarchical and cyclical structure of the real data rather than only Euclidean statistics, adding an optimal-transport loss; reports a smaller generalization-error bound than Euclidean distribution matching and gains over state-of-the-art distillation methods on standard benchmarks.

TGDD — TGDD: Trajectory Guided Dataset Distillation with Balanced Distribution

Fengli Ran, Xiao Pu, Bo Liu et al. · AAAI 2026paper ↗code ↗

Reformulates distribution matching as dynamic alignment along the model's training trajectory rather than a single static feature space, capturing how feature representations evolve during training and adding a distribution-constraint regularizer to reduce class overlap, reporting a 5.0-point accuracy gain on high-resolution benchmarks with no added optimization overhead across ten datasets.

DAVDD — Decoupled Audio-Visual Dataset Distillation

Wenyuan Li, Guang Li, Keisuke Maeda et al. · arXiv 2025Audio–visual & omnipaper ↗

Proposes DAVDD, a decoupled audio-visual distillation framework built on distribution matching: a bank of pretrained, frozen audio-visual encoder pairs feeds lightweight decoupler MLPs that split each modality's features into shared and private representations; a Common Intermodal Matching loss aligns the shared representations across modalities while a Sample-Distribution Joint Alignment combines instance-level contrastive alignment with EMA class prototypes, and private representations are kept out of the cross-modal loss entirely to protect modality-specific cues. On VGGS-10K at 10 images-per-class it reports 56.2% accuracy versus 54.0% for AVDD and 68.2% for the full dataset (Table 1).

PACE — Learning from Dense Events: Towards Fast Spiking Neural Networks Training via Event Dataset Distillation

Shuhan Ye, Yi Yu, Qixin Zhang et al. · arXiv 2025Other datapaper ↗

PACE (Phase-Aligned Condensation for Events) is the first dataset distillation framework for spiking neural networks and event-based vision; its ST-DSM module uses residual membrane potentials to densify spike-based features and match amplitude and phase between synthetic and real event streams fine-grained in space and time, while a plug-and-play quantizer (PEQ-N) keeps the output compatible with standard event-frame pipelines. On N-MNIST the paper reports 84.4% accuracy (about 85% of full-data training) with more than 50x faster training and 6000x less storage, and stronger gains than coreset selection and other distillation baselines on DVS-Gesture and CIFAR10-DVS.

DEDA — Diversity-Enhanced Distribution Alignment for Dataset Distillation

Hongcheng Li, Yucan Zhou, Xiaoyan Gu et al. · ICCV 2025paper ↗

Matches both class-wise means and covariance matrices of pretrained-model features between real and synthetic data in a Gaussian-distribution-alignment scheme, then adds a regularizer that maximizes diagonal and minimizes off-diagonal covariance terms in the last feature layer specifically, countering the low diversity and gradient starvation that mean/BN-statistics-only alignment produces; reports state-of-the-art results on CIFAR-10/100, Tiny-ImageNet and ImageNet-1K with no added compute.

OPTICAL — OPTICAL: Leveraging Optimal Transport for Contribution Allocation in Dataset Distillation

Xiao Cui, Yulei Qin, Wengang Zhou et al. · CVPR 2025paper ↗

Argues that sample-generation-based distillation methods give every real instance equal, uniform contribution when shaping each synthetic sample and so ignore instance-level real-synthetic relationships, and reformulates the matching objective as a bi-level matching-and-approximating problem where an optimal-transport matrix allocates contributions from real instances before the synthetic samples are refined against that allocation; reports gains across seven datasets and three architectures as a plug-in compatible with multiple distillation frameworks.

TimeDC — Less is More: Efficient Time Series Dataset Condensation via Two-fold Modal Matching--Extended Version

Hao Miao, Ziqiao Liu, Yan Zhao et al. · VLDB 2025Time seriespaper ↗code ↗

TimeDC condenses time series with two-fold modal matching, decomposition-driven frequency matching to preserve spectral structure and curriculum training-trajectory matching (with a buffer of precomputed expert trajectories) to preserve temporal dependencies, aiming for effective and generalized condensation rather than a single-domain surrogate objective. The paper reports effectiveness and efficiency gains over prior time-series condensation baselines across real datasets.

DSDM — Diversified Semantic Distribution Matching for Dataset Distillation

Hongcheng Li, Yucan Zhou, Xiaoyan Gu et al. · MM 2024Other datapaper ↗code ↗

Matches both class-wise Gaussian prototypes and covariance matrices of pretrained-model features between real and synthetic data, rather than means alone, to increase within-class diversity of the distilled set, reporting state-of-the-art distribution-matching results on both image and speech datasets.

CollabDM — One-Shot Collaborative Data Distillation

William Holland, Chandra Thapa, Sarah Ali Siddiqui et al. · ECAI 2024paper ↗code ↗

CollabDM captures the global data distribution across distributed clients in a single round of client-server communication, rather than merging independently-distilled local sets (which the paper shows degrades under client data heterogeneity). Reports outperforming the state-of-the-art one-shot distillation method on skewed distributed data and demonstrates benefit on 5G network attack detection.

D3S — Large Scale Dataset Distillation with Domain Shift

Noel Loo, Alaa Maalouf, Ramin Hasani et al. · ICML 2024paper ↗code ↗

Reframes large-scale dataset distillation as a domain-shift problem between the synthetic and real data distributions, derives a universal upper bound on the distillation loss under this framing and optimizes it efficiently, reporting state-of-the-art results and improved cross-architecture generalization on Tiny-ImageNet, ImageNet-1K and ImageNet-21K.

LQM — Dataset Condensation with Latent Quantile Matching

Wei Wei, Tom De Schepper, Kevin Mets · CVPR 2024 WorkshopGraphspaper ↗

Shows that matching only the mean of latent feature embeddings, as in standard distribution matching, lets very different distributions appear identical, and proposes Latent Quantile Matching, which instead matches the quantiles of the embedding distributions to minimize a goodness-of-fit statistic; matches or beats prior distribution-matching methods on image and graph-structured datasets and improves continual graph learning.

IID — Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation

Wenxiao Deng, Wenbin Li, Tianyu Ding et al. · CVPR 2024paper ↗code ↗

Adds a class-centralization constraint that pulls same-class synthetic features together and a covariance-matching constraint that aligns local feature covariance matrices, addressing dispersed within-class features and mean-only matching in prior distribution-matching methods, reporting up to 6.6 points on CIFAR-10 and 2.5-2.9 points on SVHN/CIFAR-100/Tiny-ImageNet over prior distribution-matching baselines.

CondTSC — Dataset Condensation for Time Series Classification via Dual Domain Matching

Zhanyu Liu, Ke Hao, Guanjie Zheng et al. · KDD 2024Time seriespaper ↗code ↗

CondTSC adapts image-style dataset condensation to time series classification with dual surrogate matching objectives computed in both the time and frequency domains (rather than time domain alone), combined with multi-view data augmentation and dual-domain training, to better capture the temporal and spectral structure that direct adaptations of image condensation methods miss. The paper reports outperforming other time-series and adapted image/graph condensation baselines, producing a condensed set that better matches the real data's distribution.

Differentially Private Dataset Condensation

Tianhang Zheng, Baochun Li · NDSS 2024 Workshoppaper ↗

Proposes two differentially-private dataset condensation algorithms: LDPDC, a linear DC method that runs on a low-end CPU, and NDPDC, which uses a neural network for representation extraction under a DP feature/gradient-matching objective. Reports LDPDC performs comparably to recent privacy-preserving generative methods, while NDPDC gives acceptable DP guarantees with only mild utility loss relative to plain distribution matching.

Dataset Distillation via Adversarial Prediction Matching

Mingyang Chen, Bo Huang, Junda Lu et al. · arXiv 2023paper ↗

Minimizes the prediction discrepancy on real data between a model trained on the full dataset and one trained on the distilled set, using a single-level adversarial framework in place of nested bi-level optimization or long trajectory unrolling. Reports distilled sets at 10% of the original size reaching 94% of full-data accuracy on average across datasets including ImageNet-1K, at 2.5x less memory and 5x less runtime than the prior state of the art, with ImageNet-1K distillation possible in 6.5GB of GPU memory.

Multi-Source Domain Adaptation meets Dataset Distillation through Dataset Dictionary Learning

Eduardo Fernandes Montesuma, Fred Ngolè Mboula, Antoine Souloumiac · ICASSP 2024paper ↗

Poses multi-source domain adaptation and dataset distillation as one joint problem (MSDA-DD), combining Wasserstein Barycenter Transport and Dataset Dictionary Learning from the MSDA literature with the distribution-matching distillation objective. Reports state-of-the-art adaptation performance on four benchmarks with as little as one distilled sample per class.

IEM — Towards Efficient Deep Hashing Retrieval: Condensing Your Data via Feature-Embedding Matching

Tao Feng, Jie Zhang, Huashan Liu et al. · arXiv 2023paper ↗

Adapts dataset condensation to deep hashing retrieval with IEM (Information-intensive feature Embedding Matching), a distribution-matching-centered method that adds model and data augmentation to strengthen the condensed hashing-space features, since retrieval training does not benefit directly from condensation methods designed for classification accuracy. Reports superior performance and efficiency relative to applying mainstream condensation methods to deep hashing retrieval.

FedLGD — Federated Learning on Virtual Heterogeneous Data with Local-global Distillation

Chun-Yin Huang, Ruinan Jin, Can Zhao et al. · TMLR 2024paper ↗code ↗

Observes that distilling each client's local data independently can worsen federated heterogeneity, so FedLGD instead uses iterative distribution matching to inpaint global information into local synthetic ("virtual") data and federated gradient matching to distill global virtual anchor points that rectify heterogeneous local training. Reports outperforming prior heterogeneous-FL methods on benchmark and real-world datasets, including at large numbers of class-imbalanced clients.

GDD-FL — Communication-Efficient Federated Skin Lesion Classification with Generalizable Dataset Distillation

Yuchen Tian, Jiacheng Wang, Yueming Jin et al. · MICCAI 2023 Workshoppaper ↗

GDD-FL condenses thousands of skin-lesion images per client into one synthetic image per class, modeling dataset features as an uncertain Gaussian distribution so the synthetic images capture diverse semantics and resist distribution drift across clients; only these few synthesized images are transmitted once to train a global model. Reports reduced communication cost with performance superior to classical federated learning and existing dataset-distillation baselines.

Data-efficient Neural Network Training with Dataset Condensation

Bo Zhao · The University of Edinburgh 2023paper ↗

PhD thesis presenting the author's own gradient-matching (DC), differentiable-Siamese-augmentation (DSA) and distribution-matching (DM) approaches to dataset condensation as one body of work on data-efficient neural network training.

On Divergence Measures for Bayesian Pseudocoresets

Balhae Kim, Jungwon Choi, Seanie Lee et al. · NeurIPS 2022paper ↗code ↗

Casts two representative dataset-distillation algorithms as approximations to Bayesian-pseudocoreset construction that minimize reverse KL divergence and Wasserstein distance respectively, giving a unifying view of divergence measures for pseudocoreset construction, and proposes a new pseudocoreset algorithm that instead minimizes forward KL divergence; shows empirically that pseudocoresets built this way better reflect the true posterior even in high-dimensional Bayesian inference problems.

GCDM — Graph Condensation via Receptive Field Distribution Matching

Mengyang Liu, Shanchuan Li, Xinshi Chen et al. · arXiv 2022Graphspaper ↗

Proposes GCDM, which views a graph as a distribution of receptive fields and condenses it by matching that distribution between the real and synthetic graphs with a maximum-mean-discrepancy loss; reports strong cross-architecture generalization to a variety of GNN models and substantially faster condensation than earlier graph condensation methods.