Method · Diffusion-based synthesis
ManifoldGD
ManifoldGD: Training-Free Hierarchical Manifold Guidance for Diffusion-Based Dataset Distillation
Ayush Roy, Wei-Yang Alex Lee, Rudrasis Chakraborty, Vishnu Suresh Lokhande
CVPR 2026 · first public 2026-02-26 · arXiv 2602.23295
In one paragraph
Argues that existing training-free diffusion guidance for distillation relies on simple mode-based centroids that are suboptimal, and instead computes a multi-scale coreset of prototypes via hierarchical divisive clustering of VAE latent features, then at every denoising step projects the mode-alignment vector onto the local tangent space of a manifold built from nearby prototypes, keeping generation manifold-faithful while staying semantically consistent; the first geometry-aware training-free distillation framework, improving FID, real-synthetic embedding distance and classification accuracy over prior training-free and training-based baselines.
Explained
What came before
MGD3 guides diffusion sampling toward K-means-derived class-mode centroids in VAE latent space using a Euclidean (Gaussian-kernel) attraction term, without any fine-tuning. Minimax and D4M represent the training-based and prototype-clustering strands respectively.
The problem
MGD3-style "mode guidance" pulls the denoising trajectory toward a centroid using an ordinary Euclidean vector in ambient latent space, but the true data manifold is a curved, lower-dimensional subset of that space; the paper proves (via a tangent/normal decomposition) that this attraction generally has a nonzero normal-space component, which pushes the trajectory off the data manifold especially as $t\to0$ when the marginal $p_t$ concentrates sharply near the manifold -- producing artifacts (e.g. a dog with legs in an unusual position, blurred textures) even though the class-level semantic direction is correct. Separately, K-means-based IPC centroids (as in D4M/MGD3) capture only one spatial scale of class structure.
The idea
Build a multi-scale set of prototypes with hierarchical (top-down, divisive bisecting K-means) clustering of class-wise VAE latents instead of flat K-means, so centroids span both coarse and fine modes; then, at every denoising step, decompose the standard mode-guidance vector into components tangent and normal to a locally-estimated data manifold (built from the noised neighborhood of the nearest prototype) and subtract the normal component, so guidance only pushes the trajectory along directions the manifold can actually support.
How it works
Conditional score decomposes as $\nabla_{x_t}\log p_t(x_t|c)=\nabla_{x_t}\log p_t(x_t) + \nabla_{x_t} \log p_t(c|x_t)$ (marginal denoising + mode guidance); mode guidance is generalized to any PSD kernel potential $\phi$ (Gaussian/RBF recovers MGD3, plus Laplace and inverse-multiquadric are tested) giving $g^t_{mode}=-\phi'(\|x_t-c\|_2)\frac{x_t-c}{\|x_t-c\|_2}$. IPC centroids $c_s$ come from a divisive (bisecting K-means) hierarchical clustering tree over VAE latents per class; a coarse-to-fine sweep from a chosen start depth $s_{start}$ down to the root picks one centroid per level (deeper splits for well-separated classes, shallower for overlapping ones), with any remaining IPC quota filled by random leaf centroids. For each chosen centroid's local latent neighborhood $\mathcal{N}_s$, forward-diffuse it to timestep $t$'s noise level to get a local point cloud $\mathcal{M}_t^{(s)}$; find the current sample $x_t$'s $K_t$ nearest neighbors in this cloud, take the empirical covariance's leading eigenvectors as the tangent-space basis, and build projector $P_{\mathcal{N}_t}=I-P_{\mathcal{T}_t}$; the corrected guidance is $g^t_{manifold}=g^t_{mode}-P_{\mathcal{N}_t}g^t_{mode}$, i.e. the normal component is subtracted, not the tangent component kept exclusively (explicit normal subtraction rather than strict tangent projection, to avoid over-smoothing per Remark 1). The neighborhood radius for tangent estimation is annealed (exponential decay found best) across timesteps, and guidance is switched off after a stop step $T_{STOP}$ (tuned, best around 20-25 of 50 steps). No fine-tuning; base generator is DiT.
Evidence
ImageNette hard-label protocol, ResNetAP-10 (Table 1), IPC10/20/50: 64.1±1.2/69.7±0.4/78.4±1.0, beating the reimplemented MGD3 61.9±0.6/66.5±0.5/77.5±1.1 and the unguided reimplemented DiT 59.1±0.7/64.8±1.2/ 73.3±0.9, and matching or exceeding several training-based methods (Minimax 62.0±0.2 at IPC10, D4M 64.8±3.6 at IPC10) under the same protocol. ImageNet-100 (Table 1), IPC10/20: 27.6±0.5/35.3±0.5 vs MGD3 26.1±0.6/33.2±0.5. Hard-label-only, full ImageNet-1k, ConvNet-6 (Table 10, appendix), IPC1/50: 3.1/21.4 vs MGD3 2.8±0.9/20.3±1.1 and DiT 2.6±0.3/18.5±1.3 -- markedly lower absolute numbers than soft-label-protocol papers (e.g. CoDA, DAP) because this is the harder hard-label-from-scratch evaluation, not directly comparable across protocol. Ablation (Table 3, ImageNette IPC10, ConvNet-6/ ResNetAP-10/ResNet-18): flat K-means 56.3/61.0/59.7 to divisive (no level-wise) 57.4/62.5/58.5 to divisive-levelwise 59.2/63.3/61.1 to full method with manifold guidance 60.5/64.0/62.3 to annealed radius 60.8/64.5/62.7 -- isolating that both the clustering upgrade and the manifold-correction term each contribute, and are complementary (agglomerative clustering alone is worst: 37.7/45.9/42.6). Kernel-choice ablation (Table 5) shows manifold guidance improves results regardless of the underlying kernel (RBF/Laplace/IMQ), evidencing the correction is a general mechanism, not tied to MGD3's specific Gaussian kernel. $T_{STOP}$ ablation (Fig. 8) shows FID and accuracy improve as guidance is applied through early-to-mid denoising up to ~step 25 of 50, then degrade if applied too late into the low-noise regime.
Limitations
Stated (Conclusion): at high noise levels the local neighborhood used to estimate the tangent space is itself corrupted by diffusion noise, biasing the tangent/normal estimate; low-rank tangent approximations struggle on highly curved manifolds, risking over-smoothing and reduced diversity; the paper mitigates these heuristically (adaptive neighborhood size/radius, ridge-regularized covariance, annealing) but states that formal analysis of projection error and curvature sensitivity is future work. Observed: evaluated only under the hard-label protocol, so its absolute numbers (e.g. ImageNet-1k IPC50 21.4%) cannot be compared to the soft-label-protocol numbers most other 2025-2026 diffusion papers report (e.g. CoDA 60.4%, DAP 62.7%) on the same benchmark; the tangent-space estimation adds a k-nearest-neighbor search and eigendecomposition at every denoising step for every sample, whose wall-clock cost relative to unguided or MGD3 sampling is not quantified in the main text; several hyperparameters (start depth $s_{start}$, neighborhood size $K_t$, radius annealing schedule, $T_{STOP}$) are tuned per dataset.
Written by the atlas from the paper's full text. Check the paper for exact numbers.
Where it sits
- Diffusion-based synthesis (Generative priors)
- Setting: Image classification
Design choices
| Prior / networks used | diffusion |
| Optimization regime | training-free |
| What is stored | pixels |
| Largest scale evaluated | imagenet-subsets, imagenet-1k |
Builds on
Abstract (verbatim from arXiv)
In recent times, large datasets hinder efficient model training while also containing redundant concepts. Dataset distillation aims to synthesize compact datasets that preserve the knowledge of large-scale training sets while drastically reducing storage and computation. Recent advances in diffusion models have enabled training-free distillation by leveraging pre-trained generative priors; however, existing guidance strategies remain limited. Current score-based methods either perform unguided denoising or rely on simple mode-based guidance toward instance prototype centroids (IPC centroids), which often are rudimentary and suboptimal. We propose Manifold-Guided Distillation (ManifoldGD), a training-free diffusion-based framework that integrates manifold consistent guidance at every denoising timestep. Our method employs IPCs computed via a hierarchical, divisive clustering of VAE latent features, yielding a multi-scale coreset of IPCs that captures both coarse semantic modes and fine intra-class variability. Using a local neighborhood of the extracted IPC centroids, we create the latent manifold for each diffusion denoising timestep. At each denoising step, we project the mode-alignment vector onto the local tangent space of the estimated latent manifold, thus constraining the generation trajectory to remain manifold-faithful while preserving semantic consistency. This formulation improves representativeness, diversity, and image fidelity without requiring any model retraining. Empirical results demonstrate consistent gains over existing training-free and training-based baselines in terms of FID, l2 distance among real and synthetic dataset embeddings, and classification accuracy, establishing ManifoldGD as the first geometry-aware training-free data distillation framework.
BibTeX (generated; prefer the venue's official entry)
@article{roy2026manifoldgd,
title = {ManifoldGD: Training-Free Hierarchical Manifold Guidance for Diffusion-Based Dataset Distillation},
author = {Ayush Roy and Wei-Yang Alex Lee and Rudrasis Chakraborty and Vishnu Suresh Lokhande},
journal = {CVPR 2026},
year = {2026}
}Nearby in Diffusion-based synthesis
Learnability-guided diffusion — Learnability-Guided Diffusion for Dataset Distillation
Jeffrey A. Chan-Santiago, Mubarak Shah · CVPR 2026notablepaper ↗
PDS — Multimodal Dataset Distillation Made Simple by Prototype-Guided Data Synthesis
Junhyeok Choi, Sangwoo Mo, Minwoo Chae · ICLR 2026coreVision–languagepaper ↗code ↗