Research line · emerging
Natively few-step objectives
Train for one or few steps from the start, instead of training a many-step model and distilling it afterwards.
What defines membership
Iterative sampling is an artifact of the training objective, and a better objective yields one-step generation directly.
How the line developed
Read top to bottom: what came before, the idea itself, the evidence for it, what improved, and where it breaks.
The idea
Shortcut Models · 2024-10core
Conditions the network on step size so one model works at any budget down to a single step, with no distillation stage.
IMM · 2025-03core
Matches distributions between interpolant marginals in one training stage, with convergence guarantees consistency models lack; 1.99 FID in 8 steps.
MeanFlow · 2025-05landmark
Targets average rather than instantaneous velocity, giving a self-contained identity for from-scratch one-step training; 3.43 FID at 1 NFE.
Evidence
SANA-Sprint · 2025-03strong-followup
The distillation-based contrast case: 1-4 steps via continuous-time consistency distillation plus an adversarial term, but requires a teacher.
Improvement
Drifting Models · 2026-02landmark
Learns the generator mapping directly and evolves its pushforward until it matches the data; natively one-step, 1.54 FID on ImageNet 256.
Pixel MeanFlow · 2026-01strong-followup
Combines one-step training with no tokenizer at all.
Combination
Distilling Drifting Transformers · 2026-06strong-followup
Applies drifting-style distillation inside RAE latents, where anisotropy makes the naive version unstable.
What it gets right
- Collapses the inference-cost gap that made diffusion expensive to serve
- No teacher model, so no inherited teacher errors and one training run instead of two
- Advancing fast, with several independent formulations converging on similar quality
Where it is weak
- Still generally behind the best many-step models at the very top of the quality range
- Mostly demonstrated on class-conditional ImageNet rather than frontier text-to-image
- Interacts badly with high-dimensional anisotropic latents, as the RAE case shows
Reported numbers
As published, with the guidance method, budget, and model size that produced them. Seethe comparison page for why these cannot be ranked naively.
| Paper | Benchmark | Value | Guidance | Budget | Params |
|---|---|---|---|---|---|
| Distilling Drifting Transformers | ImageNet 256x256 gFID (NFE=1) | 1.48 | extrapolation-based | 16 epochs distillation | 839M (inherited from RAE) |
| Drifting Models | ImageNet 256x256 gFID (NFE=1) | 1.61 | not stated | not stated | not stated |
| Drifting Models | ImageNet 256x256 gFID (NFE=1) | 1.54 | not stated | not stated | not stated |
| IMM | ImageNet 256x256 gFID (NFE=8) | 1.99 | not stated | not stated | not stated |
| MeanFlow | ImageNet 256x256 gFID (NFE=1) | 3.43 | not stated | not stated | not stated |
Competing answers
Open problems it has not solved
- Rectified-flow-style objectives aim for straight, easy-to-distill transport paths, but anisotropic high-dimensional latents (like RAE's) can force curved, unstable trajectories even under a straight-path objective.
- Should the noise source distribution for diffusion/flow training be a simple isotropic Gaussian, or something structured (a mixture, a learned distribution) that better matches the target latent's actual geometry?
Also in this line
Any-OPD, CrossFlow, Parallel Decoding Distillation, Second-Order Drifting