Research line · ascendant
Continuous-token autoregression
Autoregress over continuous vectors, scoring each with a small diffusion head instead of a softmax over a codebook.
What defines membership
Autoregression is worth keeping but quantization is not; per-token continuous prediction removes the codebook entirely.
How the line developed
Read top to bottom: what came before, the idea itself, the evidence for it, what improved, and where it breaks.
Before
GIVT · 2023-12core
Drops the codebook with a Gaussian-mixture head rather than a diffusion loss, predating MAR with the same underlying objection to quantisation.
The idea
MAR · 2024-06landmark
Shows vector quantisation is unnecessary by scoring continuous tokens with a small per-token diffusion loss.
Evidence
Resurrect MAR · 2025-07strong-followup
Isolates the tokenizer and bidirectional architecture, not the objective, as what actually limits masked autoregressive models.
Improvement
Transfusion · 2024-08landmark
Runs the same per-token diffusion loss on continuous image tokens inside a transformer that also does next-token text prediction.
At scale
Fluid · 2024-10landmark
Scales the recipe to text-to-image at up to 10.5B parameters with random generation order. Continuous tokens win at every scale tested, though the paper stops short of claiming the margin grows, and reports GenEval plateauing past 3.1B.
NextStep-1 · 2025-08core
A 14B model with a flow-matching head reaching state of the art among autoregressive text-to-image systems.
What it gets right
- No quantization loss, so no reconstruction ceiling from the codebook
- Keeps autoregression's flexible generation order and language-model compatibility
- Scaling evidence now exists at frontier text-to-image size
Where it is weak
- Per-token diffusion head adds sampling cost
- Still sequential, so slower than parallel denoising at comparable quality
Reported numbers
As published, with the guidance method, budget, and model size that produced them. Seethe comparison page for why these cannot be ranked naively.