Research line · contested
Discrete-token autoregression
Quantize an image into discrete tokens and predict them with a language-model-style autoregressive transformer.
What defines membership
Images should be modeled the way text is: as sequences of discrete tokens from a learned codebook.
How the line developed
Read top to bottom: what came before, the idea itself, the evidence for it, what improved, and where it breaks.
The idea
VAR · 2024-04landmark
Redefines the order as coarse-to-fine next-scale prediction rather than raster next-token; first autoregressive model to beat diffusion transformers on ImageNet FID while showing GPT-style scaling laws.
Evidence
GigaTok · 2025-04strong-followup
Scales a tokenizer to 3B parameters and finds better reconstruction does not monotonically improve downstream generation.
Improvement
Tokenizer Post-Training · 2025-09emerging
Post-trains the tokenizer on the generative distribution it will actually have to decode, closing a train/test gap.
Masked Bit Modeling · 2026-02emerging
Predicts at the bit level to sidestep vocabulary-size and codebook-collapse limits.
What it gets right
- Reuses the entire language-model stack, including scaling laws and serving infrastructure
- Natural fit for unified understanding-and-generation models
Where it is weak
- Quantization discards information and imposes a hard reconstruction ceiling
- Codebook collapse and vocabulary sizing remain persistent engineering problems
- Largely displaced by continuous-token variants in recent results
Reported numbers
As published, with the guidance method, budget, and model size that produced them. Seethe comparison page for why these cannot be ranked naively.
| Paper | Benchmark | Value | Guidance | Budget | Params |
|---|---|---|---|---|---|
| VAR | ImageNet 256x256 gFID | 1.73 | guidance ratio 2.0 + rejection sampling | 350 epochs | 2.0B |
Competing answers
Also in this line
dRAE