Generative Vision Atlas

Research lines

A line is a family of work sharing one core bet. Lines are grouped below by the design question the bet answers. A paper can belong to more than one line, and lines that answer the same question compete with each other directly.

Medical imaging is kept as a separate domain with its own 5 lines, because its constraints and its evidence bar are different. Go to medical imaging →

representation

What space to generate in

Contrastive language-image pretraining

dominant5 papers · since 2021-02

Learn visual features by aligning images to their captions across web-scale data.

The default choice. Most systems you encounter use this.

Self-distillation representations

dominant7 papers · since 2021-11

Learn visual features with no labels, no language and no pixel reconstruction, by having a network predict its own slowly-updated teacher.

The default choice. Most systems you encounter use this.

VAE-latent diffusion

dominant12 papers · since 2021-12

Compress images with a reconstruction-trained autoencoder, then run diffusion or flow matching in that latent space.

The default choice. Most systems you encounter use this.

Agglomerative multi-teacher distillation

ascendant5 papers · since 2023-12

Distil several specialist foundation models into one backbone rather than choosing between their objectives.

Gaining ground quickly, with results that are holding up.

Continuous-token autoregression

ascendant6 papers · since 2023-12

Autoregress over continuous vectors, scoring each with a small diffusion head instead of a softmax over a codebook.

Gaining ground quickly, with results that are holding up.

Representation-space generation

contested11 papers · since 2022-04

Freeze a vision foundation model and generate directly inside its feature space; train only a decoder back to pixels.

Live disagreement about whether it is the right bet. Not a verdict of failure.

Single-stage pixel transformers

contested17 papers · since 2025-07

Drop the tokenizer entirely and generate raw pixels with one plain transformer, in one stage, with no cascade.

Live disagreement about whether it is the right bet. Not a verdict of failure.

Semantic-plus-detail hybrids

emerging9 papers · since 2024-12

Take a foundation-model representation and give it back the low-level detail it discards, with a residual branch, detail queries, or a VAE compressing it.

Young, promising, and not yet independently replicated at scale.

Editing in a representation latent

emerging4 papers · since 2025-12

Perform the edit inside a foundation-model feature space rather than a reconstruction latent.

Young, promising, and not yet independently replicated at scale.

Cascaded and multiscale pixel diffusion

superseded6 papers · since 2021-05

Keep generation in pixels, but handle resolution progressively: generate small, then refine upward.

Largely displaced, kept for the history it explains.

objective

What to train the model to predict

Flow matching and rectified flow

dominant5 papers · since 2022-09

Regress the velocity field of a straight path between noise and data, rather than reversing a stochastic noising process.

The default choice. Most systems you encounter use this.

Discrete-token autoregression

contested5 papers · since 2024-04

Quantize an image into discrete tokens and predict them with a language-model-style autoregressive transformer.

Live disagreement about whether it is the right bet. Not a verdict of failure.

Natively few-step objectives

emerging11 papers · since 2024-10

Train for one or few steps from the start, instead of training a many-step model and distilling it afterwards.

Young, promising, and not yet independently replicated at scale.

Normalizing-flow revival

emerging3 papers · since 2024-12

Return to exact-likelihood invertible models, scaled up with transformers and trained in a latent space.

Young, promising, and not yet independently replicated at scale.

Transition matching

emerging4 papers · since 2025-06

Model generation as discrete-time Markov transitions over a continuous state, unifying flow matching and continuous-token autoregression.

Young, promising, and not yet independently replicated at scale.

training signal

What shapes the model beyond the base objective

Representation-aligned latents

ascendant6 papers · since 2024-10

Keep the autoencoder, but apply pressure from a vision foundation model so its latent (or the generator's hidden states) become semantically structured.

Gaining ground quickly, with results that are holding up.

Reinforcement learning and preference alignment

ascendant4 papers · since 2025-05

Fine-tune generators against reward models or human preference with policy-gradient methods, rather than only maximizing likelihood.

Gaining ground quickly, with results that are holding up.

inference

What happens at sampling time

Guidance and sampling

ascendant8 papers · since 2022-07

Change what happens at inference time — guidance formulation, stochasticity, step schedule — without retraining the model.

Gaining ground quickly, with results that are holding up.

Training-free attention manipulation

ascendant5 papers · since 2022-08

Edit by intervening in attention maps and features at sampling time, with no training and often no change to the latent at all.

Gaining ground quickly, with results that are holding up.

Inversion for flow models

ascendant3 papers · since 2024-10

Recover the noise that produced a real image so it can be re-generated with a change, and control the error that recovery introduces.

Gaining ground quickly, with results that are holding up.

Evaluation and benchmarks

ascendant7 papers · since 2025-03

Measure whether generators actually do what their scores claim, and fix the benchmarks when they stop tracking human judgment.

Gaining ground quickly, with results that are holding up.

conditioning

How the condition reaches the generator

Editing in the VAE latent

dominant5 papers · since 2022-11

Train an editor that reads an instruction and rewrites the image inside a standard reconstruction-trained latent.

The default choice. Most systems you encounter use this.

In-context editing

dominant7 papers · since 2025-01

Edit by putting the reference image into the model's own token sequence and letting attention handle it, rather than through adapters or inversion.

The default choice. Most systems you encounter use this.

Cross-attention into a frozen language model

contested4 papers · since 2022-04

Inject visual information through dedicated gated cross-attention layers interleaved into a frozen language model.

Live disagreement about whether it is the right bet. Not a verdict of failure.

Adapter-based conditioning

contested6 papers · since 2023-02

Bolt a small trainable side network onto a frozen pretrained diffusion model to inject a new control signal, instead of retraining or restructuring the backbone itself.

Live disagreement about whether it is the right bet. Not a verdict of failure.

Agentic and search-augmented generation

emerging5 papers · since 2022-09

Give the generator tools, search, or a planning loop, so it can render subjects its training data never covered.

Young, promising, and not yet independently replicated at scale.

architecture

How understanding and generation are arranged

Encoder plus projector

dominant3 papers · since 2023-04

Bolt a frozen vision encoder onto a language model with a small trained projection, and treat the projected patches as prompt tokens.

The default choice. Most systems you encounter use this.

Unified understanding and generation

ascendant14 papers · since 2023-12

Build one model that both interprets images and produces them, instead of separate understanding and generation systems.

Gaining ground quickly, with results that are holding up.

Native-resolution vision-language models

ascendant4 papers · since 2024-09

Process images at their own resolution and aspect ratio, with position encodings general enough to handle the variation.

Gaining ground quickly, with results that are holding up.

Editing inside a unified model

ascendant4 papers · since 2025-05

Treat editing as one more thing a single multimodal model does, with no editing-specific pipeline.

Gaining ground quickly, with results that are holding up.