Research line · dominant
Editing in the VAE latent
Train an editor that reads an instruction and rewrites the image inside a standard reconstruction-trained latent.
What defines membership
Editing is a supervised generation problem: given enough instruction-image-result triplets, a model learns what to change and what to leave.
How the line developed
Read top to bottom: what came before, the idea itself, the evidence for it, what improved, and where it breaks.
The idea
InstructPix2Pix · 2022-11landmark
Manufactures the training data the task needed, with a language model writing instructions and a prompt-to-prompt pipeline rendering before and after images.
Evidence
MagicBrush · 2023-06core
Human annotators required to leave unrelated regions untouched; concludes even tuned models fall short of practical use.
Improvement
Emu Edit · 2023-11core
One editor across region, free-form and recognition tasks, steered by learned task embeddings.
Step1X-Edit · 2025-04core
A multimodal language model parses the instruction and a diffusion decoder executes it; source of GEdit-Bench.
At scale
FLUX.1 Kontext · 2025-06landmark
Concatenates reference and target tokens into one sequence in a rectified-flow model, and concedes multi-turn character drift as its named limitation.
What it gets right
- By far the best-populated line, with mature tooling and open weights
- Handles open-vocabulary instructions rather than a fixed operation set
- Inherits every improvement to the underlying text-to-image model
Where it is weak
- Preservation of untouched regions is learned statistically, never guaranteed
- Drift compounds across multi-turn edits
- Depends on synthetic training triplets whose quality bounds the result