Unified understanding and generation

Transfusion · 2024-08Continuous-token autoregressionUnified understanding and generation

Runs next-token prediction for text and a diffusion objective for continuous image latents inside one transformer, with one set of weights and two losses. The cleanest demonstration that unifying understanding and generation does …

Chameleon · 2024-05Unified understanding and generation

An early-fusion, token-based decoder-only transformer that represents text and images as one unified token vocabulary and one interleaved sequence from the start of pretraining, so there is no separate conditioning module at all —…

LLaDA-Image · 2026-09VAE-latent diffusion

A 6B diffusion transformer trained from scratch alongside a frozen vision-language understanding module, released with weights, code and the full training recipe over 220 million samples — an unusually complete open release. The T…

HiDream-O1-Image · 2026-05Editing inside a unified modelSingle-stage pixel transformers

Maps raw image pixels, text tokens and task-specific condition tokens into one shared token space processed by the same transformer blocks through joint self-attention, eliminating the VAE and the separate text encoder entirely. E…

Emu3.5 · 2025-10Editing inside a unified modelUnified understanding and generation

The closest thing to a genuinely single network among open unified models: one autoregressive transformer over interleaved text and vision tokens with one training objective. A discrete-diffusion adaptation module accelerates deco…

HunyuanImage 3.0 · 2025-09Unified understanding and generation

An 80B mixture-of-experts model that unifies understanding and generation in one backbone, with autoregressive text generation and a diffusion head operating on continuous VAE latents for images, closer to Transfusion's hybrid tha…

NextStep-1 · 2025-08Continuous-token autoregression

Scales MAR's continuous-token idea to a 14B autoregressive model with a 157M flow-matching head, reaching state of the art among autoregressive text-to-image systems. An ICLR 2026 oral, and the strongest evidence that continuous-t…

OmniGen2 · 2025-06Editing inside a unified modelIn-context editingUnified understanding and generation

Uses two decoding pathways with unshared parameters for text and image, rather than forcing one shared path, and covers text-to-image, editing, and in-context generation in one model. A CVPR 2026 paper and the leading fully open u…

Show-o2 · 2025-06Unified understanding and generation

Refines the single-transformer recipe that runs autoregressive text and diffusion image generation in one set of weights, and moves to 3D-VAE latents. Evidence the native-unified line is maturing rather than stalling.

BAGEL · 2025-05Editing inside a unified modelUnified understanding and generation

A mixture-of-transformer-experts model with one expert for understanding and one for generation, routed over shared token sequences, trained on large-scale interleaved data. Unified at the sequence level, but not literally one net…

BLIP3-o · 2025-05Unified understanding and generation

A fully open study of unified-model design choices that lands on a diffusion transformer generating CLIP image features (not VAE latents), trained understanding-first then generation-second so understanding ability is preserved; s…

MetaQuery · 2025-04Encoder plus projectorUnified understanding and generation

Bridges a frozen, understanding-only multimodal LLM to a diffusion image decoder with a small set of learnable query tokens, so the MLLM's weights (and its reasoning/understanding ability) never have to change to gain generation. …

Janus-Pro · 2025-01Unified understanding and generation

Decouples the visual encoder used for understanding from the one used for generation while keeping a single shared language backbone, then scales data and model size. One of the most reused open unified baselines of the period.

Janus · 2024-10Unified understanding and generation

Diagnoses the conflict in single-encoder unified models, where understanding wants semantic abstraction and generation wants reconstructable detail, and resolves it with two separate visual encoders feeding one shared language bac…

Emu3 · 2024-09Unified understanding and generation

Tokenizes text, images, and video into one discrete vocabulary and trains a single decoder-only transformer purely on next-token prediction — no diffusion, no compositional modules — arguing this single objective is enough to matc…

Emu2 · 2023-12Representation-space generationUnified understanding and generation

A 37B multimodal model that autoregressively predicts the next CLIP visual embedding alongside text tokens, then decodes embeddings back to pixels with a diffusion decoder — an autoregressive, MLLM-scale continuation of unCLIP's s…

STARFlow2 · 2026-05Normalizing-flow revivalUnified understanding and generation

Argues an autoregressive normalizing flow is structurally just an autoregressive transformer, then fuses a vision-language stream with a flow stream through residual connections so flow-generated visual tokens land in the same key…

UniEval · 2025-05Evaluation and benchmarks

The first benchmark that evaluates understanding and generation together, which is the only way to detect the failure mode unified models are most prone to: gaining generation ability while quietly losing understanding.

TokenFlow · 2024-12Semantic-plus-detail hybrids

Argues understanding and generation genuinely need different granularities of visual information, and resolves the conflict with a dual-codebook design — a CLIP-style semantic encoder and a pixel encoder, kept aligned through a sh…

UniSpace · 2026-08Editing in a representation latent

An 8B mixture-of-experts unified understanding, generation and editing model built on a reparameterized frozen vision-transformer feature space rather than a VAE pathway. Flagged as needing an independent verification pass before …