Generative Vision Atlas

Research line · contested

Single-stage pixel transformers

Drop the tokenizer entirely and generate raw pixels with one plain transformer, in one stage, with no cascade.

What defines membership

No learned latent is necessary. Large patches plus the right prediction target let one end-to-end model find the manifold a VAE was trained separately to provide.

The bet

Every latent line in this atlas shares one premise: that raw pixel space has too many spatially-correlated dimensions to model directly, so a separately trained encoder must first project images onto a lower-dimensional manifold. This line denies it. The claim is that the denoising or flow-matching objective already forces the network to discover that manifold, and that large patches plus the right prediction target let a single end-to-end model do the compression internally — with no VAE, no cascade, and no second training stage.

The payoff, stated in almost the same words across PixNerd, JiT, DiP and HyperDiT: you lose the tokenizer’s reconstruction ceiling, the blur and colour artifacts it imposes, and the non-end-to-end pipeline it requires.

The arc

PixNerd is the technical origin, in July 2025, replacing the VAE with a per-patch neural field and reaching ImageNet-256 FID 2.15 in one stage. Four months later JiT makes the argument the field actually heard — that ε-prediction, not pixels, caused pixel-space instability, and that x-prediction on large patches makes a plain ViT competitive. JiT is usually cited as where this line begins. It is not, and the distinction matters for anyone tracing priority.

What follows splits into three kinds of work. Architecture: PixelDiT and DiP, both at CVPR 2026, separate global semantics from texture refinement across two levels — a pattern HyperDiT extends with semantic anchors and foundation-model register tokens. Diagnosis: PixelREPA finds that REPA, which reliably accelerates latent diffusion transformers, actively hurts JiT; Register Guidance finds register tokens help pixel-space transformers more than latent ones. Both say the same thing from different directions — pixel space is not latent space with more dimensions, and interventions do not transfer between them. Efficiency: MOSAIK cuts 70% of FLOPs with content-aware patch sizing, and the empirical text-to-image study reports 3.2–4.8× faster inference than latent baselines, inverting the efficiency argument that motivated latents in the first place.

Parallel Rollout Approximation is the line’s most useful piece of evidence, because it is not diffusion at all. If the tokenizer is genuinely dispensable, that should not depend on the generative objective — and a pixel-space autoregressive model at 511M parameters reaching FID 1.94 suggests it does not.

Why the FID numbers here cannot be ranked

The papers on this page report ImageNet-256 FIDs of 1.51, 1.52, 1.56, 1.58, 1.61, 1.79, 1.82, 1.94 and 2.15. That spread looks like a leaderboard and is not one. The entries differ in parameter count (511M to 952M), training budget (160 to 600 epochs), guidance scale (2.75 to 3.5) and guidance interval, and every one of those moves FID by more than the gaps between neighbouring rows. The guidance notebook shows a single frozen model moving further under sampler changes alone than these architectures move from each other. Read the within-paper ablations instead; see why the numbers cannot rank the approaches.

Two further cautions specific to this line. The two lowest numbers, from PixSGR and Observation Operators, are single-version preprints, and Observation Operators shares four authors with HyperDiT, so it is one group’s continuing programme rather than independent confirmation.

What holds up, and what does not

The reconstruction-ceiling argument holds: no paper here has been refuted, and removing a lossy component removes its failure modes. The efficiency objection that justified latents is now contested in the right direction, by measurement rather than assertion.

What does not yet hold is the scope. Nearly every headline number on this page is class-conditional ImageNet. Text-to-image evidence is one systematic study old. And the line’s own strongest paper is candid about the gap: PixelDiT states plainly that pixel data and noise distributions are more complex than latent ones and that no mature training recipe exists for them. Attention cost still scales badly with resolution — several papers here exist only to patch that, which is itself a measure of how binding the constraint remains.

A correction this atlas owes the reader

Until September 2026 this line held two papers, both from MIT, and its own text told readers that pixel-space results were “mostly from one group”. That was not a fact about the field. It was this atlas’s coverage gap stated as one. The verified affiliations now on these pages span NVIDIA, Google DeepMind, Yandex Research, Alibaba, Tencent Youtu, KAIST, NTU Singapore, UESTC, Nanjing University, NUS, Peking University, HKU, Stability AI, Stanford, Caltech and ByteDance Seed.

Anyone reading this line should also read cascaded and multiscale pixel diffusion, which made a version of the same argument years earlier and was largely ignored. SiD2 reached FID 1.48 on ImageNet 512 in pixel space in 2024 — still unbeaten at that resolution by anything on this page.

How the line developed

Read top to bottom: what came before, the idea itself, the evidence for it, what improved, and where it breaks.

The idea

PixNerd · 2025-07landmark

Per-patch neural fields replace the VAE outright, single-scale and single-stage, at ImageNet 256 FID 2.15. Four months before JiT and independent of it, though its diagnosis differs: it blames the two-stage pipeline's accumulated error, not the prediction target, and uses velocity prediction throughout.

JiT · 2025-11strong-followup

Argues epsilon-prediction rather than pixels caused pixel-space instability, and that x-prediction on large patches makes a plain ViT competitive with tokenizer-based DiTs. The paper that made the argument the field actually heard, though not the first to make it.

Evidence

PixelREPA · 2026-03strong-followup

REPA, which accelerates latent DiTs, makes JiT worse: FID 5.14 against vanilla JiT's 4.37 at 200 epochs on ImageNet 256, with a diversity-collapse pattern. Its masked-adapter fix moves B/16 from 3.66 to 3.17 and reaches 1.81 at H/16.

Register Guidance · 2026-05strong-followup

Register tokens help pixel-space transformers more than latent ones, so pixel space is not just latent space with more dimensions.

Parallel Rollout Approximation · 2026-06emerging

The same bet under an autoregressive rather than diffusion objective, suggesting the tokenizer's dispensability is not a property of diffusion specifically.

Pixel-space survey · 2026-07core

A review of the area, which is itself evidence the line stopped being a handful of papers.

Improvement

There is No VAE · 2025-10core

Keeps a self-supervised pretrained encoder but discards the VAE, hitting FID 1.58 at about 30 percent of DiT's training compute and yielding the first pixel-space consistency model.

PixelDiT · 2025-11core

Separates global semantics from texture refinement across two DiT levels, and carries the recipe to 1024px text-to-image.

DiP · 2025-11core

Attacks the efficiency objection directly: a lightweight detailer head claims up to 10x faster inference for 0.3 percent more parameters.

Pixel MeanFlow · 2026-01strong-followup

Adds one-step generation, removing the latent at sampling time as well as training time.

Self-Sample Guidance · 2026-07strong-followup

A self-guidance head on a frozen backbone improves both JiT and PixelREPA for under one percent extra compute, which is a larger gain than several architectural changes here.

HyperDiT · 2026-05core

Semantic anchors, scale-aware position embeddings and foundation-model registers reach FID 1.56, at the cost of depending on its own anchor mechanism to converge at all.

MOSAIK · 2026-08emerging

Content-aware variable patch size cuts 70 percent of FLOPs, addressing the resolution scaling objection rather than working around it.

Observation Operators · 2026-08emerging

FID 1.52, from the same group as HyperDiT rather than an independent replication, and reported with no guidance scale, interval or NFE anywhere in the text.

PixSGR · 2026-09emerging

FID 1.51 and 1.60 at 256 and 512, the lowest reported here and the least scrutinised. Its own no-guidance ablation sits at 2.22, so most of that headline comes from the sampler rather than the architecture.

At scale

Pixel T2I empirical study · 2026-08core

The first systematic text-to-image training recipe, reporting 3.18x to 4.75x end-to-end speedups against three named latent baselines on the same backbones, measured as wall-clock latency at 1024px on one H800.

What it gets right

  • Removes an entire component, and with it the tokenizer's failure modes and its reconstruction ceiling
  • No train/test mismatch between real and generated latents
  • The strongest existing challenge to the premise every latent line shares
  • Now reports inference speedups over latent baselines, inverting the argument that created latents

Where it is weak

  • Attention cost still scales badly with resolution; several papers here exist only to patch that
  • Every headline number is class-conditional ImageNet; text-to-image evidence is one paper old
  • PixelDiT states plainly that no mature training recipe exists for pixel data and noise distributions
  • The two lowest FIDs are single-version preprints whose authors overlap, and one of them reports no guidance setting at all while the other's headline is mostly guidance: its no-CFG ablation is 2.22 against a reported 1.51

Reported numbers

As published, with the guidance method, budget, and model size that produced them. Seethe comparison page for why these cannot be ranked naively.

PaperBenchmarkValueGuidanceBudgetParams
JiTImageNet 256x256 gFID1.82CFG-interval600 epochs~2B

Competing answers

Open problems it has not solved

Also in this line

HiDream-O1-Image