emerging
Pixel-Space Diffusion via Observation Operators
· 2026-08 · arXiv:2608.21885
Why it matters
ImageNet 256 FID 1.52, among the strongest pixel-space results reported. Shares four authors with HyperDiT, so it is one group's continuing programme rather than independent confirmation.
What this paper does
read: full textBefore this
Pixel-space diffusion models had begun reparameterizing what the network predicts (epsilon, v, or x0, as in JiT), but every such reparameterization still supervised the network against the full clean image at every noise level. The paper argues this is a mismatch: as noise decreases during sampling, only coarse image structure is reliably inferable at first, and fine detail only becomes recoverable later, yet fixed full-image supervision forces the network to fit undeterminable fine detail even under heavy noise.
The problem
How to align the training supervision signal with the actual order in which image structure becomes inferable during denoising, rather than always supervising against the full-resolution clean image regardless of noise level.
The idea
Replace the fixed clean-image target with a time-indexed observation of the clean image that starts coarse and sharpens to the full image as noise decreases, so at every noise level the network is only asked to predict what is actually recoverable at that noise level. The observation operator is built from a family of Gaussian-Lanczos smoothing kernels whose scale is a function of time, and the network's decoder is restructured to refine features through the same coarse-to-fine sequence of scales.
How it works
The backbone follows a two-stage Patch-DiT / pixel-decoder design similar to PixelDiT: a 26-block, 1152-hidden-dimension Patch-DiT encoder, followed by a 4-block GL-CoDA decoder in which each block pools pixel features, extracts structural responses at a Gaussian-Lanczos scale (four scales total, applied in coarse-to-fine order, with later blocks taking the difference from the previous scale's response to avoid redundancy), and refines decoder features with these responses via lightweight adapters. The observation operator combines a Lanczos-windowed sinc kernel (sharper scale selectivity) with Gaussian smoothing (suppresses ringing), with kernel width narrowing over time (parameters f0=2, sigma0=1, linear schedule, 15-tap kernel support) until it becomes the identity at t=1. Training uses a path-consistent flow-matching loss against a velocity target that accounts for the observation operator's own time-derivative, predicted directly as a velocity, plus a REPA representation-alignment loss; everything is trained end to end, prediction target is velocity, and nothing in the description is stated as frozen (REPA implies an external frozen encoder is used for its alignment target, consistent with standard REPA practice, though the paper's own wording for that encoder was not captured in the fetched text).
Evidence
On ImageNet 256x256, the model (798M parameters) reaches gFID 1.95 at 80 epochs and gFID 1.52 at 260 epochs, with sFID 4.61, IS 300.0, precision 0.78, recall 0.64 at the 260-epoch point. On ImageNet 512x512 (798M parameters), gFID reaches 1.58, sFID 4.86, IS 299.8, precision 0.79, recall 0.69. Critically, the fetched paper does not report CFG scale, guidance interval, sampling step count, or NFE for any of these results -- this is a real gap given the atlas's standard of requiring guidance and budget alongside any FID, and it means the 1.52 and 1.58 numbers cannot currently be compared on equal footing with the other papers in this batch, all of which report their guidance settings. An ablation shows the combined Gaussian-Lanczos kernel outperforms Gaussian-only (FID 2.10) or Lanczos-only (FID 2.08) versions (FID 1.95, all at what appears to be the 80-epoch ablation setting), and that coarse-to-fine ordering of the decoder's scale injection is important: reversing it to fine-to-coarse costs 0.35 FID (1.95 to 2.30).
Limitations
The paper states one explicit limitation: 'Our experiments focus on natural images; extending the proposed scale-aware framework to specialized domains such as medical imaging is a promising direction for future work.' No other limitations about computational cost, failure cases, or missing guidance/step reporting are discussed in the fetched text.
Why it matters
It reframes the object of reparameterization: instead of only changing what quantity the network predicts (as JiT does), it changes what the network is supervised against at each noise level, arguing this is a genuinely different and complementary axis to prediction-target reparameterization. If the coarse-to-fine supervision principle generalizes, it offers a training-efficiency lever that composes with, rather than competes against, the prediction-target fixes already established in this line.
Abstract, in the authors' own words
Pixel-space diffusion models directly model image distributions but remain difficult to optimize. Recent methods alleviate this challenge through target reparameterization, while still relying on a fixed clean-image target throughout denoising. Through empirical analysis, we identify a scale-time mismatch: image structures become predictable from coarse to fine as noise decreases, whereas existing models are forced to predict the full image even under high noise, resulting in low-SNR gradients that hinder optimization. To resolve this mismatch, we propose Observation Operator Diffusion, a unified framework that aligns both the supervision trajectory and feature refinement with the intrinsic recovery order of image structures. Specifically, we replace fixed full-image supervision along the standard flow path with a time-indexed observation trajectory that evolves from coarse structures to the full image during denoising. This trajectory is instantiated with a family of Gaussian-Lanczos operators at varying observation scales, yielding a path-consistent training objective. We further introduce GL-CoDA, a decoder that injects scale-specific Gaussian-Lanczos observations across decoding stages for coarse-to-fine feature refinement. Extensive experiments show that the proposed approach converges substantially faster while consistently improving generation quality, achieving an FID of 1.52 on ImageNet-256.
Research line
Builds on
Nothing recorded yet.
Built on by
Nothing recorded yet.
This is a emerging entry — a short-form summary. Full paper-page explanations (before/problem/ core idea/representation/architecture/objective/conditioning/training/inference/results/ ablations/limitations) are written for landmark and core papers first; see PROJECT_STATE.md for the schedule.