Generative Vision Atlas

Conditioning & control option

In-context sequence concatenation

Concatenate a reference image's tokens directly onto the generation sequence and let ordinary attention mix them, with no adapter, inversion, or task-specific head.

Mechanism

A reference image is encoded through the same pathway the model already uses for the image it is generating (the same VAE encoder, the same patchify step), and its tokens are concatenated directly onto the generation sequence — no adapter network, no inversion, no task-specific head. Ordinary self-attention (already present for joint text/image attention) does the rest. OminiControl is the earliest paper this atlas has verified using this exact mechanism, for general DiT control (subject-driven generation, spatial control) at “about 0.1% additional parameters” versus a trained adapter, via “a unified sequence processing strategy that combines condition tokens with image tokens” plus a dynamic position-encoding scheme for spatially-aligned versus non-aligned tasks. FLUX.1 Kontext applies the same idea specifically to editing about seven months later and became the open reference point for it.

Trade-offs

Parameter cost: close to zero beyond the base model — this is the mechanism’s whole appeal relative to adapter-based conditioning, which needs a new trained module per condition type. Flexibility: one mechanism handles both spatially-aligned control (edges, depth) and non-aligned reference conditioning (subject-driven generation, style transfer, editing) without switching architectures. Quality/precision: ICEdit reports state-of-the-art instruction editing from roughly 0.1% of the data and 1% of the trainable parameters earlier adapter- or fine-tune-based editors needed — but preserving untouched regions of the reference exactly, and grounding a referring expression when several similar objects are present, both remain open weaknesses of this mechanism (see line-in-context-editing and semantic-latent-detail-loss).

Introduced by

Used by (3)

ICEdit, FLUX.1 Kontext, OminiControl

Alternatives on this axis

adaLN / adaLN-Zero modulation, Adapter-based spatial conditioning, CLIP text encoder as conditioning source, Cross-attention conditioning, Decoupled cross-attention (image-prompt adapter), Identity-preserving face conditioning, Joint (shared) attention conditioning, Native autoregressive token conditioning, Learnable query-token bridge, Frozen large-language-model text encoder (T5-style), Vision-language model as text encoder

← All Conditioning & control options