core
Seedream 4.0: Toward Next-generation Multimodal Image Generation
Team Seedream · ByteDance · 2025-09 · arXiv:2509.20427
Why it matters
Folds text-to-image, editing, and multi-image composition into one system with native 4K output and dense text rendering. Representative of the industry-wide collapse of 'generation' and 'editing' into a single model rather than two pipelines.
What this paper does
read: full textBefore this
Text-to-image systems such as Stable Diffusion and the FLUX series, and the instruction-editing tools built on top of them, existed as separate pipelines, and diffusion transformers still relied on VAE designs whose token counts limited how efficiently a model could scale to high resolution.
The problem
How to fold text-to-image synthesis, instruction-based editing, and multi-image composition into one framework that natively generates images from 1K up to 4K resolution without the inference cost such unification usually adds.
The idea
Pair an efficient diffusion transformer with a high-compression VAE that sharply cuts the number of latent tokens, then post-train one model jointly on text-to-image and editing data with a fine-tuned vision-language model handling task routing and prompt rewriting in front of it.
How it works
The paper describes an efficient, scalable DiT backbone paired with a VAE built for a high compression ratio that reduces image-token count considerably, but it does not disclose the exact compression ratio or the specific DiT architectural changes, so neither the space of the edit nor its exact dimensionality can be pinned down from the text. A separate VLM, built on Seed1.5-VL and used only as an input-side "PE model," reads the prompt and any reference images and produces captions, task-routing decisions, and aspect-ratio estimates that are then fed to the DiT; it sits in front of generation rather than acting as a preservation mechanism inside it. Joint capability across text-to-image and editing comes from a three-stage multimodal post-training recipe, continued training for instruction following, supervised fine-tuning for reference-to-edit consistency, and RLHF for human preference alignment. Critically, the paper gives no architectural description of how unedited regions are protected during an edit, such as masking, attention injection, or explicit latent blending. It states only that the SFT stage "improves the consistency between the reference and edited images considerably," which indicates consistency is a learned property of the joint training data rather than an enforced constraint.
Evidence
Ranked first on both the text-to-image and image-editing leaderboards of the Artificial Analysis arena as of September 18, 2025. On the paper's own MagicBench 4.0 human evaluation, 325 text-to-image prompts, 300 single-image edit prompts, and 100 multi-image edit prompts, it outperforms comparators by close to 20 percent on the combined GSB metric for multi-image editing. On the automatic DreamEval benchmark, 128 sub-tasks and 1,600 prompts across three difficulty tiers, performance holds at the easy and medium tiers but drops at the hard tier, most visibly in single-image editing. Generating a 2K image takes up to 1.4 seconds without the VLM routing stage (the abstract's headline figure of 1.8 seconds does not match this number stated in the paper body), and the paper reports more than a 10x inference acceleration versus Seedream 3.0.
Limitations
The authors concede that hard-tier prompts, particularly single-image edits, still expose weak multimodal understanding and reasoning, and state this gap is meant to close by scaling the model and its data further. As an API-only system with no released weights or code, the paper also withholds the VAE compression ratio and the specific DiT modifications, and it describes no explicit mechanism for protecting unedited regions, so how well spatial consistency holds outside the reported benchmarks cannot be verified from the paper itself.
Why it matters
It is a concrete industry data point for the shift from separate generation and editing pipelines to one jointly trained model, showing that unification does not have to cost inference speed, while the hard-tier regression shows that collapsing the two tasks into one model has not yet closed the gap on compositionally demanding edits.
Abstract, in the authors' own words
We introduce Seedream 4.0, an efficient and high-performance multimodal image generation system that unifies text-to-image (T2I) synthesis, image editing, and multi-image composition within a single framework. We develop a highly efficient diffusion transformer with a powerful VAE which also can reduce the number of image tokens considerably. This allows for efficient training of our model, and enables it to fast generate native high-resolution images (e.g., 1K-4K). Seedream 4.0 is pretrained on billions of text-image pairs spanning diverse taxonomies and knowledge-centric concepts. Comprehensive data collection across hundreds of vertical scenarios, coupled with optimized strategies, ensures stable and large-scale training, with strong generalization. By incorporating a carefully fine-tuned VLM model, we perform multi-modal post-training for training both T2I and image editing tasks jointly. For inference acceleration, we integrate adversarial distillation, distribution matching, and quantization, as well as speculative decoding. It achieves an inference time of up to 1.8 seconds for generating a 2K image (without a LLM/VLM as PE model). Comprehensive evaluations reveal that Seedream 4.0 can achieve state-of-the-art results on both T2I and multimodal image editing. In particular, it demonstrates exceptional multimodal capabilities in complex tasks, including precise image editing and in-context reasoning, and also allows for multi-image reference, and can generate multiple output images. This extends traditional T2I systems into an more interactive and multidimensional creative tool, pushing the boundary of generative AI for both creativity and professional applications. We further scale our model and data as Seedream 4.5. Seedream 4.0 and Seedream 4.5 are accessible on Volcano Engine https://www.volcengine.com/experience/ark?launch=seedream.
Research lines
Method note — the shared flow-matching interpolation
Every flow-matching / rectified-flow paper in this atlas trains toward a straight-line path between a noise sample x₀ and a data sample x₁:
Builds on
Nothing recorded yet.
Built on by
Nothing recorded yet.