core

Step1X-Edit: A Practical Framework for General Image Editing

Shiyu Liu, Yucheng Han, Peng Xing · StepFun · 2025-04 · arXiv:2504.17761

Why it matters

An open instruction-editing system pairing a multimodal LLM that parses the instruction with a DiT that executes it, and the source of GEdit-Bench — the real-world editing benchmark most 2025-26 editing papers now report.

What this paper does

read: full text

Before this

Proprietary models such as GPT-4o and Gemini2 Flash showed strong instruction-based editing. Open-source efforts including OmniGen and ACE++ still fell short in generalization, edit accuracy, and image quality, and existing instruction-editing datasets such as AnyEdit and OmniEdit lacked quality and diversity.

The problem

Close the gap between open and closed-source instruction-based image editing, by building both a training data pipeline at scale and an open architecture capable of using it.

The idea

Let a multimodal LLM read the instruction together with the reference image and turn that understanding into conditioning for a diffusion transformer that paints the edit, and build the data pipeline needed to train that system at scale.

How it works

Qwen-VL (Qwen2.5-VL) processes the reference image and the instruction. Its output token embeddings, after discarding prefix tokens, pass through a lightweight token-refiner connector and substitute the T5 text embedding normally fed to the downstream DiT, FLUX. The mean of Qwen's output embeddings is separately projected into a global visual guidance vector. The edit happens in VAE latent space, meaning the target image is VAE-encoded, Gaussian noise is added to it, and its tokens are concatenated along the token dimension with the VAE-encoded reference-image tokens into one doubled-length sequence the DiT processes jointly. The fetched text describes no mechanism for protecting unedited regions, no mask, no attention replacement, no preserved KV cache, no latent blending, and no locality loss. Preservation of the untouched parts of the image is left entirely to the MLLM's semantic guidance and the DiT's learned behavior, not to any architectural constraint.

Evidence

Trained on more than 11 million filtered instruction-image triplets retained from over 20 million generated across 11 editing categories, built with Florence-2 annotation, SAM-2 segmentation, and Flux-Fill inpainting among other tools. On the paper's own GEdit-Bench (606 real-world instructions, 11 categories, English and Chinese), scored with VIEScore SC/PQ/O by GPT-4.1, Step1X-Edit reaches English O=6.701 against GPT-4o's 7.534 and Gemini2 Flash's 6.315, and beats the open-source baseline OmniGen's O=5.061 by a wide margin. On GEdit-Bench-CN, O=6.861 against GPT-4o's 7.302. A 55-participant user study gives Step1X-Edit user-preference scores of 6.544 and 6.939 on the intersection and full subsets, close to GPT-4o's 6.961 and 7.134.

Limitations

The paper has no dedicated limitations section and does not discuss the absence of a region-preservation mechanism as a design tradeoff. Its own numbers show it only approaches, not matches, proprietary systems, since GPT-4o leads on every GEdit-Bench axis reported.

Why it matters

It established the working recipe, an MLLM driving a DiT edit executor, and the GEdit-Bench benchmark that a large share of 2025-26 open instruction-editing papers now build on and report against.

Abstract, in the authors' own words

In recent years, image editing models have witnessed remarkable and rapid development. The recent unveiling of cutting-edge multimodal models such as GPT-4o and Gemini2 Flash has introduced highly promising image editing capabilities. These models demonstrate an impressive aptitude for fulfilling a vast majority of user-driven editing requirements, marking a significant advancement in the field of image manipulation. However, there is still a large gap between the open-source algorithm with these closed-source models. Thus, in this paper, we aim to release a state-of-the-art image editing model, called Step1X-Edit, which can provide comparable performance against the closed-source models like GPT-4o and Gemini2 Flash. More specifically, we adopt the Multimodal LLM to process the reference image and the user's editing instruction. A latent embedding has been extracted and integrated with a diffusion image decoder to obtain the target image. To train the model, we build a data generation pipeline to produce a high-quality dataset. For evaluation, we develop the GEdit-Bench, a novel benchmark rooted in real-world user instructions. Experimental results on GEdit-Bench demonstrate that Step1X-Edit outperforms existing open-source baselines by a substantial margin and approaches the performance of leading proprietary models, thereby making significant contributions to the field of image editing.

Research lines

Editing in the VAE latentdominant In-context editingdominant

Method note — the shared flow-matching interpolation

Every flow-matching / rectified-flow paper in this atlas trains toward a straight-line path between a noise sample x₀ and a data sample x₁:

x_t = (1-t)\,x_0 + t\,x_1, \qquad v_\theta(x_t, t) \approx x_1 - x_0

Builds on

Nothing recorded yet.

Built on by

Nothing recorded yet.