Generative Vision Atlas

landmark

Flow Matching for Generative Modeling

Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, Matt Le · Meta AI, Weizmann Institute · 2022-10 · arXiv:2210.02747

Why it matters

Introduces a simulation-free objective for continuous normalizing flows: regress a vector field that generates any fixed conditional probability path between noise and data, with the straight-line (optimal transport) path as a simple special case that trains faster and samples in fewer steps than diffusion.

What this paper does

read: full text

Before this

Continuous normalizing flows were trained by maximum likelihood, which required simulating the flow's ODE during training and did not scale to large models or datasets. Score-based diffusion models offered simulation-free training but only for a restricted family of Gaussian diffusion probability paths, such as variance-preserving or variance-exploding paths, not for probability paths in general.

The problem

Train continuous normalizing flows in a simulation-free way for a much broader family of probability paths between a noise distribution and the data distribution than the paths diffusion models use, and determine whether some other choice of path trains faster or samples more efficiently than the diffusion paths already in use.

The idea

Define a simple, per-example conditional probability path with a known vector field, and regress a network directly onto that conditional vector field rather than the intractable marginal one. The paper proves the conditional and marginal objectives have identical gradients, and shows that a straight-line optimal-transport path between noise and data is a valid and efficient special case.

How it works

This trains fully unconditional, class-conditional models directly on ImageNet pixels, with no latent encoding, using a U-Net architecture adapted from Dhariwal and Nichol. The network regresses a velocity vector field, not noise and not clean data, that generates the chosen conditional probability path. For the optimal-transport path the conditional field has a closed form, the displacement from the current point toward the data point scaled by a factor that goes to zero as the path reaches the data, giving straight-line, constant-speed trajectories rather than the curved, backtracking trajectories diffusion paths produce.

Evidence

No classifier or classifier-free guidance is used anywhere in this paper, since the models are unconditional or class-conditional ImageNet generators, so these numbers are guidance-free by construction. On ImageNet 32 by 32 the optimal-transport variant reaches 3.53 bits per dimension and FID 5.25. On ImageNet 64 by 64 it reaches 3.31 bits per dimension and FID 14.00. On ImageNet 128 by 128 it reaches FID 21.52. Training is also more efficient, the ImageNet 64 model needs 157000 iterations against 540000 for the diffusion baseline at matched throughput, and the ImageNet 128 model needs 475000 iterations against 4.36 million. At sampling time the optimal-transport path needs fewer function evaluations to reach a matched error threshold, for example 156 against a diffusion score-matching baseline's 232 on CIFAR-10, and 162 against 436 on ImageNet 64.

Limitations

The authors note explicitly that the conditional path being optimal transport does not imply the resulting marginal vector field is itself an optimal-transport solution. Their theoretical guarantees also depend on regularity assumptions, such as the data density staying bounded away from zero and decaying fast enough at infinity, that are convenient rather than verified properties of real image distributions.

Why it matters

This reframed diffusion training as one special case of a much larger, simulation-free family, and showed that regressing a velocity field along a straight path trains faster and samples in fewer steps than the curved diffusion paths used before it. Predicting a velocity or drift field rather than noise or clean data is now a distinct training-objective choice this atlas tracks, and the straight-path formulation here is the direct ancestor of the rectified-flow objectives used in later production text-to-image systems.

Abstract, in the authors' own words

We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths. Flow Matching is compatible with a general family of Gaussian probability paths for transforming between noise and data samples -- which subsumes existing diffusion paths as specific instances. Interestingly, we find that employing FM with diffusion paths results in a more robust and stable alternative for training diffusion models. Furthermore, Flow Matching opens the door to training CNFs with other, non-diffusion probability paths. An instance of particular interest is using Optimal Transport (OT) displacement interpolation to define the conditional probability paths. These paths are more efficient than diffusion paths, provide faster training and sampling, and result in better generalization. Training CNFs using Flow Matching on ImageNet leads to consistently better performance than alternative diffusion-based methods in terms of both likelihood and sample quality, and allows fast and reliable sample generation using off-the-shelf numerical ODE solvers.

Research line

Flow matching and rectified flowdominant

Design-axis choices

Objective

Flow matching / rectified flow

Method note — the shared flow-matching interpolation

Every flow-matching / rectified-flow paper in this atlas trains toward a straight-line path between a noise sample x₀ and a data sample x₁:

xt=(1−t) x0+t x1,vθ(xt,t)≈x1−x0x_t = (1-t)\,x_0 + t\,x_1, \qquad v_\theta(x_t, t) \approx x_1 - x_0

Builds on

Nothing recorded yet.

Built on by

Challenges / competes with