Research line · ascendant
Reinforcement learning and preference alignment
Fine-tune generators against reward models or human preference with policy-gradient methods, rather than only maximizing likelihood.
What defines membership
Likelihood on web data is the wrong final objective; what to optimize is a reward reflecting prompt adherence and human preference.
How the line developed
Read top to bottom: what came before, the idea itself, the evidence for it, what improved, and where it breaks.
The idea
Flow-GRPO · 2025-05core
Makes online RL work for flow models by converting the ODE to an SDE; lifts SD3.5-Medium's GenEval from 63% to 95%.
Improvement
DiffusionNFT · 2025-09core
Optimizes the forward process directly, so it works with any black-box sampler and needs only clean images.
Neon · 2025-10strong-followup
Extrapolates away from self-training degradation, turning model collapse into a usable signal.
What it gets right
- Very large gains on prompt-adherence benchmarks for modest compute
- Orthogonal to the representation and objective questions, so it composes with any line
Where it is weak
- Optimizes whatever the reward model measures, including its biases
- Reward hacking and diversity collapse are recurring failure modes
- Benchmark gains may overstate real quality gains, especially given GenEval's measured drift
Also in this line
EditScore