Research line · ascendant
Agglomerative multi-teacher distillation
Distil several specialist foundation models into one backbone rather than choosing between their objectives.
What defines membership
No single pretraining objective is best at everything, so the right move is to inherit several at once through distillation.
How the line developed
Read top to bottom: what came before, the idea itself, the evidence for it, what improved, and where it breaks.
The idea
AM-RADIO · 2023-12core
Distils CLIP, DINOv2 and SAM into one student backbone.
Evidence
Improvement
RADIOv2.5 · 2024-12strong-followup
Fixes resolution and mode-switching failures in the first recipe.
At scale
C-RADIOv4 · 2026-01emerging
Now distils DINOv3, SigLIP 2 and SAM 3, the same three encoders the generative-latent work independently converged on.
What it gets right
- Sidesteps the self-distillation versus contrastive argument by taking both
- One backbone to deploy instead of several specialists
- Its choice of teachers is a useful read on what the field currently rates
Where it is weak
- A student is bounded by its teachers and inherits their blind spots
- Not yet evaluated as a generative latent, which is the use this atlas cares about most
- Distillation adds a training stage and a set of hyperparameters of its own