Research line · dominant
Contrastive language-image pretraining
Learn visual features by aligning images to their captions across web-scale data.
What defines membership
Language supervision is what makes a representation semantically meaningful and directly promptable, and that is worth more than what it costs in spatial precision.
How the line developed
Read top to bottom: what came before, the idea itself, the evidence for it, what improved, and where it breaks.
The idea
CLIP · 2021-02landmark
Contrastive image-text pretraining at web scale, producing the zero-shot transfer that made vision-language models practical.
Evidence
Perception Encoder · 2025-04core
Finds the most useful embeddings sit in intermediate layers rather than at the output, which complicates the idea of reading a frozen encoder at a single point.
Improvement
SigLIP · 2023-03core
Swaps the softmax contrastive loss for a pairwise sigmoid, removing the dependence on very large batches.
SigLIP 2 · 2025-02core
Adds captioning and self-distillation objectives to the contrastive recipe, specifically to fix weak dense and localization features.
AIMv2 · 2024-11strong-followup
A third route: autoregressive prediction of patches and text together, rather than contrastive alignment.
What it gets right
- Directly promptable and language-aligned, which self-distilled features are not
- The default front end for every vision-language model
- SigLIP 2 is the frozen encoder Scale-RAE chose for text-to-image
Where it is weak
- Consistently beaten by DINOv2 as an alignment target in head-to-head ablations
- Contrastive training optimizes a global match, which historically cost dense and localization quality
- Caption quality bounds what the representation can learn