Analysis & theory · Meta-learning through unrolled training
On Implicit Bias in Overparameterized Bilevel Optimization
Paul Vicol, Jonathan P. Lorraine, Fabian Pedregosa, David Duvenaud, Roger B. Grosse
ICML 2022 · first public 2022-07
In one paragraph
Studies how the algorithmic choices of bi-level optimization — cold-start versus warm-start inner-loop initialization, and the hypergradient approximation used — implicitly bias which solution gradient-based bi-level methods converge to, in settings including hyperparameter optimization, meta-learning and dataset distillation, and shows warm-start solutions can retain substantial information about the outer objective even in low-dimensional problems.
Where it sits
- Meta-learning through unrolled training (Bi-level performance matching)
- Setting: Image classification
BibTeX (generated; prefer the venue's official entry)
@article{vicol2022implicit,
title = {On Implicit Bias in Overparameterized Bilevel Optimization},
author = {Paul Vicol and Jonathan P. Lorraine and Fabian Pedregosa and David Duvenaud and Roger B. Grosse},
journal = {ICML 2022},
year = {2022}
}Nearby in Meta-learning through unrolled training
2024-10
Shiye Lei, Sen Zhang, Dacheng Tao · NeurIPS 2024notableOther datapaper ↗code ↗
2023-11
2019-12