Trustworthy DD · Meta-learning through unrolled training
Towards Robust Dataset Learning
Yihan Wu, Xinda Li, Florian Kerschbaum, Heng Huang, Hongyang Zhang
arXiv 2022 · first public 2022-11-19 · arXiv 2211.10752
In one paragraph
Poses learning a dataset such that any classifier naturally trained on it is adversarially robust as a tri-level optimization problem, avoiding the cost of adversarial training on the downstream task and showing the robustness property transfers between models and data. Proves the method learns a provably robust dataset under an abstraction that separates robust from non-robust features, and reports effectiveness across different network initializations and architectures on MNIST, CIFAR-10 and Tiny-ImageNet.
Where it sits
- Meta-learning through unrolled training (Bi-level performance matching)
- Setting: Image classification
Abstract (verbatim from arXiv)
Adversarial training has been actively studied in recent computer vision research to improve the robustness of models. However, due to the huge computational cost of generating adversarial samples, adversarial training methods are often slow. In this paper, we study the problem of learning a robust dataset such that any classifier naturally trained on the dataset is adversarially robust. Such a dataset benefits the downstream tasks as natural training is much faster than adversarial training, and demonstrates that the desired property of robustness is transferable between models and data. In this work, we propose a principled, tri-level optimization to formulate the robust dataset learning problem. We show that, under an abstraction model that characterizes robust vs. non-robust features, the proposed method provably learns a robust dataset. Extensive experiments on MNIST, CIFAR10, and TinyImageNet demostrate the effectiveness of our algorithm with different network initializations and architectures.
BibTeX (generated; prefer the venue's official entry)
@article{wu2022towards,
title = {Towards Robust Dataset Learning},
author = {Yihan Wu and Xinda Li and Florian Kerschbaum and Heng Huang and Hongyang Zhang},
journal = {arXiv preprint arXiv:2211.10752},
year = {2022}
}Nearby in Meta-learning through unrolled training
Shiye Lei, Sen Zhang, Dacheng Tao · NeurIPS 2024notableOther datapaper ↗code ↗