Trustworthy DD
TrustDD
Towards Trustworthy Dataset Distillation
Shijie Ma, Fei Zhu, Zhen Cheng, Xu-Yao Zhang
PR 2024 · first public 2023-07-18 · arXiv 2307.09165
In one paragraph
Observes that dataset distillation has only ever targeted closed-world in-distribution classification, ignoring out-of-distribution detection, and proposes Trustworthy Dataset Distillation (TrustDD), which distills both in-distribution samples and outliers so the condensed set trains models competent at both tasks; Pseudo-Outlier Exposure corrupts in-distribution samples to stand in for real outlier data when none is available. Reports POE surpasses the state-of-the-art Outlier Exposure method and that TrustDD generalizes better to open-world scenarios than preceding DD.
Where it sits
- Setting: Image classification
Abstract (verbatim from arXiv)
Efficiency and trustworthiness are two eternal pursuits when applying deep learning in real-world applications. With regard to efficiency, dataset distillation (DD) endeavors to reduce training costs by distilling the large dataset into a tiny synthetic dataset. However, existing methods merely concentrate on in-distribution (InD) classification in a closed-world setting, disregarding out-of-distribution (OOD) samples. On the other hand, OOD detection aims to enhance models' trustworthiness, which is always inefficiently achieved in full-data settings. For the first time, we simultaneously consider both issues and propose a novel paradigm called Trustworthy Dataset Distillation (TrustDD). By distilling both InD samples and outliers, the condensed datasets are capable of training models competent in both InD classification and OOD detection. To alleviate the requirement of real outlier data, we further propose to corrupt InD samples to generate pseudo-outliers, namely Pseudo-Outlier Exposure (POE). Comprehensive experiments on various settings demonstrate the effectiveness of TrustDD, and POE surpasses the state-of-the-art method Outlier Exposure (OE). Compared with the preceding DD, TrustDD is more trustworthy and applicable to open-world scenarios. Our code is available at https://github.com/mashijie1028/TrustDD
BibTeX (generated; prefer the venue's official entry)
@article{ma2023towards,
title = {Towards Trustworthy Dataset Distillation},
author = {Shijie Ma and Fei Zhu and Zhen Cheng and Xu-Yao Zhang},
journal = {PR 2024},
year = {2023}
}