Analysis & theory
No Free Lunch
No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"
Nicholas Carlini, Vitaly Feldman, Milad Nasr
arXiv 2022 · first public 2022-09-29 · arXiv 2209.14987
In one paragraph
A rebuttal of Dong et al. (2022, "Privacy for Free"), arguing its empirical evaluation and theoretical analysis of dataset condensation's privacy benefit both have major flaws, so the original paper does not provide statistically significant evidence that DC improves privacy over a naive baseline. Points out that DP-SGD, the standard privacy-preserving training approach, simultaneously gives better accuracy and a provably lower membership-attack success rate than the condensation-based claims it disputes.
Where it sits
- Setting: Image classification
Builds on
Abstract (verbatim from arXiv)
New methods designed to preserve data privacy require careful scrutiny. Failure to preserve privacy is hard to detect, and yet can lead to catastrophic results when a system implementing a ``privacy-preserving'' method is attacked. A recent work selected for an Outstanding Paper Award at ICML 2022 (Dong et al., 2022) claims that dataset condensation (DC) significantly improves data privacy when training machine learning models. This claim is supported by theoretical analysis of a specific dataset condensation technique and an empirical evaluation of resistance to some existing membership inference attacks. In this note we examine the claims in the work of Dong et al. (2022) and describe major flaws in the empirical evaluation of the method and its theoretical analysis. These flaws imply that their work does not provide statistically significant evidence that DC improves the privacy of training ML models over a naive baseline. Moreover, previously published results show that DP-SGD, the standard approach to privacy preserving ML, simultaneously gives better accuracy and achieves a (provably) lower membership attack success rate.
BibTeX (generated; prefer the venue's official entry)
@article{carlini2022free,
title = {No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"},
author = {Nicholas Carlini and Vitaly Feldman and Milad Nasr},
journal = {arXiv preprint arXiv:2209.14987},
year = {2022}
}