Analysis & theory · Meta-learning through unrolled training
New Properties of the Data Distillation Method When Working With Tabular Data
Dmitry Medvedev, Alexander D'yakonov
AIST 2020 · first public 2020-10-19 · arXiv 2010.09839
In one paragraph
This paper applies the original backpropagation-through-time dataset distillation method to tabular data and finds that a model trained on the distilled samples can outperform one trained on the original dataset, but that the distilled data generalizes poorly to models with different hyperparameters than the one used during distillation. It shows that distilling with multiple architectures simultaneously mitigates this poor cross-hyperparameter generalization.
Where it sits
- Meta-learning through unrolled training (Bi-level performance matching)
- Setting: Other data types
Abstract (verbatim from arXiv)
Data distillation is the problem of reducing the volume oftraining data while keeping only the necessary information. With thispaper, we deeper explore the new data distillation algorithm, previouslydesigned for image data. Our experiments with tabular data show thatthe model trained on distilled samples can outperform the model trainedon the original dataset. One of the problems of the considered algorithmis that produced data has poor generalization on models with differenthyperparameters. We show that using multiple architectures during distillation can help overcome this problem.
BibTeX (generated; prefer the venue's official entry)
@article{medvedev2020properties,
title = {New Properties of the Data Distillation Method When Working With Tabular Data},
author = {Dmitry Medvedev and Alexander D'yakonov},
journal = {AIST 2020},
year = {2020}
}Nearby in Meta-learning through unrolled training
Shiye Lei, Sen Zhang, Dacheng Tao · NeurIPS 2024notableOther datapaper ↗code ↗