On Learning Representations for Tabular Data Distillation

Inwon Kang, Parikshit Ram, Yi Zhou, Horst Samulowitz, Oshani Seneviratne

arXiv 2025 · first public 2025-01-23 · arXiv 2501.13905

paper ↗catalogued✓ abstract read

In one paragraph

Studies representation learning for tabular dataset distillation, where feature heterogeneity and non-differentiable learners (tree ensembles, nearest-neighbor predictors) break image-style pipelines, and proposes TDColER, a column-embedding representation-learning framework layered on top of existing distillation schemes, evaluated on TDBench, a new benchmark of 226,890 distilled tabular datasets and 548,880 downstream models. Reports TDColER improving distilled-data quality by 0.5-143% across seven tabular learning models.

Where it sits

Setting: Other data types

Abstract (verbatim from arXiv)

Dataset distillation generates a small set of information-rich instances from a large dataset, resulting in reduced storage requirements, privacy or copyright risks, and computational costs for downstream modeling, though much of the research has focused on the image data modality. We study tabular data distillation, which brings in novel challenges such as the inherent feature heterogeneity and the common use of non-differentiable learning models (such as decision tree ensembles and nearest-neighbor predictors). To mitigate these challenges, we present $\texttt{TDColER}$, a tabular data distillation framework via column embeddings-based representation learning. To evaluate this framework, we also present a tabular data distillation benchmark, ${{\sf \small TDBench}}$. Based on an elaborate evaluation on ${{\sf \small TDBench}}$, resulting in 226,890 distilled datasets and 548,880 models trained on them, we demonstrate that $\texttt{TDColER}$ is able to boost the distilled data quality of off-the-shelf distillation schemes by 0.5-143% across 7 different tabular learning models.

BibTeX (generated; prefer the venue's official entry)

@article{kang2025learning,
  title   = {On Learning Representations for Tabular Data Distillation},
  author  = {Inwon Kang and Parikshit Ram and Yi Zhou and Horst Samulowitz and Oshani Seneviratne},
  journal = {arXiv preprint arXiv:2501.13905},
  year    = {2025}
}