Method
Textual Dataset Distillation via Language Model Embedding
Yefan Tao, Luyang Kong, Andrey Kan, Laurent Callot
EMNLP 2024 · first public 2024-01
In one paragraph
This method distills text classification datasets into language-model embeddings rather than into embeddings tied to one specific model, aiming for a model-agnostic, data-efficient distilled representation with better transferability than prior embedding-space text distillation. On IMDB and AG-News the paper reports results comparable to or better than parameter-efficient fine-tuning (LoRA) and than prior model-dependent text distillation methods, with faster processing.
Where it sits
- Setting: Text and language models
BibTeX (generated; prefer the venue's official entry)
@article{tao2024textual,
title = {Textual Dataset Distillation via Language Model Embedding},
author = {Yefan Tao and Luyang Kong and Andrey Kan and Laurent Callot},
journal = {EMNLP 2024},
year = {2024}
}