Video

IDTD — Video Set Distillation: Information Diversification and Temporal Densification

Yinjie Zhao, Heng Zhao, Bihan Wen et al. · arXiv 2024Videopaper ↗

This paper introduces video set distillation, jointly reducing within-sample redundancy (across frames of one video) and inter-sample redundancy (across videos in a set), which prior key-frame selection, dataset pruning or single-video distillation methods each address only one of. Its IDTD method uses a Feature Pool and Feature Selectors to preserve diversity across samples and a Temporal Fusor to keep temporal information dense within each synthesized video, reporting state-of-the-art results in video dataset distillation.

ProtoBlend — Efficient Video Dataset Distillation via Cluster-Guided Prototype Blending

Chongle Ren, Guang Li, Wenbo Huang et al. · arXiv 2026Videopaper ↗

Builds distilled video sets without any gradient-based optimization of the stored clips: a teacher model selects one high-confidence temporal segment per source video, cluster-guided allocation partitions the selected clips per class in the teacher's feature space and assigns one distilled slot per cluster, and each slot is blended with an in-cluster anchor clip while their teacher predictions are mixed with the same coefficient. Reports a competitive accuracy-efficiency trade-off against iteratively optimized video distillation methods on four trimmed action-recognition benchmarks.

Static-dynamic video DD — Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement

Ziyu Wang, Yue Xu, Cewu Lu et al. · CVPR 2024coreVideopaper ↗code ↗

This paper gives the first systematic study of video distillation, finding that the temporal dimension of synthetic clips is usually not well learned and contributes little, and responds by disentangling static and dynamic information: a static learning stage distills videos into still images via gradient matching, and a dynamic fine-tuning stage compensates motion with a separately stored, learnable dynamic memory block rather than distilling every frame. The paper reports state-of-the-art results on video datasets at multiple scales with a notably smaller memory storage budget than prior methods.

PRISM — PRISM: Video Dataset Condensation with Progressive Refinement and Insertion for Sparse Motion

Jaehyun Choi, Jiwan Hur, Gyojin Han et al. · CVPR 2026notableVideopaper ↗

PRISM treats a condensed video as one fully coupled spatiotemporal structure rather than disentangling it into separate static and dynamic components, starting from minimal temporal anchors and progressively inserting key-frames only where linear interpolation between them fails to capture non-linear motion, guided by gradient misalignments between synthetic and real data. The paper reports competitive accuracy with state-of-the-art storage efficiency on standard video-distillation benchmarks by allocating representational capacity only where the motion needs it.

Temporal Saliency-Guided Distillation: A Scalable Framework for Distilling Video Datasets

Xulin Gu, Xinhao Zhong, Zhixing Wei et al. · arXiv 2025Videopaper ↗

Optimizes synthetic video clips directly against a single pretrained model rather than through a bi-level inner training loop, and adds a temporal-saliency-guided filtering mechanism that uses inter-frame differences to concentrate distillation on informative temporal cues while suppressing redundant frames. Reports state-of-the-art results on standard video dataset-distillation benchmarks.

ProtoBlend — Efficient Video Dataset Distillation via Cluster-Guided Prototype Blending

Chongle Ren, Guang Li, Wenbo Huang et al. · arXiv 2026Videopaper ↗

Builds distilled video sets without any gradient-based optimization of the stored clips: a teacher model selects one high-confidence temporal segment per source video, cluster-guided allocation partitions the selected clips per class in the teacher's feature space and assigns one distilled slot per cluster, and each slot is blended with an in-cluster anchor clip while their teacher predictions are mixed with the same coefficient. Reports a competitive accuracy-efficiency trade-off against iteratively optimized video distillation methods on four trimmed action-recognition benchmarks.

Condensing Action Segmentation Datasets via Generative Network Inversion

Guodong Ding, Rongyu Chen, Angela Yao · CVPR 2025notableVideopaper ↗

Condenses procedural video datasets for temporal action segmentation into compact latent codes using a generative prior learned from the dataset and network inversion, reducing storage across both the temporal and channel dimensions, plus a diversity-driven sampling step that selects representative action sequences to cut video-wise redundancy; on the Breakfast dataset it reports over 500x storage reduction while retaining 83% of full-dataset segmentation performance, and improves downstream incremental-learning performance.

Adaptive Latent Trajectory Anchoring for Action Segmentation Dataset Condensation

Artheme Gauthier-Villar, Guodong Ding, Angela Yao · ECCV 2026Videopaper ↗

Condenses action-segmentation video datasets by replacing VAE-based iterative latent optimization with deterministic DDIM latent trajectories anchored by sparse latent points on the noise manifold, and adaptively reallocates the anchoring budget across segments by per-segment reconstruction difficulty; reports performance parity with full-data training at a 2.4% condensation ratio on the Breakfast dataset.

Static-dynamic video DD — Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement

Ziyu Wang, Yue Xu, Cewu Lu et al. · CVPR 2024coreVideopaper ↗code ↗

This paper gives the first systematic study of video distillation, finding that the temporal dimension of synthetic clips is usually not well learned and contributes little, and responds by disentangling static and dynamic information: a static learning stage distills videos into still images via gradient matching, and a dynamic fine-tuning stage compensates motion with a separately stored, learnable dynamic memory block rather than distilling every frame. The paper reports state-of-the-art results on video datasets at multiple scales with a notably smaller memory storage budget than prior methods.

IDTD — Video Set Distillation: Information Diversification and Temporal Densification

Yinjie Zhao, Heng Zhao, Bihan Wen et al. · arXiv 2024Videopaper ↗

This paper introduces video set distillation, jointly reducing within-sample redundancy (across frames of one video) and inter-sample redundancy (across videos in a set), which prior key-frame selection, dataset pruning or single-video distillation methods each address only one of. Its IDTD method uses a Feature Pool and Feature Selectors to preserve diversity across samples and a Temporal Fusor to keep temporal information dense within each synthesized video, reporting state-of-the-art results in video dataset distillation.

Latent Video Dataset Distillation

Ning Li, Antai Andy Liu, Jingran Zhang et al. · CVPR 2025 WorkshopVideopaper ↗code ↗

This method stores condensed video data as codes in the latent space of a state-of-the-art variational encoder, rather than in pixel space, and picks which latents to keep with a diversity-aware selection strategy plus a training-free compression step on top. The paper reports new state-of-the-art results on all tested video-distillation benchmarks, including a 2.6-point gain on HMDB51 at IPC 1 and a 7.8-point gain on MiniUCF at IPC 5.

Temporal Saliency-Guided Distillation: A Scalable Framework for Distilling Video Datasets

Xulin Gu, Xinhao Zhong, Zhixing Wei et al. · arXiv 2025Videopaper ↗

Optimizes synthetic video clips directly against a single pretrained model rather than through a bi-level inner training loop, and adds a temporal-saliency-guided filtering mechanism that uses inter-frame differences to concentrate distillation on informative temporal cues while suppressing redundant frames. Reports state-of-the-art results on standard video dataset-distillation benchmarks.

PRISM — PRISM: Video Dataset Condensation with Progressive Refinement and Insertion for Sparse Motion

Jaehyun Choi, Jiwan Hur, Gyojin Han et al. · CVPR 2026notableVideopaper ↗

PRISM treats a condensed video as one fully coupled spatiotemporal structure rather than disentangling it into separate static and dynamic components, starting from minimal temporal anchors and progressively inserting key-frames only where linear interpolation between them fails to capture non-linear motion, guided by gradient misalignments between synthetic and real data. The paper reports competitive accuracy with state-of-the-art storage efficiency on standard video-distillation benchmarks by allocating representational capacity only where the motion needs it.

DAViD — Dynamic-Aware Video Distillation: Optimizing Temporal Resolution Based on Video Semantics

Yinjie Zhao, Heng Zhao, Bihan Wen et al. · arXiv 2025Videopaper ↗

Trains a reinforcement-learning agent, updated with a teacher-in-the-loop reward, to predict a per-class temporal resolution for synthetic video clips instead of assuming uniform temporal redundancy across classes, which the paper identifies as a limitation of prior video distillation methods. Reports substantial accuracy improvements over existing video dataset distillation methods.

SFVD — Distill Video Datasets into Images

Zhenghao Zhao, Haoxuan Wang, Kai Wang et al. · arXiv 2025Videopaper ↗

Identifies that the temporal dimension explodes the parameter count in video dataset distillation and proposes SFVD, which distills each class into a small set of highly informative single frames and reconstructs clip-length sequences from them via differentiable interpolation, restricting optimization to the frames themselves and mixing in sampled real clips through a channel-reshaping layer during matching; reports up to a 5.3-point improvement over prior methods on MiniUCF.