GC-Bench: An Open and Unified Benchmark for Graph Condensation

What came before

Individual graph condensation papers (GCond, DosCond, SGDD, GCDM/DM, SFGC, GEOM, KiDD, Mirage) each report results under their own protocol and condensation-ratio choices, and a concurrent benchmark (GCondenser) covers only node-level homogeneous-graph methods with limited evaluation dimensions, leaving no unified, cross-family comparison of effectiveness, transferability and efficiency.

The problem

Reported progress cannot be trusted at face value because methods differ in condensation-ratio ranges, backbones, and evaluated datasets, and several dimensions relevant to real deployment — behavior under a much broader ratio sweep, transfer to non-node-classification tasks, transfer across backbone architectures (including graph transformers), sensitivity to initialization, and actual memory/time cost — are not reported by the original papers at all.

The idea

Build one open library and evaluation harness that runs 12 representative methods across 6 mechanism categories (coreset, gradient matching, trajectory matching, distribution matching, kernel ridge regression, computation-tree compression) on 12 datasets (5 homogeneous + 2 heterogeneous node-level, 5 graph-level) and both node- and graph-level tasks, under a shared protocol, and use it to answer six specific research questions about effectiveness, structure-sensitivity, task transfer, architecture transfer, initialization, and efficiency rather than only headline accuracy.

How it works

No new condensation objective is proposed. GC-Bench standardizes datasets/splits, backbones and evaluation protocol, then re-runs each method (using authors' released code/hyperparameters where possible) across a wider range of condensation ratios than the original papers used, records node- and graph-classification accuracy, and additionally measures: transfer of a node-classification-condensed graph to link prediction / node clustering / anomaly detection; transfer across 5 GNN backbones plus MLP and Graph Transformer when condensation used a different backbone; sensitivity to 5 initialization schemes (random noise, random sample, center, k-center, k-means); and wall-clock condensation time plus peak CPU/GPU memory to reach best validation performance.

Evidence

Table 2 (node classification, various GCNs/backbones, hard/method-native labels, broadened ratio sweep): GC methods reach lossless or super-original accuracy in 5/7 datasets at some ratio (e.g. Citeseer r=0.18%, ACM r=0.003%, DBLP r=0.002%), and distribution-matching methods (GCDM, DM) underperform gradient- and trajectory-matching methods on 6/7 datasets. Critically, at the largest ratios tested (e.g. ogbn-arxiv r=5.00%, Flickr r=5.00%, DBLP r=0.186%, Reddit r=5.00%), GCDM, DM, DosCond, GCond and SGDD all hit out-of-memory (OOM) errors, while SFGC and (mostly) GEOM survive — directly contradicting the impression from individual papers that these methods scale gracefully to larger budgets; GEOM itself OOMs on Reddit r=5.00% and r=1.00% despite reporting a "lossless" ogbn-arxiv r=5% result in its own paper. Table 3 (graph classification, GIN): KiDD (kernel ridge regression) wins 18/25 cases but depends entirely on structure rather than the backbone, so its results collapse when the downstream model is switched from GIN to GCN — the paper's key cross-model ablation for that method. RQ3 (Fig. 2): all condensed datasets lose substantial accuracy when transferred from node classification to link prediction, clustering, or anomaly detection, with gradient-matching methods (GCond, DosCond, SGDD) transferring better than the structure-free trajectory-matching methods (SFGC, GEOM). RQ4 (Fig. 3): graphs condensed with SGC as backbone transfer well across architectures, but graphs condensed with Graph Transformer transfer poorly in both directions; increasing GCond's gradient-matching steps reduces cross-architecture transfer, implying more matching steps encode more backbone-specific bias. RQ5: initialization choice (e.g. Center vs. Random Noise on Cora) changes convergence speed by up to 71% but has little effect on final accuracy.

Limitations

Evaluates only node classification and graph classification as condensation objectives (explicitly named as the study's own scope limit); Mirage cannot directly hit the paper's target condensation ratios and had to be matched to DosCond's disk usage instead; some KRR/CTC configurations return no result due to search recursion limits ("-" entries); and, like any benchmark, its conclusions are only as current as the 12 methods and hyperparameter settings it reproduced (e.g. no diffusion- or generative-prior graph condensation method is included).