Abstract
<title>Abstract</title> <p>Gene regulatory network inference methods are commonly evaluated against a reference network whose recorded edges are treated as truth. Reference networks, however, encode different biological relations and cover different parts of the regulatory system. This study measures the practical consequence of that choice in one controlled empirical setting. We used the mouse hematopoietic stem cell erythroid-lineage data distributed with BEELINE: 1,071 cells, the 300 most variable genes, 20 transcription factors, and 5,980 directed transcription-factor-to-gene candidate edges. Six fixed edge-scoring rules were evaluated without refitting or changing the candidate universe. Only the reference labels changed: a cell-type-specific ChIP-seq network, a non-specific ChIP-seq network, or a STRING-derived network. The references marked 580, 124, and 82 candidate edges as positive, respectively. Their pairwise Jaccard indices ranged from 0.009 to 0.177. The best scoring rule was lagged correlation under the cell-type ChIP reference but Spearman correlation under both other references. Across the three reference pairs, 24 of 45 head-to-head method comparisons reversed direction (53.3%). Pairwise Kendall rank correlations ranged from −0.467 to 0.467. A transcription-factor-block bootstrap gave an overall median reversal fraction of 48.9% (95% percentile interval 22.2–62.2%). These results do not identify a universally best inference method. They show, in a narrow and reproducible case study, that a benchmark can change its conclusion when the biological meaning and coverage of the reference change. Benchmark reports should therefore treat reference choice as an experimental factor, not as fixed background.</p>