Abstract
<title>Abstract</title> <p>Background Biomedical Knowledge Graphs (bioKGs) are increasingly used to integrate heterogeneous biomedical data, including genomic interactions, disease associations, and drug–target relationships. Property graph databases, such as Neo4j, provide an effective framework for representing and querying these data through the Cypher query language. However, writing Cypher queries requires substantial technical expertise, limiting the accessibility of bioKGs to domain experts. Although Large Language Models (LLMs) have recently shown promise for translating natural language (NL) questions into Cypher queries, progress in the biomedical domain remains constrained by the limited availability of representative benchmarks and reproducible evaluation frameworks. Results We present bio2C, a FAIR-compliant biomedical benchmark comprising 2,500 curated NL-Cypher pairs covering diverse biomedical information needs, graph schemas, and progressively more expressive Cypher constructs. Building upon this resource, we introduce a modular Retrieval-Augmented Generation framework that systematically combines graph schema descriptions, retrieved NL-Cypher examples, and query execution outputs, thereby establishing reproducible baselines for future Text2Cypher research. We evaluate the resulting pipelines using both proprietary (GPT) and open-source (LLaMA) LLMs together with their fine-tuned counterparts. Experimental results show that retrieving representative NL-Cypher examples consistently outperforms schema-based prompting, with particularly pronounced improvements for fine-tuned models. The best-performing configuration improves execution accuracy by 31.6 percentage points over current schema-based prompting strategies. Conclusions This work provides two complementary resources for biomed-ical Text2Cypher research: a high-quality FAIR benchmark and a modular evaluation framework for systematically studying retrieval-augmented prompting strategies. Together, they establish reproducible baselines for developing and comparing future Text2Cypher systems while demonstrating that carefully curated benchmarks and representative retrieval examples substantially improve Cypher generation over bioKGs. The benchmark, source code, and evaluated pipelines are publicly available through Zenodo and GitHub.</p>