Two-Stage Synthetic-to-Real Transfer Learning for Automated Mammography Report Generation Using Vision-Language Models

<title>Abstract</title> <p>Automated mammography report generation is critically constrained by the scarcity of publicly available paired mammogramreport datasets. While structured annotations exist in datasets such as VinDr-Mammo and CBIS-DDSM, the free-text radiologyreports required for supervised vision-language model (VLM) training are absent from nearly all public mammography resources.In this work, we propose a two-stage synthetic-to-real transfer learning framework that addresses this data scarcity problem.In Stage 1, we develop a label-to-report synthesis pipeline that converts structured BI-RADS annotations into 400 clinicallyvalidated synthetic radiology reports, which are used to fine-tune MedGemma-4B via multimodal Low-Rank Adaptation (LoRA).In Stage 2, the synthetically pretrained model is further fine-tuned on 407 real radiologist reports from the DMID dataset.Weadditionally construct a BI-RADS-grounded Retrieval-Augmented Generation (RAG) module to provide domain-specificclinical context. Evaluated on the DMID test set (n = 52) against real radiologist reports, our two-stage approach achievesBLEU-4 of 0.312, ROUGE-L of 0.671, METEOR of 0.685, CIDEr of 0.729, and BERTScore of 0.917—outperforming ninebaseline methods including zero-shot VLMs (Qwen2.5-VL-7B, Qwen3-VL-8B, LLaVA-1.6-7B), pipeline approaches (CLIP+GPT2,MedCLIP+GPT2), and direct fine-tuning without synthetic pretraining. Our approach surpasses the previously reported stateof-the-art AMRG framework across all metrics (ROUGE-L: +17.9%, METEOR: +11.4%, CIDEr: +25.3%), while reducinghallucination rates from 73.1% (zero-shot) to 21.2%. Data efficiency analysis demonstrates that synthetic pretraining reducesreal data requirements by approximately 2×, with 100 real examples sufficient to surpass the AMRG benchmark. All code,synthetic datasets, and model weights are released for reproducibility.</p>