HistAgent: an evidence-grounded agent for spatial molecular reasoning from routine histology

<title>Abstract</title> <p>Routine hematoxylin and eosin (H&amp;E) staining is a widely used histology test in clinical practice and biomedical research, but it does not provide direct spatial molecular information. Spatial transcriptomics (ST) provides such data while preserving the tissue context, yet its high cost, technical complexity, and limited accessibility restrict its broad application. Methods for inferring spatial molecular information from H&amp;E images could bridge this gap; however, current visual-omics algorithms often exhibit limited cross-dataset generalizability and do not sufficiently facilitate traceable biological interpretation. Here, we present HistAgent, a unified framework that couples a visual-omics foundation model with a spatial artificial intelligence (AI) module. The foundation model, trained on 2.23 million paired H&amp;E–ST spots, generates ranked molecular readouts from local and contextual H\&amp;E images. The spatial AI agentic module structures these readouts as evidence cards comprising data on the inferred cell composition, functional programs, and spatial context, thereby enabling question-driven multi-turn analysis. In tests, HistAgent recovered and interpreted literature-defined spatial findings, spanning immune, stromal, tumor, and anatomical states across tissues at lower cost and in less time than conventional ST workflows, and its molecular readouts accurately reproduced five standard ST analyses. Its intermediate visual-omics representations supported whole-slide predictions of molecular and histologic subtypes, tumor grade, and prognostic risk. Providing the module access to an evidence bank constructed from experimentally measured ST profiles enabled further natural-language and H&amp;E–image retrieval as well as an analysis of molecular states. Overall, HistAgent enables generalizable spatial molecular analyses and traceable biological interpretation from routine histology.</p>