Abstract
<title>Abstract</title> <p>The increasing scale and complexity of high-performance computing (HPC) systems make performance diagnosing of large-scale parallel applications increasingly challenging. Current tools provide detailed metrics across ranks, but identifying performance bottlenecks still relies heavily on expert interpretation. Large language models (LLMs) offer the potential to automate performance interpretation, yet they have to deal with massive performance data from scale of parallel applications. In this paper, we present Semperf, an LLM assisted approach for performance diagnosis of parallel programs. Semperf constructs a rank-profile matrix to capture function-level execution characteristics across processes based on metrics. It clusters ranks to identify representative performance patterns. The selected ranks and performance features are analyzed by an LLM to generate explanations of metrics even in extreme-scale. Experiments on JEuler3D.m, BT benchmark, and JUPITER application at scales up to 102,400 processes show that Semperf can effectively reveal performance patterns and infer causes. Our results demonstrate that combining structured performance features with LLM-assisted semantic reasoning provides a scalable and interpretable approach for performance analysis in extreme-scale HPC applications</p>