Hyperparameter Analysis in Retrieval-Augmented Generation Systems Applied to the Public Prosecutor’s Office of the State of Espírito Santo Corpus

<title>Abstract</title> <p>This research paper proposes a methodology based on Design of Experiments to analyze and opti-mize hyperparameters of a Retrieval-Augmented Generation system applied to the normative corpusof the Public Prosecutor’s Office of the State of Espírito Santo. The central objective is to identifywhich critical factors—such as document segmentation strategy (chunking), retrieved context volume(k), and Large Language Model choice—significantly impact generated response quality. Evaluationemploys the LLM-as-a-Judge paradigm supported by the Retrieval Augmented Generation Assess-ment framework. Due to the non-parametric nature of the data, statistical analysis is conducted viaAligned Rank Transform. Results reveal that hyperparameters significantly impact response quality.</p>