Abstract
<title>Abstract</title> <p>Background: Chronic pancreatitis (CP) is a complex, progressive inflammatory disorder characterized by irreversible damage to the pancreatic parenchyma, leading to exocrine and endocrine dysfunction. Patients with CP frequently experience debilitating symptoms including chronic pain, malabsorption, and diabetes mellitus, significantly impacting their quality of life. A critical challenge in CP management is the limited accessibility to accurate, up-to-date medical information, which is essential for informed decision-making. In recent years, advanced large language models (LLMs) such as ChatGPT-4 and DeepSeek have emerged as potential sources of medical knowledge, offering rapid responses to health-related queries. However, the reliability and clinical utility of these AI-driven platforms in the context of CP remain underexplored, particularly in comparison to established online medical resources. Aims: This study aimed to conduct a comprehensive evaluation of the performance of two leading AI models DeepSeek and ChatGPT-4 alongside conventional online medical platforms in addressing CP-related inquiries. The assessment focused on two key domains: (1) guideline-based knowledge derived from the 2020 American College of Gastroenterology (ACG) Clinical Guidelines, and (2) patient-specific concerns extracted from real-world clinical follow-up records. By comparing AI-generated responses to physician-curated information, this research sought to determine the potential role of LLMs in supporting CP patient education and clinical decision support. Methods: A rigorous dual-phase methodology was implemented to evaluate both clinical and patient-centered aspects of information delivery. In the first phase, a panel of board-certified gastroenterologists systematically assessed the AI and traditional platform responses using the validated Likert scale, with particular attention to four critical dimensions: diagnostic and therapeutic accuracy, completeness of information, comprehensiveness of coverage and presentation, and safety considerations. The second phase involved direct feedback from a cohort of CP patients, who rated the responses based on perceived empathy and overall satisfaction. This approach enabled a multidimensional comparison of information quality across different platforms. Results: The evaluation demonstrated that both DeepSeek and ChatGPT-4 achieved accuracy levels comparable to physician-generated responses, with DeepSeek showing a trend toward more complete and systematic explanations of complex CP-related concepts. Notably, the AI models provided detailed pathophysiological explanations and treatment rationales compared to conventional platforms. However, patient evaluations revealed uniformly neutral satisfaction ratings across all information sources, underscoring the inherent difficulties in communicating chronic disease management strategies effectively. This finding suggests that while AI platforms may provide factual information, significant challenges remain in meeting patients' emotional and psychological support needs. Conclusion: This study provides compelling evidence that advanced LLMs like DeepSeek and ChatGPT-4 can serve as valuable supplementary tools for CP information dissemination, offering accurate and comprehensive responses to both guideline-based and patient-specific queries. The findings highlight the models' potential to enhance patient education and support clinical decision-making, particularly in resource-limited settings. However, the neutral patient satisfaction scores emphasize the irreplaceable role of human clinicians in providing empathetic, personalized care. The research underscores the necessity of maintaining physician oversight to ensure AI-generated content adheres to current clinical guidelines while advocating for continued development of more patient-centered AI communication strategies. These insights contribute significantly to the ongoing discourse on optimal integration of artificial intelligence in chronic disease management systems.</p>