Back to Search View Original Cite This Article

Abstract

<jats:p>Large Language Models are increasingly used in consumer-facing mental health tools, many of which claim that prompt engineering alone can ensure safe therapeutic behavior. This study evaluates that assumption by testing 20 proprietary and open‑source LLMs on high‑risk psychiatric scenarios, using prompts grounded in behavioral therapy principles. Prompt engineering reduced some predictable risks, such as explicit endorsement of self‑harm, but consistently failed in ambiguous or clinically nuanced situations. Models frequently validated harmful statements, colluded with hallucinations, minimized symptoms, or used stigmatizing language, including in the newest and largest models. These failures reflect structural limitations such as lack of memory, insufficient contextual reasoning, and training‑related biases. Prompt engineering alone is therefore insufficient for safe AI‑mediated psychotherapy; clinician‑guided fine‑tuning, integrated safety mechanisms, and system‑level oversight will be required. This work provides early evidence motivating deeper clinician‑led evaluation and safety‑oriented model development.</jats:p>

Show More

Keywords

models prompt engineering language used

Related Articles

PORE

About

Connect