Abstract
<title>Abstract</title> <p>The interpretability of machine learning models is a critical concern as AI systems are deployed in sensitive domains such as religious discourse classification. This study replicates and extends a trustworthiness evaluation experiment on the Religion subset of the 20 Newsgroups dataset (alt.atheism vs. soc.religion.christian), comprising 1,310 training and 328 test instances. Four classifiers-Logistic Regression with L2 regularisation (LR), Support Vector Machine with RBF kernel (SVM), Random Forest (RF), and K-Nearest Neighbours (K-NN)-were trained using binary bag-of-words features (98,022 dimensions). Three model-agnostic explanation methods-LIME, Parzen windows, and a Greedy approach-were evaluated against a Random baseline in a controlled data-trusting experiment with 25% untrustworthy features across five rounds. In this experimental setting, LIME achieved the highest trustworthiness-identification performance for RF, LR, and SVM (F1 = 0.993-1.000). We also analyse why Greedy attains high precision but very low recall (0.06–0.07), and we report stability trends under repeated runs with larger evaluation samples. In addition, we examine LIME on image classification (InceptionV3 and ResNet18) through a focused superpixel-based case study. Overall, the findings indicate that LIME is a strong practical option for this dataset and protocol, while the evidence remains bounded by the binary text setting, sparse BOW representation, and limited image scope.</p>