Signed Layer-Level Evidence in Transformer Predictions, and a Diagnostic for When It Applies

<title>Abstract</title> <p>Two tokens can carry the same softmax probability yet be reached very di erently: one backed by near-unanimous layer agreement, the other surviving strong internal opposition. No output-level score separates them, because every such score is a function of the logit gap alone and is blind to evidence mass. We introduce Language Evidence Flow (LEF), which splits each token prediction into signed per-layer evidence in a single forward pass and, in retrieval-augmented generation, sepa rates evidence coming from parametric memory from evidence coming from retrieved documents. We prove an identi cation boundary characterising exactly when output-level scores cannot re veal internal con ict. Across eleven models and six architecture families (1.4B14.7B), LEF detects RAG source disagreement on four of six families, but con ict as an error detector proves model-speci c. That gap motivates ScopeGate, a permutation test asking whether a score is predictive of error on a given model task pair. ScopeGate is detector-agnostic, so we ran it across detectors, scoring choices and models: eleven of thirty-six combinations fail. On Mistral-7B-Instruct no detector predicts error, predictive entropy included, and for con ict the rst-token and mean token verdicts disagree on three of nine models. Validity depends on the model, the detector and the scoring protocol together, and one AUROC number hides all three. We report the least favourable comparison plainly: entropy outperforms con ict on single hop factual QA in nine of eleven con gurations. ScopeGate is calibrated, needs roughly 100 labelled examples, and is released as a standalone module</p>