Abstract
<title>Abstract</title> <p>The manifold hypothesis—that high-dimensional data concentrate near a low-dimensional, smoothly varying manifold—has become an organizing metaphor for interpreting neural representations, and recent work reports rich geometric and even specific low-dimensional manifolds inside single-cell foundation models. We ask whether the internal representations of protein language models (pLMs) admit analogous manifolds. Applying a broad detection battery—intrinsic dimension estimators (TwoNN, MLE, PCA participation ratio), nonlinear dimensionality reduction (Isomap, LLE, diffusion maps, UMAP), persistent homology, trustworthiness and geodesic-connectivity diagnostics, and confound controls—to ESM-2 (8M–3B) and to ProtBERT and ProtT5, we find little evidence for such structure. The intrinsic dimension of pLM representations is high in absolute terms (best-layer TwoNN ≈63 for ESM-2 650M), grows with model size, and does not stabilize across neighborhood scales, contradicting a single global manifold. Nonlinear manifold coordinates never outperform a linear probe on the raw embeddings for secondary structure, remote homology, stability, or localization, and ∼64–128 linear dimensions are needed to saturate performance. Persistent homology recovers no robust loops beyond the noise floor, and the apparent clustering of protein families largely dissolves after regressing out sequence length, amino-acid composition, and homology (family silhouette 0.31→0.05). Crucially, the same battery cleanly detects manifolds in positive controls (a Swiss roll, a single-cell cell-cycle loop, a torus), so the negative result is not a failure of method. We conclude that pLM representations are better described as high-dimensional and predominantly linear than as low-dimensional manifolds, and we discuss why the discrete, homologystructured geometry of protein sequence space differs from the continuous developmental manifolds seen in single-cell data.</p>