Automatic Discovery of Intra-Class Sub-Structure for Supervised Tabular Classification: Offline Clustering vs. Joint Sub-Center Training

<title>Abstract</title> <p>Class labels in real-world tabular data are often coarser than the underlying data-generating process. We revisit, with a rigorous empirical study (5 seeds per dataset), the idea of automatically discovering this hidden intra-class sub-structure to improve supervised tabular classification. First, we show that the conventional two-stage approach - cluster penultimate features per class with k-means, relabel, and retrain - is unreliable: across ten benchmarks it helps on only 2/10 datasets and degrades the rest by up to 2.8 points, with paired significance testing confirming the degradation is statistically significant on 6/10 datasets. A full sweep of this pipeline over k in {2, 3, 5, 10} shows the degradation worsens monotonically with k on 8/10 datasets; the sole exception is Letter, where accuracy improves significantly with k, peaking at k=5 (+0.57pp over base, p=0.004). Second, replacing it with a joint sub-center output layer trained end-to-end removes this failure mode almost entirely (worst degradation 0.03pp, statistically indistinguishable from the base network on 9/10 datasets) and improves accuracy where the offline method fails. The mechanism self-limits to one or two effective sub-components per class (mirroring expert collapse in mixture-of-experts). A parameter-matched, non-marginalized control shows this safety margin is not yet distinguishable from output-layer width alone (p&gt;0.05 on all 10 datasets), a null result we report transparently. We further propose an inexpensive RF-gap heuristic that predicts, across 17 datasets, whether offline clustering will help; it predicts the correct direction on 13/17 (76%), only marginally above the 12/17 (70.6%) achieved by a trivial "always skip" baseline, and is not statistically robust (r=0.37, p=0.15). Finally, we report a training-budget artifact that had spuriously suggested penultimate features beat a network's own end-to-end output. We therefore position this work as a rigorous empirical and diagnostic study, aiming to guide practitioner choices rather than introducing a new architecture.</p>