Abstract
<title>Abstract</title> <p>Expert di erentiation and collapse in Mixture-of-Experts (MoE) is governed by how gradi ent signals couple across experts during training. We introduce Γ — the cross-expert gradient alignment — as a diagnostic metric and close a complete causal chain through 7 experiments on Mixtral 8×7B (47B parameters, 32 layers, softmax top-2 routing). Γ is de ned as the mean pairwise cosine similarity of router-logit gradients across all expert pairs: Γ > 0 (turbulent — synergistic reinforcement), Γ ≈ 0 (laminar — independent learning), Γ < 0 (negative-aligned — zero-sum competition). Native Mixtral yields Γ = −0.107 across all 32 layers, establishing that MoE training under standard con gurations is fundamentally zero-sum expert competition. Three perturbation experiments (LoRA continual learning, gate unfreezing, noise sweep 0100%) all fail to cross Γ through zero, demonstrating the negative-aligned state is a struc tural attractor independent of speci c weight values. A 2×2 factorial decomposition (soft max/sigmoid × top-k normalization on/o ) reveals three opposing forces: architectural posi tive alignment from the shared FFN (+0.030), softmax denominator coupling ∂σi/∂zj = −σiσj (−0.044), and top-k zero-sum normalization Σw = 1 (−0.124), summing to the net Γ = −0.107. Only the simultaneous removal of both softmax and top-k normalization allows Γ to cross zero (+0.030). Top-k ablation (k = 1,2,3,4,6,8) reveals an inverted-U relationship: Γ = 0.000 at k = 1 (perfectly laminar, no inter-expert competition), maximally negative at k = 2 (−0.107), then progressively weaker as the competition pool dilutes (k = 8: −0.045). Tile MosaicModel validation con rms the picture: softmax routing yields Γ = −0.031 (softmax coupling only, no top-k), while hard routing with equal weights achieves Γ = −0.009 (near-laminar). We formalize the Competitive Routing Lemma: any routing mechanism that selects k ≥ 2 experts and normalizes their combination weights necessarily produces Γ < 0, with the strength of negative alignment following an inverted-U relationship with the normalization pool size k. The three-force adversarial model provides a uni ed gradient-alignment account of MoE training dynamics.</p>