An absolute information floor for softmax heads, with the sharp constant 1/8

A softmax-to-softmax Kullback-Leibler divergence is a centred log-moment-generating function: KL(softmax z || softmax a) = log (sum_c softmax(z)_c e^(a_c - z_c)) - sum_c softmax(z)_c (a_c - z_c) (klDivₛoftmaxₑq), together with the partition-ratio identity sum_c softmax(z)_c e^(a_c - z_c) = (sum_c e^(a_c)) / (sum_c e^(z_c)) it rests on (sumₛoftmaxₘulₑxp)

Gibbs' inequality for the finite Kullback-Leibler sum: 0 <= sum_c p_c log(p_c / q_c) for strictly positive probability vectors (klDivₙonneg)

Topsoe's compensation identity for a finite mixture: sumᵢ pᵢ KL(qᵢ || Q) = Iₖ + KL(qbar || Q) for every strictly positive reference distribution Q (sumₖlDivₑq); hence Iₖ <= sumᵢ pᵢ KL(qᵢ || Q) for every Q, i.e. the realized conditional information is a MINIMUM over reference distributions (condInfoₗeᵣef)

Hoeffding's lemma on a finite weighted space: for a weight vector w on Fin m and v with values in [lo, hi], log (sum_c w_c e^(v_c)) <= sum_c w_c v_c + (hi - lo)² / 8 (logₛumₑxpₗe)

KL(softmax z || softmax a) <= (max_c (a_c - z_c) - min_c (a_c - z_c))² / 8, for every number of classes, with no dependence on the marginal and none on the biases (klDivₛoftmaxₗe)

THE SOURCE'S CONJECTURE (arXiv:2607.09487v1, conj:generalfloor, 'Information floor, general K'), PROVED, WITH AN ABSOLUTE CONSTANT. For every m, every real inner-product space, every finite context distribution, arbitrary residual rows delta_c and arbitrary per-class biases b_c: Iₖ <= (R_Delta² / 8) Dₖ with R_Delta = max_(c,c') ||delta_c - delta_c'|| (info_floor); the same bound holds with Dₖ replaced by the dispersion about ANY reference point (info_floorᵣef), and in the stronger envelope form Iₖ <= (1/8) E[max_(c,c') <delta_c - delta_c', x - h>]² (info_floorₘaster). In the source's own normalisation max_c ||delta_c|| <= R this reads Dₖ >= 2 Iₖ / R² (info_floorₙorm, disp_geₒfₙormₗe), i.e. c(m, qbar) = 2 – independent of m, of qbar, of the ambient dimension and of the biases

THE COMPONENT-LEVEL FLOOR, the statement the source's Remark rem:floor calls 'the natural next theorem'. For any linear map P with <delta_c - delta_c', y> = <delta_c - delta_c', P y> for all c, c' and all y – the orthogonal projection onto spandelta_c - delta_c' being the motivating case – Iₖ <= (R_Delta² / 8) sumᵢ pᵢ ||P(hᵢ - x)||² (info_floorₚroj). The floor loads only on the component of the dispersion lying in the span of the residual-row differences; the remaining trace is free

The binary margin form, which is the quantity the source measures directly: for m = 2 and u = delta₀ - delta₁, Iₖ <= (1/8) sumᵢ pᵢ <u, hᵢ - x>², i.e. Var(u^T h) >= 8 Iₖ (info_floorₘargin)

The bias-variance identity for the dispersion in an inner-product space: sumᵢ pᵢ ||hᵢ - x||² = Dₖ + ||E h - x||² (dispAtₑq), hence the barycentre minimises the dispersion (dispₗe_dispAt)

SHARPNESS OF THE CONSTANT 1/8. An explicit configuration – d = 1, m = 2, two contexts at +- log(5/4) of weight 1/2 each, residual rows (1, 0) so that R_Delta = 1 exactly, zero biases, conditionals exactly (5/9, 4/9) and (4/9, 5/9) with uniform marginal – attains Iₖ > (1/8 - 1/1000) R_Delta² Dₖ (sharp_witness). So the constant 1/8 of row IF6 cannot be replaced by anything below 31/250 = 0.124: it is optimal to within 0.8%. The same configuration has strictly positive information (sharp_witnessₚos), so row IF6 is not vacuous

Negative controls. (i) The constant cannot be lowered: the claim Iₖ <= (31/250) R_Delta² Dₖ is FALSE at the witness configuration (notᵢnfo_floorₛmaller). (ii) The diameter hypothesis cannot be dropped: residual rows (2, 0) have diameter 2, and using R_Delta = 1 for them makes the floor false by a factor of about four (diameterₕypothesisₙeeded)

Negative control (no reverse floor): with coincident residual rows the realized conditional information is exactly 0 while the dispersion is strictly positive (noᵣeverse_floor), so no inequality Dₖ <= C Iₖ can hold; and since the same Dₖ supports two configurations with different Iₖ, the R_Delta factor in row IF6 is doing real work

m-INDEPENDENCE, REFUTING THE SOURCE'S PREDICTED SHAPE. The balanced split delta = (1, 1, 0, 0) at m = 4, with the same contexts and the same diameter, has EXACTLY the same realized conditional information as the m = 2 witness (condInfo4ₑq_condInfoW) and hence attains the same ratio (sharp_witness_four). The source predicts c(m, qbar) = Theta(min_c qbar_c / m) = Theta(1/m²), which would make the attainable constant at m = 4 about a quarter of the one at m = 2

The source's binary constant never exceeds 4: 16 qbar (1 - qbar) <= 4 for every qbar (source_constantₗe_four), so the sharp margin floor Var(u^T h) >= 8 I of row IF8 implies the source's proved Var(u^T h) >= 16 qbar (1 - qbar) I with at least a factor 2 to spare at every qbar, and unboundedly as qbar -> 0 or 1 (margin_dominatesₛource)

THE ERRATUM. The source reports 'the median margin variance is 21x its floor' against its own floor 16 qbar (1 - qbar) I. Measured against the sharp floor 8 I of row IF8, a pair reported at rho x its floor is at most (rho/2) x the sharp floor (erratumₘedianᵣatio), so the reported 21x becomes AT MOST 10.5x (erratumₐtₜwentyone)