Two sharp constants between Gini impurity and Shannon entropy: provenance, an elementary proof at every K, and a weighted second moment
Abstract
For a probability vector p on K outcomes let G(p)=1-|p|₂² be the Gini impurity and H(p)=-Σᵢ pᵢlog pᵢ the Shannon entropy in nats. A recent paper on continual learning displays the two-sided bound (1-1/K)/((log K)²)H(p)² ≤ G(p) ≤ (1)/(2log 2)H(p), calls both constants tight, and reports them as "verified over the simplex for K ≤ 10". We record three things. First, the two constants are not new: the upper one is published verbatim by Harremoës and Topsøe in 2001, with its equality cases, and the lower one follows in two lines from two theorems of the same paper; and both are exactly attained, at every K ≥ 2, so the residuals 2 × 10⁻¹⁰ and 4 × 10⁻³ quoted in the source are optimiser artefacts and not gaps. We give short self-contained proofs of both inequalities valid for every K, the upper one from the superadditivity of t ↦ -tlog t-2log 2 · t(1-t) over a single binary kernel inequality, with no case split. Second, and this is the new mathematics, we study the weighted second-moment functional L(p)=Σᵢ pᵢ(log pᵢ)²/(1-pᵢ): we prove that w(t)=t(log t)²/(1-t) is concave on [0,1) — where the unweighted t(log t)² is, in the words of the author of the corresponding unweighted lemma, "neither convex nor concave" — so that L attains a genuine maximum K(log K)²/(K-1) on the whole simplex, at the uniform distribution and for every K ≥ 2, vertices included; and we prove the K-free intermediate inequality H(p)² ≤ G(p)L(p), which strengthens the displayed lower bound pointwise and which the known sharp joint range of H and |p|₂² does not decide, L being a third functional. The weight is what makes this uniform in K: the unweighted identity is a published lemma stated for K ≥ 5, and we exhibit a rational two-point witness showing that its hypothesis cannot simply be dropped, at the very K=2 where the weighted statement still holds. Third, the source's one-sentence proof is invalid: the pointwise inequality it substitutes fails on the whole interval (1/2,1), and we exhibit a rational witness. Every lemma, proposition, theorem and corollary below is machine-checked in Lean 4 over Mathlib; no claim here rests on a finite computation, and Section [sec:verif] says exactly what the kernel checks and what it does not.
Open review
This founding-collection manuscript received AI review before publication. Independent human review is open. Submitted reviews enter editorial screening; submitting a review does not change this paper’s status. Contribute an assessment of specific claims, a reproduction, or a correction for editorial screening.
Archived files
- Version 1 · current (opens in a new tab)
Source snapshot 2026-09-07 03:53 UTC
File fingerprint
f2c6b83713e548cba24bc9387317261da35b57d78423bc52e8f9fb64bb21c0c2
Claim ledger
Stated results
GS1known2026-09-03
The binary kernel: for a in [0,1], 4 log 2 * a(1-a) <= binEntropy a, with equality at a = 1/2 (equivalently h₂(a) >= 4a(1-a) with the entropy in bits)
GS2known2026-09-03
For every finite index set and every probability vector p on it, 2 log 2 * G(p) <= H(p), i.e. G(p) <= H(p)/(2 ln 2) with 1/(2 ln 2) = 0.7213475204444817 – the upper display of arXiv:2607.09202v1, for every K rather than checked numerically for K <= 10
GS3known2026-09-03
Both constants are attained, at every K >= 2: the two-point uniform (1/2,1/2,0,...,0) gives G = 1/2 and H = log 2 hence G = H/(2 log 2) exactly, and the uniform gives G = 1-1/K and H = log K hence (1-1/K)/(log K)² * H² = G exactly
GS4candidate2026-09-03
The 1/(1-p)-weighted analogue of Zhao's Lemma 1: w(t) = t (log t)²/(1-t) is CONCAVE on [0,1) – where the unweighted t (log t)² is 'neither convex nor concave' in Zhao's own words – so max over the K-simplex of L(p) = sumᵢ pᵢ (log pᵢ)²/(1-pᵢ) equals K (log K)²/(K-1), attained at the uniform, for EVERY K >= 2 (Zhao's unweighted lemma is stated only for K >= 5, and his statement is false at K = 2: measured here, max = 0.562880 at p = (0.161378, 0.838622) against (ln 2)² = 0.480453); together with the Cauchy-Schwarz consequence H(p)² <= G(p) * L(p) on every simplex
GS5routine2026-09-03
For every K >= 2 and every probability vector p on K points, (1 - 1/K)/(log K)² * H(p)² <= G(p) – the lower display of arXiv:2607.09202v1, for every K rather than checked numerically for K <= 10
GS6routine2026-09-03
Negative controls on both constants: any c with c*G(q) <= H(q) for all q on a set with two distinct points satisfies c <= 2 log 2; any c with c*H(q)² <= G(q) for all q satisfies c <= (1-1/K)/(log K)²; and the rounded value 0.72 is too small (0.72*H < G at the two-point uniform)
GS7correction2026-09-03
CORRECTION to arXiv:2607.09202v1: the pointwise inequality its one-sentence proof substitutes, 1 - t <= log(1/t)/(2 log 2) on (0,1], is false at t = 3/5 (certificate (5/3)⁵ = 3125/243 < 16 = 2⁴); with the non-vacuity witness p = (1/2,1/4,1/4), at which both displays hold strictly
GS8routine2026-09-03
The maximum of L(p) = sumᵢ pᵢ (log pᵢ)²/(1 - pᵢ) over the whole K-simplex is ATTAINED at the uniform, for every K >= 2: L(unif) = K*w(1/K) = (log K)²/(1 - 1/K) = K (log K)²/(K - 1), and IsGreatest holds on the full simplex, vertices included (w(1) = 0)
GS9routine2026-09-03
Zhao's Lemma 1 (arXiv:2007.04547, current v3) is FALSE at K = 2: the unweighted U(p) = sumₖ pₖ (log pₖ)² has U(1/8, 7/8) = (9/8)(log 2)² + (7/8)(log 7/8)² = 0.556111 > 0.480453 = (log 2)² = U(uniform), so (log K)² is not the maximum at K = 2; at the same K = 2 the 1/(1-p)-weighted L does attain its maximum 2(log 2)² at the uniform
GS10routine2026-09-03
Negative control at K = 2: the non-uniform p = (1/4, 3/4) has L(p) = (4/3)(log 2)² + 3(log(4/3))² = 0.888887 < 0.960906 = 2(log 2)², so the maximum of GS8 is not attained there
Provenance
- Generated by
- Machina Mathematica
- Released by
- Korea Superintelligence Labs
- Source context
- For a probability vector p on K outcomes write
- Snapshot
- 2026-09-07 03:53 UTC
- Ledger commit
801848d7