posfloor

Based's second-order Taylor attention weight has an exact positivity floor of 1/2: a key can never be suppressed below half its unnormalized weight, where softmax's weight goes to zero. Order two is the minimal positivity-preserving Taylor truncation and the one with the largest floor, and that floor forces a selectivity tax: to single out one key against N others, Based needs a query-key alignment growing like sqrt(N) where softmax needs only log(N).

Based (Arora et al. 2024) approximates the softmax score exp(t), t = q . k, by its second-order Taylor sum, realized exactly by the feature map phi(x) = [1, x, (x outer x)/sqrt(2)] so that phi(q) . phi(k) = 1 + t + t^2/2.

The 1/2 positivity floor

1 + t + t^2/2 is a parabola with vertex at t = -1 and minimum value exactly 1/2 (its discriminant 1 - 2 is negative, so it has no real root). Every Based weight satisfies w(t) >= 1/2 for all real t, tight at q . k = -1, whereas exp(t) has infimum 0. A Based key can never be pushed below weight 1/2.

Order two is not arbitrary. A Taylor partial sum of exp is globally positive if and only if its order is even; among the even truncations, order two carries the largest floor (global minima 0.5 at order 2, 0.27 at order 4, 0.15 at order 6). Based's order two is the minimal positivity-preserving truncation, with the strongest anti-suppression floor.

The sqrt(N) selectivity tax

Every non-target key contributes at least the floor 1/2 to the Based denominator. So to give a target key normalized weight w against N - 1 maximally suppressed distractors, the target's unnormalized weight must reach w (N - 1) (1/2)/(1 - w), and inverting the parabola gives the required alignment

t_target = -1 + sqrt(2 (N - 1) w / (1 - w) - 1) = Theta(sqrt(N)) .

Softmax, whose distractors can be driven toward zero, needs only log((N - 1) w/(1 - w)) = Theta(log N). Measured for w = 1/2, the required Based alignment is 6.9, 14.9, 31.0, 127 at N = 64, 256, 1024, 16384 (tracking sqrt(N)), versus softmax's 4.1, 5.5, 6.9, 9.7, a gap growing from 1.7x to 13x and widening without bound. Placing a Based target at the predicted alignment reaches weight exactly 0.5 in a direct simulation. The fidelity ratio exp(t)/(1 + t + t^2/2) is above 1 for t > 0 and below 1 for t < 0, so Based under-weights true matches and over-weights distractors, the same anti-selectivity the floor enforces.

Layout

• floor.py: the 1/2 positivity floor, the even-order positivity with order two's largest floor, and the fidelity envelope.
• selectivity.py: the sqrt(N) required-alignment law versus softmax's log(N), and the end-to-end weight check.
• test_posfloor.py: the feature-map identity, the floor, the even/odd truncation minima, the fidelity sign, the sqrt(N) scaling, and the achieved target weight.

Reproduce

python floor.py
python selectivity.py
python test_posfloor.py

Note

The Based paper motivates the Taylor feature map for recall and throughput and acknowledges its approximation error, but states neither the exact 1/2 floor nor the selectivity cost. That the even Taylor partials of exp are globally positive is classical (Szego), and this repository does not claim it. Deliberately not claimed either is the feature-Gram recall capacity (d + 1)(d + 2)/2: that is the textbook dimension of the degree-two polynomial kernel and is already used by symmetry-aware Taylor-attention work, so it is not part of the contribution. What is here is the exact 1/2 weight floor tied to Based, its identity as the maximal-margin minimal positive truncation, and the resulting exact sqrt(N) versus log(N) selectivity tax with its measured 13x gap and end-to-end certificate. The named extension is the same law under the normalized denominator with a distribution of distractor alignments, and a trained-model measurement of how close real Based heads run to the floor.