ssdcond

Mamba-2's SSD token-mixing matrix is the inverse of a bidiagonal shift, so it is exactly invertible in one O(L) pass and its condition number is bounded by (1 + a)/(1 - a) independently of sequence length. Among the sequence-mixing family it is the member that is simultaneously full-rank and uniformly well-conditioned: softmax matrices are singular and the undecayed causal accumulator has condition number growing as 4L/pi.

Mamba-2's state-space duality writes a scalar-gated SSM as y = M x with a lower-triangular 1-semiseparable mixing matrix. For a constant scalar gate a in [0, 1), M_ij = a^{i-j} for i >= j.

M is the inverse of a bidiagonal shift

M = (I - a S)^{-1} exactly, with S the first lower shift (verified, max |M(I - a S) - I| = 8e-17). The inverse operator T = I - a S is bidiagonal, so:

• The SSD mixing inverts in one first-order recurrence x_i = y_i - a_i y_{i-1}, in 2N - 1 operations, no matrix solve.
• T is near-Toeplitz with symbol 1 - a e^{i theta}, singular values in [1 - a, 1 + a], so M = T^{-1} has singular values in [1/(1 + a), 1/(1 - a)].

Conditioning is length-independent

As the length grows,

sigma_max(M) -> 1/(1 - a),   sigma_min(M) -> 1/(1 + a),   cond_2(M) -> (1 + a)/(1 - a),

with no dependence on L. Measured: cond_2 is flat in length and lands on the limit (a = 0.5 -> 3.00, a = 0.9 -> 19.00, a = 0.99 -> 199), and the singular extremes reach 1/(1 - a) and 1/(1 + a) to four digits, approaching from inside the band so finite length is strictly better conditioned than the limit.

The contrast is the point. The undecayed accumulator a = 1, the linear-attention regime, is full-rank but has cond_2 ~ 4L/pi (measured 41.3, 326.6, 2608 at N = 32, 256, 2048, matching 4N/pi to 0.16%), growing without bound. A softmax attention matrix is row-stochastic and generically singular. The SSD decayed matrix is the family member that is both full-rank and uniformly well-conditioned, and the bound diverges as a -> 1, matching the a = 1 blow-up smoothly.

The systems consequence: exact O(L) inversion at any precision

Because the inverse operator is bidiagonal and well-conditioned, the round-trip inversion does not accumulate error with length. In fp64 the relative error is machine epsilon and flat, 1.16e-16 at L = 10^6. In fp32 it is 1.1e-7 and in bf16 7.4e-3, both larger but still flat in L (identical at L = 10^4 and 10^5), set by the operator conditioning rather than the sequence length. So an SSD layer can be reconstructed exactly from its output at arbitrary context length in one linear pass, at a fixed precision cost that does not grow with the sequence, which the ill-conditioned undecayed accumulator and the singular softmax matrix cannot offer.

Layout

• conditioning.py: the exact M = (I - a S)^{-1} identity, the length-independent cond_2 -> (1 + a)/(1 - a), the singular-extreme band, and the a = 1 contrast.
• inversion.py: the exact O(L) inverse recurrence and its fp64/fp32/bf16 round-trip error, machine epsilon and flat in length.
• test_ssdcond.py: the inverse identity, the flat-in-length conditioning, the limit, the singular band, the 4L/pi undecayed growth, and the exact and low-precision inversion.

Reproduce

python conditioning.py
python inversion.py
python test_ssdcond.py

Note

The spectral core here is classical: a bidiagonal Toeplitz matrix has singular values in [1 - a, 1 + a] (Kac-Murdock-Szego, Avram-Parter). This repository does not claim that. What is here is the state-space-duality reading and its measured consequences: identifying the SSD mixing matrix as (I - a S)^{-1} exactly, the length-independent cond_2 -> (1 + a)/(1 - a) as a statement about SSD versus softmax and versus the undecayed accumulator, the exact O(L) invertibility, and the length-independent inversion error down to bf16 measured out to L = 10^6. The result is for the scalar-gate (state-dim-1) SSD; the multi-state case M = L . (C B^T) is d-semiseparable and its joint conditioning bound is the named extension. It is distinct from the undecayed full-rank statement (the a = 1 endpoint here, which is degenerate with cond = 4L/pi): this is a conditioning and invertibility law in the decayed a < 1 regime.