energygate

Griffin's RG-LRU is the unique energy-orthonormal leaky integrator: its sqrt(1 - a^2) input scale makes the impulse response unit energy at every gate, so it preserves variance on white input, while the natural convex gate leaks a fixed fraction per step and collapses in the long-memory regime. The price of that white-noise orthonormality is a DC gain that diverges: RG-LRU amplifies slow or correlated input by exactly (1 + a)/(1 - a).

The RG-LRU recurrence (De et al. 2024) with a constant per-channel gate a in (0, 1) and white input is h_t = a h_{t-1} + sqrt(1 - a^2) u_t.

Energy orthonormality and the convex-gate collapse

The impulse response h_n = sqrt(1 - a^2) a^n has squared L2 energy (1 - a^2) sum_n a^{2n} = 1 for every a, and this fixes the input scale: g^2/(1 - a^2) = 1 has the unique nonnegative root g = sqrt(1 - a^2). So RG-LRU is energy-orthonormal, and its stationary variance under unit white input is exactly 1 (measured 1.000000 at every gate).

The natural baseline, the convex gate h_t = a h_{t-1} + (1 - a) u_t, is not. From equal unit variances one step gives Var = a^2 + (1 - a)^2 = 1 - 2 a (1 - a), an exact per-step leak of 2 a (1 - a), and its stationary variance is exactly (1 - a)/(1 + a) (measured 0.053, 0.005, 0.0005 at a = 0.9, 0.99, 0.999), which collapses to 0 as a -> 1, exactly where the gate is meant to hold a long memory.

The DC-gain tradeoff

Unit impulse energy fixes only the response to white input. The response to a constant input is sqrt(1 - a^2) sum_n a^n = sqrt(1 - a^2)/(1 - a), an amplitude gain of sqrt((1 + a)/(1 - a)), so the DC power gain is

(1 + a)/(1 - a) ,

which diverges as a -> 1 (measured power gain 19, 199, 1999 at a = 0.9, 0.99, 0.999). The convex gate, whose weights sum to 1, has DC gain exactly 1. So RG-LRU is variance-preserving on white input and variance-amplifying on slow input, trading DC neutrality for white-noise orthonormality. The two gates share the same pole a and the same memory time-constant; the difference is entirely in gain and energy.

Layout

• variance.py: the unit-energy impulse response, the uniqueness of the sqrt(1 - a^2) scale, the RG-LRU unit stationary variance, and the convex gate's 2 a (1 - a) shrink and (1 - a)/(1 + a) collapse.
• dcgain.py: the RG-LRU (1 + a)/(1 - a) DC power gain and the convex gate's DC neutrality.
• test_energygate.py: the impulse energy, the scale uniqueness, the one-step variance, the stationary-variance simulation, and the DC-gain tradeoff.

Reproduce

python variance.py
python dcgain.py
python test_energygate.py

Note

The Griffin paper motivates the sqrt(1 - a^2) input scale as keeping the recurrence's variance controlled, a one-line design rationale. The elementary facts, that a geometric sum gives 1/(1 - a^2) energy and 1/(1 - a) DC gain, are standard. What is here is the exact reading of the choice: sqrt(1 - a^2) is the unique input scale making the impulse response unit energy, so variance preservation holds exactly and only for white input; the convex-gate baseline leaks exactly 2 a (1 - a) per step to a (1 - a)/(1 + a) collapse; and the cost of white-noise orthonormality is a DC power gain (1 + a)/(1 - a) that diverges as the gate approaches one, so RG-LRU amplifies slow or correlated input. This last tradeoff is the consequence the paper does not state. It is exact real analysis on the variance and energy, distinct from the shipped gatebias (a bf16 precision result on a decay gate) and from the discretization and overflow ships. The named extension is the input-dependent gate a_t = exp(-c softplus(Lambda) r_t), where the white-input preservation holds per channel conditional on the gate path and the DC amplification becomes a bound in E[a].