The sink channel is built up through depth, then torn down by the final MLP

Qwen2.5 residual streams carry a single massive-activation channel, the attention-sink coordinate, whose magnitude dwarfs every other channel. This measures its life-cycle in residual-write coordinates: each block's write w = block_output - block_input (attention plus MLP) builds that channel up through the body of the network, and then the final block tears it back down toward zero. Because that one channel dominates the residual norm, its teardown is what makes the residual norm appear to collapse at the output, the late norm drop seen in massiveact and erracc. The teardown is done by the MLP, not attention.

Method and oracle

For each block, capture its output and its attention (o_proj) and MLP (down_proj) writes at the last token, on Qwen2.5-0.5B/1.5B/3B. Identify the massive channel as the largest-magnitude coordinate of a mid-network residual. Measure the write direction cos(w, block_input) both on the full residual and with the massive channel removed from both vectors, the residual norm change with and without that channel, and the channel's own value and the MLP and attention contributions to it. Oracle (additivity): the write equals attention plus MLP output to fp32 round-off (max 1e-5), so the residual path is exactly the sum of the two sublayers. fp64 inner products, eight prompts, CPU. (A tempting second "oracle," that the norm change equals 1 + ratio^2 + 2*ratio*cos, is not reported as a check: that is the law of cosines and holds for any vectors by definition; it is a coordinate identity, not a validation.)

Result: the collapse is one channel, torn down by the MLP

model	final cos (full)	final cos (no massive ch)	final norm x (full)	final norm x (no ch)	massive ch in -> out	MLP write / attn write to ch
0.5B	-0.60	-0.20	0.83	1.27	63 -> 13	-42 / +3
1.5B	-0.50	-0.39	0.87	0.96	-182 -> -110	+49 / -4
3B	-0.49	-0.32	0.85	0.98	-285 -> -204	+105 / -6

Two facts. First, the massive channel is torn down at the final block: its magnitude drops sharply (63 to 13, 182 to 110, 285 to 204), and that teardown is the MLP writing against the channel's own sign (a negative write on the positive 0.5B channel, positive writes on the negative 1.5B and 3B channels) while attention barely touches it. Second, that single channel accounts for the norm collapse. With the channel removed, the final block's effect on the norm flips from a 13-17% drop to roughly flat or growing: 0.5B goes from x0.83 to x1.27 (the rest of the residual is reinforced, not erased), and 1.5B and 3B go from x0.87 and x0.85 to x0.96 and x0.98. The "the last block collapses the residual norm" picture is really "the last MLP tears down the one dominant channel."

The final-block write direction stays somewhat anti-aligned even without the massive channel (cos -0.20 for 0.5B, -0.39 and -0.32 for 1.5B and 3B), so a weaker distributed anti-alignment exists beyond the one channel in the larger models; but the norm collapse itself is essentially all the massive channel.

Pre-registration scorecard (PREREG.md, committed before results)

	prediction	outcome
P1	cos rises through the body (blocks reinforce)	partly. The mid-body write is positively aligned, but that alignment is substantially the massive channel growing; a distributed reinforcement remains only weakly. Reframed as the channel build-up
P2	final block erases, collapsing the norm	reframed. The norm collapse is real (x0.83 to 0.87) but is the massive channel's teardown, not a whole-residual erase; removing that channel, the final block does not collapse the norm (x0.96 to 1.27)
P3	the erase is MLP-driven	holds cleanly. The MLP writes against the massive channel (opposing its sign) while attention does not, all three sizes
P4	pattern holds across sizes	holds for the channel build-up and MLP teardown; the "collapse is entirely one channel" is strongest at 0.5B and partial at 1.5B/3B

Scope

Qwen2.5-Instruct 0.5B/1.5B/3B, last-token residual writes, fp32, CPU. The massive channel is the sink coordinate that massiveact characterizes; this study measures its trajectory and the sublayer that removes it, so its relationship to massiveact's sink-decay and erracc's late norm drop is the same phenomenon in write coordinates, not an independent mechanism (those studies are separate repos). The additivity oracle is exact; the norm identity is definitional and is not claimed as a validation.

Reproduce

python run.py cpu    # per-block write geometry, massive-channel decomposition, MLP/attn split -> results/main.json

License

MIT.