fix(engine): honor session max_seq_len; stop decode no-op past ctx 4096

[geisten]

Problem

A geist_session created with opts.max_seq_len larger than the model-load default (4096) silently degenerated once context crossed position 4096: geist_session_decode_step returned GEIST_OK at ~0.54 ms/tok while the forward pass never actually ran.

Repro (before this change, on a Metal-enabled build):

GEIST_BENCH_PP=4096 GEIST_BENCH_TG=16 ./bin/.../bench_session_throughput
# decode 0.54 ms/tok  (vs ~50 ms/tok at pp3072)

Root cause

Three stacked failures:

1. transformer_session_alloc read m_max, kv-mode, and sampler opts but never max_seq_len. The state-level RoPE cos/sin tables and per-session KV caches stayed sized to st->max_seq_len, set from the model-load default (4096) at state-create time.
2. No bounds check anywhere — GEIST_E_TOO_MANY_TOKENS was in the enum but never raised. Positions >= max_seq_len indexed past the RoPE tables and memcpy'd past the KV buffers.
3. ops->prefill returns void and op_decode_step returns a -1 sentinel the engine ignored, so the failure was swallowed and reported as GEIST_OK.

Fix

• Grow-on-session-create (arch_state.c): a session requesting a larger window bumps st->max_seq_len and rebuilds the shared RoPE tables (grow-only; allocate_runtime_rope is now re-runnable) before sizing its KV caches. Multi-session safe — bigger tables serve smaller windows, and each session records its own capacity in the new sess->max_seq_len.
• Enforcement (forward.h): transformer_check_kv_room() guards all four KV-appending paths (text prefill, audio prefill, verify_forward, decode/audio-advance) → GEIST_E_TOO_MANY_TOKENS instead of out-of-bounds writes.
• Error surfacing (session.c): prefill failure detected via kv_len shortfall; decode -1 now returns the backend error instead of GEIST_OK.
• Bench (bench_session_throughput.c): sizes its window from PP+TG and honors GEIST_BENCH_BACKEND (was hardcoded cpu_neon).

Verification (cpu_neon + cpu_scalar — this is engine-level, not Metal-specific)

• pp4096+tg16 decode 0.54 → 64.4 ms/tok; pp4096+tg64 and pp6144+tg16 both complete at sane rates.
• Token parity 32/32 between cpu_neon and cpu_scalar at positions 4080–4111, crossing the old cliff.
• Oversized prefill now rejected up-front — self-check added to test_session_lifecycle_int.
• make format-check clean; make test-unit green (22 passed, 0 failed); existing test_multi_session_int / test_known_answer_e2e / test_pin_prefix_int green.

Known ceiling

KIVI KV mode's fixed 64 KB scores arena caps max_seq_len at 16384 (fails cleanly; marked with a ponytail: comment and upgrade path). FP32/INT8 KV modes are unaffected.

🤖 Generated with Claude Code