nemotron-h-abi-e2e.md names dense_attn::AttnBlock as NemotronH's device seam, but that block applies RoPE and reads rms_norm_eps, which resolves to 0.0 on this checkpoint

[localai-bot]

Claim

.agents/specs/nemotron-h-abi-e2e.md §2 names dense_attn::AttnBlock as the device attention seam for NemotronH. It cannot serve this architecture, and routing it there would reproduce the defect #810 just removed from the runner.

Three measurements, all at origin/main:

1. It is Qwen3-shaped and config-driven.

// include/vllm/model_executor/models/dense_attn_block.h:335
inline DBuf AttnBlock(Dev d, const Qwen3DenseAttnWeights& w, const HfConfig& cfg, ...)

Its own comment says "One Qwen3 dense self-attention block (qwen3.py::Qwen3Attention.forward)". Geometry comes off HfConfig.

2. It applies RoPE on the default path.

// dense_attn_block.h:496 — the DEFAULT branch
vt::RopeNeox(d.q, q3, k3, si.positions.t(), MakeRopeArgs(cfg));

NemotronH has no RoPE at all. models/nemotron_h.py:473-486 @ pin 555967922 is qkv → split → attn → o_proj with no rotation step, and the file contains zero occurrences of rope/rotary. Our own nemotron_h_forward.h records this as kNemotronHAttentionHasNoRope, and llama.cpp corroborates it independently with LLAMA_ROPE_TYPE_NONE (llama-model.cpp:2410-2412).

The released config.json does ship rope_theta: 10000 and partial_rotary_factor: 1.0 — both inert for this architecture, which is exactly what makes this trap quiet: the fields are present, so a config-driven block will happily use them.

3. It reads an eps field this checkpoint does not ship.

// src/vllm/transformers_utils/hf_config.cpp:551
cfg.rms_norm_eps = GetDouble(text, "rms_norm_eps", 0.0);

Nemotron-3.5-Lightning's config ships layer_norm_epsilon: 1e-05 and norm_eps: 1e-05 — no rms_norm_eps. So the field resolves to 0.0, silently.

Why this matters beyond one model

This is the same shape as #810: a shared function reconstructing behaviour from HF-config fields the model does not ship. #810's fix was to read from the model's own published spec instead. Routing NemotronH's attention through a config-driven Qwen3 block would reintroduce the pattern one layer up, and two of its three failure modes are silent — an eps of 0.0 and an applied-but-inert rotation both produce numbers, not errors.

Note also that a wrongly-applied RoPE may not move a token on a short prompt, so a token gate would not catch it. The spec's own §6b already records this: applying rope_theta/partial_rotary_factor "would be numerically plausible, would not change a single tensor SHAPE, and on a short prompt might not move a token".

What the tree already does instead

Model-local attention blocks are the established idiom, not an exception — none of these is allowlisted:

src/vllm/model_executor/models/granite.cpp:84    GraniteAttnBlock
src/vllm/model_executor/models/gemma.cpp:42
src/vllm/model_executor/models/gemma2.cpp:123
src/vllm/model_executor/models/gemma3.cpp:108
src/vllm/model_executor/models/glm4.cpp:80
src/vllm/model_executor/models/gemma4.cpp:206

Each documents its deltas from the dense block. A NemotronHAttnBlock taking its geometry from NemotronHParams — no RoPE, no qk-norm, eps from the field the checkpoint actually ships — is that idiom, not a parallel path.

What done looks like

1. Correct the spec's §2 seam claim, with this evidence, so no later implementer is sent the same way. The spec is authoritative but not infallible; this is the clause working as intended.
2. Whatever implements NemotronH's device attention writes a model-local block in the Granite/Gemma idiom and documents its deltas.
3. Consider whether hf_config.cpp:551 defaulting rms_norm_eps to 0.0 rather than refusing is right in general. A silent 0.0 epsilon is a numerically plausible wrong answer, and this is the second row it has been noticed on.

Found while scoping A2α of #810.