Skip to content

granitemoehybrid rotates only when it says so — position_embedding_type is the switch, not a restatement - #373

Merged
chrishayuk merged 1 commit into
mainfrom
position-policy-granite
Sep 1, 2026
Merged

granitemoehybrid rotates only when it says so — position_embedding_type is the switch, not a restatement#373
chrishayuk merged 1 commit into
mainfrom
position-policy-granite

Conversation

@chrishayuk

Copy link
Copy Markdown
Owner

The first fix of census experiment #2 — and the recorded inspection for it was wrong.

What the census said, and what the reference says

position_embedding_type: "rope" on granite-4.0-micro was classified as a checked default that "restates the scheme already in use". Checking it against transformers instead of against its name says the opposite:

position_embedding_type (`str`, *optional*):
    Positional embedding type to be used; defaults to None.
    Allowed options: `[None, "rope"]`
self.rotary_emb = GraniteMoeHybridRotaryEmbedding(config) \
                  if config.position_embedding_type == "rope" else None

The default is no positional encoding at all. "rope" is the opt-in that turns rotation on, not a restatement of anything.

rope_theta is declared either way — granite-4.0-micro ships 10000000 — so a resolver that reads the theta and rotates is right about this checkpoint by luck, and wrong about any granitemoehybrid that omits the opt-in: it would rotate every position against the model's own instruction. That is the use_sliding_window bug one key over, and it was found the same way.

Scope, and the control that bounds it

Only granitemoehybrid gates rotation on this key. granite, granitemoe and granitemoeshared construct their rotary unconditionally and never mention it, so a family-wide rule would turn every dense Granite into a NoPE model — a far larger wrong answer than the one being fixed. control_a_dense_granite_still_rotates_without_the_key is that bound, and it covers all three model_types.

An out-of-contract value takes the same path as absence, because that is what the reference does — HF compares against "rope" and sends every other string down the else branch. Not hypothetical: the BERT lineage spells absolute in this same leaf, and the same key appears in 23 model families.

Calibrated, not assumed

With the check disabled, the two adversarial arms fail with Rope { theta: 10000000.0 } on all four layers — the bug stated in its own output — while all three control arms stay green. Red first, then green.

Carriage

Represented, not Parsed. The effect lands on every layer's PositionPolicy, so probe_position_embedding_type asks the graph what it believes rather than trusting the parser to have read the key. A stack that rotates nowhere answers null — an answer, not a failure to answer, the same way probe_sliding_window reports "no window anywhere".

position_policy_for_layer's default body lifts to a free function so the override can narrow the decision and then defer to it. Rust gives an override no way to call the default it replaced, and copying the body would leave two resolvers to keep in agreement.

Census

From cached headers, no payload bytes:

ibm-granite/granite-4.0-micro    16 → 15 blocking
ibm-granite/granite-4.2-3b        0 blocking (unchanged)
Qwen3 ×6, 0.6B–32B                0 blocking (unchanged)

granite-4.0-micro's remaining layer_types mismatch is pre-existing and unrelated — it is a hybrid Mamba/attention stack whose 40 declared attention layers resolve to a different kind.

Gates

fmt --all --check · check --workspace --all-targets (a new ModelConfig field, so larql-server's exhaustive literal matters) · clippy with each touched crate's own CI flags · full suites for larql-models, larql-vindex, larql-server · coverage policy for larql-models and larql-vindex, both re-measured after llvm-cov clean.

…_embedding_type is the switch, not a restatement

The census recorded this key as a "checked default" that "restates the
scheme already in use". Checking it against the reference instead of
against its name says the opposite. `GraniteMoeHybridConfig` documents

    position_embedding_type (`str`, *optional*):
        Positional embedding type to be used; defaults to None.
        Allowed options: `[None, "rope"]`

and `modeling_granitemoehybrid.py` builds

    self.rotary_emb = GraniteMoeHybridRotaryEmbedding(config)
                      if config.position_embedding_type == "rope" else None

So the default is NO POSITIONAL ENCODING and `"rope"` is the opt-in that
turns rotation on. `rope_theta` is declared either way — granite-4.0-micro
ships 10000000 — so reading the theta and rotating is right about that
checkpoint by luck and wrong about any granitemoehybrid that omits the
opt-in, rotating every position against the model's own instruction. The
same shape as `use_sliding_window`, one key over, and found the same way.

Scoped to `granitemoehybrid` alone. `granite`, `granitemoe` and
`granitemoeshared` construct their rotary unconditionally and never
mention the key, so a family-wide rule would turn every dense Granite
into a NoPE model — a far larger wrong answer than the one being fixed.
`control_a_dense_granite_still_rotates_without_the_key` is that bound.

An out-of-contract value takes the same path as absence, because that is
what the reference does: HF compares against `"rope"` and sends every
other string down the `else` branch. The BERT lineage spells `absolute`
in this same leaf, so the case is not hypothetical.

Calibrated rather than assumed — with the check disabled, the adversarial
arms fail with `Rope { theta: 10000000.0 }` on all four layers, which is
the bug stated in its own output. Three control arms stay green.

Carried at `Represented`, not `Parsed`: the effect lands on every layer's
`PositionPolicy`, so `probe_position_embedding_type` asks the graph what
it believes rather than trusting the parser to have read the key. A stack
that rotates nowhere answers null — an answer, not a failure to answer.

`position_policy_for_layer`'s default body lifts to a free function so
the override can narrow the decision and then defer to it. Rust gives an
override no way to call the default it replaced, and copying the body
would leave two resolvers to keep in agreement.

Census, from cached headers:

    ibm-granite/granite-4.0-micro    16 -> 15 blocking
    ibm-granite/granite-4.2-3b        0 blocking (unchanged)
    Qwen3 x6, all sizes               0 blocking (unchanged)
@chrishayuk
chrishayuk merged commit 2abef4b into main Sep 1, 2026
43 of 45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant