Support transformers >= 5.9.0

[antonpibm]

transformers 5.9.0 introduced a breaking change that causes test failures and possibly issues in Granite-Switch
Compatibility with version 5.9.0 should be supported and validated

To reproduce remove the "transformers>=5.5.1,<5.9.0", constraint in pyproject.toml and run all tests

[barvhaim]

@antonpibm i'm running the tests, will post here the error as well

[barvhaim]

Investigated on a fresh branch (bugfix/transformers-5.9-compat). Bumped the constraint to >=5.5.1,<5.10.0, ran uv sync --upgrade-package transformers (installed 5.9.0).

Findings

• tests/unit/: 86 passed
• Test collection across tests/unit/, tests/hf/, tests/composer/: 905 collected, no import errors
• tests/hf/: fails at the first test (test_generation.py::TestSingleSwitchGeneration::test_generates_tokens)

Error

TypeError: create_causal_mask() got an unexpected keyword argument 'input_embeds'
src/granite_switch/hf/modeling_granite_switch.py:286

Root cause

transformers.masking_utils.create_causal_mask changed signature between 5.8 and 5.9:

1. input_embeds → inputs_embeds (added s)
2. cache_position parameter removed (docstring in 5.9: "Deprecated and unused")

Single call site in our code: src/granite_switch/hf/modeling_granite_switch.py:286.

Question on direction

The fix itself is small, but there's a compatibility decision to make. The current constraint range (>=5.5.1) allows installs on 5.5–5.9. Two options:

Option A — Bump minimum to >=5.9.0

• Simplest patch (just rename the kwarg, drop cache_position)
• Drops support for users still on 5.5–5.8

Option B — Version-dispatch at the call site

• Keeps the wide compatibility window
• ~5 extra lines using packaging.version to switch kwarg names

Like

  import transformers
  from packaging.version import parse as parse_version

  _TRANSFORMERS_GE_5_9 = parse_version(transformers.__version__) >= parse_version("5.9.0")

  # ... later, at the call site:
  mask_kwargs = {
      "config": self.config,
      "attention_mask": attention_mask,
      "past_key_values": past_key_values,
      "position_ids": position_ids,
  }
  if _TRANSFORMERS_GE_5_9:
      mask_kwargs["inputs_embeds"] = mask_shape_proxy
  else:
      mask_kwargs["input_embeds"] = mask_shape_proxy
      mask_kwargs["cache_position"] = cache_position
  causal_mask = create_causal_mask(**mask_kwargs)

Which would you prefer? Happy to open a PR either way.

I haven't run tests/composer/ or tests/vllm/ yet (no GPU on my current machine), so there may be additional 5.9 breakages beyond this one — will check the composer suite next on CPU.

[antonpibm]

@barvhaim Thanks for looking into that! couple of thoughts:

1. It is absolutely mandatory to run the GPU tests
2. About the direction, in the branching on call site strategy (Options B), where would this code be located?
3. Option A - is limiting to transformers>=5.9.0 too strict? as it was only released last week. I think we need to understand this better - does our other dependencies support it (e.g. vllm 0.19.1 and HF) or we need to also apply addition restrictions to our other dependencies? are there environments where this is not supported?

[barvhaim]

@antonpibm

2. About the direction, in the branching on call site strategy (Option B), where would this code be located?

Inline in src/granite_switch/hf/modeling_granite_switch.py:286 — it's the only call site (single import + single call, confirmed via grep). No new util module.

Module-level near the existing transformers imports:

import transformers
from packaging.version import parse as _parse_version

_TRANSFORMERS_GE_5_9 = _parse_version(transformers.__version__) >= _parse_version("5.9.0")

At the call site:

mask_kwargs = {
    "config": self.config,
    "attention_mask": attention_mask,
    "past_key_values": past_key_values,
    "position_ids": position_ids,
}
if _TRANSFORMERS_GE_5_9:
    mask_kwargs["inputs_embeds"] = mask_shape_proxy
else:
    mask_kwargs["input_embeds"] = mask_shape_proxy
    mask_kwargs["cache_position"] = cache_position
causal_mask = create_causal_mask(**mask_kwargs)

Verified against the 5.9 source that create_causal_mask still only reads batch/seq-length/dtype/device from inputs_embeds, so the existing (B, S, 1) mask_shape_proxy stub stays valid.

Re: (1) GPU tests — acknowledged, will run before opening the PR.
Re: (3) ecosystem compatibility — investigating, will report back.

[barvhaim]

3. Option A — is limiting to transformers>=5.9.0 too strict? as it was only released last week. I think we need to understand this better — does our other dependencies support it (e.g. vllm 0.19.1 and HF) or we need to also apply additional restrictions to our other dependencies? are there environments where this is not supported?

Investigated. Short answer: Option A is viable — no resolver-level blockers, no transitive constraints we'd violate. Details below.

Release timing. transformers 5.9.0 uploaded 2026-05-20 (8 days ago). One minor cycle after 5.8.0 (2026-05-05).

Direct dependents in our pyproject.toml:

Dep (extra)	transformers constraint	OK with 5.9?
vllm 0.19.1 ([vllm], [dev], [tutorials])	>=4.56.0, no upper bound	Yes
vllm 0.20.x ([vllm20], [dev-vllm20])	same	Yes
accelerate>=0.20.0 ([hf])	no pin	Yes
mellea==0.6.0 ([tutorials])	<5,>=4.53.2; extra=="hf"	N/A — we install bare mellea, not mellea[hf], so the constraint never activates
sentence-transformers>=3.0.0 ([tutorials])	<6.0.0,>=4.41.0	Yes
chromadb>=0.4.0 ([tutorials])	no transformers dep	N/A

Empirical resolve test. Created a scratch project mirroring our pyproject.toml with transformers>=5.9.0,<5.10.0 and ran uv pip compile --python-platform linux for every extra:

Extra	Result	transformers	torch	vllm
[hf]	OK	5.9.0	2.12.0	—
[vllm]	OK	5.9.0	2.10.0	0.19.1
[vllm20]	OK	5.9.0	2.11.0	0.20.2
[tutorials]	OK	5.9.0	2.10.0	0.19.1 + mellea 0.6.0 + sentence-transformers 5.5.1
[dev]	OK	5.9.0	2.10.0	0.19.1
[dev-vllm20]	OK	5.9.0	2.11.0	0.20.2

All six resolve cleanly. No additional dep restrictions needed.

Python/torch. transformers 5.9.0 requires python>=3.10, torch>=2.4 — well within our python>=3.11,<3.14 and torch>=2.10.0.

Caveats:

1. 5.9.0 is 8 days old — recency risk is real but not resolver-detectable. The GPU test run (per your point 1) is what will actually answer "does it work."
2. mellea[hf] (which we don't use) still pins transformers<5. Any user combining us with mellea[hf] in a single env would conflict — worth flagging to the mellea team regardless of A vs B.

Recommendation. Option A is acceptable. Option B remains the conservative choice (~5 extra lines, keeps 5.5–5.8 users working) — only worth picking if we know of concrete consumers pinned below 5.9.

[barvhaim]

As per GPU tests, I am still onboarding, will run when I get GPU cluster