chore: Check that we are compatible with transformers v5 by sjrl · Pull Request #10587 · deepset-ai/haystack

sjrl · 2026-02-13T10:08:56Z

Related Issues

fixes Support Transformers v5 #10453

Proposed Changes:

Updated failing unit tests due to a mock needing to be updated
hf_device_map is not always present anymore. Seems to only be set when mixed device loading is actually configured. Updated code to check if hf_device_map is present and if not then we fall back to using the regular device attribute.
text2text-generation no longer exists as a valid HF pipeline anymore. It seems we should always use text-generation.
- Updated defaults in HuggingFaceLocalGenerator to match changes made in the Chat variant. Needed to make some sort of change since the default task was text2text-generation which fails when using transformers v5.
- Yes text2text-generation was removed as a valid HF Pipeline in this PR Delete Text2TextPipeline, SummarizationPipeline and TranslationPipeline huggingface/transformers#43256 even though it still exists in the code base. Just not as straightforward to use anymore so I'll list it as a deprecated option in our components if users are using transformers v5+

How did you test it?

Notes for the reviewer

All of these changes are compatible with both transformers v4 and v5 (v4 tested locally). Not straightforward to test for both versions in our CI.

Checklist

I have read the contributors guidelines and the code of conduct.
I have updated the related issue with new insights and changes.
I have added unit tests and updated the docstrings.
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
I have documented my code.
I have added a release note file, following the contributors guidelines.
I have run pre-commit hooks and fixed any issue.

vercel · 2026-02-13T10:09:02Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
haystack-docs	Ignored	Preview	Feb 16, 2026 2:37pm

coveralls · 2026-02-13T10:13:35Z

Pull Request Test Coverage Report for Build 22065664803

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0 changed or added relevant lines in 0 files are covered.
45 unchanged lines in 5 files lost coverage.
Overall coverage decreased (-0.002%) to 92.64%

Files with Coverage Reduction	New Missed Lines	%
components/rankers/transformers_similarity.py	5	90.83%
utils/hf.py	5	87.44%
components/generators/chat/hugging_face_api.py	7	96.65%
components/readers/extractive.py	8	95.65%
components/generators/chat/hugging_face_local.py	20	85.29%

Totals
Change from base Build 22059650324:	-0.002%
Covered Lines:	15218
Relevant Lines:	16427

💛 - Coveralls

…to specify text-generation no matter what type of model is being used.

julian-risch

Looks quite good to me already! My main change request is to mention that the default model has been changed from "google/flan-t5-base" to "Qwen/Qwen3-0.6B". Other than that just two small questions and a simplification with walrus operator.

julian-risch · 2026-02-16T10:35:25Z

haystack/components/generators/chat/hugging_face_local.py

                    huggingface_pipeline_kwargs["model"], token=huggingface_pipeline_kwargs["token"]
                ).pipeline_tag  # type: ignore[assignment]  # we'll check below if task is in supported tasks

        if task not in PIPELINE_SUPPORTED_TASKS:


We could add a check here for task == "text2text-generation" and trigger a DeprecationWarning. However, as the deprecation comes directly from huggingface transformers, it's not strictly necessary. Still could be helpful for Haystack users.

Same for HuggingFaceLocalGenerator

We'd also need to check for the transformers version technically since it's only deprecated for v5+. So I'm not entirely sure if it's worth the effort.

I'll look into to see if there is an easy way to make this check

haystack/components/generators/hugging_face_local.py

releasenotes/notes/support-transformers-v5-3b43f560e0405d62.yaml

julian-risch · 2026-02-16T10:41:19Z

haystack/components/extractors/named_entity_extractor.py


        self._model_name_or_path = model_name_or_path
-        self.tokenizer: AutoTokenizer | None = None
+        self.tokenizer: TokenizersBackend | SentencePieceBackend | None = None


Will instances of AutoTokenizer still work as expected or is there a breaking change that we should mention in the release notes?

Hmm this is just a type change so nothing should break, but it's possible imports of these types would fail if they weren't present before. I'll double check locally.

You're right the imports aren't present so I think I'll go with ignoring the type issues from mypy so we can support both versions

Updated to make type Any so mypy is satisfied and works in both major versions of transformers

haystack/components/rankers/transformers_similarity.py

haystack/components/readers/extractive.py

…ack into support-transformers-v5

Fix a few tests

b412d21

github-actions bot added topic:tests topic:build/distribution labels Feb 13, 2026

sjrl added 2 commits February 13, 2026 11:47

text2text-generation task is deprecated. It seems like you just need …

1c23973

…to specify text-generation no matter what type of model is being used.

Fix the fact that hf_device_map is not always present anymore

cc3adf9

github-actions bot added the type:documentation Improvements on the docs label Feb 13, 2026

sjrl added 3 commits February 13, 2026 13:50

Fix for how to handle multi-device model loading

4473a98

fix tests and improve docstrings

5474eca

Add reno

2a17159

sjrl marked this pull request as ready for review February 16, 2026 10:03

sjrl requested a review from a team as a code owner February 16, 2026 10:03

sjrl requested review from julian-risch and removed request for a team February 16, 2026 10:03

sjrl added 2 commits February 16, 2026 11:19

fix mypy

211e307

Fix import errors

64b3dec

julian-risch requested changes Feb 16, 2026

View reviewed changes

Merge branch 'main' into support-transformers-v5

2230faa

vercel bot deployed to Preview February 16, 2026 11:56 View deployment

sjrl added 5 commits February 16, 2026 14:53

PR comments and avoid imports only available in v5

e2e16a8

Merge branch 'support-transformers-v5' of github.com:deepset-ai/hayst…

8bc2234

…ack into support-transformers-v5

Update reno

91a2ab6

Remove import not present in v4

bc6417f

Add a new check

c096d30

Conversation

sjrl commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Checklist

Uh oh!

vercel bot commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 22065664803

Warning: This coverage report may be inaccurate.

Details

💛 - Coveralls

Uh oh!

julian-risch left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sjrl commented Feb 13, 2026 •

edited

Loading

vercel bot commented Feb 13, 2026 •

edited

Loading

coveralls commented Feb 13, 2026 •

edited

Loading