Fix text-generation pipeline not forwarding tokenizer to generate - #47702
Open
alwaysprince05 wants to merge 1 commit into
Open
Fix text-generation pipeline not forwarding tokenizer to generate#47702alwaysprince05 wants to merge 1 commit into
alwaysprince05 wants to merge 1 commit into
Conversation
When stop_strings is set on the pipeline's generation config, generate needs the tokenizer to decode the stop strings. Forward the pipeline's tokenizer via generate_kwargs.setdefault so explicit user-passed tokenizers still win. Covers the pipeline half of huggingface#34571.
Contributor
CI recapDashboard: View test results in Grafana |
Rocketknight1
approved these changes
Aug 4, 2026
Rocketknight1
left a comment
Rocketknight1
left a comment
Member
There was a problem hiding this comment.
This makes sense, but can you check if self.tokenizer is always defined in that pipeline, and it doesn't sometimes have self.processor or something like that? I think that one always uses tokenizer, but just in case
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
When
stop_stringsis set on the pipeline's generation config,generateneeds the tokenizer to decode the stop strings. The text-generation pipeline wasn't forwarding its tokenizer, so this raised aValueError.This adds
generate_kwargs.setdefault("tokenizer", self.tokenizer)before thegeneratecall, so the pipeline's tokenizer is forwarded while explicit user-passed tokenizers still win.This covers the pipeline half of #34571, as suggested by @qgallouedec.
Test
Added
test_stop_strings_from_generation_config, which setsstop_stringson the pipeline's generation config and checks the output stops correctly. It fails without the fix and passes with it.make styleandmake fix-repopass. The only local test failures in this file are pre-existing ones that need access to the gatedgoogle/gemma-3-270m-itmodel.