Error when tokenizer is set to string: AttributeError: 'str' object has no attribute 'pad_token_id'

[TrevinAvery]

I'm using the SageMaker HuggingFace inference toolkit, with the image-text-to-text task. I am getting the following error from this line in transformers/pipelines/base.py:

AttributeError: 'str' object has no attribute 'pad_token_id'

This toolkit passes a string for the tokenizer parameter to the pipeline function. The passed string matches model_dir (as seen here).

I expect the string to either 1) be used to load a valid tokenizer object, 2) be dropped because it is unused, or 3) throw an error because it is invalid. However, instead, it just passes the string forward to pipeline_class (in this case ImageTextToText), which does not accept a string for the tokenizer parameter.

Suggested fix:

When evaluating load_tokenizer, if it is false, it should either set tokenizer to None or raise an exception.

[Rocketknight1]

Is this happening with Gemma3?

[zucchini-nlp]

It should have been fixed by recent patch on the release branch, we had a bug where the tokenizer wasn't added to model-mapping