-
Notifications
You must be signed in to change notification settings - Fork 25.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make AutoProcessor a magic loading class for all modalities #18963
Conversation
The documentation is not available anymore as the PR was closed or merged. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your PR! Simple and elegant. I can't think of a repository that would have a feature extractor and a tokenizer, but no processor.
Looks good to me! @patrickvonplaten, could you give it a look?
# At this stage, there doesn't seem to be a `Processor` class available for this model, so let's try a | ||
# tokenizer. | ||
try: | ||
return AutoTokenizer.from_pretrained( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess the order doesn't matter too much here since if both tokenizer and feature extractor would be present then the model also should have a processor?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I don't see when we could have both without a processor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for re-enabling this!
…ace#18963) * Make AutoProcessor a magic loading class for all modalities * Quality
What does this PR do?
This PR re-enables a feature initially part of #14465 : the fact that
AutoProcessor
is a class loading the right processing class for any model (so processor, tokenizer or feature extractor). You can thus do:or