-
Notifications
You must be signed in to change notification settings - Fork 25.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Extend typing to path-like objects in PretrainedConfig
and PreTrainedModel
#8770
Extend typing to path-like objects in PretrainedConfig
and PreTrainedModel
#8770
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
Good idea! This could be done for tokenizers as well, no? (the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Works for me!
I have extended the same modifications to the tokenizers, as suggested by @thomwolf , and to auto classes too. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, thanks a lot!
…nedModel` (huggingface#8770) * update configuration_utils.py typing to allow pathlike objects when sensible * update modeling_utils.py typing to allow pathlike objects when sensible * black * update tokenization_utils_base.py typing to allow pathlike objects when sensible * update tokenization_utils_fast.py typing to allow pathlike objects when sensible * update configuration_auto.py typing to allow pathlike objects when sensible * update configuration_auto.py docstring to allow pathlike objects when sensible * update tokenization_auto.py docstring to allow pathlike objects when sensible * black
What does this PR do?
In my experience, I often call the
from_pretrained
andsave_pretrained
methods of models and configurations with a path-like variable rather than a string. Since the paths are then used by variousos
functions, this works just fine: however, the relevant variables are typed as strings only, raising warnings when using an IDE 😧 .This PR extends the typing to
Union[str, os.PathLike]
when relevant insidePretrainedConfig
andPreTrainedModel
methods.Since passing a path-like object is already tacitly supported in most cases, no significant changes to the code are necessary. In a few places, the relevant variable needs to be turned to a string in order to support functions such as
is_remote_url
.Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.
Maybe (documentation): @sgugger