-
Notifications
You must be signed in to change notification settings - Fork 25.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[tokenizers] convert_to_tensors: don't reconvert when the type is already right #8283
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, this looks like a good additional check. @mfuntowicz, what do you think?
ping |
Looks good to me. Thanks for handling this one @stas00 and sorry for the delay. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, I find it better this way too. Thanks @stas00!
…eady right (huggingface#8283) * don't reconvert when the type is already right * better name * adjust logic as suggested * merge
I was trying to fix this warning:
which appeared when running:
This appears to have happened since
convert_to_tensors
was called with data which was already a tensor of the right type.test_batch_encoding_with_labels_jax
I understand
lambda
isn't welcome, so I had to define a few helper functions for numpy/jax.partial
would have done the trick, butisinstance
doesn't accept keyword args.@LysandreJik, @mfuntowicz