Skip to content

update to_py_obj to support np.number#14064

Merged
sgugger merged 1 commit intohuggingface:masterfrom
PrettyMeng:np_number_to_py_obj
Oct 19, 2021
Merged

update to_py_obj to support np.number#14064
sgugger merged 1 commit intohuggingface:masterfrom
PrettyMeng:np_number_to_py_obj

Conversation

@PrettyMeng
Copy link
Copy Markdown
Contributor

What does this PR do?

Update to_py_obj function to make it support np.number type.

Fixes # See an example below:

tokenizer.decode(110)
'no'
tokenizer.decode(np.array([110])[0])
Traceback (most recent call last):
File "", line 1, in
File "/Users/prettymeng/anaconda3/envs/ptuning/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 3179, in decode
return self._decode(
File "/Users/prettymeng/anaconda3/envs/ptuning/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 527, in _decode
text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
TypeError: Can't convert 110 to Sequence

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@PrettyMeng
Copy link
Copy Markdown
Contributor Author

@sgugger Hi! This is an reopened PR for #14041, could you please check whether the failed tests are related to my PR?

@sgugger sgugger merged commit 3892d09 into huggingface:master Oct 19, 2021
@sgugger
Copy link
Copy Markdown
Collaborator

sgugger commented Oct 19, 2021

All good, thanks a lot for your contribution!

@PrettyMeng PrettyMeng deleted the np_number_to_py_obj branch February 17, 2022 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants