Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix type hints pickling in python 3.6 #818

Merged
merged 1 commit into from
Nov 10, 2020

Conversation

lhoestq
Copy link
Member

@lhoestq lhoestq commented Nov 9, 2020

Type hints can't be properly pickled in python 3.6. This was causing errors the run_mlm.py script from transformers with python 3.6

However Cloupickle proposed a fix to make it work anyway.
The idea is just to implement the pickling/unpickling of parameterized type hints. There is one detail though: since in python 3.6 we can't use isinstance on type hints, then we can't use pickle saving functions registry directly. Therefore we just wrap the save_global method of the Pickler.

This should fix huggingface/transformers#8212 for python 3.6 and make run_mlm.py support python 3.6

cc @sgugger

@lhoestq lhoestq merged commit 02669ea into master Nov 10, 2020
@lhoestq lhoestq deleted the fix-type-hints-pickling-in-py36 branch November 10, 2020 09:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pickle error
1 participant