Make safetensors the default #2120
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
There was a problem hiding this comment.
Looks great! Something that we're doing in transformers as well that you might want to do here, is to continue testing for safe_serialization=False in the common pathways.
By removing testing of the serialization of pytorch_model.bin (which we're doing as the default isn't explicitly defined in the tests and the default just changed), we're risking breaking that code without noticing.
You implemented another test, do you think maybe others would need implementing? That's what we did on transformers side (not merged yet): huggingface/transformers#27242
Good point.
I think this is something where test coverage could be helpful, as it would reveal if we have code paths that are no longer taken but which used to be taken. |
|
@LysandreJik @BenjaminBossan I've updated all of our tests that do anything with |
| If `safe_serialization` is `True`, models will be saved with `safetensors` while the rest are saved using native | ||
| `pickle`. |
| return f"{func.__name__}_{param_based_name}" | ||
|
|
||
|
|
||
| @parameterized_class(("use_safetensors",), [[True], [False]], class_name_func=parameterized_custom_name_func) |
There was a problem hiding this comment.
TIL about parameterized_class.
What does this PR do?
Similar to transformers, makes
safetensorsthe default and a library requirement 馃Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@LysandreJik @SunMarc @BenjaminBossan