Skip to content

Fix issue with transformers library huggingface #11027

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 5, 2025

Conversation

alexcbb
Copy link
Collaborator

@alexcbb alexcbb commented Mar 12, 2025

Description

I was trying to modifiy a RLDS dataset built on top of TFDS following this repo : https://github.com/kpertsch/rlds_dataset_builder
I needed to extract some features from images with models from the transformers library of HuggingFace but was facing an issue during the import : raise ValueError('{}.__spec__ is None'.format(name)) ValueError: datasets.__spec__ is None

And more specifically this one : transformers/utils/import_utils.py", line 120, in <module> _datasets_available = _is_package_available("datasets")

It verifies if datasets (the HF library) is available by looking at the __spec__ attribute. As tfds is overwritting datasets by a mock, it does not create the attribute, which causes the issue.

In the PR I fixed the issue by simply creating the needed attribute in the Mock in order to solve the problem.

Here are the versions of the libraries involved in the problem :
tensorflow-datasets 4.9.3
transformers 4.50.0.dev0

Transformers library in the last versions verify if datasets contains a _spec_ to check if it is available
@pierrot0 pierrot0 self-assigned this Jun 5, 2025
@pierrot0 pierrot0 added the copybara-import Internal label for PR management label Jun 5, 2025
@copybara-service copybara-service bot merged commit c317c90 into tensorflow:master Jun 5, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
copybara-import Internal label for PR management
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants