Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'open_images_v4' with 'array_record' raise ValueError #5419

Open
seongmoon729 opened this issue May 18, 2024 · 0 comments
Open

'open_images_v4' with 'array_record' raise ValueError #5419

seongmoon729 opened this issue May 18, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@seongmoon729
Copy link

/!\ PLEASE INCLUDE THE FULL STACKTRACE AND CODE SNIPPET

Short description
When I try to load the 'open_images_v4' dataset with 'array_record' file format, it raises ValueError.

Environment information

  • Operating System: ubuntu 20.04

  • Python version: 3.8.13

  • tensorflow-datasets/tfds-nightly version: tensorflow-datasets=4.9.2

  • tensorflow/tf-nightly version: no tensorflow

  • Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ? Yes, I checked with tfds-nightly-4.9.2.dev202308090034.

Reproduction instructions

import os
os.environ.pop('TFDS_DATA_DIR', None)
import tensorflow_datasets as tfds

builder = tfds.builder('open_images_v4', file_format='array_record', data_dir='./tensorflow_datasets/')
builder.download_and_prepare()
ds = builder.as_data_source()['train']
print(ds[0])

Link to logs
Traceback (most recent call last):
File "test_tfds.py", line 27, in
batch = next(ds)
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/data_sources/array_record.py", line 83, in iter
yield self[i]
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/data_sources/array_record.py", line 95, in getitem
return self.dataset_info.features.deserialize_example_np(
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 159, in deserialize_example_np
return self.decode_example_np(example_data, decoders=decoders)
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 112, in decode_example_np
return self._decode_example_generic(
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 81, in _decode_example_generic
flatten_decoded = [
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 82, in
_decode_feature( # pylint: disable=g-complex-comprehension
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/top_level_feature.py", line 209, in _decode_feature
return decoder.decode_example_np(example, **decode_kwargs)
File "/home/moon/.local/miniforge/envs/filtering/lib/python3.8/site-packages/tensorflow_datasets/core/features/tensor_feature.py", line 234, in decode_example_np
return example_data.reshape(shape)
ValueError: cannot reshape array of size 8 into shape (4,)

Expected behavior
Load the 'open_images_v4' dataset without exception and print a data

Additional context
Add any other context about the problem here.

@seongmoon729 seongmoon729 added the bug Something isn't working label May 18, 2024
@fineguy fineguy self-assigned this May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants