Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Tensorflow Dataset #134

Merged
merged 10 commits into from
Apr 23, 2024
Merged

Introduce Tensorflow Dataset #134

merged 10 commits into from
Apr 23, 2024

Conversation

BillHuang2001
Copy link
Collaborator

@BillHuang2001 BillHuang2001 commented Apr 15, 2024

Description

Introduce Tensorflow Dataset and deprecate the old torchvision dataset.

Motivation:

  1. Torchvision requires torch to co-exist with JAX, which is a pain to manage. Tensorflow dataset is mostly standalone (download the dataset require tensorflow, if already downloaded, tensorflow is not required), and tensorflow/jax co-existence is much simpler.
  2. Tensorflow dataset has a much larger dataset coverage, not only vision tasks.
  3. The default data format is the same as flax/haiku. For example, the default format for image is (N, H, W, C), while torchvision gives (N, C, H, W).
  4. Torch's dataloader is really bad.

Checklist

  • I have formatted my Python code with black.
  • I have good commit messages.
  • If adding new algorithms, problems, operators:
    • Added related test cases.
    • Added docstring to explain important parameters.
    • Added entries in the docs.

@BillHuang2001 BillHuang2001 merged commit 25f0026 into main Apr 23, 2024
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant