Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace 'decord' with 'av' in VideoClassificationPipeline #29747

Merged
merged 8 commits into from Mar 26, 2024

Conversation

Tyx-main
Copy link
Contributor

@Tyx-main Tyx-main commented Mar 20, 2024

What does this PR do?

As the feature request shows, the dependency 'decord' can only be used in x86 instead of aarch. For those users who want to use VideoClassificationPipeline on aarch, we decide to replace 'decord' with 'av' which is a perfect adaptation for the two above-mention architecture.

Fixes # (issue)
feature request

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@amyeroberts

Test on V100

To make sure the codes is correct, I test the VideoClassificationPipeline via tests/pipelines/test_pipelines_video_classification.py on V100

Here is my command(the current dir is the root dir of transformers):

TRANSFORMERS_TEST_BACKEND="torch" TRANSFORMERS_TEST_DEVICE="cuda:0" TRANSFORMERS_TEST_DEVICE_SPEC="spec.py" python -m pytest -v tests/pipelines/test_pipelines_video_classification.py

The codes of spec.py:

import torch
DEVICE_NAME = "cuda:0"
MANUAL_SEED_FN = torch.cuda.manual_seed_all
EMPTY_CACHE_FN = torch.cuda.empty_cache
DEVICE_COUNT_FN = torch.cuda.device_count

And the version of 'av' is 11.0.0 which is the default version of 'pip install av'

Finaly we can get the 'pass' via testing tests/pipelines/test_pipelines_video_classification.py:
test

Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks for adding this and making our pipelines more accessible 🔥

To resolve the failing quality checks:

  • Some are unrelated to this PR. A fix was pushed on main, could you rebase to include this?
  • Some are just linting checks - running make fixup and pushing the changes should resolve

@Tyx-main
Copy link
Contributor Author

Awesome! Thanks for adding this and making our pipelines more accessible 🔥

To resolve the failing quality checks:

  • Some are unrelated to this PR. A fix was pushed on main, could you rebase to include this?
  • Some are just linting checks - running make fixup and pushing the changes should resolve

Thanks for your review!I will fix the remaining problems.

@Tyx-main
Copy link
Contributor Author

@amyeroberts sorry for waiting so long! I haved fix all the problems, could you have a look?

Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great - thanks!

@amyeroberts amyeroberts merged commit b32bf85 into huggingface:main Mar 26, 2024
21 checks passed
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

hovnatan pushed a commit to hovnatan/transformers that referenced this pull request Mar 27, 2024
…e#29747)

* replace the 'decord' with 'av' in VideoClassificationPipeline

* fix the check of backend in VideoClassificationPipeline

* adjust the order of imports

* format 'video_classification.py'

* format 'video_classification.py' with ruff

---------

Co-authored-by: wanqiancheng <13541261013@163.com>
itazap pushed a commit that referenced this pull request May 14, 2024
* replace the 'decord' with 'av' in VideoClassificationPipeline

* fix the check of backend in VideoClassificationPipeline

* adjust the order of imports

* format 'video_classification.py'

* format 'video_classification.py' with ruff

---------

Co-authored-by: wanqiancheng <13541261013@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants