Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Video decoder in plugin #5477

Merged
merged 53 commits into from
Jun 17, 2024
Merged

Video decoder in plugin #5477

merged 53 commits into from
Jun 17, 2024

Conversation

awolant
Copy link
Contributor

@awolant awolant commented May 21, 2024

Category:

New feature

Description:

Adds video decoder based on the Video Codec SDK 12.0 and is distributed with nvidia-dali-video package.

Additional information:

This PR adds the basic minimal video decoder based on Video Codec SDK 12.0. It can decode h264 and h265 videos passed as encoded binary buffers. Can be used with external source operator as an input. There is one argument: end_frame - it tells what is the index of a last frame that should be decoded.

Affected modules and functionalities:

Video plugin, video plugin tests.

Key points relevant for the review:

  • Does it pick up the proper Video Codec SDK.
  • This is very minimal but working version. For now, it is not user facing code. I would like to use it as a base to add more features in the future. Please, keep that in mind.

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: N/A

Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
…functions

Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
Signed-off-by: Joaquin Anton <janton@nvidia.com>
if (spec.HasArgument("end_frame")) {
end_frame_ = spec.GetArgument<int>("end_frame");
} else {
DALI_FAIL("Not passing `end_frame` argument is not yet implemented.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick, to get rid two negations from the sentence.

Suggested change
DALI_FAIL("Not passing `end_frame` argument is not yet implemented.");
DALI_FAIL("Currently, `end_frame` argument is required.");

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

yuv_to_rgb(pFrame, nPitch, reinterpret_cast<uint8_t *>(dpFrame),
sample.decoder_->GetWidth() * 3, sample.decoder_->GetWidth(),
sample.decoder_->GetHeight(), full_range, cuStream);
CUDA_CALL(cudaStreamSynchronize(cuStream));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this synchronization needed here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was leftover from the more extended version where we might want to do some postprocessing here. Removed it for now, I will add it if needed later.

Signed-off-by: Albert Wolant <awolant@nvidia.com>
Signed-off-by: Albert Wolant <awolant@nvidia.com>
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Copy link
Contributor Author

awolant commented Jun 6, 2024

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15614667]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15614667]: BUILD FAILED

Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Copy link
Contributor Author

awolant commented Jun 6, 2024

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15620547]: BUILD STARTED

Signed-off-by: Albert Wolant <awolant@nvidia.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15620547]: BUILD FAILED

@awolant
Copy link
Contributor Author

awolant commented Jun 7, 2024

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15650237]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15650237]: BUILD FAILED

Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Copy link
Contributor Author

awolant commented Jun 10, 2024

!build

2 similar comments
@awolant
Copy link
Contributor Author

awolant commented Jun 10, 2024

!build

@awolant
Copy link
Contributor Author

awolant commented Jun 16, 2024

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15879104]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15879104]: BUILD FAILED

Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Copy link
Contributor Author

awolant commented Jun 17, 2024

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15887381]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [15887381]: BUILD PASSED

@awolant awolant merged commit 3119508 into NVIDIA:main Jun 17, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants