Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix VideoReader CPU only variant #3660

Merged
merged 4 commits into from
Feb 8, 2022

Conversation

awolant
Copy link
Contributor

@awolant awolant commented Feb 7, 2022

Signed-off-by: Albert Wolant awolant@nvidia.com

Category:

Bug fix

Description:

VideoReaderCpu was unnecessarily dependent on the GPU. Removed the dependency and added proper tests for CPU only test suite.

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Checklist

Tests

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: DALI-2588

Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Copy link
Contributor Author

awolant commented Feb 7, 2022

!build

@JanuszL
Copy link
Contributor

JanuszL commented Feb 7, 2022

I would update test_dali_cpu_only.py as experimental.readers.video is not tested there.

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3907820]: BUILD STARTED

@awolant
Copy link
Contributor Author

awolant commented Feb 7, 2022

I would update test_dali_cpu_only.py as well as experimental.readers.video.

Not sure what you mean here? There are no python tests for this at this time.

@JanuszL
Copy link
Contributor

JanuszL commented Feb 7, 2022

I would update test_dali_cpu_only.py as well as experimental.readers.video.

Not sure what you mean here? There are no python tests for this at this time.

Sorry, I was not precise. Here you test the decoder, but I wanted to make sure that the experimental.readers.video operator is tested for CPU only cases as well. And this needs to be done in test_dali_cpu_only.py.

@awolant
Copy link
Contributor Author

awolant commented Feb 7, 2022

Operator is tested with C++ tests as well and I adjusted them to be picked up by the cpu only test job as well in this PR.

@JanuszL
Copy link
Contributor

JanuszL commented Feb 7, 2022

Operator is tested with C++ tests as well and I adjusted them to be picked up by the cpu only test job as well in this PR.

I think we should have at least a basic test inside test_dali_cpu_only.py, otherwise it will be hard to track where each operator is tested.

@@ -21,11 +21,11 @@


namespace dali {
class FramesDecoderTest : public VideoTestBase {
class FramesDecoderTest_CpuOnlyTests : public VideoTestBase {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think gtest is against putting _ in test names.

https://github.com/google/googletest/blob/main/docs/faq.md#why-should-test-suite-names-and-test-names-not-contain-underscore

Also, why is this name change necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use this sufix to pick up tests we want to run in the CPU only test. This is already existing mechanism, so I just used it.

Possible change in that to comply to gtest recommendation is out of scope for this, I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the main idea of the slice test here was to check CPU implementation than to check if it can run without GPU. We have only one such native test with that intention - CApiTest.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pattern that we use will work regardless of _.

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3907820]: BUILD FAILED

Signed-off-by: Albert Wolant <awolant@nvidia.com>
@@ -603,6 +604,9 @@ def create_wav_files():
check_no_input(fn.readers.nemo_asr, manifest_filepaths=[nemo_asr_manifest], dtype=types.INT16, downmix=False,
read_sample_rate=True, read_text=True, seed=fixed_seed)

def test_video_reader():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant awolant changed the title Fix CPU video tests Fix VideoReader CPU only variant Feb 7, 2022
@awolant
Copy link
Contributor Author

awolant commented Feb 7, 2022

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3908645]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3908645]: BUILD PASSED

@awolant
Copy link
Contributor Author

awolant commented Feb 7, 2022

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3911220]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3911220]: BUILD PASSED

Comment on lines +133 to +135
EXPECT_TRUE(strstr(
e.what(),
make_string("Failed to open video file at path ", path).c_str()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about
EXPECT_THROW(FramesDecoder(path), DALIException)
?
I don't think you can match the error message, so up to you

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I wanted to make sure that the message is the one I want and it reflects what happen, so I'm going to leave it for now.
If this keeps reoccurring we can think on some solution that would do both.

@awolant awolant merged commit cf0a0a7 into NVIDIA:main Feb 8, 2022
cyyever pushed a commit to cyyever/DALI that referenced this pull request Feb 21, 2022
* Fix CPU video tests

Signed-off-by: Albert Wolant <awolant@nvidia.com>
cyyever pushed a commit to cyyever/DALI that referenced this pull request May 13, 2022
* Fix CPU video tests

Signed-off-by: Albert Wolant <awolant@nvidia.com>
cyyever pushed a commit to cyyever/DALI that referenced this pull request Jun 7, 2022
* Fix CPU video tests

Signed-off-by: Albert Wolant <awolant@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants