Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial unit tests for SVT-AV1 encoder #260

Merged
merged 3 commits into from May 23, 2019

Conversation

Projects
None yet
6 participants
@Cidana-Developers
Copy link
Contributor

commented May 20, 2019

  1. Added end-to-end test, which will execute conformance test
  2. Added unit tests for modules including tx, quant/dequant,
    entropy coding;
  3. Added api test, which will check with different parameters
Initial unit tests for SVT-AV1 encoder
1. Added end-to-end test, which will execute conformance test
2. Added unit tests for modules including tx, quant/dequant,
entropy coding;
3. Added api test, which will check with different parameters
@@ -28,6 +37,11 @@ if (MSVC OR MSYS OR MINGW OR WIN32)
option(BUILD_SHARED_LIBS "Build shared libraries (DLLs)." ON)
endif()

if(${CMAKE_SYSTEM_NAME} MATCHES "Windows")

This comment has been minimized.

Copy link
@1480c1

1480c1 May 20, 2019

Contributor
Suggested change
if(${CMAKE_SYSTEM_NAME} MATCHES "Windows")
if(MSVC)

Please use msvc instead of matching windows since this will also affect mingw and other compilers that does not accept /* style flags

This comment has been minimized.

Copy link
@Cidana-Developers

Cidana-Developers May 22, 2019

Author Contributor

Done.

# Define helper functions and macros used by Google Test.
include(../../third_party/googletest/cmake/internal_utils.cmake)

if(${CMAKE_SYSTEM_NAME} MATCHES "Windows")

This comment has been minimized.

Copy link
@1480c1

1480c1 May 20, 2019

Contributor
Suggested change
if(${CMAKE_SYSTEM_NAME} MATCHES "Windows")
if(MSVC)

Ditto here.

This comment has been minimized.

Copy link
@Cidana-Developers

Cidana-Developers May 22, 2019

Author Contributor

Done.

# Define helper functions and macros used by Google Test.
include(../../third_party/googletest/cmake/internal_utils.cmake)

if(${CMAKE_SYSTEM_NAME} MATCHES "Windows")

This comment has been minimized.

Copy link
@1480c1

1480c1 May 20, 2019

Contributor
Suggested change
if(${CMAKE_SYSTEM_NAME} MATCHES "Windows")
if(MSVC)

This comment has been minimized.

Copy link
@Cidana-Developers

Cidana-Developers May 22, 2019

Author Contributor

Done.

@1480c1

This comment has been minimized.

Copy link
Contributor

commented May 20, 2019

Can you also add a separate job for the unit tests? It seems that a lot of them failed but since it's in the coveralls section, it does not mark it as failed.
https://travis-ci.com/Cidana-Developers/SVT-AV1/jobs/201454816#L7209

@1480c1

This comment has been minimized.

Copy link
Contributor

commented May 20, 2019

Something like:

diff --git a/.travis.yml b/.travis.yml
index c6cae3d..b88951d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -114,7 +114,7 @@ matrix:
       stage: Coveralls
       os: linux
       compiler: gcc
-      env: COVERALLS_PARALLEL=true build_type=debug CMAKE_EFLAGS="-DCOVERAGE=ON"
+      env: COVERALLS_PARALLEL=true build_type=debug CMAKE_EFLAGS="-DCOVERAGE=ON -DBUILD_TESTING=ON"
       script:
         - *base_script
         - *coveralls_script
@@ -123,7 +123,7 @@ matrix:
       stage: Coveralls
       os: osx
       compiler: clang
-      env: COVERALLS_PARALLEL=true build_type=debug CMAKE_EFLAGS="-DCOVERAGE=ON"
+      env: COVERALLS_PARALLEL=true build_type=debug CMAKE_EFLAGS="-DCOVERAGE=ON -DBUILD_TESTING=ON"
       script:
         - *base_script
         - *coveralls_script
@@ -136,3 +136,15 @@ matrix:
       script:
         - *base_script
         - valgrind -- SvtAv1EncApp -enc-mode 8 -w 720 -h 486 -fps 60 -i akiyo_cif.y4m -n 150 -b test1.ivf
+    - name: Unit Tests
+      stage: test
+      compiler: gcc
+      os: linux
+      env: build_type=debug CMAKE_EFLAGS="-DBUILD_TESTING=ON"
+      script:
+        - *base_script
+        - cd $TRAVIS_BUILD_DIR/Build/linux/$build_type
+        - cmake -j $(if [ $TRAVIS_OS_NAME = osx ]; then sysctl -n hw.ncpu; else nproc; fi) --build . --target SvtAv1UnitTests;SvtAv1ApiTests;SvtAv1E2ETests;TestVectors
+        - SvtAv1UnitTests
+        - SvtAv1ApiTests
+        - SvtAviE2ETests
\ No newline at end of file

also, add build_testing to the cmake flags

@kodabb

This comment has been minimized.

Copy link

commented May 20, 2019

ah another giant PR, another example for #238

Fix platform check issue of CMakeLists.txt in test
1. Using MSVC instead of matching windows.
@Cidana-Developers

This comment has been minimized.

Copy link
Contributor Author

commented May 22, 2019

Can you also add a separate job for the unit tests? It seems that a lot of them failed but since it's in the coveralls section, it does not mark it as failed.
https://travis-ci.com/Cidana-Developers/SVT-AV1/jobs/201454816#L7209

Sure, will do it.

@andrey-norkin andrey-norkin requested review from hassount and kylophone May 22, 2019

Add separate jobs to run unit test.
Add two jobs to run unit test on linux and mac. Currently the unit
tests will fail in certain cases, set it "allow to fail" for now.

@hassount hassount merged commit 964642b into OpenVisualCloud:master May 23, 2019

3 checks passed

Travis CI - Pull Request Build Passed
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@kylophone

This comment has been minimized.

Copy link
Collaborator

commented May 23, 2019

ah another giant PR, another example for #238

Yeah, big initial import, but it's pretty self contained. @Cidana-Developers, I think these will be more incremental going forward. Is that right?

@Cidana-Developers

This comment has been minimized.

Copy link
Contributor Author

commented May 24, 2019

ah another giant PR, another example for #238

Yeah, big initial import, but it's pretty self contained. @Cidana-Developers, I think these will be more incremental going forward. Is that right?

Yes, no problem.

@Cidana-Developers Cidana-Developers deleted the Cidana-Developers:refine_stage1 branch May 27, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.