Running Audacity tests

[christian-heusel]

Hello everyone,

I really fear I am missing something but so far I didn't manage to run any meaningful tests for Audacity:

==> Starting check()...
Internal ctest changing into directory: /build/audacity/src/build
Test project /build/audacity/src/build
No tests were found!!!

The tests are executed like this:

  ctest --test-dir build --output-on-failure

I tried to explicitly build with -D audacity_has_tests=ON (even though it should be on by default), and switch around the release targets and also explicitly set them like it is done in the CI, sadly to no avail:

  ctest -C Release --test-dir build --output-on-failure

Any help would be appreciated and don't fear to just RTFM me if I just didn't find the right doc page! 🤗

[matoro]

I am also running into this issue and cannot figure out what is different in CI!

[matoro]

Never mind, I got it. It's because of the rule in CMakeLists.txt that makes audacity_has_tests dependent on audacity_conan_enabled, and I guess just disables it silently otherwise. I don't understand why this was added in #2536 . But I have worked around it with this patch:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index e631e18ad..02e05b147 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -222,15 +222,12 @@ cmd_option( ${_OPT}has_url_schemes_support
    "Build custom URL schemes support into Audacity"
    Off)
 
-include( CMakeDependentOption )
-
-cmake_dependent_option(
+cmd_option(
    ${_OPT}has_tests
    "Enables automated testing support"
-   On
-   "${_OPT}conan_enabled"
-   Off
-)
+   On)
+
+include( CMakeDependentOption )
 
 cmake_dependent_option(
    ${_OPT}has_audiocom_upload

I don't know if this was on purpose or if it would be OK to remove this dependency? I think the reasoning might have been for pulling in catch2 via conan, but everything works fine if catch2 is installed via system package manager.

[crsib]

It won't build on Arch, for example.

[christian-heusel]

@matoro thanks a ton for figuring this out, I was pretty much stuck with it 🤔

I don't know if this was on purpose or if it would be OK to remove this dependency?

Yes I'd agree that this dependency is unexpected! Could you maybe create a PR from your patch so this change gets upstreamed?

It won't build on Arch, for example.

Why not? 🤔 I build it just fine and initially posted this issue (I am one of the Arch Linux Package Maintainers for Audacity)

[crsib]

I have a report from an Arch user that the system version of Catch2 does not work because it is Catch3 now.

We have no immediate plans to update the library used for the unit testing. We absolutely have no plans to support both Catch 2 and Catch 3.

[crsib]

Please see #5771 for reference

[christian-heusel]

See my #5771 (comment) over there, we have a package for catch2-v2 😄