Change test runner to ctest

[felixschurk]

Some thoughts about testing/performance etc..

As a general regard for the tests and how we call it.
I thought a bit about it and for me personally it is quite confusing how one does run all test (unit tests, integration tests etc.) as well as how one could build only a specific test.
E.g. when I would like to work on a feature which affects only a subset of tests it would be beneficial to only run the corresponding tests when developing, and then have a simple interface to run all test when one would be ready.
Also on this approach I think most of the IDEs should be able to recognize the proper tests and then also coverage etc should be quite simple to add in the future.

What could one do?

I guess the most forward way would be to use CTest as it is bundled with CMake and thus already part of our toolchain.
With CTest one can also execute shell scripts, python files etc. meaning we do not need to rework existing tests, only how they are executed/build with CMake.

How would it look like? Some demo repository

I just searched and found ctest-demo as a simple sample repository (not affiliated with me or something). Maybe to have a feeling do:

git clone https://github.com/tgamblin/ctest-demo.git
cd ctest-demo
cmake -S . -B build
cmake --build build
ctest --test-dir build

This would give a first overview how it is looking/how it might be looking.
If one would like to see a bit bigger example this repository I worked on and we used there the CTest for the test invokation (Mainly liked because its a bit larger than the sample, but yet I am able to understand everything :D).

I played around already a bit with the taskwarrior repository, and I should be able to implement it here.

What would need to be done?

• adding all existing tests to the CMakeLists
• update build pipelines to pickup new schema
• update documentation for the developer
• remove obsolute run_all and problems scripts
• update documentation in test/README

@djmitche and @ryneeverett if you guys think it's a reasonable idea to go this direction, I would start working on it, probably with a separate issue.

Originally posted by @felixschurk in #3351 (comment)