Require OPENMS_TEST_VERBOSE_SUCCESS env variable set if output of successful test executions should be printed

[timosachsenberg]

Description

Output from successful tests clutter the log. In most cases this information is not needed. Optionally can now be enabled by setting the OPENMS_TEST_VERBOSE_SUCCESS environment variable

Checklist:

• Make sure that you are listed in the AUTHORS file
• Add relevant changes and new features to the CHANGELOG file
• I have commented my code, particularly in hard-to-understand areas
• New and existing unit tests pass locally with my changes
• Updated or added python bindings for changed or new classes. (Tick if no updates were necessary.)

How can I get additional information on failed tests during CI:

If your PR is failing you can check out

• http://cdash.openms.de/index.php?project=OpenMS and look for your PR. If you click in the column that lists the failed tests you will get detailed error messages.
• Or click on the action: e.g., for clang-format linting

Note:

• Once you opened a PR try to minimize the number of pushes to it as every push will trigger CI (automated builds and test) and is rather heavy on our infrastructure (e.g., if several pushes per day are performed).

Advanced commands (admins / reviewer only):

• /rebase will try to rebase the PR on the current develop branch.
• /reformat (experimental) applies the clang-format style changes as additional commit
• setting the label NoJenkins will skip tests for this PR on jenkins (saves resources e.g., on edits that do not affect tests)

[timosachsenberg]

rebuild jenkins

[cbielow]

is this only intended for strings?
And documenting the behaviour would be nice, I think.

[timosachsenberg]

it is for all successful lines as these clutter our CI logs and are automatically truncated

[jpfeuffer]

But I think he has a point. There is also "testRealSimilar" and others.

[timosachsenberg]

ok I will exclude those lines as well.
were is the best place to document this?

[cbielow]

No ideal place comes to mind, but maybe installation instructions, at the very end (Testing your OpenMS/TOPP installation)
?

May I suggest to query getenv only once in the ctor of Classtest and caching the result. Otherwise getenv will be called potentially hundreds of times, each time iterating though potentially hundreds of environment variables.

[timosachsenberg]

Thanks for the input.
I am looking into alternatives now. E.g., if we can use the ctest verbosity level (which is passed to the test in OpenMS) to disable / enable output of successful tests.