Add error summary at the end of ament test

[gerkey]

As an occasional contributor, I'm frequently tripped up when the last output from ament test is happy and I forget to check for earlier failures. I know that I can run ament test_results to get the summary, but I guess that I just can't get over the expectation that ament test should provide me some sort of summary or at least a red light vs. green light at then of its run (at present, a test failure somewhere in the run doesn't even produce a non-zero return code at the end).

E.g., ros2/cli_tools#6 is a case where that mistake on my part lead @mikaelarguedas and I to spend time investigating a non-existent problem.

Proposal: modify ament test to give the ament test_results output when it's done. If compute overhead for collating the results is a concern, then an option could be added to disable the summary (but the default should be to enable it). If this sounds like a good change, I'm happy to provide a PR.

[dirk-thomas]

That sounds like a good improvement to me. The summary should probably only show the results from the packages processed by the test verb.

[gerkey]

Though not strictly related to the idea of printing the test summary, I'm also looking at making ament test return non-zero when a test fails. I found this chunk of code:

https://github.com/ament/ament_tools/blob/master/ament_tools/verbs/test/cli.py#L50-L63

which seems intended to do exactly that. But it always returns zero, apparently because of this chunk of code:

https://github.com/ament/ament_tools/blob/master/ament_tools/verbs/test_pkg/cli.py#L76-L89

which explicitly discards the return code from testing an individual package.

I don't know the history here. @dirk-thomas, should I try to (re-)enable proper return code handling, or not bother?

[dirk-thomas]

Please see ros-infrastructure/ros_buildfarm#367 (comment) for a rational why ament test intentionally returns zero if the tests have been run even if some tests failed.

[dirk-thomas]

Closing since the repo is archived.