Print a list of failed tests at the end of test summary

[eskultety]

Is your feature request related to a problem? Please describe.
Consider the following execution

$ avocado run /bin/false /bin/true /bin/false
JOB ID      : latest
JOB LOG     : /home/gitlab-runner/avocado/job-results/job-2021-12-17T16.52-9b59568/job.log
SPAWNER     : process

Test ID      End Time            Run Time Status
2-/bin/true  1970/01/01 00:00:49 0.010845 PASS
3-/bin/false 1970/01/01 00:00:49 0.010837 FAIL
1-/bin/false 1970/01/01 00:00:49 0.011155 FAIL
RESULTS     : PASS 1 | ERROR 0 | FAIL 2 | SKIP 0 |WARN 0 | INTERRUPT 0 | CANCEL 0

From an output like ^this it is very simple to tell which tests passed and which failed, but do it for hundreds of tests and then one has to take scrollback and look for a needle in the haystack to find which tests have failed.

Describe the solution you'd like
Since failed tests is all everyone cares about when looking at the results, it would be both very convenient and fast to just look at a summary which might look like this:

...
RESULTS     : PASS 1 | ERROR 0 | FAIL 2 | SKIP 0 |WARN 0 | INTERRUPT 0 | CANCEL 0
Failed tests:
   3-/bin/false
   1-/bin/false

Describe alternatives you've considered

1. Having the following the config file:

[job.output.testlogs]
statuses = ['FAIL', 'ERROR']

Avocado will tell me what the logfile for a given failed tests contained which can be very verbose as the developer is primarily interested in the list of failed tests so that they can go and investigate on their own not mentioning the fact the the log files may not contain all the necessary data for an analysis.

2. grepping jobs

$ avocado jobs show | grep FAIL

This could be a somewhat satisfactory workaround, but you have to run 2 commands (run, jobs) which will produce 2 summaries with the latter being slightly confusing (naturally because of using grep):

$ avocado run /bin/false /bin/true /bin/false
...
RESULTS    : PASS 1 | ERROR 0 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

$ avocado jobs show | grep FAIL
1-/bin/false 1970/01/01 00:13:55 0.010567 FAIL
3-/bin/false 1970/01/01 00:13:55 0.010627 FAIL
RESULTS     : PASS 1 | ERROR 0 | FAIL 2 | SKIP 0 |WARN 0 | INTERRUPT 0 | CANCEL 0

I think we can definitely do better (by just using a single command) and just place the list of failed test at the end of the existing job summary.

[beraldoleal]

Since we are in Feature Freeze this week, moving this to the next sprint.

[beraldoleal]

Hi @eskultety let me know what you think of #5249 for this feature.

[ozer550]

Hi there, I would like to help with this issue, The avocado project is very overwhelming for me and I would love to help in any way possible. Can someone guide me?

[richtja]

Hi @ozer550, thank you for your interest in Avocado and welcome to the Avocado community. I think this issue is already solved by @beraldoleal in #5249. But you can pick some of our low-hanging-fruit which are good for new contributors to get familiar with Avocado. Also, please read our Contributor’s Guide to get more information about contribution to Avocado project. If you need any help, don't hesitated to ask on our communication channels.

[ozer550]

Thanks for your guidance @richtja I would like to start with PR #5219.