sanitycheck in shippable states faillure but reports 0 errors

[b0661]

I get the failure report below while #5799 is tested in shippable.
I don´t understand why

"ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)"

is a failure. Is this message hiding the "real" error message?

from https://app.shippable.com/github/zephyrproject-rtos/zephyr/runs/12571/1/console

unit_testing tests/unit/scripts/extract_dts_includes/test FAILED: failed
----------sanity-out/unit_testing/tests/unit/scripts/extract_dts_includes/test/handler.log----------
==22062== Memcheck, a memory error detector
==22062== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==22062== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==22062== Command: /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/sanity-out/unit_testing/tests/unit/scripts/extract_dts_includes/test/testbinary
==22062==
==22062==
==22062== HEAP SUMMARY:
==22062== in use at exit: 0 bytes in 0 blocks
==22062== total heap usage: 5 allocs, 5 frees, 5,997 bytes allocated
==22062==
==22062== All heap blocks were freed -- no leaks are possible
==22062==
==22062== For counts of detected and suppressed errors, rerun with: -v
==22062== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
----------sanity-out/unit_testing/tests/unit/scripts/extract_dts_includes/test/handler.log----------

I also got this https://app.shippable.com/github/zephyrproject-rtos/zephyr/runs/12591/1/tests in #6761, which is a cut-out PR #5799

Sanitycheck / unit_testing:tests/unit/scripts/extract_dts_includes/test / unit_testing/tests/unit/scripts/extract_dts_includes/test

failure: handler_crash

Thx
Bobby

[b0661]

Valgrind message was hiding the "real" error message. Seems to be fixed now.

Thx
Bobby

[agross-oss]

What exactly was the fix? I don't see changes here.

[b0661]

What exactly was the fix?

No fix from my side - just an assumption because I could see the error message in the console output when testing. I had a programming error in a python script.

[agross-oss]

Ah ok. This needs to be more transparent. This has happened in the past and was hard to track down due to failures that required a ton of digging to find the culprit.

[b0661]

I`m confused:
test passes in PR #5799 but fails in the cut-out PR #6761 with "ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)"

Local run with "./scripts/sanitycheck -C -R -v -t extract_dts_includes"
passes in both PRs ( #5799 and #6761 ). I removed pychache in between the tests of the different branches.

See https://app.shippable.com/github/zephyrproject-rtos/zephyr/runs/13023/1/console for the #6761 test result.

[b0661]

Retested #6761 - no faults anymore.

I spotted that there is a timeout value given for the test and the matrix build was extended from 4 to 5 in between.

[nashif]

can't reproduce