Incorrect test summary when testcase name includes colon (e.g. in different namespaces)

[5p4k]

What kind of issue is this?

• PlatformIO Core.

---

Configuration

Operating system: Linux Mint 21.1 Vera
PlatformIO Version: PlatformIO Core, version 6.1.5

Description of problem

Testcases which have a colon in their name are not correctly counted in the summary of pio test. This affects in particular all testcases functions residing in a different namespace. The summary always declares "0 test cases: 0 succeeded".

Steps to Reproduce

Create a project with a unique test located in a different namespace, e.g. test::dummy (see below) invoked with a fully qualified namespace.
Alternatively, it's enough to test the following line:

test/test_main.cpp:5:test::dummy:FAIL: Expression Evaluated To FALSE

against this TESTCASE_PARSE_RE regex.

Actual Results

test/test_main.cpp:5:test::dummy:FAIL: Expression Evaluated To FALSE

-----------------------
1 Tests 1 Failures 0 Ignored 
FAIL

---------------- esp32:* [PASSED] Took 40.03 seconds ----------------

============================== SUMMARY ==============================
Environment    Test    Status    Duration
-------------  ------  --------  ------------
esp32          *       SKIPPED   00:00:40.027
============= 0 test cases: 0 succeeded in 00:00:40.027 =============

Expected Results

---------------- esp32:* [FAILED] Took 11.78 seconds ----------------

============================== SUMMARY ==============================
Environment    Test    Status    Duration
-------------  ------  --------  ------------
esp32          *       FAILED    00:00:11.781

______________________________ esp32:* ______________________________
test/test_main.cpp:5:test::dummy:FAIL: Expression Evaluated To FALSE

======== 1 test cases: 1 failed, 0 succeeded in 00:00:11.781 ========

If problems with PlatformIO Build System:

The content of platformio.ini:

[env:esp32]
platform=espressif32
framework=espidf
board=esp32dev

Source file to reproduce issue: test/test_main.cpp

#include <unity.h>

namespace test {
    void dummy() {
        TEST_ASSERT(false);
    }
}

extern "C" void app_main() {
    UNITY_BEGIN();
    RUN_TEST(test::dummy);
    UNITY_END();
}

[5p4k]

One possible fix would be to match as the test name everything before PASS|IGNORE|FAIL. In that case it suffices to change TESTCASE_PARSE_RE to the following:

^(?P<source_file>[^:]+):(?P<source_line>\d+):(?P<name>.+?):(?P<status>PASS|IGNORE|FAIL)(:\s*(?P<message>.+)$)?$

This will match the whole line, so it might be more resistant to creative test case names.

On the other hand, perhaps it would be better to change to ; as a separator. A semicolon is definitely not going to be part of any method name.

[ivankravets]

Thanks for the report. Please re-test with pio upgrade --dev.

[5p4k]

@ivankravets Thanks for the fix -- the dev version works!