Wrong test window output with custom test harness instead of libtest

[ilslv]

Environment

• IntelliJ Rust plugin version: 0.4.173.4714-221
• Rust toolchain version: 1.62.0-x86_64-apple-darwin
• IDE name and version: CLion 221.5921.27
• Operating system: macOS 12.3.1

Problem description

I'm currently integrating support for --format=json feature to cucumber testing framework. This is done via emulating libtest's json output. While serial test execution works fine, concurrent tests lead to some tests being skipped inside the IDE window.

Steps to reproduce

Serial execution

• Console output:

{"type":"suite","event":"started","test_count":8}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::3: Background Given 1 sec"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::3: Background Given 1 sec"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::14: Given 2 secs"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::14: Given 2 secs"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::15: When 2 secs"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::15: When 2 secs"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::16: Then 2 secs"}
{"type":"test","event":"failed","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::16: Then 2 secs","stdout":"Step panicked. Captured output: Too much!\nWorld(\n    4,\n)"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::3: Background Given 1 sec"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::3: Background Given 1 sec"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::6: Given 1 sec"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::6: Given 1 sec"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::7: When 1 sec"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::7: When 1 sec"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::8: Then unknown"}
{"type":"test","event":"ignored","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::8: Then unknown"}
{"type":"suite","event":"failed","passed":6,"failed":1,"ignored":1,"measured":0,"filtered_out":0,"exec_time":7.034503}

• IDE window:

Concurrent execution:

Execution of same tests, but concurrently, leads to some tests being skipped inside the IDE window. Basically, we output the same test events, but in a different order. cucumber guarantees, that every test case has unique name and started event is outputted before ok/failed/ignored.

• Console output:

{"type":"suite","event":"started","test_count":8}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::3: Background Given 1 sec"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::3: Background Given 1 sec"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::3: Background Given 1 sec"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::6: Given 1 sec"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::3: Background Given 1 sec"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::14: Given 2 secs"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::6: Given 1 sec"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::7: When 1 sec"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::14: Given 2 secs"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::15: When 2 secs"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::7: When 1 sec"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::8: Then unknown"}
{"type":"test","event":"ignored","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::5: Scenario 1 sec::8: Then unknown"}
{"type":"test","event":"ok","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::15: When 2 secs"}
{"type":"test","event":"started","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::16: Then 2 secs"}
{"type":"test","event":"failed","name":"Basic /Users/work/Work/cucumber/tests/features/wait/rule.feature::11: Rule rule::13: Scenario 2 secs::16: Then 2 secs","stdout":"Step panicked. Captured output: Too much!\nWorld(\n    4,\n)"}
{"type":"suite","event":"failed","passed":6,"failed":1,"ignored":1,"measured":0,"filtered_out":0,"exec_time":7.045812}

• IDE window: