[Feature]: Show the Test Annotation in the Merged Report Even Though the Test are Being Skipped

[aldys33]

🚀 Feature Request

I just notice that if my test got skipped, the annotation is missing from the report. This only happen when I use merge-report, which is I'm utilizing sharding.

---

I think it's better to have annotation still got attached in the report, despite the test being success, failed, or even skipped as well for the merged report.

Example

Condition when the test got skipped

Condition when the test is failed/success

Motivation

I believe there are user that utilize annotation for several purpose. Like for me, it is a place to store Test ID that connects with the AIO. So it's important to preserve the annoation in the report regardless of how the test execution is going

[dgozman]

@aldys33 Could you please share a repro exhibiting the issue that we can run locally?

[aldys33]

yeah, will create a repo to demonstrate on that one

[aldys33]

@dgozman Please refer to the following repo https://github.com/aldys33/Playwright.Annotation.Demo/actions
I did create a sample run with and without sharding. Please note that when using sharding, the annotation is not showing

[dgozman]

@aldys33 Thank you for the repro, I can see the bug now.

---

Investigation notes: a test is skipped in one of the shards and there is no onTestEnd event in the blob reporter that contains annotations.

[aldys33]

Thanks, @dgozman