[Feature]: trace: retain-all-failures

[olivierbeaulieu]

🚀 Feature Request

We use test traces to debug flaky tests. However, the current trace options do not allow me to capture traces for a test that fails on the first try, and then passes on retry without using the on option, which generates way more traces than needed.

What I want is essentially retain-on-failure, but without the last part of its description - "When test run passes, remove the recorded trace.".

I want all have all traces of failed tests, regardless of whether they passed on retry, and want no traces for successful attempts.

Example

playwright.config.ts would have { use: { trace: 'retain-all-failures' } }

Motivation

It will make it easier to address flaky tests.

[Nav-2d]

I think for your use case, retain-on-failure will work.

'retain-on-failure': Record trace for each test. When test run passes, remove the recorded trace.

It means if the test passes on retry, it will remove the passed trace file but not remove the failed first attempt.

What issue are you facing when you use retain-on-failure?