Skip to content

docs: clarify behavior of each trace and video mode#41127

Merged
yury-s merged 3 commits into
microsoft:mainfrom
yury-s:docs-trace-modes
Jun 4, 2026
Merged

docs: clarify behavior of each trace and video mode#41127
yury-s merged 3 commits into
microsoft:mainfrom
yury-s:docs-trace-modes

Conversation

@yury-s
Copy link
Copy Markdown
Member

@yury-s yury-s commented Jun 3, 2026

Summary

  • Clarify what each trace and video mode records and keeps, so the difference between retain-on-failure and the other modes is unambiguous (follow-up to removing retain-all-failures in chore(trace): remove retain-all-failures trace and video mode #41123).
  • Add comparison and scenario tables (flaky vs. always-failing) for both options to the recording options guide.
  • Note that retain-on-failure already keeps a failed run's trace/video even when a later retry passes.

yury-s added 2 commits June 3, 2026 13:53
Add a comparison table and a scenario table to the recording options
guide, and tighten the per-mode descriptions so the difference between
retain-on-failure and the other modes is unambiguous.
Mirror the trace mode clarification for the video option: tighten the
per-mode descriptions and add comparison and scenario tables to the
recording options guide.
@yury-s yury-s changed the title docs(trace): clarify behavior of each trace mode docs: clarify behavior of each trace and video mode Jun 3, 2026
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@yury-s
Copy link
Copy Markdown
Member Author

yury-s commented Jun 3, 2026

Rendered section:

Trace modes

The trace option supports several modes that differ in which runs are recorded and which recordings are kept after the test finishes. The initial run of a test is the "first run"; subsequent runs caused by retries are "retries".

Mode Records a trace on Keeps the trace when
'off' never
'on' every run always
'retain-on-failure' every run that run failed
'retain-on-first-failure' first run only the first run failed
'retain-on-failure-and-retries' every run that run failed, or it is a retry
'on-first-retry' first retry only always
'on-all-retries' every retry always

The following table shows which traces are kept in a few common scenarios, assuming retries: 2 is configured:

Mode Passes on first run Fails, then passes on retry Fails on every run
'off'
'on' first run first run + retry all three runs
'retain-on-failure' first run all three runs
'retain-on-first-failure' first run first run
'retain-on-failure-and-retries' first run + retry all three runs
'on-first-retry' first retry first retry
'on-all-retries' first retry both retries

:::note
'retain-on-failure' already keeps the trace of a failed run even when a later retry passes, so a flaky test that recovers still leaves the failing run's trace for debugging.
:::

Comment thread docs/src/test-use-options-js.md Outdated
| `'on-first-retry'` | — | first retry | first retry |
| `'on-all-retries'` | — | first retry | both retries |

:::note
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure this note helps at all. Same for the video note.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, let's drop

| [`property: TestOptions.trace`] | Playwright can produce test traces while running the tests. Later on, you can view the trace and get detailed information about Playwright execution by opening [Trace Viewer](./trace-viewer.md). Options include: `'off'`, `'on'`, `'retain-on-failure'` and `'on-first-retry'` |
| [`property: TestOptions.video`] | Playwright can record [videos](./videos.md) for your tests. Options include: `'off'`, `'on'`, `'retain-on-failure'` and `'on-first-retry'` |

#### Trace modes
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic!

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

Test results for "MCP"

7230 passed, 1103 skipped


Merge workflow run.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

Test results for "tests 1"

1 failed
❌ [webkit-library] › library/client-certificates.spec.ts:346 › browser › should not intercept TLS for origins without a client certificate @webkit-ubuntu-22.04-node20

5 flaky ⚠️ [chromium-library] › library/video.spec.ts:719 › screencast › should work with video+trace `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/popup.spec.ts:261 › should not throw when click closes popup `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-ubuntu-22.04-node20`
⚠️ [webkit-page] › page/page-autowaiting-basic.spec.ts:58 › should await form-get on click `@webkit-ubuntu-22.04-node20`
⚠️ [playwright-test] › ui-mode-trace.spec.ts:189 › should show snapshots for steps `@windows-latest-node20`

39527 passed, 771 skipped


Merge workflow run.

@yury-s yury-s merged commit 08e2d03 into microsoft:main Jun 4, 2026
45 of 46 checks passed
@yury-s yury-s deleted the docs-trace-modes branch June 4, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants