Skip to content

v2.0.5 — A Quarantine Convention for Flaky Tests

Latest

Choose a tag to compare

@idavidov13 idavidov13 released this 17 Nov 06:24
2664b16

Rules layer

  • A flaky test that cannot be fixed now is quarantined, never silently skipped. The agent investigates, then proposes a declaration-form test.fixme with a quarantine annotation (owner=@handle; until=YYYY-MM-DD); a human approves it and names the owner and the expiry. The procedure is debugging step 7 and references/quarantine.md.
  • ESLint enforces it for every spec under tests/ (quarantine/annotation, lint/quarantine.mjs): a skip passes only with the annotation or, for a contract mismatch, test.skip with // FIXME: <ticket-url> on the line above. A bare skip, the in-body forms (test.skip(true, ...), testInfo.skip()) and test.step.skip() fail npm run lint. The expiry date is checked for form, not for having passed.

Install: npm create agentic-playwright@latest . -- --demo

Full changelog: v2.0.4...v2.0.5