Catch flaky Playwright tests before they gaslight your CI.
Parallel isolated retries · Wilson stats · evidence packs · optional AI (redacted only)
One command proves the flake (local demo — no keys, no DB, no cloud):
git clone https://github.com/KhyFee/Deflake.git
cd Deflake && npm install && npm run demo
# exit 2 = flake detected → open .deflake/runs/| Pain | What people usually do | What Deflake does |
|---|---|---|
| CI fails once, passes on re-run | “Re-run” and hope | N isolated attempts in parallel |
| “Is it flaky or broken?” | Gut feel | Wilson confidence + stable/flake/fail buckets |
| Flakes die with zero context | Screenshot somewhere | Markdown / HTML / JUnit / SARIF evidence packs |
| AI fixes that invent secrets | Paste full traces | AI opt-in only, redacted evidence, never auto-patches |
Built for Playwright teams who are tired of green-on-retry green.
git clone https://github.com/KhyFee/Deflake.git
cd Deflake
npm install
npm run demoExpect exit code 2 when the intentional flaky suite is detected. Artifacts land under .deflake/runs/.
Zero-config: works without Python, AI keys, or a database. Those only enhance output when present.
Packages: monorepo packages are
@khyfee/deflake(+ core / reporters).
GitHub Packages (npm): configured — needs a token withwrite:packagesonce, then:# .npmrc @khyfee:registry=https://npm.pkg.github.com //npm.pkg.github.com/:_authToken=YOUR_GITHUB_TOKEN npm i @khyfee/deflakenpmjs.org global
npx @khyfee/deflakestill pending public registry publish.
From the monorepo after npm install && npm run build (or wire your path to the CLI):
# in your app with Playwright tests
npx tsx path/to/Deflake/packages/cli/src/cli.ts init
npx tsx path/to/Deflake/packages/cli/src/cli.ts doctor
npx tsx path/to/Deflake/packages/cli/src/cli.ts run --grep "checkout" --attempts 10Workspace package name after build: @khyfee/deflake (deflake bin in packages/cli).
| Code | Meaning |
|---|---|
0 |
Stable pass |
1 |
Stable (deterministic) fail |
2 |
Flake detected (or inconclusive with --fail-on-flake) |
3 |
Config / runtime error |
Wire demo or run into GitHub Actions and treat 2 as the signal your suite is lying.
| Command | Purpose |
|---|---|
init |
Write deflake.config.json |
check / doctor |
Environment diagnostics |
list |
Discover tests |
run |
Parallel attempts → summary + artifacts |
report / compare |
Regenerate or diff runs |
upload |
Idempotent dashboard ingest |
demo |
Seeded intentional flake fixture |
- CLI workers launch isolated Playwright children
- TypeScript aggregation + reporters (md / html / junit / sarif / github summary)
- Optional Python offline triage
- Optional Next.js dashboard — stores signed results; never runs browsers
Privacy defaults:
- Argv arrays only (no shell string execution)
- Secrets redacted before disk / dashboard / AI
- AI requires
DEFLAKE_AI=1and never auto-applies patches - Supabase RLS when you turn the dashboard on
npm install
npm run build
npm run dev -w @deflake/webOpen /demo for seeded UI. POST /api/v1/runs accepts Bearer project tokens.
| Path | Role |
|---|---|
packages/cli |
@khyfee/deflake |
packages/core |
Pool, stats, fingerprints, rules |
packages/reporters |
md / html / junit / sarif / github |
packages/triager |
Optional Python offline triage |
apps/web |
Next.js dashboard |
fixtures/* |
Intentional flaky + stable suites |
supabase/ |
Auth / RLS schema |
docker/ |
Playwright + triager images |
- Teams fighting flakes that only show up under load or rare races
- OSS maintainers who want evidence-backed triage before blaming the contributor
- Interview / portfolio demos: one command that proves intermittent failure math
Not for: reinventing Playwright itself, or “just increase retries: 3 forever” without understanding.
- Publish
@khyfee/deflaketo npm (npx deflake demozero-clone) - One-click GitHub Action wrapper
- Public demo video / GIF of flaky-suite detection
- More fixture patterns (timing, order, network)
Open an issue if you want one of these first.
See CONTRIBUTING.md. Default branch is main. PRs with a failing/flake fixture + a green path are favorites.
MIT © KhyFee
If Deflake saves you from a flaky CI rabbit hole, star the repo so other teams can find it — and open an issue with the flake that still hurts.