Skip to content

v0.26.0

Latest

Choose a tag to compare

@PhenX PhenX released this 19 Aug 22:35
65ce9ba

This release turns Piwi's run history into decisions: run only the tests your change touches, order the flaky ones first, let the platform open a PR when a locator breaks, and watch every step stream live as it happens.

Full diff: v0.25.0…v0.26.0

✨ Highlights

  • 🌿 Branch as a first-class dimension — Baselines, analytics, and history are now keyed on the git branch, making it a queryable, analysis-bearing dimension. The reporter resolves the branch through a CI fallback chain and captures the PR number, so every run knows where it came from.
  • 🎯 Test selections — Build data-driven test subsets from run history, browse them in a new Selections tab, and expose them to agents through MCP tools. piwi select / piwi run resolve a selection, --require-selection can gate a run on one, and selections come with health & drift analytics plus suggestions.
  • 🔍 Impact-from-diffpiwi run impact --base <ref> maps a changed-file set to the tests it affects, so CI can run just the tests your diff can break.
  • ⚡ Fail-fast ordering & balanced shardspiwi run --fail-fast runs the least-reliable tests first to surface failures sooner, and --shard splits a selection into duration-balanced shards.
  • 🩹 Auto-heal broken locators — When a locator breaks, Piwi can automatically open a fix PR with a git-applyable edit — backed by a settings toggle, healing chips, a "PR opened" notification, and retention controls.
  • 📡 Live step streaming — Watch each test's steps stream onto the run page in real time, with per-attempt outcomes shown as attempt chips and distinct statuses for timed-out vs. interrupted runs.
  • 🚦 Flaky-as-failure & "why didn't this run?" — Runs can now fail when flaky tests are detected (PIWI_FAIL_ON_FLAKY_TESTS), and the dashboard explains why tests didn't run, linking the blocking failure that stopped them.
  • 🔗 Public share links — Share an execution or a failure cluster with a public link, no login required.
  • 🔔 Notifications — Global (project-wide) channels, browser push subscriptions, auth-less support, and digest delivery.
  • 🧱 API surface hardening (pre-1.0) — List endpoints now return a consistent { items } envelope, every error carries a machine-readable errorCode with normalized status codes, and query parsing is unified behind clamp-and-400 helpers.
  • 📊 Cleaner charts — Test runs render as per-run bars (dropping the unovis dependency), and flaky tests are no longer double-counted.
  • 🤖 Text-only AI models — Diagnosis now works with models that have no vision: it retries without images and falls back gracefully.

Features

Branches & selections

  • Add branch as a first-class dimension for baselines and analytics (2a13b17), queryable and analysis-bearing across the app (7278f0c, 3d51f25)
  • reporter: resolve branch via a CI fallback chain and capture the PR number (c0c4e3e)
  • Add test selections — data-driven test subsets from run history (f46f251); model, resolver, API and demo (c17aaa2)
  • app: gate on a selection, add selection MCP tools, and a Selections tab (f2af353)
  • app: rank a selection's materialized order for fail-fast (d248692)
  • app: selection suggestions, an ids predicate, and duration-balanced shards (c54bcfa)
  • app: selection health & drift analytics, fail-fast ordering (4221b09, f84f803)
  • reporter: add piwi select / piwi run and resolveSelection (f4dfc7c)
  • reporter: gate --require-selection and the run-the-right-tests skill (7eeaa31)
  • reporter: piwi select/run --shard for duration-balanced shards (7d97cae)
  • db: persist run-level shardIndex (551d2fc)

Impact & fail-fast

  • app: impact-from-diff — map a changed-file set to the tests it affects (62feeb5)
  • piwi run impact --base — run the tests your diff affects (091a828, c6faf2e)
  • reporter: piwi run --fail-fast — least-reliable tests first (79f1a64)

Auto-heal broken locators

  • app: auto-heal broken locators by opening a fix PR when a locator breaks (4ef18c2, ab4e09b)
  • app: hand back a git-applyable edit for healed locators (e39f98e)
  • app: finish auto-heal — settings UI, PR-opened notification, healing chips, retention (e681bde)

Live runs

  • app: live step streaming, attempt chips, fail-on-flaky, and distinct run statuses (fb86633)
  • app: stream live per-test steps to the run page (ed407a3) and show them inline on each running test row (018be70)
  • app: record per-attempt outcomes and show attempt chips (ead8c28)
  • app: explain why tests did not run and link blocking failures (142b13d, 9dbe7f4)
  • reporter: fail the run when flaky tests are detected (84fc7da)
  • reporter: keep timed-out and interrupted runs distinct (253d83e)

Sharing, notifications & AI

  • app: public share links for executions and failure clusters (1ec2a90, 7118f63)
  • notifications: auth-less support, global subscriptions, digests, delivery fixes (803ac3c); global channels and browser subscriptions (bfb7182)
  • ai: support text-only models and live diagnosis E2E tests (ad40d16)
  • auth: harden auth rate limiting and polish the first-admin setup flow (2459da9, 0f8f686)

API surface & UI

  • app: return a consistent { items } envelope from list routes — projects (368d262), project/run listings (90a0e5c), content CRUD (afaba0e), settings/user CRUD (6c0b364), flaky-tests & clusters (8f95263), performance & slow-tests (e52176b), cluster commits (bff5ece), and the low-fanout routes (0848327)
  • app: add a machine-readable errorCode to every API error (04351ce)
  • app: unify query parsing with clamp-and-400 helpers (d87df0e)
  • ui: chart test runs as per-run bars and drop the unovis dependency (b536c9e, e3e8bd9)
  • demo: seed did-not-run reasons and a serial-group cascade (7785502)
  • docs: make the split hero a scene and recapture it from the current app (4288b38)

Bug Fixes

  • ai: retry without images when the model has no vision (0499501)
  • app: stop double-counting flaky tests in the test-runs bar graph (bee0d16, 20699d3)
  • app: stop the failure-evidence count showing NaN (8575223) and count attempts in the summary-tile fallback (5aa1384)
  • app: store one canonical per-case timeout status and match both spellings everywhere (0280015)
  • app: let the recorded content type decide evidence classification (75f3f60)
  • app: drain the event batch past a worker-less step event (2fa54a1)
  • app: render the execution History tab from the SSR payload (66ae5ac, 62eb745)
  • app: read projects/menu items on the auto-heal page so reload survives (37f42b1)
  • app: read runId from the desktop import response (2abd141)
  • app: normalize error status codes across the API (6e12373) and settle pre-1.0 wire/API-surface inconsistencies from a full audit (9bcd6ec, 9eecaba)
  • app: serve icons from local bundles and bake head metadata into the demo shell (0afa861, 7d71ae5)
  • app: lower the dashboard Node floor to 22 (95686ae, 3b4d880)
  • reporter: honor PIWI_FAIL_ON_FLAKY_TESTS in wrapConfig (79d97ca)
  • reporter: stream step events queued before the run starts (7221370)
  • db: map Postgres integer-boolean columns to booleans at the Drizzle layer (f53d54b)
  • settings: extract items from the projects-menu API response (1c25d68)
  • ci: answer 404 before writing the demo server headers (8a98e2d) and bust the poisoned Nuxt build cache for the docs-URL change (1a7ccc0)
  • docs: add favicon, sitemap and robots.txt and de-linkify config examples (17fc828); commit CNAME so the Pages deploy keeps the piwitests.dev domain (0d8fe88, 1da1fb7)
Demo / mock-server parity fixes — bringing the in-browser demo up to the real server's behavior
  • demo: align the ingest pipeline with the server persist (2b5c7dc)
  • demo: enforce project scope on entity endpoints (991212a)
  • demo: surface 4xx status codes from demo handlers (b1be9d6)
  • demo: shape user creation like the server (94a94a2) and validate quarantine requests (d1186f4), tags/markers/subscriptions (9555e6c) like the server
  • demo: forward flaky-tests filters (918d7d2) and performance/run-limit query params (8ef8d5b) like the server
  • demo: honor timeout thresholds in timeout opportunities (a921512) and send the test timeout on simulated completions (52c6e7a)
  • demo: match the admin cleanup response shape (76024a0), the AI-usage response and compute hasScmToken (78ffb44), and the diagnosis context endpoint shapes (130de38)
  • demo: return stored diagnoses unless force is set (4a230d4)
  • demo: strip scmToken from project responses (ca97e56)
  • demo: accept expiresAt and generate API keys with real randomness (8cbc007)
  • demo: carry tags, test meta and AI usage on simulated runs (46c6912)
  • demo: persist setup steps and emit didNotRunTests on finish (c06524e); materialize did-not-run cases for interrupted runs (3d34c83)
  • demo: persist shard tokens and drop them on shard finish (81eed14)
  • demo: tolerate re-begin on running runs and revive interrupted ones (80b4b3e)
  • demo: mirror run-label patching and wasted-waits resolution (c1f1972)
  • demo: point never-fired cluster spans at the correct runs (843404c) and recompute cluster counts when runs are deleted (71e1431)
  • demo: seed app settings so setup status reports AI active (2dd06c8), cluster merge suggestions (df98038), and the partial quarantine streak part-way (d5ecf3e)
  • demo: surface constraint violations in the dev-seed loader (143d498) and resolve dev-seed/screenshot deps from the workspace (2370948)
  • demo: write the generated visual diff into the seed output dir (2ea51c8)
  • demo: drop the vestigial auth session route (481bcb9) and the unused auth-stub constant (70c65ef)