Skip to content

v0.56.0

Choose a tag to compare

@PhenX PhenX released this 19 Oct 18:58
9e56f3d

This release closes the findings of a security audit and a code audit across the dashboard, the reporter, the desktop app, the editors and Piwi Picker: sign-in, tokens, uploads and outbound requests are bounded and checked, concurrent shards merge into one run, and healing suggests only locators related to the failing one.

Full diff: v0.55.0…v0.56.0

✨ Highlights

  • 🔒 Audit findings closed — The high-severity findings of the security audit and the medium-severity findings of the code audit are fixed, along with the remaining ones, dead code and unused API routes; the dashboard refuses to be framed by another site, and link unfurls go through safeFetch.
  • 🔑 Sign-in and tokens — Passwords are hashed with scrypt at N=2^17 and rehashed on sign-in, sign-out revokes every session of the account, the login throttle counts concurrent attempts, and run stream and Jira webhook tokens are stored as digests.
  • 📦 Bounded uploads and traces — Trace archive inflation is bounded and digests are verified, upload caps count the bytes read, reports are extracted into private folders, and traces are stored under the hash the server computes.
  • 🧩 Shards and deliveries — Shards that arrive together merge into one run that settles once, and a batch the reporter sends again is stored once.
  • 🩹 Healing suggestions — Locator healing and its ARIA fallback suggest only elements related to the failing locator, never the failing locator's own element.
  • 🖥️ Desktop hardening — The desktop app trusts only its spawned sidecar, creates its key, token and discovery file owner-only, checks the bundled Node download against the release's SHA-256 list, and gives each reproduce, bisect and lab job its own worktree.
  • 🤖 AI providers — The Claude CLI runs with its tools off, for one turn, in an allowlisted environment, and AI step resolutions are bounded and rate-limited per caller.

Features

  • ui: disable a Jira connection's inbound webhook and sync the tracker statuses from settings (3364860)
  • demo: add a relabeled field whose locator fix is copied, not applied (b306936)

Performance Improvements

  • app: read each run's eligibility once for the quarantine release streaks (ede13ba)
  • app: set regression and flaky signals with one update per batch of 500 executions (a564454)
  • db: restore the test_run_id index of test_runs_cases (84ce9ca)

Bug Fixes

Audit & web security

  • app: close the high-severity findings of the security audit (ed6ca78)
  • app: close the medium-severity findings of the code audit (bc7e9b1)
  • app: close the remaining audit findings, dead code and unused API routes (105e127)
  • app: refuse to be framed by another site with X-Frame-Options and frame-ancestors (25b66a5)
  • app: unfurl links through safeFetch and connect only to the addresses it checked (cff9680)
  • app: build SCM links only from http and https repository URLs (5e638f5)
  • app: read every project's SCM repository from a pinned URL, never from run metadata (c7413e2)
  • app: keep run stream tokens out of API responses and store them as digests (7b50646)
  • app: keep a Jira webhook token as its digest and look the connection up by it (1f4ee07)

Sign-in & permissions

  • auth: hash passwords with scrypt at N=2^17, encode the cost, and rehash on sign-in (e78fe06)
  • auth: revoke every session of the account on sign-out (8b5de6a)
  • auth: count concurrent login attempts against the throttle before checking the password (7ebe557)
  • auth: count OAuth and extension lookup misses before the lookup runs (482551d)
  • auth: ask for PIWI_SETUP_TOKEN on the first-admin form and generate it in one-click deploys (e9e0654)
  • auth: keep first-admin setup open while no administrator exists (0db18e7)
  • auth: keep the last administrator under concurrent demotions on PostgreSQL (eb9cdbd)
  • auth: create a ticket's share link only when the issue is filed, with share:create (edc01cb)
  • auth: store gate verdicts and mark incidents only with run:submit or run:control (cd1c291)

Uploads, storage & traces

  • app: bound trace archive inflation and verify trace and resource digests (21d53ef)
  • app: count upload caps on the bytes read and check case-file tokens first (367b874)
  • app: extract uploaded reports a few files at a time into private folders (c87e6fa)
  • app: serve only report archives from inside a .gz, read as a bounded stream (19eb66e)
  • app: validate an uploaded run and create it before storing its reports (a297539)
  • app: store every uploaded trace in the blob store under the hash the server computes (8df29bd)
  • app: delete stored traces no execution links in the nightly sweep (6980bd8)
  • app: keep attachments of one execution that share a file name (2728fbe)
  • app: answer local storage exists() for regular files only (bad3546)

Ingestion & data integrity

  • app: merge shards that arrive together into one run and settle it once (10b4993)
  • app: merge concurrent shards and deliveries, and fix data-integrity bugs (f29362b)
  • app: store a batch delivery once when the reporter sends it again (4c40033)
  • app: open a run with more executions than the database binds parameters (e287d70)
  • app: keep a cluster's last seen run when an older run is imported (d37cc4e)
  • app: retry an outbox row after 1, 5, 15, 60 and 240 minutes, as documented (301e895)
  • app: parse error texts in linear time, whatever their blank lines (a3098db)
  • db: store a cluster's time to resolution as bigint on PostgreSQL (3780c8a)

Locator healing & generated code

  • app: suggest only locators related to the failing one in locator healing (e91a3a2)
  • app: suggest only related elements in the ARIA healing fallback (208515c)
  • app: keep the failing locator's own element out of healing suggestions (6e6e699)
  • app: quote every generated locator string and edit tests only with canonical locators (1a33c6b)
  • extension: quote the strings of generated code with core's quoteJsString (e6b2fca)

Analytics, flaky tests & sharding

  • app: leave bisect, reproduction and incident runs out of flaked-again and quarantine streaks (63ca8d9)
  • app: rank the Analytics flakiest tests over the whole scope before the top-50 cut (b7565e0)
  • app: read one default branch in Analytics, the project tabs and the flaky leaderboard (f977ec8)
  • app: read fixed-before candidates with no landed fix last on PostgreSQL too (72f259a)
  • app: recommend timeouts from finished runs the flakiness use reads (b51a6e8)
  • app: weigh a test with no recorded duration at the median when balancing shards (426dd61)

AI

  • ai: run the Claude CLI with its tools off, one turn and an allowlisted environment (c76b841)
  • ai: send the Claude CLI a text-only diagnosis prompt and list no image modality for it (6039245)
  • ai: bound every field of an AI step resolution and rate-limit each caller (4b4f188)
  • ai: read a Groq stream's token usage from x_groq.usage (759e6df)

Notifications

  • notifications: escape every value a Slack message interpolates (806f4a4)
  • notifications: re-check the caller and reload their scope every minute on the event stream (fa11a79)
  • notifications: send email only to verified addresses and rate-limit saved-channel tests (1e1535b)
  • notifications: show a global webhook's origin, not its full URL, to users who do not own it (ab9086f)

Reporter

  • reporter: never fail a green run from onEnd, and keep Playwright's terminal output (97c4d83)
  • reporter: drop the credentials a git remote URL carries before sending it (cadf902)
  • reporter: mask the credentials a page's URL fragment carries before it is uploaded (ba553b3)
  • reporter: keep the crash-recovery file per dashboard, owner-only, until its replay succeeds (33c53a0)
  • reporter: name test files relative to the Playwright config's folder, not the working directory (8b46231)
  • reporter: count tests that share a title and a line once each (7981dcd)
  • reporter: read a failure's stack frames in linear time (58a77bf)
  • reporter: start Playwright and package managers from the piwi CLI on Windows too (148e7fa)
  • reporter: bound a Nitro request's logs and keep those of its server-side $fetch calls (6efa123)
  • reporter: list only the routes a Nitro route handler matched in the route manifest (309e013)
  • reporter: give each .NET coverage request id a random secret (82efa04)
  • reporter: guard the .NET capture buffer the parallel tasks of a request share (efb74e4)
  • reporter: record a .NET request's leftover work at run level once its coverage is collected (8ef99d4)

Desktop

  • desktop: trust only the spawned sidecar and allow only known IDE launchers (7ca9de4)
  • desktop: create the key, the token and the discovery file owner-only from the start (9beb8a0)
  • desktop: compare the desktop session token in constant time (626cfba)
  • desktop: bundle Node 24.22.0 and check its download against the release's SHA-256 list (2337b13)
  • desktop: give each reproduce, bisect and lab job its own worktree (12a3e69)
  • desktop: stop local runs with their webServer on quit and remove their repro specs (2ab8363)
  • desktop: keep waiting for a slow server, and show its state on the splash and in the dashboard (54b9cba)
  • desktop: let the dashboard window load its pages from devUrl under tauri dev (3402b48)

Piwi Picker & editors

  • extension: send the desktop token only to the app that proves it holds it (26ce69d)
  • extension: pair the desktop app on 127.0.0.1 whatever loopback name was typed (6aa87bd)
  • extension: give a replay started from an extension page to the first tab that asks (afc8d72)
  • extension: put a no-break space before the colon of the French unproven-app message (328e85f)
  • ide: build run commands in the editor service and wait for trust in a JetBrains project (48d7204)
  • ide: escape the instance's text in a test's JetBrains gutter tooltip (abe8b6d), and in the JetBrains status bar and recording tooltips (f9f9aa8)
  • ide: keep downloaded traces in a private per-user folder, and fetch a pruned one again (6f7fb56)

Dashboard

  • ui: keep the API console's bearer token for the tab session and clear it on sign-out (a898eba)
  • ui: let an API key expire at the end of the chosen day in the viewer's time zone (51d84da)
  • ui: show why a streaming diagnosis could not start, and follow one already running (a1a4bfb)
  • ui: reload an execution's running diagnosis on Refresh and poll it until it lands (5a40ba6)
  • ui: fetch the trace snapshots and the healing of the execution being shown (7d34193)
  • ui: show the network drawer's body of the request it is open on (a2a8a03)
  • ui: keep only the latest answer of the performance tab, diagnosis context and search (07c9b25)
  • ui: apply an Analytics drill-down once, not on every change of the project page's URL (b86429f)
  • ui: keep the inbox new queue's cut for the whole browser session (9994e65)
  • ui: copy a failure, a cluster or a diagnosis as plain text where rich copy is unavailable (28209fe)
  • ui: let ctrl and cmd-click open test and run links, with the app's base path (76c7a6b)
  • ui: link view page source to the matched page's file under apps/application (7265023)

Deploys

  • app: give a Coolify deploy a site URL with its scheme (918ad32)
  • app: take a Fly.io instance's site URL from its app name (9396ab3)
  • app: turn Nitro instrumentation off when NODE_ENV is not set (e8c7479)
Demo, CI & dependencies — the in-browser demo, the release pipeline and the Docker image
  • demo: disable open trace for a run imported into the demo, and say why (c6f7f41)
  • demo: keep notification links and icon under the demo's base and open no event stream there (bae1863)
  • demo: keep the outbound OpenAPI fetch out of the demo service worker (90f5bb8)
  • ci: keep publishing and signing secrets away from pull-request runs of the release (ef53a77)
  • ci: start the server image before publishing it, and move latest only to the newest release (3066076)
  • ci: build the dashboard without the Nuxt build cache on CI (3d3559a)
  • ci: read the tool configs as a local run does when running knip (27c65fa)
  • deps: bump nodemailer to 10.1.7 and sharp to 0.35.5 (ea7fdf4)
  • deps: install the locked sharp and libsql versions in the Docker image (8118246)