v0.15.0

Added

• A CI-onboarding skill, testsprite-ci. agent install (and setup) now installs a third skill by default, which walks a coding agent through putting TestSprite on every pull request: how the repo deploys previews (platform-deployed → on: deployment_status, deployed in the workflow → on: pull_request, read from the repo and gh api …/deployments), which environment the gate signs in with, ci init github plus the placeholder edit, the TESTSPRITE_API_KEY secret, and proving the gate on one real, non-draft pull request — with the pitfalls that cost real time (draft and conflicting pull requests never run, shape A is per commit, cancelling the job does not stop the execution, a shell-exported TESTSPRITE_API_URL beats --profile). --skill testsprite-ci installs it alone; agent list/agent status show it. The docs gain complete shape-A and shape-B workflow examples.
• ci run: the CI gate, in one command. testsprite ci run --project <id> --env <name> [--target-url <url>] runs the project's tests, waits, and decides the exit code with CI defaults: failed/blocked tests exit 1, a timed-out run 7, missing test ids 4, nothing dispatched 5 (--allow-empty accepts an empty selection, never a refused batch), and everything that did not dispatch shares one skipped count that warns unless --fail-on-skipped. The JSON envelope, --summary-file, the GitHub job summary, annotations and GITHUB_OUTPUT step outputs all come from one summary, written before the exit code. Opt-in JUnit (--report junit) carries the same rows, skipped tests included.
• A CI run is one execution, like a GitHub App run. ci run asks the backend to seed every selected test into one execution and pace dispatch itself (admission: 'all'), so a CI run is one row in the Portal instead of a first batch plus a rate-deferred retry. A backend that cannot honor this is refused with exit 7 (admission-not-honored), never silently cut to the per-key budget. Needs backend v0.21.0.
• In a pull request, ci run tests that pull request's preview the way the GitHub App does. Under GitHub Actions the pull request is read from the job (a deployment_status event is looked up by commit with GITHUB_TOKEN); with --env X --target-url U the run happens in a temporary clone of X at U named github-pr-<N>, is stamped with the PR so the Portal links back to it, and the environment is released when the run finishes. A recorded-session (SSO) template is refused before charging. Draft pull requests and deployments with no open pull request are skipped with a ::notice::, zero-valued step outputs and exit 0. ci run reads only the deployment's commit, never its state or environment — the workflow's if: selects which deployment_status events to gate (see the documentation). Outside a pull request the run keeps test run's semantics.
• ci run --env <unknown> is refused before anything dispatches, listing the project's environment names — even with --allow-empty, so a typo can never be a permanently green check. The check fails closed: an environment list that cannot be read is that error (bad key → exit 3, backend down → 10, interrupted → the signal), never a run that dispatches unchecked; a backend with no environment surface for the project (V2, or an older route-less backend) leaves the name to the batch's own validation.
• --priority accepts high, medium and low (any case) alongside p0..p3, in test create, test update, plan files and create-batch. They map to p0, p1 and p2 (p3 is treated as low), which is how the Portal's High/Medium/Low are stored.
• test plan put protects structured steps. test get marks steps that carry structured fields with *. A backend that refuses to drop structured fields (412 structured_loss) now names the affected steps, and the new --force overrides it; --expected-step-count works as before. Older backends ignore the extra field and keep working.
• test run --all on a backend project names the integration phase. One line says how many existing integration tests run after the wave (0.2 credits each) and whether a first run may generate new ones; --output json carries the same integration object. test list shows backend/int in the TYPE column for assembled integration tests.
• A run blocked because its stored code could not be replayed shows replay refused (<reason>) in the text run card.
• The CI job summary is the execution report — the GitHub App comment, in Actions. On a backend that serves execution reports, ci run fetches the report for the batch's execution (polling up to --execution-report-timeout, default 60s) and writes its Markdown to the job summary: verdict, AI narrative, per-test error details, suggested fix prompts, screenshots, evaluation notes. The structured report rides along as summary.executionReport (JSON envelope and --summary-file), and the step outputs execution-report-file / execution-report-markdown-file name its JSON and Markdown under $RUNNER_TEMP for later steps. Older backends, or a report that is not ready in time, keep the results table. When the CLI's own verdict disagrees with the report (a run it could not read or that timed out, a refused test, a retry error), the summary opens with a one-line CI gate warning and appends the rows the report cannot carry, so it never reads "all passed" under a red check. --no-execution-report keeps the local table and reads nothing; no report is read after a run timed out; the report files are written 0600; an interrupt while waiting for the report (request or retry sleep) falls back to the table instead of dropping the finished batch's summary and outputs, then exits with the signal's code (130/143). A red run whose failures the report also shows keeps the report as is — the CI gate warning appears only when the counts differ.

Changed

• ci run telemetry names its outcome. The one command-outcome event now carries ciRunOutcome (gated, skipped_draft, skipped_no_pr, refused_admission, refused_env, refused_preview_name, refused_loopback, refused_billing, refused_other, empty), the execution's UUID (executionId) when the gated batch names exactly one, and reportReady (the execution report reached the job summary, or the local table did). Before, a draft-PR skip and a green gate were the same success event, and the event could not be joined to the execution. Still no URLs, names or messages — see "Telemetry" in the documentation for the full field list.

• test run --all --wait no longer adopts another environment's in-flight run. When the backend refuses a case as already in flight, the batch used to attach to that run and report its verdict as its own, whatever environment or target URL it was testing — routine once a CI gate runs the whole project on every push in a github-pr-<N> clone. The batch now applies the same gate the single run has had since 0.14.0: it reads the in-flight run, compares its environment id with the requested --env (or the project default) and its target with --target-url, and attaches only on a match; anything else stays a hard conflict, named on stderr with the test wait command to follow it. Runs that record no environment (V2, older backends) still attach when no --env was asked for.

• test run --all keeps the skipped tests of a retried batch. When a rate-deferred retry answered with its own skippedFrontend / skippedIntegration buckets, only the first batch's were reported; the retry's are now folded into the same skipped count, output, summary and JUnit rows. The exit code is unchanged (skipped tests warn unless --fail-on-skipped).

• ci init github writes a PR gate that tests the pull request's preview, calling the CLI directly. The generated workflow no longer uses TestSprite/testsprite-action or tests the project's configured environment on push/pull_request. It runs testsprite ci run --project <id> --env <name> --target-url <preview> after one placeholder step that you replace with where the preview URL comes from: delete it and switch the trigger to deployment_status when your platform deploys every PR, or put your deploy commands in it. The workflow installs @testsprite/testsprite-cli@latest, skips fork and draft PRs, ignores production deployments, passes GITHUB_TOKEN, and always uploads the summary file. New --env <name> picks the environment whose sign-in each PR run uses; an unknown name is refused, an SSO (recorded-session) environment gets a warning because it cannot sign in on a preview, and without it the project's default is written by name. --timeout is written only when given. --path must be a .yml/.yaml file directly under .github/workflows/. ci init warns when another workflow already runs TestSprite. Removed: --filter (add it to the ci run line yourself). JSON summary: adds environment and otherTestspriteWorkflows, timeoutSeconds is null when not given; drops action and filter. Requires the ci run command. The generated gate passes --fail-on-skipped; its concurrency group is declared on the job and keyed on the pull request (shape B) or the commit (shape A — Vercel's deployment ref is the commit SHA), so a new pull-request push cancels the previous job (shape B only) while dropped deployment events never cancel a running gate; cancelling only stops the job's CLI, the run it started keeps executing. --force keeps the previous file as .bak and names it as backupPath, since the preview step is hand-edited. The job is named testsprite-tests, which is the name to require in branch protection; GitHub matches required status checks by job name across all workflows, so it must not collide with your own test or e2e job.

• A new project no longer dead-ends. Someone who creates a project without a coding agent was never told that TestSprite can propose the first tests: project create printed the record and stopped, setup's next steps said to write a test by hand, and an empty project answered test list with "No tests." and test run --all with a bare exit 5. Each of those moments now names the way in — test plan generate --project <id> then test plan accept (with project docs upload first for a backend project), and test create --plan-template for anyone who prefers to write the plan themselves. The hints print only in text mode, on stderr, so --output json output is unchanged, including its error envelope. Printing a hint runs and charges nothing; generating a plan uses credits. test run --all --local <port> on an empty project is pointed at the hand-written plan instead, since generation cannot reach an app on your machine, and the suggested run command keeps the target that run used (--env <name> for a saved local environment, --target-url, or --local <port> with a non-default --local-host), so it does not fall back to the project's default environment; project create --local gets no second hint: its record's own next steps now start from test create --plan-template and set projectId in the plan file instead of passing --project, which --plan-from ignores. The always-on testsprite-onboard skill description and its codex one-liner now say the suite can be generated, matching the skill body.

Deprecated

• ci doctor and ci connect are hidden and deprecated. Platform checks and linking belong to the coding agent, not to CLI commands: the testsprite-ci skill reads the same deployment history (gh api repos/<owner>/<repo>/deployments) and runs the platform's own CLI (npx vercel link --project <name> --non-interactive, npx vercel git connect --non-interactive), and ci doctor's name-based preview check misjudged custom environment names. Both commands still run with the same behaviour and exit codes; each run prints one stderr line naming the equivalent commands. They are removed in the release after this one, together with src/lib/ci-linkage.ts.

Fixed

• Replacing the credentials file retries a transient Windows lock. On Windows, renaming the temp file over ~/.testsprite/credentials can fail with EPERM, EBUSY or EACCES while another program briefly holds the file open; the CLI now retries for up to two seconds before failing. Persistent errors, other error codes and non-Windows hosts behave as before.
• Reading the credentials file no longer fails with EPERM on Windows. Hardening the file's permissions removed ACL inheritance in a way that could lock the owner out of their own credentials, so every later read failed. Inheritance is restored, doctor repairs a file already locked that way, and the connectivity and tunnel checks no longer re-read an unreadable file. Community fix by @aythin and @jangjos-128 (#356).
• Replay routing records with null fields are accepted. A normal replay reports null reason and caveat, which the response schema rejected. plan put also no longer reads the test before writing, so write-only API keys keep working; --expected-step-count behaves as before.