You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Environments from the CLI.project env list | create | update | delete | set-default <project-id> manage a project's environments over /api/cli/v1/projects/{id}/env with the existing read:projects / write:projects scopes; passwords come from --password-file and are never printed, and delete refuses the default environment (the server soft-deletes, so run history keeps naming it). --env <name> on test run (including --all), test rerun and test result --history targets or filters by environment; an unknown name is a server-side validation error that lists the project's environment names, never a silent fallback. project create and project update accept --local <port> (with --local-host and --skip-preflight) for an app on this machine — the CLI builds the loopback URL and probes the port first; a loopback --url is refused and pointed at --local, and a bind-all 0.0.0.0 / :: is refused everywhere. Every run now names an environment on the server (--target-url and --local match by origin or create one), and run, result and history payloads carry environment {id, name}; the history table gains an ENV column and authMode reports account / otp / manual / public. Requires a backend that serves the environment facade; against an older backend the project env verbs return not-found and --env is rejected.
TESTSPRITE_CLIENT — let a wrapper identify itself. A tool driving the CLI (the GitHub Action sets github-action/<ref>) can set TESTSPRITE_CLIENT=<name>/<version>; the value is appended to the User-Agent on every API request and the telemetry beacon (testsprite-cli/<version> (<client>)) and reported as the telemetry client field. The value is validated against a tight <name>/<version> grammar; anything else is ignored silently and never echoed. Without the variable every request is byte-identical to before.
CI context and outcome counts on the telemetry event. The one per-invocation event now carries ciProvider (github / gitlab / circleci / buildkite / other / none), and under GitHub Actions ciEvent and repoHash — a salted, truncated hash of the repository, never the name. test run --all, testlist run and test run <id> --wait add outcome counts (accepted / conflicts / deferred / skipped, passed / failed / blocked / timedOut) and conflictReason, the most frequent reason a case did not dispatch; ci init adds platform, force, workflowExisted and projectResolved. All fields are low-cardinality numbers and closed enums; the privacy contract (no URLs, ids, messages, argument values or repository names) is unchanged, and so are the opt-outs.
Pinned-version warning in GitHub Actions. A ci init-generated workflow pins cli-version, so the job keeps running that release forever. When the backend reports that the running version has fallen below its minimum supported version, a job under GITHUB_ACTIONS=true now gets one ::warning title=TestSprite:: annotation naming the pinned version, the floor, and the fix (testsprite ci init github --force) — on stderr, so --output json on stdout stays parseable. The plain TTY advisory is unchanged off Actions. This keys off the version floor the backend already sends on every response; the npm "newer version available" check is (still) skipped in CI, so the warning does not fire for a pin that is merely behind the latest release.
Changed
Batch billing refusals exit 12, like a single run.POST /tests/batch/run and testlist run may now report per-case insufficient_credits / billing_hold conflicts, and both are named in the conflict summary with the server's message. When nothing dispatched, nothing was rate-deferred, and every conflict is insufficient_credits, test run --all (with or without --wait) and testlist run exit 12 with the same INSUFFICIENT_CREDITS envelope and billing nextAction a single test run prints — previously a generic "already in flight" conflict, exit 6. A backend that answers that case with a 402 envelope directly is mapped identically. A batch refused entirely for a billing hold comes back from the current backend as the standard 403 FEATURE_GATED envelope (details.reason: 'billing_hold') and exits 13 with the server's nextAction — the same as a single test run; per-case billing_hold conflicts appear only on a mixed batch, and an older backend that folds every case into them still exits 6 with the hold named. Every other conflict mix keeps its current exit code.
The CI summary counts never-dispatched tests as skipped, never as failed. The --summary-file JSON (on test run, batch test rerun, and testlist run) gains a skipped field, and failed now counts only dispatched runs that did not pass. Previously a test that never ran — rate-deferred, conflicted (already in flight), not found, or skipped by the engine — was counted under failed, so a batch could exit 0 while its own artifact claimed failed: 1 and the checks tab showed a red annotation: three surfaces telling two stories. The annotations split the same way — a dispatched run that failed or timed out still annotates as ::error::, while a never-dispatched test annotates as ::warning:: — and the job-summary headline reads N/M passed (F failed, S skipped, T timed out). Migration: a pipeline that parses the summary JSON and gates on failed > 0 no longer fails on conflict/not-found/deferred members; gate on the exit code (which already covers every case where nothing ran: deferred → 7, all-conflict → 6, all-not-found rerun → 4, zero-dispatch → 5), or on failed + skipped to keep the old behavior.
test rerun --all fails (exit 5) when the resolved rerun set is empty, instead of a silent exit 0. No tests matching --filter/--status/--skip-terminal — or a project with no tests — used to print one stderr line and exit 0 with no CI artifact: a rerun gate that greens on zero runs. It now emits the summary + annotation and fails with exit 5, exactly like test run --all (0.8.0); pass the new --allow-empty to opt back into exit 0 when an empty sweep is expected.
Fixed
A batch rerun where every id has no replayable run exits 4, not 0.test rerun a b --wait with both ids in notFound dispatched nothing, printed two red annotations and an artifact claiming two failures — and exited 0, so a CI gate passed on zero runs. It now exits 4 (NOT_FOUND, the same code testlist run uses for a --case miss) with a nextAction pointing at test run <id> for a first fresh run, on both the --wait and non---wait paths. A mixed batch is unchanged: notFound ids surface as skipped rows without failing an otherwise-passing batch.
An empty frontend stepSummary renders as n/a, not 0/0. A frontend run card whose step summary carried no steps printed steps 0/0 (passed=0, failed=0), the same shape a backend run uses to say "not applicable"; it now prints n/a like the backend card. Community contribution (#336).
A month-restricted cron schedule is no longer described as "daily". The human-readable schedule description ignored the month field, so 0 9 * 1 * read as "every day at 09:00"; the month restriction is now part of the description. Community contribution (#319).
DOCUMENTATION.md manual-authentication commands match the current setup / auth surface. Community contribution (#342).