Skip to content

v0.14.0

Latest

Choose a tag to compare

@testsprite-cli-release testsprite-cli-release released this 06 Oct 06:23

Added

  • Explicit wait controls for automatic tunnels. test create --run, test create-batch --run, test rerun and testlist run accept --no-wait to keep the cloud dispatch path. Rerun and test-list run accept --skip-preflight to skip the local port probe.
  • Dry-run tunnel previews. Credentialed test run --dry-run can read test/environment metadata to show its automatic tunnel target; it never creates a tunnel or starts runs. Explicit targets and credentialless previews remain offline.
  • Automatic tunnels for saved loopback environments. Frontend test run (one/many/--all), test rerun, test create --run, test create-batch --run and testlist run open a tunnel when the selected or default environment URL is localhost, 127.0.0.1 or [::1]. They preserve the stored URL, wait for completion, and cancel unfinished runs on an owned tunnel by default. --no-wait and unreadable environment lists keep the previous server behavior. Rerun/test-list responses must confirm the tunnel client id; older servers trigger a cancellation attempt and a clear error.
  • Environment-aware test status reads. test list --env <name> selects frontend statuses by environment and labels the text table with that selection. The new ENV column names the environment behind each row's status, while test get shows the headline environment beside its status. JSON keeps status and passes through statusByEnvironment and headlineEnvironment. Against a server that does not support the filter, --env fails with UNSUPPORTED (exit 7) instead of printing unfiltered statuses; an empty page carries no support signal and prints as an empty list.
  • Environment inspection and variables. project env get <project-id> <name> shows the URL, default status, safe login summary and custom variables. project env create / update accept repeatable --var KEY=VALUE; update merges keys, and JSON list/get responses include variables. Local validation rejects invalid, duplicate, reserved and oversized keys or values before sending a request.
  • Clear environment credentials. project env update --clear-credentials removes the stored username and password and disables account login. It cannot be combined with credential flags. Portal-managed login modes remain read-only in the CLI.
  • Project sign-in modes. project sign-in get/set reads or changes public, test-account, or SSO sign-in per environment. project env create --sign-in also selects OTP at creation, with --otp-channel and manual --session-ttl. SSO selection requires the per-account manual-login feature flag (FEATURE_GATED when off); login is captured once in the Portal, and the CLI warns when no valid session exists. With the matching backend, project env update and project update refuse a username or password on an SSO or OTP environment (pointing at project sign-in set), project env update refuses them on a temporary environment, and both refuse --local on an SSO or OTP environment. On an environment with no stored account, project env create (without --sign-in) and project env update now require --username and --password together (exit 5) instead of storing half an account on a public environment. Needs backend v0.19.0 or later: an older backend answers project sign-in with endpoint-not-available, and ignores --sign-in on project env create, creating an environment without that sign-in mode.

Changed

  • Frontend test status headlines the default environment. With backend v0.19.0 or later, test list and test get without --env report each frontend test's latest result in the project's default environment instead of its latest run in any environment, and --status filters on that status. A run recorded under another environment (--env <name>, or the temporary environment of a --target-url run) no longer changes the listed status; test result still shows that run, and test list --env <name> reads another environment.
  • Local telemetry remains compatible. local=true still records every explicit --local invocation, including borrowed tunnels, rejected invocations and project creation, and additionally records automatically opened tunnels. No new telemetry fields.
  • --local defaults new values to localhost. Project/environment writes use http://localhost:<port>; test runs preserve a selected loopback environment's host, or the unique saved loopback host on that port, before falling back to localhost. If the environment list is unavailable, test shorthand retains the legacy 127.0.0.1 spelling. Existing URLs and exact-origin matching remain unchanged.
  • Loopback URLs use the existing tunnel path. Project/environment --url accepts HTTP localhost, 127.0.0.1 and [::1] origins with an explicit port and no path; test run --target-url accepts these targets through a tunnel. HTTPS loopback origins and other private/unspecified/link-local addresses remain refused. Explicit --local with a public environment reaches the backend, which can apply that environment's sign-in configuration or return its existing refusal. A stderr advisory names the selected sign-in environment and local origin.
  • Environment and tunnel guidance names saved environments. Retry and success hints use test run <id> / --env <name>. test run --help and tunnel start --help suggest a built server for heavy dev servers that can take a minute per page through a tunnel.
  • Additive text output preserves existing order. Environment columns are appended to project/test/schedule tables, and new environment detail lines follow the existing write-confirmation lines.
  • project update reports default environment edits. New servers return the edited environment name and applied effects, which text output now explains. Older server responses keep their existing text output. Environment text write confirmations preserve their existing id and account lines before appending new login details.
  • Project environment summaries. project list adds DEFAULT ENV and ENVS columns, and project get shows the default environment, environment count, and an environment-list hint when the count is positive. JSON passes through the server's defaultEnvironment and environmentCount fields unchanged. Older servers and V2 projects omit the detail lines; missing list values show -, while a reported zero count stays 0.
  • Project schedule environments. schedule create and schedule update accept --env <name> to pin a project schedule to a named environment. A new project schedule without it inherits the project's live default at each tick; test-list schedules inherit their list's environment bindings and reject --env on create with a testlist update --project-env hint. List/get and create/update text show pinned or inherited mode — always read back from the API's response, never from the flags sent — while JSON passes through the API fields unchanged. An unknown name returns the API's validation error and available names. When existing schedule reads prove that the server is too old, the CLI refuses (exit 7) before creating or updating anything. Empty or unreadable capability reads retain response confirmation: an unconfirmed create is best-effort removed, with its exact id reported; an unconfirmed update reports the schedule id and requested fields that may have been applied.
  • Recursive secret redaction in errors and debug traces. Any field named like password, token, secret, apiKey/api_key, authorization, cookie, or credential(s) — at any depth, inside arrays too — is replaced with [REDACTED] in an error envelope's details (--output json and text) and in --debug request tracing; --dry-run previews are built from presence flags and field names, so they never carry a credential value. This is defence in depth: the backend already redacts its own error envelopes, but the CLI now redacts again on the way out. Success output is never touched: generated test code, plans and fixtures print exactly as stored — byte-identical, even a literal Authorization: Bearer … header or a password fixture key — because that's the caller's own data, not server error prose. Ordinary fields that merely contain one of those words — a presence flag like hasCredentials, a path like credentialsPath, a pagination cursor like nextToken — are left untouched either way. New helper: src/lib/redact.ts.
  • test run list is a clear usage error, not a run against a test literally named "list". Someone guessing at a "list prior runs" subcommand now gets pointed at testsprite test result <test-id> --history (exit 5, no request sent) instead of a confusing not-found a moment later.

Fixed

  • Tunnel teardown cancels pending targets. DNS lookups and candidate fallback cannot open target sockets after stop, control disconnect or CloseTunnel. Intentional stop no longer reports a spurious control disconnect, and target teardown no longer reports an aborted stream as an error. Tunnel retry cleanup also survives an error callback throwing, and established data-plane errors are reported in verbose diagnostics with the credential redacted.
  • JUnit timing works without a run start timestamp. Reports use createdAt when the backend omits startedAt, retaining startedAt precedence and zero when timing is unavailable.
  • Latest results explain environment status differences. test result (text mode) keeps showing the newest run of any environment and, when available, appends the default environment's latest status for a run from another environment. Older servers and failed advisory reads remain compatible; JSON and history are unchanged.
  • Blocked local schedules show their reason. Result summaries and waited run cards display the backend's explanation when a cloud schedule cannot reach an environment on this machine.
  • Tunnel interrupt opt-out describes the possible outcome. --no-cancel-on-interrupt explains that the cloud run continues and is billed, steps needing this machine will fail after the tunnel closes, and the run can still pass if it needs no further local loads; it names the result and wait commands to check later.
  • Tunnel batches preserve dispatch and timeout contracts. Several fresh-run IDs without a tunnel retain the previous exit-5 refusal; inert tunnel flags are rejected. Rerun/list batches share one timeout including capacity waits, and missing tunnel confirmation reports cancelled runs honestly.
  • Create-with-run checks tunnels before creating tests. Known targets are resolved, probed and checked for tunnel permission before create writes; late failures preserve created IDs and per-row reasons in stdout.
  • Environment preflight stays bounded. Known list/create metadata avoids per-test lookup fan-out; advisory reads use a five-second window without retries. Only backend tests with their own loopback environment are skipped in tunnel batches, and JSON includes the skipped IDs. No-wait hints print once per environment.
  • Environment support probes preserve missing-resource errors. A missing route signals an old server; a missing project/environment keeps NOT_FOUND and exit 4.
  • Environment password validation remains active in dry runs. project env create/update retain their password/password-file mutual-exclusion validation; project create/update preserve password precedence.
  • Environment writes report unsupported flags and terminal conflicts immediately. project env create / update verify that the response applied --var and --clear-credentials before reporting success; mixed updates warn that the URL may already have changed. Environment and project writes retain one automatic retry for transient 409 conflicts, using the same idempotency key.
  • Project password flags retain their previous precedence. With both --password and --password-file, project create/update keep the supplied password and print a one-line stderr warning; the ignored file is never read, including under --dry-run. project env create/update keep their mutual-exclusion validation.
  • Old-server environment features fail clearly. project env get maps a missing route to UNSUPPORTED (exit 7). Environment variable and credential-clear writes probe support before changes where possible and still verify response echoes when a read is inconclusive.
  • V3 advisories no longer say frontend --target-url is ignored. auth status and doctor retain the rerun advisory; point-of-use target mismatch warnings still follow the actual server response.
  • The auto-resume advice no longer suggests --local for an https loopback environment. --local runs over plain http, so an environment at https://localhost:8443 now gets --env <name> alone instead of a --local 8443 that would point at the wrong scheme.
  • test plan generate now says what an interruption actually does. The 0.7.0 notes (and the help, the first-run hint, the Ctrl-C and --timeout messages and the onboard skill) said the server keeps working after a detach. Only the stage already running does; the next stage starts only when the command is re-run, because the client drives the stage ladder. The copy on all five surfaces now says so ("paused, your place is kept, re-run to continue"), one stage i/N <name> done <elapsed> line is printed as each stage finishes (text mode), and the partial object printed on interrupt, --timeout, request timeout, rate limit and the stuck fuse gains stagesRemaining (the stages this run had not started yet; null when the run only attached to a stage it did not start). The --timeout envelope carries the same list under details.stagesRemaining, and under --output json the INTERRUPTED envelope printed on Ctrl-C now tells the same paused-stage story (nextAction names the stage and the continue command; details gains projectId and stagesRemaining) instead of the run-detach text pointing at test wait. The stuck-fuse line counts every start attempt and no longer contradicts the partial printed beside it. Exit codes and the ladder's requests are unchanged.

Deprecated

  • --local-host on project/environment writes. The flag remains accepted for compatibility and is hidden from help. Prefer an explicit loopback --url when the host matters; test runs still accept the override.