Repository navigation
Added
test runtakes several test ids.testsprite test run <id> <id> …dispatches any number of ids as batches: ids are grouped per project (read from each test) and sent 50 per request, one chunk after another, through the same wave-ordered batch as--all, with--wait,--report,--summary-fileand the shared exit-code precedence. If a later chunk fails, the command exits with that chunk's exit code and the output (and itsnextAction) still lists every run id the earlier chunks accepted; without--waitthose runs are not polled, with--waitthey are (see Changed). A JUnit report across several projects uses each row's own project id as theclassname. Backend-only ids go through the batch even when the project's default environment is a loopback URL, since backend tests do not use the tunnel.test run --all --status <list>runs only the test cases whose latest status (the status of their latest result, as in the Portal's Latest Status column) matches one of the comma-separated values (passed|failed|blocked|cancelled|not_run;not_runis a test case that never ran).draft,ready,queued,runningandunknownare rejected (exit 5).--env <name>only picks where to run; it does not change which test cases are selected.test run --all --status failed,blockedruns what did not pass. An empty, whitespace-only or commas-only--statusis a usage error (exit 5) ontest runandtest rerun, before any request, including under--dry-run.--allow-emptynow covers--filter/--statusmatching nothing.test list --statusandtest delete-batch --all --statustakenot_run(a test case that never ran: draft or ready) and list the same five values astest run --status(passed|failed|blocked|cancelled|not_run). The older values (draft,ready,queued,running,unknown) still work until 1.0 but are no longer listed;test rerun --statusis unchanged.test run --skip-dependencies(backend test cases only) runs the named test cases without the test cases they depend on, on a single id, several ids and--localruns. The backend confirms it: a response toPOST /tests/{id}/runsorPOST /tests/batch/runcarriesskipDependencies: trueexactly when the request did and the backend handled the field, and omits the key otherwise. A batch whose response lacks the echo is cancelled and exits 7 (skip-dependencies-not-honored); a single or--localrun prints an[advisory]and carries on (a Portal run of a test case also includes the test cases it depends on), as it does for an unconfirmed--target-url.
Changed
- No more 61-second resend; runs queue on the server.
test run --all,test run <id> <id> …andtest rerun --allno longer resend deferred ids after 61 seconds under--wait, and the CLI's own in-flight limit is gone: runs queue on the server as Portal runs do. Each request starts one run, so a large batch is accepted whole and its results showqueued(text progress and JSON) until they start.deferred[]can still appear in rare cases (a request spanning several projects where one project's lock stayed busy, or a test case that went in flight during dispatch); those ids are listed (stdoutdeferred, stderr line, not-dispatched, JUnit and summary rows), never resent, accepted runs are still awaited under--wait, and the exit code is 7. The 60 run triggers per minute per API key and the 409 for a test case that already has a run in flight are unchanged;ci runis unaffected. test run --wait: one rule when a request fails part-way through a batch (a one-time behaviour change). This covers several ids and--allwith--filter/--status. A request that fails after earlier requests were accepted now stops the dispatch (nothing further is sent) and, under--wait, polls every run already accepted (and every in-flight run it joined) to its verdict before exiting: stdout,--summary-fileand the JUnit report carry those real verdicts, and the ids never sent are printed on stderr and written asnot dispatchedrows (stdout lists them innotRunTestIds). The exit code is the failed request's own, and it outranks rate-deferred (7), a failed verdict (1), credits (12) and conflict (6). What differs from before: a failed later chunk under--waitused to list its earlier chunks' runs asnot awaitederrors without polling them; it now exits with that request's code, for example 11 for a rate limit or 6 for a precondition. Without--waitnothing changes: the accepted run ids and the ids never sent are listed and nothing is polled. A run that sends a single request and fails is still rethrown unchanged.ci run,test rerunandtest flakyare not affected.- Docs and shipped skills match the current CLI. The
testsprite-verifyskill now accepts a loopback environment--urland treats--expected-versionas optional, the README and DOCUMENTATION.md coverci,completion,test open,tunnelandauth remove, and docs links point at docs.testsprite.com. --helpis accurate and shorter. About 50 help texts were corrected against the code and the current backend, with no behavior change, and every command has a one-line summary so parent tables list one line per command (test --help149 → 42 lines, with the deprecatedrerunandflakyno longer listed). Highlights:test cancelandtest runagree on Ctrl-C (a run through a tunnel the CLI opened is cancelled by default);test runlists exits 12/13 andtestlist rungains an exit-code list;--page-sizealone returns one page;schedule deletekeeps past runs andschedule update --envkeeps the pin when omitted;project auto-authis "(paid plans)"; the root help lists the environment variables it reads.
Deprecated
-
test rerunis deprecated and removed in 0.17.0. It keeps working with the same requests, output and exit codes; each invocation now prints one[deprecated]line to stderr (before argument validation, also under--dry-run), a success JSON object on stdout ends with adeprecationsarray of{ command, replacement, removedIn }(replacementis a template:test run, to be used with your own ids and flags), and its own--helpstarts with[deprecated]. It is a hidden alias:test --helpno longer lists it. Thedeprecationskey appears in stdout JSON only: not in--summary-file, JUnit reports or error envelopes, and not in text output. A server-sent deprecation advisory on a rerun response is dropped so the notice appears once. -
test flakyis deprecated and removed in 0.17.0, with the same three signals. It has no replacement; runtest run <test-id>again to sample a test.Instead of Use test rerun <id> [--env] [--wait] …test run <id>with the same flagstest rerun <id> --no-auto-healtest run <id> --no-auto-healtest rerun a b ctest run a b ctest rerun --all --project <p> [--filter] [--status]test run --all --project <p>with the same flags;--statustakes onlypassed|failed|blocked|cancelled|not_runtest rerun --all --skip-terminaltest run --all --status not_runtest rerun <be-id> --skip-dependenciestest run <be-id> --skip-dependenciestest flaky <id> --runs Nno replacement; run test run <id>againDifferences you will notice after switching:
test runstarts a fresh run, so a user-authored case replays instead of being re-authored (the server's auto-heal default applies); the JSON has noclosure,notFoundoradvisories; an unknown id fails the whole multi-id batch (exit 4) instead of appearing innotFound[]; and new runs carry nocreatedFrom: rerun:<id>.
Fixed
- A batch run names the
no_producerandproducer_in_flightrefusals. A backend test case that needs a variable no test case captures, or whose producer is running in another run, was reported as1 not dispatched (1 not dispatched)and described as already in flight. It now readsno producer for its variablesorproducer running in another run, with the server's message where it sends one. Under--waitsuch a test case is now reported with that reason as blocked before dispatch (exit 6), as a singletest runreports it, instead of being polled and printed as a bareblocked(exit 1). Its latest status isblockedeither way: the server records the refusal as a blocked result, as the Portal shows. Ano_producertest case is told to add a producer instead of to wait. test delete-batch --all --status ''no longer deletes every test in the project. An empty--statuswas read as no filter. An empty, whitespace-only or commas-only--statusis now a usage error (exit 5) before any request, including under--dry-run, as ontest run.