Command Reference

Entry point: testo (pyproject.toml → testo_core.cli.app:main).

Parent index: Index. Architecture: Architecture Overview. Run lifecycle: QA Strategies.

testo --help

Help is grouped into panels: Run and report, Config, Diagnostics, Maintenance, About.

Extras: pip install testo-core is the CLI with pytest, Playwright, command and junit stages. Add [bdd] for behave and behavex stages (Behave, BehaveX and Allure-Behave), [api] for the REST API and UI backend, [db] for Postgres or MySQL, [metrics] for InfluxDB, [otel] for OpenTelemetry export, [mcp] for the MCP server. A behave stage without the extra fails with exit 3 and executable not found: behave (... pip install 'testo-core[bdd]').

`testo run`

Execute one cycle or all cycles from testosterone.yaml.

testo run --cycle sample-pytests
testo run --cycle sample-pytests --ci
testo run --cycle sample-pytests --stream
testo run --cycle sample-pytests --workers 8 --force

Options

Flag	Short	Description
`--cycle`		Cycle name from `cycles:`; use `all` for every cycle (sorted), each trigger evaluated separately
`--plan`	`-p`	Deprecated alias for `--cycle` (hidden)
`--config`	`-c`	Path to `testosterone.yaml` (default: discovery)
`--stream`		Tail each stage stdout live (vs post-mortem Rich panel)
`--ci`		NDJSON events on stdout instead of Rich UI
`--no-persist`		Skip optional run-history DB writes
`--no-report-db`		Skip archiving cycle artifacts to report DB after run
`--async-report-db`		Archive in background thread with join timeout; ignored when `--ci` is set
`--workers`	`-w`	Override parallel workers (e.g. BehaveX); 1 or more
`--force`	`-f`	Run even when trigger would skip the cycle
`--coverage`		Code coverage report the tests write (repeatable): Cobertura or JaCoCo XML, LCOV, or a Go cover profile. Read when the stages finish; its line coverage is stored with the run and shown on the run page, the PR comment and the digest. A missing file only logs a warning. Not with `--cycle all`

[!warning] Roadmap flags — not implemented yet --tag, --fail-fast, --dry-run and --reporter were previously listed here but are not present in testo_core/cli/commands/run.py. The API layer already accepts fail_fast and reporter_override on POST /api/v1/cycles/{cycle}/executions (surfaced in the React Run panel); the CLI flags remain roadmap items. (Doc corrected 2026-07-04 during the Phase 5 UI redesign.)

Only one run of a cycle uses its artifacts folder at a time. A second testo run of the same cycle (another terminal, another CI job on the runner, the API) waits and says another run of cycle 'x' is using ... (pid N); waiting for it to finish; see Engine Fixes - 2026-11-01.

Expected terminal output

Interactive (default): Rich panels for plan start, each stage, and finish. On success:

Run finished successfully.

On failure:

Run exited with code 1.

--ci: One NDJSON object per line (plan_started, stage_started, stage_finished, plan_finished, etc.), no Rich styling.

Exit codes

See Architecture Overview § Exit code contract. testo run returns the engine exit code directly.

`testo ingest`

Record test results that another step already wrote (any runner, any language) as a run of a cycle. Nothing is executed: the files are parsed into the same run history as testo run, so Compare, trends, testo flaky, reporters and the PR comment work on them.

testo ingest 'reports/**/*.xml' --cycle web-unit          # record locally
testo ingest junit.xml --cycle api --tier integration --ci
testo ingest test-results/results.json --cycle web-e2e --stage e2e --tier e2e   # Playwright JSON
testo ingest allure-results --cycle api                    # a folder: the files in it
testo ingest                                               # find the result files, cycle = folder name
testo ingest 'reports/*.xml' --cycle web-unit \
  --server https://testo.example.com --token "$TESTO_API_TOKEN"   # team server

Flag	Default	Meaning
`REPORTS...`	found	Files, folders (their files) or quoted globs, relative to `--repo` (absolute paths must be inside it). Each file is recognised by its content; see the table below. None given: every result file under `--repo` that git doesn't track (fixtures are tracked) and isn't in a dependency folder (`node_modules`, `.venv`, ...); exit `2` when there is none
`--cycle`	folder name	Cycle name the run is recorded under: letters, digits, `.`, `_`, `-`. Reuse it so runs build a history. Default: the `--repo` folder's name, as `testo init` and `pytest --testo` use
`--stage`	`junit`	Stage name the tests are grouped under
`--tier`	`unit`	Test-pyramid tier: `unit`, `integration`, `e2e`
`--repo`	`.`	Directory the report paths are relative to
`--config` / `-c`	discovered	`testosterone.yaml` for `reporters:` and paths; not required
`--ci`	off	NDJSON events on stdout, as `testo run --ci` (with `--server`: one `ingest_uploaded` event)
`--no-persist`	off	Parse only, record nothing
`--server`	`$TESTO_SERVER_URL`	Upload to a team server instead of recording locally
`--token`	`$TESTO_API_TOKEN`	The server's API token
`--coverage`	none	Code coverage report to store with the run (repeatable, relative to `--repo`), as `testo run --coverage`. Checked first: no match or an unknown format is exit `2`. With `--server` only the totals are sent (the server recomputes the percentage)

Files it reads, with the command that writes each one (no extra package needed except where noted):

Runner	Write results with	Read as
pytest	`pytest --junitxml=junit.xml` (or `pytest --testo`)	JUnit XML
Jest	`npx jest --json --outputFile=jest.json` (or `equipment: jest`)	Jest JSON: `describe` blocks, retried tests flagged flaky, test files that failed to load. Prefer it to `jest-junit`, which drops a test file that fails to compile
Vitest	`npx vitest run --reporter=json --outputFile=vitest.json` (or `equipment: vitest`)	Jest-style JSON, labelled `vitest`
Playwright	`npx playwright test --reporter=json` (`PLAYWRIGHT_JSON_OUTPUT_FILE=results.json`)	Playwright JSON: retries, flaky tests, traces
Cypress	`npx cypress run --reporter junit` (or `equipment: cypress`)	JUnit XML
Mocha, WebdriverIO	their `junit` reporter	JUnit XML
JUnit 4/5, TestNG, Kotest (Maven, Gradle)	built in: `target/surefire-reports/.xml`, `build/test-results//.xml`	JUnit XML
.NET (xUnit, NUnit, MSTest)	`dotnet test --logger trx`	TRX: data rows one result each
Go	`go test -json ./... > go-test.json`	`go test -json`: subtests, build failures as broken
Rust	`cargo nextest run --profile ci` with JUnit output configured	JUnit XML
PHPUnit, Pest	`--log-junit junit.xml`	JUnit XML
RSpec	`rspec --format progress --format json --out rspec.json`	RSpec JSON: failure class and backtrace, pending examples as skipped, load errors as broken
Minitest	`minitest-reporters`	JUnit XML
XCTest, Swift Testing (Xcode 16+)	`xcrun xcresulttool get test-results tests --path Tests.xcresult > xcresult.json` after `xcodebuild test -resultBundlePath Tests.xcresult`	`xcresulttool` JSON: failure messages, retries flagged flaky
Android instrumentation (Espresso, UI Automator, Compose)	built in: `./gradlew connectedAndroidTest` writes `app/build/outputs/androidTest-results/connected/*/.xml`	JUnit XML, labelled `android` with the device it ran on
k6	`k6 run --summary-export=k6-summary.json script.js` (or `JSON.stringify(data)` from `handleSummary`)	k6 summary: each check and each threshold is one result
Robot Framework	`robot` (writes `output.xml`)	Robot `output.xml`: tags, failure messages
GoogleTest, Catch2	`--gtest_output=xml:gtest.xml`, `-r junit`	JUnit XML
Anything with an Allure adapter	its `allure-results/` folder	Allure results, with their attachments

A file that is none of these is skipped with a note in the stage log. See Runner Coverage - 2026-10-26.

Exit codes follow testo run: 0 every test passed or was skipped, 1 a test failed or broke, or every matching file was unreadable (truncated, empty, not a report), 2 bad arguments or no file matched, 3 the server could not be reached or refused the upload (wrong token). With --server, the job's CI provenance (provider, pipeline, job, commit, branch, repository) is sent with the files. Under the hood this is a one-stage cycle with equipment: junit run through CycleRunService; see Ingest and Team Server - 2026-10-22. For Playwright JSON (each retry as a step, flaky tests flagged, traces and screenshots copied from inside --repo as Allure attachments) see Playwright JSON Import - 2026-10-22.

`testo report`

Build unified Allure reports from artifacts; subcommands for archives and native framework HTML.

# Default: generate + local Allure dashboard (opens browser unless --no-open)
testo report --cycle sample-pytests

testo report --generate-only --out artifacts/report
testo report --format json --summary-out summary.json
testo report list
testo report open --id <uuid>
testo report compare
testo report native
testo report native flow-tests --no-open
testo report pyramid <run-id>

Default callback options

Flag	Short	Default	Description
`--artifacts`	`-a`	`artifacts`	Artifacts root
`--cycle`		latest	Restrict to one cycle
`--generate-only`		false	Write HTML only; no HTTP server
`--host`		`127.0.0.1`	Allure dashboard host
`--port`		`8080`	Port (auto-picks free port if busy; `0` = always free)
`--out`	`-o`	`artifacts/report`	Generated HTML output dir
`--format`	`-f`	`html`	`html`, `json`, or `junit`
`--summary-out`			Path for json/junit summary file
`--no-history`		false	Skip injecting prior Allure history
`--open` / `--no-open`		open	Start dashboard after generate
`--trend`		`1`	Merge history from N prior archived runs

Typical success output: messages about collecting results, running allure generate, and a local URL such as http://127.0.0.1:8080/....

Subcommands

Command	Purpose
`testo report list`	Table of archived runs (needs DB); columns: id, cycle, created, exit, tests, pass, fail
`testo report open --id <uuid>`	Extract archive and regenerate/serve Allure
`testo report compare`	Rich diff + Allure comparison (optional baseline/current UUIDs)
`testo report native [ROUTINE]`	List or open BehaveX/pytest native HTML under stage dirs
`testo report markdown [--cycle NAME \| --run-id ID] [--out FILE] [--run-url URL] [--baseline-branch BRANCH]`	The cycle's latest run (or one run) as GitHub-flavored Markdown: headline counts, per-stage table, failing tests, and per-test changes since its baseline: the latest run on `--baseline-branch`, else on the pull request's recorded target branch (`ci_base_ref`), else the cycle's previous run. When the run stored coverage, the headline adds it and its change against the same baseline (`coverage 81.2% (▼ 0.4)`). The first line is a hidden `<!-- testo-summary:<cycle> -->` marker. Exit `2` when no run is recorded. The GitHub Action uses it for the job summary and the PR comment
`testo report markdown --ai`	Also adds an AI summary of the failures (a quoted block before the footer, HTML and `@` escaped) to a failed run, generated and stored like the UI's (see `testo ai`). When the provider is off or doesn't answer, one line says why; the exit code doesn't change
`testo report annotations [--cycle NAME \| --run-id ID] [--workspace DIR] [--limit N] [-c CONFIG]`	The run's failing tests as GitHub Actions `::error file=…,line=…` workflow commands, so each failure shows on its line in the pull request's "Files changed". The location is the first stack-trace frame inside `--workspace` (default `.`), tried relative to the workspace, each stage's `target_repo` from the config, and the run's target repo. At most `--limit` (default `10`, GitHub's per-step cap) lines, then a `::notice` with the rest. Exit `2` when no run is recorded. The GitHub Action prints it by default. See Alerts Where Developers Look - 2026-10-22
`testo report pyramid RUN_ID [-c CONFIG]`	ASCII unit/integration/e2e test pyramid for a completed run (`run_id`, not the archive UUID); stage→tier is the stage's `tier:` (default inferred from `equipment`: pytest→unit, behave→integration, behavex→e2e, playwright→e2e, jest→unit, vitest→unit, cypress→e2e) as recorded in the run when it executed; `-c` is only read for older runs that predate recorded tiers. See Architecture Overview § Test pyramid.

`testo cycles` (alias: `testo plans`)

Inspect cycles in config (hidden alias plans for backward compatibility).

testo cycles list
testo cycles show sample-pytests
testo cycles show sample-pytests -c ./testosterone.yaml

list: Rich table — Name, Description, stage count.

show: Cycle title plus table of stages (equipment, target repo, args, timeout).

Config errors print [fail]config error: … and exit 2.

`testo config`

Subcommand	Description
`testo config validate`	Load YAML; optional `--check-executables` for pytest/behave/behavex (and `npx` for playwright, jest, vitest and cypress) on PATH. Warns about keys nothing reads (`unknown key 'cycles.web.stages[0].timout_s' is ignored; did you mean 'timeout_s'?`); `--strict` makes them an error (exit 2)
`testo config init`	Write starter `testosterone.yaml` with one active `smoke-test` cycle running `pytest -q tests` and commented-out marker/BehaveX examples (`--path`, `--force`)
`testo config db`	Set `database.url` (PostgreSQL prompts or `--url`)

testo config validate
testo config validate --no-check-executables
testo config validate --strict   # unknown keys fail, for CI
testo config init --path testosterone.yaml --force
testo config db --url sqlite:////tmp/testo-reports.db

Validate success:

ok — version=1 cycles=6 reporters=4 defaults_target=.

Deprecated: testo config-db → use testo config db.

Editor autocomplete

schema/testosterone.schema.json is a JSON Schema for testosterone.yaml. Editors with a YAML language server (VS Code with the Red Hat YAML extension, JetBrains IDEs, Neovim) then suggest keys and built-in equipment, and underline typos as you type. testo config init and testo init add this line at the top of the file; add it yourself to an existing config:

# yaml-language-server: $schema=https://raw.githubusercontent.com/taltal-beep/testosterone/main/schema/testosterone.schema.json

testo config validate is still the final word. The schema describes the cycles: layout only, and accepts plugin equipment and reporter names. Why and how it is generated: Config JSON Schema.

`testo diff` / `testo summary`

Compare two archived report runs by UUID (from testo report list).

testo diff <BASELINE_ID> <CURRENT_ID>
testo summary   # same family; see help for metrics-only variants

Rich terminal diff of metrics and (for full diff) extracted Allure deltas. Reads the run history database (SQLite by default; testo-core[db] for Postgres).

Prefer testo report compare for Rich diff plus Allure visual comparison pipeline.

Note: this operates on ReportArchive UUIDs (testo report list), a separate id space from run-history run_ids — see Architecture Overview § Persistence. The React Compare page (GET /api/v1/analytics/delta + /analytics/delta/cases) reaches equivalent per-stage and per-test detail by run_id instead, per Delta Comparison Policy.

`testo flaky`

Tests whose outcome keeps flipping across a cycle's recent runs.

testo flaky --cycle fake-api               # last 20 runs
testo flaky --cycle fake-api --window 50 --json
testo flaky --cycle nightly --fail-on-flaky  # exit 1 when any test is flaky and not quarantined (CI gate)
testo flaky --cycle nightly --new --json     # only tests the latest run made flaky

Flag	Default	Meaning
`--cycle`	required	Cycle whose run history to read
`--window`	`20`	How many of the cycle's most recent runs to look at (min 2)
`--new`	off	Only tests the cycle's latest run made flaky: flaky in the window ending at that run, not in the window ending at the run before. Used for alerts and the action's `flaky-issues`
`--json`	off	Print the report as JSON
`--fail-on-flaky`	off	Exit `1` when any flaky test that isn't quarantined is found
`--config`	discovery	`testosterone.yaml` whose `quarantine:` rules mark tests; without a config found, nothing is marked. Exit `2` if the given file is missing or invalid

A test is flaky when, within the window, it passed on a retry inside one run, passed and failed on the same commit, or both passed and failed with its outcome changing at least twice on one branch (skipped runs are ignored; runs that recorded no branch count as one branch). The Why column ("reasons" in --json) says which. One change is a regression or a fix, which testo summary shows. A test that failed each of its last 3 runs is listed under "Broken, not flaky" ("broken" in --json) instead, and --fail-on-flaky ignores it; see Better Flaky Signal - 2026-11-01. Exit 2 when fewer than two runs with per-test results exist. Reads run-history records (run_ids) and each run's artifact snapshot, the same data as GET /api/v1/analytics/flaky?cycle=<name> and the Flaky tests card on the cycle page. See Open Source Features - 2026-10-19.

The table's Quarantined column (and "quarantined" in --json) shows which flaky tests match the cycle's quarantine: rules or the list kept in this machine's run history (testo quarantine). For the others it prints a testo quarantine add --cycle <cycle> "<name>" line per test, which needs no config file, and the quarantine: snippet for those who keep the list in testosterone.yaml. A quarantined test's failures don't fail testo run, testo ingest or pytest --testo; see Flaky Test Quarantine - 2026-10-22 and Quarantine From the UI - 2026-11-01.

`testo ui`

Open the web UI (Dashboard, Cycles, Runs, Compare, AI settings) on this machine's run history: the same database and ./.testo/ data the CLI uses.

pip install 'testo-core[api]'
testo ui                     # http://127.0.0.1:8000, opens the browser
testo ui --port 9000 --no-open

Flag	Default	Meaning
`--port`, `-p`	`8000`	Port on 127.0.0.1
`--open` / `--no-open`	`--open`	Open the default browser once the server is up

It starts testo-api bound to 127.0.0.1 only (for a shared server use the Team Server) and serves the UI that ships inside the wheel, or the build TESTO_UI_DIR points at. The UI's own origin (http://127.0.0.1:<port>, http://localhost:<port>) is added to TESTO_CORS_ORIGINS for this process so it can start runs; every other origin is still refused. Exit 2 when the [api] extra is missing or no built UI is found (a source checkout builds one with scripts/build_ui_bundle.sh, which needs Node 22).

`testo mcp`

Serve the run history to a coding agent (Claude Code, Claude Desktop, Cursor, any MCP client) over the Model Context Protocol, so it can ask what failed, what's flaky and what changed instead of reading logs. Same history as the CLI: DATABASE_URL, else database.url in the config, else the SQLite file next to the project. Design: MCP Server - 2026-11-01.

pip install 'testo-core[mcp]'
testo mcp --config /path/to/project/testosterone.yaml     # stdio; the agent starts it

Flag	Default	Meaning
`--config`	auto-discover	The project's `testosterone.yaml`. Its folder becomes the working directory, so the database and run snapshots are found wherever the agent starts the server. Its `quarantine:` rules mark flaky tests that are already quarantined
`--allow-writes`	off	Also register the write tools. Without it the server is read-only and they don't exist

Add it to the agent's MCP config. Claude Code: claude mcp add testosterone -- testo mcp --config /path/to/project/testosterone.yaml, or a .mcp.json in the project. Claude Desktop (claude_desktop_config.json) and Cursor (.cursor/mcp.json) take the same block:

{
  "mcpServers": {
    "testosterone": {
      "command": "testo",
      "args": ["mcp", "--config", "/path/to/project/testosterone.yaml"]
    }
  }
}

Or let testo mcp install write it: in the project folder, testo mcp install adds the block to .mcp.json (Claude Code), --client cursor to .cursor/mcp.json, --client vscode to .vscode/mcp.json (under servers, "type": "stdio") and --client claude-desktop to Claude Desktop's own claude_desktop_config.json. Project files name testo and the config relative to the project, so they can be committed; Claude Desktop's gets absolute paths. Other servers in the file are kept, an older testosterone entry is replaced, --allow-writes passes through, and a file that isn't a JSON object is left alone (exit 2).

Use the full path to testo (e.g. .venv/bin/testo) when it isn't on the agent's PATH, and set DATABASE_URL under "env" for a Postgres history.

Tool	Returns
`list_cycles()`	Every recorded cycle with its latest run's outcome and counts, and configured cycles never run
`list_runs(cycle=None, limit=20)`	Recent runs, newest first (limit 1 to 100)
`get_run(run_id)`	One run: counts, stages, failing tests (new breakages first), same-cause groups, rerun commands
`failing_tests(cycle)`	The latest run's failing tests: first line of the message, failing since, new or flaky, marked cause, rerun commands
`flaky_tests(cycle, window=20)`	Flaky tests in the last runs with why (passed on retry, mixed results on one commit, flips on one branch), a `PFPF` history and whether they're quarantined; tests failing each of their last 3 runs come back as `broken`
`test_history(cycle, test)`	One test (name, full name or key; a partial name that matches one test) in each of the last 20 runs
`compare_runs(baseline_id, current_id)`	Count and timing deltas, then per-test regressions, fixes, added and removed tests, regressions first
`project_health(cycle=None)`	The project health score, grade, signals, per-cycle health and top findings
`mark_failure_cause(cycle, message, cause, note="")`	`--allow-writes` only. Marks why failures with this message happen in the cycle (`product_bug`, `test_bug`, `environment`, `flaky`, `other`), as the run page does; later failures like it inherit the cause

Answers are compact: messages are cut to their first line (300 characters), lists to 50 entries, and logs and traces are never returned. A bad argument or an unknown run, cycle or test comes back as a tool error the agent can read. The server reads the local history only; to ask a team server, run testo mcp where its database is reachable (DATABASE_URL). Exit 2 when the [mcp] extra is missing (The MCP server needs the mcp extra: pip install 'testo-core[mcp]') or --config doesn't exist.

`testo demo`

See the UI with something in it before you have results of your own. Writes eight JUnit XML files for a small made-up shop app into a new folder, records them with testo ingest as eight runs of the demo-shop cycle, then starts testo ui on that history.

pip install 'testo-core[api]'
testo demo                          # temporary folder, opens http://127.0.0.1:8000
testo demo --dir ./testo-demo --no-ui

Flag	Default	Meaning
`--dir`	a new temporary folder	Where the demo history goes; must be new or empty
`--ui` / `--no-ui`	`--ui`	Start the UI once the runs are recorded
`--port`, `-p`	`8000`	Port for the UI
`--open` / `--no-open`	`--open`	Open the default browser

The runs show a flaky test (test_apply_coupon), a new failure in the latest run (test_redirect_after_login), a test that got slower (test_results_are_ranked) and one that was fixed (test_badge_count). The demo has its own SQLite file and .testo/ folder inside --dir (it sets DATABASE_URL and TESTO_HOME for the commands it starts), so your real history is never touched. Exit 2 when --dir is not empty; otherwise the exit code of the failing testo ingest or testo ui.

`testo try`

See the pull request comment your latest results would get, before you add anything to CI. It records the result files as a run in a throwaway copy of your history and prints what testo report markdown (the job summary and the GitHub Action's comment) says about them. The command-line partner of the /try page.

pytest --junitxml=test-results/junit.xml; testo try    # the files your last run left here
testo try reports/junit.xml --cycle api                # these files, compared with cycle `api`
testo try --from-github                                # the latest GitHub Actions run's artifacts
testo try --from-github --run-id 123456 --out comment.md
testo try --run                                        # run the tests first, the way CI would

Flag	Default	Meaning
`REPORTS...`	found	Result files, folders or globs; none = every result file in `--repo` that git doesn't track, as `testo ingest` finds them
`--repo`	`.`	The project
`--cycle`	folder name	The cycle to compare with
`--from-github`	off	Download the artifacts of the latest finished workflow run with the GitHub CLI (`gh run download`) and preview the result files in them. The workflow must upload them (`actions/upload-artifact`)
`--run-id`	latest	With `--from-github`: this run
`--baseline-branch`	default branch	Compare with this branch's latest run, as a pull request into it would
`--out`, `-o`	stdout	Write the comment to a file
`--run`	off	Run the project's tests first with the command `testo init` puts in CI (its result-file flag included; output on stderr), then preview what they wrote. Not with `--from-github` or `REPORTS`

Nothing is recorded: the run goes into a new temporary folder with its own SQLite file and .testo/ (deleted afterwards). When your history is a local SQLite file (DATABASE_URL or database.url, else ./testo_history.db), it is copied in first, so the preview shows what changed since your earlier runs; otherwise it is a first run's comment. No config is read, so reporters and alerts don't fire. Exit 0 with a preview, even when tests failed; 2 when there are no result files or --from-github can't download them.

`testo notify`

Send a recorded run to Slack and/or Microsoft Teams: pass/fail counts, then new regressions, newly flaky tests, failing tests and fixed tests (10 names per list).

export TESTO_SLACK_WEBHOOK_URL=...         # from a secret, never on the command line
testo notify --to slack --cycle nightly                      # only when it needs attention
testo notify --to slack --to teams --cycle nightly --when change --run-url "$CI_JOB_URL"
testo notify --to teams --cycle nightly --dry-run            # print the payload, send nothing
testo notify --to slack --digest week --run-url https://testo.example.com   # weekly digest (cron)

Flag	Default	Meaning
`--to`	required	`slack` or `teams`; repeat for both. URLs come from `TESTO_SLACK_WEBHOOK_URL` and `TESTO_TEAMS_WEBHOOK_URL`
`--cycle` / `--run-id`	latest run	Which recorded run to send
`--when`	`failure`	`failure`: the run failed or made a test flaky. `change`: the run's result flipped, or a test regressed, got fixed or became flaky. `always`: every run
`--title`	cycle name	Name in the message
`--run-url`	none	"View run" link
`--dry-run`	off	Print the JSON payloads instead of posting
`--digest`	none	`week` or `day`: instead of one run, a digest of every cycle (or `--cycle`) over that period against the one before: runs, green builds, test pass rate, line coverage, tests that turned flaky, still fail, got fixed or got slower. `--run-url` becomes the "Open Testosterone" link; `--when` and `--run-id` don't apply. Run it from cron or a scheduled CI job; a period with no runs sends nothing

Exit 0 when sent or when there is nothing to report, 2 for a bad flag, a missing webhook variable or no recorded run, 3 when a webhook rejects the message or can't be reached (the error never includes the URL). The slack and teams reporters send the same message right after testo run. See Alerts Where Developers Look - 2026-10-22.

`testo issues sync`

Open one GitHub issue for each flaky test of a cycle, assigned to its CODEOWNERS when asked, and close it once the test is stable. The same pass runs on its own after every recorded run (testo run, testo ingest, the team server's ingest and runs) when TESTO_ISSUES=github is set; this command runs it now and prints what it did.

export TESTO_GITHUB_TOKEN=...              # from a secret, never on the command line
testo issues sync --cycle nightly --dry-run   # what it would open and close, changes nothing
testo issues sync --cycle nightly             # every flaky test without an open issue
testo issues sync --cycle nightly --new --repo acme/shop

Flag	Default	Meaning
`--cycle`	required	Cycle whose flaky tests to sync
`--dry-run`	off	Print what would be opened or closed; only reads issues (works without a token on a public repository)
`--new`	off	Open issues only for tests the cycle's latest run made flaky (`testo flaky --new`), as the run hook does. Without it, every flaky test in the last 20 runs is a candidate
`--repo`	`TESTO_ISSUES_REPO`, else the repository the runs recorded	`owner/name` on GitHub (or GitHub Enterprise Server through `GITHUB_API_URL`)

Open. Title Flaky test: <name> (<cycle>), labels TESTO_ISSUES_LABELS (default flaky-test). The body has the pass/fail counts and flip history, the last failure message in a code block, the CODEOWNERS owners, a link to the test's page when TESTO_UI_URL is set, and a hidden  marker. A test with an open issue that has its marker, or the same title (issues the GitHub Action's flaky-issues opened), gets no second one. At most 5 issues are opened per pass; the rest wait for the next one. With TESTO_ISSUES_ASSIGN=1 CODEOWNERS @user owners are assigned (teams and emails can't be); if GitHub rejects an assignee the issue is opened unassigned.
Close. An open issue with the cycle's marker is commented on and closed as completed when its test is no longer flaky in the last 20 runs and passed each of its last TESTO_ISSUES_CLOSE_AFTER runs (default 10). Issues without the marker are never touched.

Exit 0 when the pass ran, 2 with no repository or (without --dry-run) no token, 3 when GitHub refused a request or couldn't be reached. After a run the pass is best-effort: an error is one testo issues: warning line on stderr and never changes the run's exit code. The token needs issues: write (CI-CD Pipeline Setup). Design: Flaky Test Issues - 2026-11-01.

`testo quarantine`

Quarantine a test without a config change: it still runs and is reported, but its failure no longer fails the run. The list lives in the run-history database (table quarantined_test), the same one the UI's Quarantine button and /api/v1/quarantine change, and every run of the cycle adds it to the quarantine: rules of testosterone.yaml.

testo quarantine add --cycle web-unit "test_refund_flaky" --reason "#412" --owner @pay
testo quarantine add --cycle web-unit "tests.payments#*"     # a glob, matched like quarantine: rules
testo quarantine list --cycle web-unit
testo quarantine remove --cycle web-unit "test_refund_flaky"
testo quarantine add --cycle web-unit "test_x" --server https://testo.example.com   # the team server's list

Command / flag	Meaning
`add TEST --cycle C`	Add `TEST` (the name or full name runs record, or an `fnmatch` glob) to cycle `C`'s list; adding it again replaces the entry. `--reason`, `--owner` are free text
`remove TEST --cycle C`	Release the entry, spelled as `list` shows it. Exit `1` when it isn't in the list
`list [--cycle C] [--json]`	Entries, newest first. `testosterone.yaml` rules aren't listed
`--server URL`	Change the list on a team server instead of this machine's (default `$TESTO_SERVER_URL`, token from `$TESTO_API_TOKEN`; writes need a `runner` account or the token). Exit `2` when the server refuses, `3` when it can't be reached

Where the list is applied: testo run, testo ingest and the server's runs and ingest, all through CycleRunService (an ingest cycle also gets the quarantine: rules of a testosterone.yaml cycle with the same name). testo ingest --server exits with the server's verdict, so the server's list decides. pytest --testo checks the session itself (below). The GitHub Action runs testo run on the job's own cached history, so it uses that history's list and the config, not a team server's. A run with --no-persist reads only testosterone.yaml. Matching and the "exit 1 only" rule are the same as for config rules; reading the list is best-effort (a database error is one log line, and the run keeps its config rules).

An entry whose test is an exact name is released on its own once the test passed, on the first try, in each of its last TESTO_QUARANTINE_RELEASE_AFTER runs of the cycle (default 10; 0 turns it off). Skipped runs don't count; globs are never released on their own. Names with *, ? or [ (pytest's test_x[1]) can be stored escaped (test_x[[]1]), which matches only that test and is released the same way. With TESTO_AUTH=accounts the audit log records CLI changes and releases (user testo, method RELEASE) next to the API's. Design: Quarantine From the UI - 2026-11-01.

Automatic quarantine (TESTO_QUARANTINE_AUTO). Off by default. Set it to a flip rate (0.3, or 30) and, after each recorded run, every test that is flaky at that rate or more and failed in that run is added to the list (user testo, the reason says the rate and why it's flaky), so the next run doesn't fail on it. A test that fails every time is broken, not flaky, and is never added; one a rule already covers is skipped. With auto-release it comes back only if it fails again after its clean runs. The run's pull request comment and job summary list the tests it added ("Quarantined automatically"); in the GitHub Action, set the variable on the step's env.

On a laptop. add on a local SQLite history (no --server, no shared DATABASE_URL) also prints the quarantine: lines for testosterone.yaml, because CI (a fresh runner or the Action's cached history) never sees this machine's list.

`testo otel`

Send a recorded run to an OpenTelemetry collector as one trace: a span for the run, one per stage and one per test, with the tests' real start and stop times. Failed tests, red stages and red runs are error spans. Datadog, Grafana Tempo, Honeycomb, Jaeger and any other OTLP backend show it. Needs pip install 'testo-core[otel]'.

export OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.example:4318
export OTEL_EXPORTER_OTLP_HEADERS="api-key=$OTLP_KEY"   # from a secret, never on the command line
testo otel --cycle nightly                               # the cycle's latest run
testo otel --run-id 3f2c... --service-name shop-tests
testo otel --cycle nightly --dry-run                     # print the spans as JSON, send nothing

Flag	Default	Meaning
`--cycle` / `--run-id`	latest run	Which recorded run to send
`--endpoint`	`$OTEL_EXPORTER_OTLP_ENDPOINT`, else `localhost`	OTLP endpoint; a base URL gets `/v1/traces` added for HTTP
`--protocol`	`$OTEL_EXPORTER_OTLP_PROTOCOL`, else `http/protobuf`	`http/protobuf` or `grpc` (`grpc` needs `opentelemetry-exporter-otlp-proto-grpc`)
`--service-name`	`$OTEL_SERVICE_NAME`, else `testosterone`	The trace's `service.name`
`--dry-run`	off	Print the spans as JSON instead of sending (no extra needed)

Headers come only from OTEL_EXPORTER_OTLP_HEADERS (or OTEL_EXPORTER_OTLP_TRACES_HEADERS). Exit 0 when sent, 2 for no recorded run, a bad --protocol, an HTTP endpoint without http:// or https://, or a missing [otel] extra, 3 when the endpoint refuses the spans or can't be reached. The otel reporter sends the same trace right after testo run. Span names and attributes: OpenTelemetry Export - 2026-10-30.

`testo badge`

A README badge for a cycle's latest run: tests | 412 passed · 2 flaky. Green when everything passed, yellow when the cycle has flaky tests (testo flaky's window), red when a test failed or broke, grey with no run.

testo badge --cycle web --out public/testo-badge.svg     # in CI, next to a published report
testo badge --cycle web --format json --out public/testo-badge.json   # a shields.io endpoint badge

Flag	Default	Meaning
`--cycle`	latest run of any cycle	Which cycle's latest run
`--out` / `-o`	stdout	File to write (folders are created)
`--format`	`svg`	`svg`: the image. `json`: what `https://img.shields.io/endpoint?url=<its URL>` reads, for shields.io's styles and logos
`--label`	`tests`	Left-hand text

Exit 0, or 2 for a bad --format. With no run of --cycle, the grey badge is still written and a warning names the closest recorded cycle (testo flaky does the same). A team server serves the same badge live at GET /api/v1/badge.svg?cycle=web and GET /api/v1/badge.json?cycle=web (cached 5 minutes). Those are reads like any other: with TESTO_PROTECT_READS on, a public README can't load them, so publish the file testo badge writes instead. Put it in a README with:

[![tests](https://you.github.io/project/testo-badge.svg)](https://you.github.io/project/)

`testo doctor`

Health check: config load, framework CLIs on PATH, Node.js + Allure Report 3 CLI, optional Docker warnings, DB probe if DATABASE_URL or config URL is set.

testo doctor
testo doctor -c testosterone.yaml

Output: Rich table with Check / Status / Detail rows (PASS, FAIL, WARN, SKIP).

Exits 2 on hard failures (config or missing required executables) — see Troubleshooting and Error Codes § Commands that use exit codes.

`testo plugins`

Lists the built-in frameworks and reporters, then every installed plugin with the package that registered it.

testo plugins            # table
testo plugins --check    # also import each plugin and verify its interface
testo plugins --json

Exits 1 when --check finds a plugin that fails to import or does not implement its interface; otherwise 0.

`testo clean`

Remove artifacts/ (from config or ./artifacts) and ./temp; optional Docker prune.

testo clean --yes
testo clean -y --docker

Without --yes / -y: refuses with exit 2. It also refuses (exit 2, nothing deleted) when the artifacts root or --temp-dir is the filesystem root, your home folder, the current folder, the config's folder, a parent of any of them, or a folder holding .git, testosterone.yaml or pyproject.toml, so artifacts_root: . can't delete the project.

`testo clean --history`

Deletes expired run history instead of artifacts: each expired run's row, its test_result rows and its report copy under static/history/<run_id>/, oldest first. Runs still RUNNING are never touched; report archives (testo report list) are separate and stay.

testo clean --history --dry-run          # list what would go, delete nothing
testo clean --history --yes              # apply history.retention from testosterone.yaml
testo clean --history -y --keep-days 30  # override (or set) the limits for this call
testo clean --history -y --keep-runs 200

The limits come from the top-level history: section (either or both; a run goes when it breaks any):

history:
  retention:
    days: 90            # runs that started more than 90 days ago
    runs_per_cycle: 200 # beyond each cycle's newest 200 runs

--keep-days / --keep-runs override the file's values. Without any limit (no history: section and no flag) it exits 2 and says so; without --yes (and not --dry-run) it exits 2. A report copy that cannot be removed is listed and the command exits 5; the rows are already gone. The API server applies the same section on its own every 6 hours while it runs (Team Server § How it works). Without a history: section nothing is ever deleted.

`testo watch`

Filesystem watcher (watchdog) re-runs a cycle after debounced changes.

testo watch --cycle sample-pytests
testo watch --cycle sample-pytests --path sample_target_repo --debounce-ms 750

Ignores .git, artifacts, venv, __pycache__, etc. Each batch invokes testo run with the same cycle.

`testo init`

Sets a project up in one command: detects its test runners, writes testosterone.yaml and the CI workflow, adds Testosterone's local files to .gitignore and records a first run. Nothing has to be edited by hand for the runners it knows.

testo init                      # detect, write, record a first run
testo init --dry-run            # print the files it would write; write and run nothing
testo init --stack jest --stack go --cycle web --no-run
testo init --interactive        # the older wizard: asks for each value, writes only the config
testo init --scan               # monorepo: a stage per runner in every project folder; asks first
testo init --scan --check       # also ask each runner which tests it would run (runs none)
testo init --pr                 # then commit the files on a new branch, push it and open a pull request

Detected from	Stack (`--stack`)	Stage it writes
`pytest.ini`, `conftest.py`, `[tool.pytest]`, pytest in `requirements.txt` / `pyproject.toml`, `tests/test_.py`	`pytest`	`equipment: pytest`
`jest` in `package.json`	`jest`	`equipment: jest` (Jest's JSON report)
`vitest` in `package.json`	`vitest`	`equipment: vitest`
`@playwright/test` in `package.json`	`playwright`	`equipment: playwright`, `tier: e2e`
`go.mod`	`go`	`command`: `go test -json ./...` into `test-results/go-test.json`
a `*.csproj` referencing a test SDK (xUnit, NUnit, MSTest)	`dotnet`	`command`: `dotnet test --logger trx`
`pom.xml`	`maven`	`command`: `mvn -B test` (or `./mvnw`), Surefire XML
`build.gradle(.kts)`	`gradle`	`command`: `gradle test` (or `./gradlew`)
`rspec` in `Gemfile`, or `.rspec`	`rspec`	`command`: `bundle exec rspec` with the JSON formatter
`phpunit/phpunit` in `composer.json`	`phpunit`	`command`: `vendor/bin/phpunit --log-junit`

Cypress is found but left out (it needs the app running first); the command says so.

Every folder (--scan). Plain testo init reads the files at the top of the project. With --scan it looks in every folder that has a project file (up to 6 levels, skipping node_modules, virtualenvs, build output and hidden folders) and writes a stage per runner per folder, named after the folder (api-pytest, libs-core-pytest, web-jest) with target_repo set so it runs there. A folder an outer stage already covers is left out and listed: Maven modules under a parent pom.xml, Gradle subprojects, packages under a root Jest or Vitest, and Python folders a bare pytest (or pytest tests) above them collects. dotnet test starts at a .sln, else in each test project's folder. The CI file installs and runs each stage in its folder. --scan prints the stages and the command each runs, then asks before writing: --yes (-y) skips the question, --dry-run prints the files, and with no terminal to answer nothing is written (exit 2).

Checked by the runners (--check). Asks each runner to list the tests it would run, without running them: pytest --collect-only, jest --listTests (test files), vitest list, playwright test --list, go test -list, dotnet test --list-tests, rspec --dry-run, phpunit --list-tests. A stage that lists no tests is left out; one whose runner isn't installed yet or fails to list is kept and flagged, so install the project's dependencies first for a full check. Listing loads test code (pytest imports conftest.py, Go and .NET compile), which is why it is opt-in. pytest uses the folder's (or the project's) .venv, Node runners the project's own node_modules/.bin; nothing is downloaded. Maven and Gradle are not checked (they can't list without running). Works without --scan too, and with --dry-run (it still lists, but writes nothing).

What it writes, in the project folder (--dir, default here):

testosterone.yaml: one cycle, named after the folder unless --cycle (the same default as pytest --testo), with a stage per runner.
The CI file (--ci; auto, the default, picks the CI the project already uses, else GitHub Actions; none writes none):
- github: .github/workflows/testosterone.yml: checkout, each language's setup and dependency install (from requirements*.txt or the pyproject.toml test extra, the Node lockfile's package manager and .nvmrc, go.mod, ...), then the GitHub Action running the cycle with comment-on-pr: "true".
- gitlab, jenkins, circleci: .gitlab-ci.yml, Jenkinsfile or .circleci/config.yml that run the tests and record them with testo ingest, as on Setup Snippets. When that file exists it is left alone: the new one goes next to it (.gitlab/testosterone.gitlab-ci.yml, Jenkinsfile.testosterone, .circleci/testosterone.yml) and the command says how to wire it in. With several runners, GitLab and CircleCI get a job per runner, each recording its own cycle (<cycle>-<runner>).
.gitignore: testo_history.db, artifacts/, .testo/ and test-results/, the ones not ignored yet.

A file that exists is kept unless --force; a kept testosterone.yaml with one cycle gives the workflow its cycle name.

The first run (--no-run skips it): result files a test run already left in the project (JUnit XML, Jest/Vitest/Playwright/RSpec JSON, .trx, go test -json, Robot output.xml, allure-results; recognised by content, files git tracks are skipped as fixtures) are recorded like testo ingest would. When there are none, the new cycle runs once. Exit code 0 once a run is recorded, even if some of its tests failed; 2 when no runner was found, --check found no tests in any stage, --scan wasn't confirmed, or an option is wrong.

Straight to a pull request (--pr). After writing (and the first run), switches to a new branch testosterone-setup (-2, -3... if taken), commits only the files it wrote (git commit -- <paths>, so anything else you had staged stays staged), pushes it to origin and opens a pull request into the default branch with the GitHub CLI (gh pr create). Without gh, or when a step fails (no origin, push refused), it prints the commands left to run; the exit code stays the first run's. The pull request's own comment has no baseline yet: once it is merged and the workflow has run on the default branch, later pull requests are compared with that run.

A team server (--server URL). The GitHub workflow also uploads every run to that team server (the Action's server-url and server-token: ${{ secrets.TESTO_API_TOKEN }} inputs). With the GitHub CLI and $TESTO_API_TOKEN set, init stores it as the repository secret (gh secret set, the value on stdin, never printed); otherwise it says which secret to add. For GitLab, Jenkins and CircleCI it prints the --server flag to add. A URL that isn't http(s)://... is exit 2.

Pull requests from forks (--forks). A fork's pull request runs with a read-only token, so the comment can't be posted from it. With --forks the test workflow sets the Action's comment-artifact: "true" (and drops pull-requests: write), and init also writes .github/workflows/testosterone-comment.yml: a workflow_run workflow that downloads the testo-comment artifact and posts it with post-comments-from. It checks out no code from the fork, posts only files that start with the <!-- testo-summary: marker (size-capped, @ mentions defused), and finds the pull request by the tested commit, never from the artifact. GitHub only; ignored for other CI systems.

Dependabot (`--dependabot`). Also writes `.github/dependabot.yml` with weekly `github-actions` updates, when the project has none. `taltal-beep/testosterone@v1` moves to each 1.x release by itself; this keeps `actions/checkout` and the setup actions current.

`testo ai`

AI failure summaries from the command line, with the same settings file as the UI's AI settings page (TESTO_AI_SETTINGS_PATH, default artifacts/ai_settings.json).

testo ai setup                    # picks a provider this machine already has
testo ai setup --provider openai_compatible --base-url http://localhost:11434/v1 --model llama3.2
testo ai status
testo ai summary [--cycle NAME | --run-id ID] [--refresh]

setup detects, in order: OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY / GOOGLE_API_KEY (Gemini's free tier through its OpenAI-compatible endpoint), then the Claude Code CLI. --provider, --model, --base-url and --key-env override. Only the name of the key's environment variable is saved, never the key. Exit 2 when nothing is found.
status prints the provider, model, where the settings came from and whether the key variable is set.
summary prints the AI summary of the latest failed run (of --cycle, or --run-id), generating and storing it on first use so the UI shows the same. Exit 2 when there is no failed run or the provider can't answer.
In CI, set TESTO_AI_PROVIDER (openai, anthropic, openai_compatible, claude_code) with TESTO_AI_MODEL, TESTO_AI_BASE_URL, TESTO_AI_KEY_ENV and TESTO_AI_TIMEOUT_S (default 60): they win over the settings file. The GitHub Action's ai-summary: "true" input adds the summary to the job summary and comment (testo report markdown --ai).

`testo users`

Accounts for a team server that runs with TESTO_AUTH=accounts (Team Server § Accounts). It writes to the database in DATABASE_URL, so run it where the server's database is reachable (for the compose file: docker compose exec server testo users …).

Command	Effect
`testo users add NAME [--role viewer\|runner\|admin] [--password-stdin]`	Create an account (default role `viewer`). Prompts for the password twice unless `--password-stdin`; at least 10 characters
`testo users list`	Usernames, roles and creation dates
`testo users set-role NAME ROLE`	Change the role
`testo users reset-password NAME [--password-stdin]`	Set a new password
`testo users sign-out NAME`	Sign the account out of every browser
`testo users remove NAME`	Delete the account (its audit log lines stay)

set-role, reset-password and sign-out all end the account's existing sessions. Exit code 2 for a bad name, password or role, or an unknown account.

`testo lease`

Give each parallel worker its own key from a resource pool (resources: in testosterone.yaml), so no two workers, on this machine or on other laptops, use the same VM, device or account at once. Guide: Shared Resource Pools.

testo lease run flow-vms -- pytest tests/flows      # hold a key while the command runs
testo lease run flow-vms -l win11 --wait 10m -- ./run_flow.sh
LEASE=$(testo lease take flow-vms --ttl 2h)          # prints "<key> <lease id>"
testo lease heartbeat "${LEASE#* }"; testo lease release "${LEASE#* }"
testo lease list                                     # who holds what, how many wait

Command	Effect
`testo lease run POOL [-l LABEL] [--wait 30m] [--note TEXT] -- CMD...`	Wait for a key, run `CMD` with `TESTO_RESOURCE_ID`, `TESTO_RESOURCE_POOL`, `TESTO_RESOURCE_LABELS`, `TESTO_RESOURCE_DATA`, `TESTO_RESOURCE_<NAME>` per data value and `TESTO_LEASE_ID` set, heartbeat while it runs, give the key back when it ends. Exits with `CMD`'s exit code
`testo lease take POOL [-l LABEL] [--wait 30m] [--ttl DURATION] [--json]`	Take a key and print `<key> <lease id>` (or JSON with the env vars). Nothing heartbeats it: it lives `--ttl` (default the pool's `lease_ttl`) unless you call `heartbeat`
`testo lease heartbeat LEASE_ID`	Keep a lease alive for another TTL. Exit `1` when it is gone
`testo lease release LEASE_ID [--clean]`	Give the key back. `--clean`: you reset it yourself, so the next taker gets `TESTO_RESOURCE_CLEAN=1`
`testo lease list [POOL] [--json]`	Each pool's keys: free (and whether it was left clean), held (by whom, since when), expired (free again after the pool's `reuse_delay`) or disabled, and how many requests wait
`testo lease disable POOL KEY [--reason TEXT]` / `enable POOL KEY`	Take a broken key out of the pool, put it back
`testo lease free POOL KEY`	Take a key back from its holder (needs `admin` on a server with accounts)
`testo lease demo [--clear]`	Fill this machine's database with made-up pools (`demo-flow-vms`, `demo-phones`, `demo-license-seats`), some keys held by invented people, one disabled and two requests waiting, to try `testo lease list` and the Resources page. Never touches a team server. `--clear` gives the keys back

Flag	Default	Meaning
`-l`, `--label`	none	Only keys with this label; repeat for several
`--wait`	`30m`	How long to wait for a free key: `90s`, `10m`, `2h`, `0` (try once) or `forever`
`--server`	`$TESTO_SERVER_URL`	Team server that keeps the leases (token from `$TESTO_API_TOKEN`). Unset: this machine's run-history database
`--config`	discovery	`testosterone.yaml` with the pools

Exit codes: 2 for an unknown pool, a bad config or no command after --; 3 when no key came free within --wait, the pool has no usable key (none match the labels, or all are disabled), or the server can't be reached; 1 when heartbeat or disable/enable/free finds nothing to act on.

`testo pool`

Compare resource pools with the keys that really exist, and add or remove keys with the pool's own provider commands. Testosterone never creates or deletes a key on its own: add, remove and sweep are dry runs unless --apply. Guide: Shared Resource Pools.

Command	Effect
`testo pool status [POOL] [--check] [--json]`	Read-only. For each pool: which keys `provider.list` shows (`yes`, `missing`, or `?` without a list command), which listed keys aren't in the pool, and each key's lease state. `--check` runs `on_check` on every free key that exists
`testo pool add POOL [KEY...] [-n N] [--apply] [--yes]`	Make keys with `provider.create`: the names given, or `N` names `<name_prefix><n>`. Stops at `max_items`. Doesn't edit your config
`testo pool remove POOL KEY... [--apply] [--yes]`	Remove keys with `provider.destroy`. Skips keys in `items`, held keys, names without `name_prefix` and keys `provider.list` doesn't show
`testo pool sweep POOL [--apply] [--yes]`	Remove leftovers: keys `provider.list` shows with `name_prefix` that aren't in the pool or held, 5 at a time
`testo pool log [POOL] [--limit 20] [--json]`	Every `create`/`destroy` run with `--apply`: when, which key, who and from where, and how it went

add, remove and sweep print the commands and run nothing without --apply; with it they ask first unless --yes. They need the verb under provider.allow, take at most 5 keys per command, and are refused when TESTO_PROVIDER_ACTIONS=0.

Flags --server and --config as for testo lease. Exit codes: 0 done (or a dry run), or every pool matches for status; 1 a key is missing, not in the pool or fails its check (status), or a command failed or every key was skipped (add/remove/sweep, also when you answer no); 2 an unknown pool, a bad config, a verb that isn't allowed, too many keys, or TESTO_PROVIDER_ACTIONS=0; 3 a provider.list command failed, the lease server can't be reached, or a command ran but logging it failed (nothing more runs).

`testo split`

Split a cycle's tests into shards that take about the same time, from each test's median time over the cycle's recent runs (longest test first, each to the shard with the least time so far). For CI matrix jobs, or one shard per key of a pool. Guide: Shared Resource Pools § Split tests by time.

testo split --cycle flows --shards 4                       # table: tests and time per shard
pytest --collect-only -q | testo split --cycle flows --shards 4 --index 2 --tests -   # shard 2's node ids
testo split --cycle flows --shards 4 --json                # every shard, with its tests

Flag	Default	Meaning
`--cycle`	required	Cycle whose test times to use
`--shards`	required	How many shards
`--index I`	none	Print only shard I's tests (1 to `--shards`), one per line
`--tests FILE`	the cycle's history	Tests to split, one per line (`-` reads stdin). pytest node ids are matched to their recorded names
`--stage`	all	Only this stage's tests
`--server`	`$TESTO_SERVER_URL`	Read the times from a team server (token from `$TESTO_API_TOKEN`)
`--window`	`20`	Recent runs to take the times from
`--allow-empty`	off	With `--index`, exit `0` and print nothing when the shard has no tests (fewer tests than shards)

Tests with no recorded time count as a typical (median) test; with no history at all the split is round robin. The split depends only on which tests there are and their times, not their order, so every shard that sees the same history picks the same tests. Exit codes: 2 for a bad --index or unreadable --tests, 3 when the server can't be reached, 1 when there is nothing to split or (without --allow-empty) the --index shard is empty, since an empty pytest $(testo split ...) would run every test. A --server (or $TESTO_SERVER_URL) that isn't an http(s)://host URL exits 2 here and in testo ingest and testo lease.

`testo affected`

Pick the tests a change can affect, and say why each one was picked. Opt-in: nothing runs fewer tests unless you use this command or pytest --testo-affected. Design: Affected Test Selection - 2026-10-30.

testo affected                                   # tests to run, each with its reasons
testo affected --base origin/develop --json      # the selection, reasons and per-file decisions
npx jest $(testo affected --list)                # nothing printed = run everything
git diff --name-only HEAD~1 | testo affected --changed -

Flag	Default	Meaning
`--base REF`	`affected.base`, else the pull request's target branch in CI, else `origin/main`	Diff the working tree (and untracked files) against the merge base with this
`--changed FILE`	git diff	Changed files, one per line, relative to the repo root (`-` reads stdin)
`--coverage FILE`	`affected.coverage`	coverage.py JSON with test contexts: `pytest --cov --cov-context=test`, then `coverage json --show-contexts`
`--config`	discovery	`testosterone.yaml` with the `affected:` section
`--list`	off	Only the tests, one per line; prints nothing when everything should run
`--json`	off	`run_all`, `run_all_reasons`, `nothing_to_run`, `tests` (with `reasons`) and `files` (each changed file's outcome)

Each changed file maps to tests by the first of: run_all_when (run everything), ignore (no test), a changed test file (it runs), rules, the coverage report. Anything else, an unknown base or an empty diff runs everything, with the reason. Exit codes: 0 for any selection, including "run everything"; 2 for a bad config, an unreadable --changed or --coverage, or both --list and --json.

affected:
  base: origin/main            # optional
  coverage: coverage.json      # optional, relative to this file
  run_all_when: ["src/core/**"]   # added to conftest.py, pyproject.toml, lockfiles, testosterone.yaml
  ignore: ["docs/**", "*.md"]
  test_files: ["test_*.py"]    # replaces the built-in names of test files
  rules:
    - changed: "src/payments/**"
      tests: ["tests/payments/**", "tests/test_api.py::test_checkout"]

`testo backup` / `testo restore`

Copy the whole run history (every database table and each run's stored results under static/history/) into one .tar.gz, and load it back, into the same or another database (SQLite, Postgres, MySQL).

testo backup                                   # ./testo-backup-<date>-<time>.tar.gz
testo backup --out /backups/testo.tar.gz --no-files
DATABASE_URL=postgresql://... testo restore testo-backup-20261101-090000.tar.gz
testo restore old.tar.gz --replace --yes       # delete the current history first

Flag	Default	Meaning
`backup --out` / `-o`	`testo-backup-<UTC time>.tar.gz`	Where to write it (written as `.partial`, then renamed)
`backup --no-files`	off	Tables only, without the per-run result snapshots (Compare, flaky and the test page need those for old runs)
`restore PATH`	required	A file `testo backup` wrote
`restore --replace`	off	Empty the tables first. Without it, a database that already has runs is refused
`restore --yes` / `-y`	off	Don't ask before `--replace`

Only columns today's tables have are written, so an older backup restores into a newer version. The AI settings file (it holds API keys) is never included; the user_account table holds password hashes, so keep backups as private as the database. Exit 0 on success, 2 for a file that isn't a backup or a database that already has runs, 3 when the database can't be reached.

`testo version`

testo version
# testo 0.1.0

# Equivalent top-level eager flag (works before/instead of a subcommand):
testo --version
testo -v

`pytest --testo`

testo-core installs a pytest plugin (entry point pytest11, testo_core/pytest_plugin.py). The pytest-testo package (integrations/pytest-testo/) installs the same thing under a pytest-* name. It does nothing unless asked:

Option	Effect
`--testo`	Record this session as a run (also `TESTO_RECORD=1`)
`--testo-cycle NAME`	Cycle to record under; implies `--testo`. Default: the rootdir's name
`--testo-tier unit\|integration\|e2e`	Pyramid tier of the tests (default `unit`)
`--testo-server URL`	Upload to a team server instead (default `$TESTO_SERVER_URL`, token from `$TESTO_API_TOKEN`)
`--testo-resource POOL`	Give each worker its own key from this resource pool (from `testosterone.yaml`, or `$TESTO_RESOURCE_SPEC` set by a stage's `resources:`). Tests read it from the variables or the `testo_resource` fixture
`--testo-resource-per worker\|test`	One key per worker (default) or a fresh one per test
`--testo-resource-label LABEL`	Only keys with this label (repeat for more)
`--testo-resource-wait DURATION`	How long to wait for a key (default `30m`); then the test fails
`--testo-shard I/N`	Run only shard I of N, split by time like `testo split` (also `$TESTO_SHARD`). Bad value or unreachable server: usage error (exit `4`), never a guess
`--testo-affected`	Run only the tests the changes since the baseline branch can affect, like `testo affected` (also `$TESTO_AFFECTED=1`); deselects the rest before `--testo-shard` splits. The summary lists what was picked and why, or why everything ran. A picked Python test that isn't collected runs everything; when only ignored files changed, no test runs and pytest exits `0`
`--testo-affected-base REF`	Branch or commit `--testo-affected` diffs against (default as `testo affected --base`)
`--testo-failed-first`	Run the tests that failed last time first, then flaky ones, then ones that failed in the last 20 runs, then the rest in pytest's order (also `$TESTO_FAILED_FIRST=1`). History comes from the same cycle as `--testo-timings` (local, or `--testo-server`). Applies after `--testo-affected` and `--testo-shard`; works with pytest-xdist. The summary says how many moved; when no history can be read, pytest's order is kept
`--testo-timings CYCLE`	Cycle whose test times `--testo-shard` uses (default `$TESTO_CYCLE`, which `testo run` sets, else the rootdir's name; from `--testo-server` / `$TESTO_SERVER_URL` when set)

The session's JUnit XML (yours if --junitxml is set, else a temporary file in a .testo-* folder of the rootdir, removed afterwards) is recorded through the same path as testo ingest, into a pytest stage, and the run writes artifacts/ like testo ingest does. One line in pytest's summary says where it went. Recording never changes pytest's exit code: a problem is reported on that line instead. The one exception is quarantine: when pytest exits 1 and every failed test matches the cycle's quarantine (the quarantine: rules of a testosterone.yaml cycle with that name, plus the stored list of testo quarantine, read from --testo-server when set), the session exits 0 and a testo: exit 0 because every failed test is quarantined … line names them. Nothing is recorded when no tests ran, or on a pytest-xdist worker (the controller records).

Other entry points (not `testo` Typer tree)

Command	Package	Role
`testo-api`	`testo_api`	FastAPI server (React UI backend)

The v1.0 uqo command (including its uqo run --config headless format with --ghost, --json, --stream-json) and the testo-ui Streamlit app were removed in v1.1; use testo run --cycle … --ci and the React frontend.

Environment variables (common)

Variable	Effect
`DATABASE_URL`	Overrides `database.url` in YAML for archives / list
`TESTO_HOME`	Where per-run report copies (`static/`, `static_reports/`) are written. Default: `./.testo` for a `pip install`, the repository root for a source checkout (Architecture Overview § Artifact layout)
`INFLUXDB_URL`, `INFLUXDB_TOKEN`, `INFLUXDB_ORG`, `INFLUXDB_BUCKET`	Push run KPIs to InfluxDB after each run
`PROMETHEUS_PUSHGATEWAY_URL`, `PROMETHEUS_JOB_NAME`	Push run KPIs to a Prometheus Pushgateway after each run
`TESTO_SLACK_WEBHOOK_URL`, `TESTO_TEAMS_WEBHOOK_URL`	Webhooks for `testo notify` and the `slack` / `teams` reporters
`TESTO_ISSUES`	`github`: after every recorded run, open an issue per newly flaky test and close fixed ones (`testo issues sync`)
`TESTO_GITHUB_TOKEN`, `GITHUB_TOKEN`	Token for those issues (`issues: write`); the first one set wins
`TESTO_ISSUES_REPO`	`owner/name` for the issues, instead of the repository the runs recorded
`TESTO_ISSUES_LABELS`	Comma-separated labels on the issues (default `flaky-test`; empty for none)
`TESTO_ISSUES_ASSIGN`	`1`: assign the test's CODEOWNERS users
`TESTO_ISSUES_CLOSE_AFTER`	Passing runs in a row before an issue is closed (default `10`)
`TESTO_QUARANTINE_RELEASE_AFTER`	Clean passing runs in a row before a stored quarantine entry for one test is released (default `10`; `0`: never). See `testo quarantine`
`TESTO_UI_URL`	Base URL of the web UI, for the test page link in issues
`GITHUB_API_URL`	GitHub API for issues (default `https://api.github.com`; GitHub Enterprise Server: `https://<host>/api/v3`)
`OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `OTEL_EXPORTER_OTLP_PROTOCOL`, `OTEL_SERVICE_NAME`	Where `testo otel` and the `otel` reporter send traces (the standard OpenTelemetry variables, `_TRACES_` forms too)
`TESTO_SERVER_URL`, `TESTO_API_TOKEN`	Team server for `testo lease`, `testo quarantine --server` (and `pytest --testo-server`)
`TESTO_LEASE_DB_URL`	Database for local leases when it isn't the run-history one
`TESTO_SHARD`	`I/N`: run only that shard of a pytest session (`--testo-shard`)
`TESTO_REDACT`	`0` turns off hiding secrets in a run's stored results and logs (on by default: API keys, tokens, passwords, credentials in URLs, private keys)
`TESTO_REDACT_PATTERNS`	Your own regular expressions to hide, one per line
`TESTO_FAILED_FIRST`	`1`: same as `--testo-failed-first`
`TESTO_CYCLE`, `TESTO_STAGE`	Set by `testo run` for each stage's process: which cycle and stage it is
Reporter tokens	e.g. `REPORTPORTAL_TOKEN`, `SLACK_WEBHOOK` via `${env:…}` in YAML

Reporter types (`reporters:` / `--reporter`)

Supported: allure, extent, otel, reportportal, slack, teams, testbeats, plus any installed reporter plugin. Configured in root testosterone.yaml; outputs often under reports/ (see repo testosterone.yaml for examples).

slack and teams post the testo notify message for the run that just finished. They need a recorded run (not --no-persist):

reporters:
  - type: slack
    webhook: ${env:SLACK_WEBHOOK_URL}   # default: $TESTO_SLACK_WEBHOOK_URL
    when: failure                        # failure | change | always
    title: Nightly API tests             # default: the cycle name
    report_url: ${env:CI_JOB_URL}        # optional "View run" link

otel sends the testo otel trace for the run that just finished (recorded runs only, [otel] extra). Options left out fall back to the OTEL_* variables:

reporters:
  - type: otel
    endpoint: https://otlp.example:4318   # default: $OTEL_EXPORTER_OTLP_ENDPOINT
    headers: ${env:OTLP_HEADERS}          # "key=value,..."; default: $OTEL_EXPORTER_OTLP_HEADERS
    protocol: http/protobuf               # or grpc
    service_name: shop-tests              # default: $OTEL_SERVICE_NAME or testosterone

testbeats posts counts only, to the slack_webhook / teams_webhook options.

After testo run, terminal hints may point to ./reports/allure, ./reports/extent, etc.

Official documentation

Reporter	Reference
Allure Report 3	https://allurereport.org/docs/v3/
ReportPortal	https://reportportal.io/docs/api-development/
ReportPortal agents	https://reportportal.io/docs/log-data-in-reportportal/test-framework-integration/

Plugins

A Python package can add a test runner (equipment:) or a reporter (reporters:) without changing Testosterone, through two entry point groups:

# the plugin's pyproject.toml
[project.entry-points."testo.frameworks"]
unittest = "testo_example_plugin:UnittestAdapter"

[project.entry-points."testo.reporters"]
json-summary = "testo_example_plugin:JsonSummaryReporter"

After pip install, the names work in testosterone.yaml and --reporter, and testo plugins lists them.

Group	Value	Interface
`testo.frameworks`	A class or factory returning an adapter	`FrameworkAdapter` in `testo_core/frameworks/base.py`: `name`, `results_subdir()`, `build_argv(target_repo, results_dir, stage_args, workers)` and `native_report(stage_dir)`
`testo.reporters`	A `BaseReporter` subclass	`testo_core/reporting/reporters/base.py`: `reporter_type` and `publish(results, context, console)`; YAML keys other than `type` arrive as `self._options`

The stage process runs build_argv's command in target_repo. Tests are counted from Allure *-result.json files written to results_dir (also exported as TESTO_SHARED_ALLURE_RESULTS_DIR), or from the stage's junit_xml globs.
A plugin stage defaults to tier: unit and gets workers passed through unchanged.
Built-in names always win; a plugin that reuses one, or a second plugin with the same name, is ignored with a warning.
Plugins are imported only when a stage or reporter uses them, or by testo plugins --check.
The API's ad-hoc run endpoint (POST /api/v1/adhoc-executions) accepts built-in frameworks only.

examples/plugin is a complete plugin with one of each. Design: Plugin API.

Related operational docs

CI-CD Pipeline Setup — testo run --ci in CI wrappers
Delta Comparison Policy — semantics for testo diff / testo summary

Generated from docs/CLI Commands/Command Reference.md. Edit it there; changes made in the wiki are overwritten.

Home

Architecture

CLI Commands

Processes & Guides

Prompts & Snippets

Roadmap & Strategy

Specs & ADRs

Testing Workflows

Vault

Archive

Command Reference

Command Reference

testo run

Options

Expected terminal output

Exit codes

testo ingest

testo report

Default callback options

Subcommands

testo cycles (alias: testo plans)

testo config

Editor autocomplete

testo diff / testo summary

testo flaky

testo ui

testo mcp

testo demo

testo try

testo notify

testo issues sync

testo quarantine

testo otel

testo badge

testo doctor

testo plugins

testo clean

testo clean --history

testo watch

testo init

Dependabot (--dependabot). Also writes .github/dependabot.yml with weekly github-actions updates, when the project has none. taltal-beep/testosterone@v1 moves to each 1.x release by itself; this keeps actions/checkout and the setup actions current.

testo ai

testo users

testo lease

testo pool

testo split

testo affected

testo backup / testo restore

testo version

pytest --testo

Other entry points (not testo Typer tree)

Environment variables (common)

Reporter types (reporters: / --reporter)

Official documentation

Plugins

Related operational docs

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

`testo run`

`testo ingest`

`testo report`

`testo cycles` (alias: `testo plans`)

`testo config`

`testo diff` / `testo summary`

`testo flaky`

`testo ui`

`testo mcp`

`testo demo`

`testo try`

`testo notify`

`testo issues sync`

`testo quarantine`

`testo otel`

`testo badge`

`testo doctor`

`testo plugins`

`testo clean`

`testo clean --history`

`testo watch`

`testo init`

Dependabot (`--dependabot`). Also writes `.github/dependabot.yml` with weekly `github-actions` updates, when the project has none. `taltal-beep/testosterone@v1` moves to each 1.x release by itself; this keeps `actions/checkout` and the setup actions current.

`testo ai`

`testo users`

`testo lease`

`testo pool`

`testo split`

`testo affected`

`testo backup` / `testo restore`

`testo version`

`pytest --testo`

Other entry points (not `testo` Typer tree)

Reporter types (`reporters:` / `--reporter`)