TestPion 0.48.0

Behaviour changes: what works differently (read before upgrading)

• Suggest assertions, Generate tests and Describe a request now open a review instead of adding their result at once. Each suggested check, test or request is checked first and shown with its verification; tick what you want, then Add to the checks, Add to the Post-response script (after your own script, never replacing it) or Open as a new request. Nothing is added before you accept. (Proposals)
• AI features need a model you set up: your Claude (Anthropic) key, or a provider of the workspace in Settings ▸ AI assistant; a model on your computer (Ollama (local), the new LM Studio (local) preset) works offline, and Local only keeps every AI call on this computer. Without a model, an AI action offers Set up AI and sends nothing. (Set up a model)
• AI calls can be capped: tokens per call, tokens per day and cost per day (Settings ▸ AI assistant ▸ Limits). A call over a cap is refused before anything is sent, with Change the limits; calls from the CLI, MCP agents and CI count and are capped too. (Cost)
• More is masked before anything reaches a model: besides secret values and sensitive field names, credentials found by their shape in any field or text (JWTs, Bearer/Basic credentials, private keys, provider API keys, credential query parameters) and personal data (e-mail addresses, phone numbers, Luhn-checked card numbers, IBANs; Mask personal data turns that off). Large contexts are sampled (first, last, failed, one of each shape) and summarised to their shape instead of being cut mid-JSON.

Let AI write the integration flow

• Right-click a collection or a folder ▸ Generate integration flow with AI… (also in a collection's Export menu, and the flow canvas's Generate with AI, which now offers From a collection or folder beside Describe in words): your model turns the folder's saved requests into one flow — create, read, update, delete in order, the ids and tokens each step returns extracted and passed on, status and key-field checks, one or two error cases — and says what it assumed. It starts from the flow TestPion builds without AI and names saved requests by id: a step naming a request that is not in the folder is dropped (the review says so), and a URL may only gain {{variables}} on the request's own host. (Let AI write the flow)
• Checked before you see it: the flow is linted (cycles, unknown dependsOn), every {{variable}} must be extracted by a step it waits for (or be one your workspace has), and it runs against a mock of the collection when it has saved examples; a problem goes back to the model up to twice.
• Reviewed on a canvas: the proposed flow opens as a preview marked AI-generated with its summary, assumptions, what TestPion changed, verification badges and YAML (edit it there; Check again verifies the edit); Try on mock, Regenerate with a hint, Discard, Accept — which writes tests/flows/<collection>-<folder>.yaml and offers Run now and Open (and Undo). Nothing is saved or run before Accept. The dialog shows the token and cost estimate and What will be sent first (requests, body shapes, variable names — never values; Metadata only sends no body values; large folders are sampled). Without a model: Set up AI / Add the key, or Use the draft without AI.
• Terminal and agents: testpion flow generate --collection <name|id> [--folder <name|id|path>] [--hint …] [--privacy metadata] [--dry-run] [--apply] [--no-ai] [--json], and the MCP tool flow_generate (it proposes; ai_apply_proposal writes). Without a model set up both return the deterministic draft with a note. (CLI, MCP)

Suggested checks and tests are proven before they are offered

• Suggest assertions runs each suggested check against the response on screen — and the one before it in History: a check that passes now but failed before (an id, a time, a token) is unticked as the value changes, and a check that pins a generated value is flagged; both offer Safer: forms (exists, type, a format regex) that are checked again when you pick one.
• Generate tests writes one tp.test per item, each run in the sandbox against the response (and the previous one); a failing test is shown unticked with its message, and pm.* / postman.* is refused and sent back to be written as tp.*.
• Run now in the review sends the request again (the usual way, under the production guard) and checks every item against the new response. Try on mock does the same for a proposed flow.
• YAML from a model is made safe first: a value starting with {{ written without quotes (url: {{baseUrl}}/pets, a YAML error models often repeat) is quoted before the flow or test file is checked — the review says Fixed N unquoted template values — and a YAML error that remains goes back to the model with its likely cause in plain words (unquoted {{, tabs, indentation, a duplicate key). The flow and test prompts say to quote such values, and the test and check prompts say what a script can read (the response time only when it is known).
• The model is told the test-file format: the flow and test prompts carry a compact reference made from the same schema and check registry the lint uses (step keys, each response check with an example, one complete flow), and a check written in the wrong shape goes back with the right shape and the allowed types.
• CLI and MCP parity: testpion ai run suggest-assertions --item request:<collection>:<request> --json (alias of generate-assertions; generate-tests of generate-pm-tests), MCP ai_run_task with the same ids.

AI features you can trust: see what is sent, cap the cost, review every change

• What will be sent… beside every AI button (the eye icon) and in the assistant's header: the model and whether it stays on this computer, the privacy level (switchable), each context section with its tokens and how a large one was fitted, what was masked and where, the token and cost estimate, and the exact messages — before anything is sent. Hovering an AI button shows the model and the estimate. (What is sent)
• Privacy levels: Full (redacted), Metadata only (shapes and names, no body values) and Names only; the default in Settings ▸ AI assistant, and each action can change it.
• Local only: Settings ▸ AI assistant ▸ Local only allows only Ollama, LM Studio or another server on 127.0.0.1 (CLI/CI: TESTPION_AI_LOCAL_ONLY=1); a cloud model is refused before anything is sent.
• Models per task: a model for small tasks (explanations, triage, commit messages) and one for generation (tests, flows, checks); CLI/CI: TESTPION_AI_PROVIDER, TESTPION_AI_MODEL, TESTPION_AI_MODEL_SMALL / TESTPION_AI_MODEL_LARGE.
• AI usage: AI Lab ▸ Usage now starts with AI features: every AI call from the app, the CLI, MCP agents and CI, per task, model and surface, with what is left of the caps; testpion ai usage, MCP ai_usage.
• Proposals, verified, reviewed in one place: suggested checks, tp tests, flows, coverage tests and requests come as a proposal whose items the engine checked first — checks against the response they were written for, scripts run once in the sandbox, test files linted and loaded, flow variables traced, JSON validated. A failing item is sent back to the model (up to twice), then shown unticked with the problem, never hidden; an answer cut at the output limit is refused instead of used. The review shows each item's diff and badges: Accept selected (Ctrl+Enter, one Undo), Regenerate with a hint, Discard. Proposals are kept a day, so one made in the app can be applied from the CLI or by an agent.
• The same AI from the terminal and for agents: testpion ai tasks|run|apply|undo|discard|proposals|usage (all with --json; ai run --dry-run prints what would be sent) and the MCP tools ai_list_tasks, ai_run_task, ai_proposals, ai_apply_proposal (confirmed), ai_undo_proposal, ai_usage. (CLI, MCP, AI features)

More

• AI settings that can't break the app: an ai value of the wrong shape in settings.json (a hand edit, another version) falls back to the defaults field by field — no model for a tier, Local only off, full privacy, masking on, no cap — instead of refusing every call.
• Proposals stay in the workspace: a proposal can only write test files under tests/, and a proposal id or a hand-edited proposal that points elsewhere is refused before anything is written or undone.
• One AI button everywhere: every AI action of the window (Explain with AI, How to fix (AI), Analyze with AI, Suggest assertions …) is the same control with What will be sent… beside it, and every AI setup problem (a missing key, no model, Local only, a cap) offers its fix the same way.
• Under the hood: the AI assistant's tasks (27, with the new flow-from-collection) moved from the desktop app into the engine: one task runner with versioned prompts, structured output (JSON schema where the provider has it, validated and repaired), verifiers, proposals and the usage ledger, used by the app, the CLI, the MCP server and (later) the cloud host. Golden cases for every AI task run with the mock provider in npm test; AI_EVAL_LIVE=1 runs them against a real model with a scorecard. (For contributors)

Downloads

See the download page for which file to choose and the installation guides.

The installers aren't code-signed yet: Windows SmartScreen and macOS Gatekeeper ask you to confirm the first launch.