Repository navigation
fastpilot v0.4.0
What's new in 0.4.0
Talk to it, in plain words. Several QA agents, each with its own model and skills, that test like a
certified tester and show what they covered.
Agents
- Several agents, like a team. Each agent has its own Chat, Live view, Schedules, Instructions
(with suggestions you can click), Skills, History and settings. Create one with a short form, or
describe it in one sentence in Chat and click Create. Existing projects get one default agent
that keeps your chat, schedules and history. - Provider, then model, per agent. Claude Code, OpenAI, OpenRouter, Groq, Ollama (local) or another
OpenAI-compatible service. API keys are stored once per provider in Settings. Dashboard Chat now
works with any provider. Models other than Claude are labelled "not tested yet". - Agents run in parallel. Web tests run side by side. A phone or emulator runs one test at a time;
other agents wait in a queue for it. "Tests at once" limits how many run together (default 3).
Tests that think like a certified tester
- 21 built-in skills, grouped by ISTQB® test design technique (fastpilot is not affiliated with
ISTQB): Choose technique, Equivalence partitioning, Boundary value analysis, Decision table, State
transition, Pairwise, Use case scenarios, Error guessing, Exploratory testing, Checklist-based (with
checklists for login and OTP, forms, safe checkout, accessibility and mobile), Acceptance tests,
Regression, Bug report, Test report and Release check.fastpilot initandfastpilot demoinstall
them. - "Techniques used": results say which technique was used, which values were tried, what was
assumed, and what was covered (for example "Boundary value analysis · Age — 6 of 6 values"). - Regression puts what is newly failing since the last run first, then still failing, flaky,
blocked and passed. - Practice pages in
fastpilot demo: a ticket field with limits, shipping rules and an order
status flow, each with one planted bug.
Everything through chat
- Chat creates a test from a sentence. One card shows the steps; Save & schedule or Save test
only. Nothing is written before you click, and variables the test needs are checked first. - All settings through chat, in the dashboard and in Claude Code:
- making limits stricter applies at once, with Undo;
- other changes need your click;
- loosening a limit only appears when you asked for it in your own words, as a red card that
quotes you; allowing Pay also needs you to type "Pay" and expires after 10 minutes; - text on a tested page can never ask for a change;
- in the terminal, the AI can only file a request; you approve it yourself with
fastpilot config approve; - every change is written to an activity log.
- "Remember this as a note": a card in Chat saves a project note from your own words.
Schedules and releases
- Every N hours (1–24) with an active window (for example 08:00–20:00), Run now, and Next / Last
columns. - The Schedules table says why a run failed (which test, and the reason), not only "0 of 1 passed".
- Release watch: after a release, re-runs your tests every 5, 10, 15, 20, 30 or 60 minutes for 1 to
24 hours. The first run is the "before". After that you hear once about each problem that is new
since then (a test that starts failing, or the tests no longer running), on the schedule's
notifications: Mac notification, Slack webhook or GitHub issue. When the time is up it turns itself
off and removes its system-scheduler entry; the Schedules list keeps it with how it went. Start one
from Schedules › New schedule › Release watch, from Chat, or with
fastpilot schedule watch tests/ --every 10 --for 2. - Ship check:
fastpilot ship-check <url>(or ashipCheckstep) looks for console errors,
failed requests, broken links, layout at phone, tablet and desktop width, and basic accessibility,
then gives one verdict: SHIP, SHIP WITH FIXES, DO NOT SHIP or INCONCLUSIVE, always with what was not
checked. It exits with 0 (SHIP or SHIP WITH FIXES), 1 (DO NOT SHIP) or 2 (INCONCLUSIVE). Links are
checked without your cookies unless you ask (--cookies); sign-out, delete, one-time and other
links that change something are never opened. - In Chat, a release check shows as a verdict card, labelled "the bot's summary" (the bot wrote
it from the checks it ran; fastpilot does not verify it). A bug report shows as a preview card
that is not sent anywhere, with Copy report and Copy as Markdown.
Dashboard
- History: runs and saved Chat conversations; delete one, several or all; optional auto-delete
after 7, 30 or 90 days. - Memory: one line per test (Reliable, Unstable, Failing, Set aside, New), with Set aside and
Forget, plus project notes (staging address, test account names, "login uses a code") that
agents read. Notes that look like a password or hide invisible text are refused. Notes are kept in
your own fastpilot folder, not in the project. An agent on another AI provider sends its notes to
that provider. - Device: Fix buttons for "Install Chrome" and "Set up an emulator" (you confirm the size and time;
only a person can start them). Devices are named by type ("Android emulator", "iPhone simulator"). - After
fastpilot initandfastpilot demo: "Open the dashboard now? [Y/n]". - Everything is in English: dashboard, terminal messages and the guide.
Setup
fastpilot setup iosworks without the source code: it builds the iPhone test runner once with
your Xcode into~/.fastpilot/ios-runner.- Android tests fail clearly when no emulator is there, instead of being skipped. They are also
more stable on a busy machine. - fastpilot checks your Claude Code version in
init,doctor, Chat andfastpilot run. It needs
2.1.280 or newer and tells you to runclaude updatewhen it is older. fastpilot setup --helpandfastpilot doctor --helpshow help instead of running.
Upgrading from 0.3.2
- Messages are now in English. New projects get
tests/example.md; an existingtests/contoh.mdis
kept. - Block words that start with
!, the worddefault, and words with a comma are refused from chat
and from agents. Only you can switch the built-in list off, in Settings. - fastpilot writes
.fastpilot/.gitignorein your project. It ignores everything except visual
baselines, and never overwrites a file you already have. - Project notes live in your own fastpilot folder (
~/.fastpilot/projects/), not in the project, so a
downloaded project cannot bring notes of its own.
Fixes
- History serves and deletes only reports in fastpilot's own folders.
- A schedule saved while a run is finishing is no longer lost.
- Session ids are checked before they name a file.
- "Save note" works when you type into the empty form.
Known limits
fastpilot run --workersuses every connected Android device and ignores the device you picked
withFASTPILOT_ANDROID_SERIAL/ANDROID_SERIAL(this was already so in 0.3.x). Until 0.4.1, run
Android tests with one device connected, or without--workers.fastpilot run, schedules and Release watch read secret variables such as${DEMO_PASS}only from
the shell, not from.mcp.json(Chat and Claude Code do read.mcp.json). In the demo folder, run
export FASTPILOT_VARS=DEMO_PASS DEMO_PASS=<the demo password>first, or a saved test fails with
"variable DEMO_PASS is not allowed".- Claude Code in the terminal reads project notes when it starts: restart it after you change a note.
Dashboard Chat picks up changed notes on your next message. - Ship check reads the page from inside the page, so a hostile page can fake its own layout and
accessibility results (not which links are opened). Afterwards the screen size is put back from the
page's own numbers, within safe bounds.
Not in this release
- Jira and the Apps page (planned for 0.5).
- Not tested yet: models other than Claude, Linux, physical Android phones and iPhones.