Repository navigation
Phonton CLI v0.22.2
phonton-cli v0.22.2 — Half the cost, fewer false passes
Same results at half the cost
Phonton used to plan a test-writing subtask for every new function, whether
or not the goal asked for tests. Those generated test files were most of the
tokens of a one-method goal, and the project's own tests already run on
every edit. Test subtasks are now planned only when a goal asks for tests.
Cloud smoke suite, DeepSeek flash, 12 tasks x 3 runs, hidden tests:
| Build | Accepted | Tokens | Cost | Wall time |
|---|---|---|---|---|
| 0.22.1 | 36/36 | 302,442 | $0.186 | 852 s |
| 0.22.2 | 36/36 | 193,129 | $0.096 | 381 s |
Write "with tests" in a goal to get them.
First run
phonton goalasks once in a terminal before running the project's own
build and tests. Without a terminal it stops before any model call unless
--allow-host-checksis given; it used to spend tokens and then fail.phonton modelsandphonton models catalogprint what is installed, what
fits this machine, and the next step (--jsonfor the full output).phonton reviewshows "review-ready" or "failed: reason" and paths
relative to the repository.
Fewer false passes on small models
A local model is only useful if "review-ready" means something. This release
runs the local-model smoke suite in CI and fixes what it found.
Local model results
benchmarks/smoke, qwen2.5-coder:3b in Ollama on a GitHub Actions
ubuntu-latest runner (4 vCPU, 16 GB, no GPU), 3 runs per task, scored by
hidden tests. A "false pass" is a run Phonton marked review-ready that the
hidden tests then failed.
| Fixture | 0.22.1 accepted | 0.22.1 false passes | 0.22.2 accepted | 0.22.2 false passes |
|---|---|---|---|---|
todo-api (4 small files) |
9/12 | 3 | 12/12 | 0 |
ledger (339-line module) |
1/12 | 5 | 2/12 | 2 |
shop (28 KB module) |
1/12 | 0 | 1/12 | 0 |
The other runs ended without a candidate: the harness refused edits that
failed checks or made no change instead of handing them over. The remaining
false passes are real semantic misses (a $ currency format, a regex that
still rejects parentheses) that only tests covering the new behavior catch.
A 3B model is good for small, focused edits; larger ones need a larger model.
Fixes
- Values scope the edit. A goal that changes a value ("from 120 to 80")
is scoped to the files containing it, plus files the goal names. The 3B
model used to editstore.js, which matched through a local variable named
todo, and the existing tests passed.todo-title-limit: 0/3 to 3/3. - Named identifiers must appear. When the goal calls or quotes an
identifier that does not exist yet (balanceOn(account, date)), a
candidate that does not add it is rejected and the model is told why. It
used to editparseDateinstead and pass the existing tests.
Diagnostics
PHONTON_LOG=debug (or any tracing filter, e.g. phonton_worker=debug)
appends to phonton.log in the Phonton home: each worker attempt, why
verification rejected it, and hunks Phonton recounted or moved. On the
cloud ledger and shop tasks (DeepSeek flash, 24 runs, all accepted) it
showed 13 hunks recounted and 14 moved across 28 worker diffs; each would
have failed verification before 0.22.1.
Reproduce
Run the workflow .github/workflows/local-smoke.yml (inputs: a released
version or source, the model, runs per task), or locally:
node benchmarks/smoke/run.mjs --arm local --runs 3 --runtime unverifiedInstall
npm install -g phonton-cli
phonton doctor