Fix the paused watcher, the dead Dashboard button, and 37B phantom tokens - #1
Merged
Merged
Conversation
… the numbers inflated Watcher: an identity-bearing lock. The pidfile held only a number, and PID numbers restart and get reused at every boot, so a lock that outlived a reboot kept naming a live process — just not ours. A lock left at pid 810 was inherited by /usr/libexec/mobilerepaird; kill(810,0) went on succeeding and every `tokenflow watch` refused to start for days, from the launch agent and the menu bar's play button alike. The data went stale behind a phantom while the UI said "paused" and the button did nothing. The lock now records the boot its pid was issued by (watch-lock.js), so a pidfile from an earlier boot is stale by construction; legacy bare-number files fall back to asking the OS who owns the number. status.json is no longer treated as evidence of liveness — a watcher block outlives the process that wrote it — and a watcher that fails to start now reports why instead of failing silently into /dev/null. Dashboard: the button opened http://127.0.0.1:<port> and hoped. With no server running the browser showed a connection error. It now probes /api/ping, starts the server when nothing is serving, shows "Starting…" while the bundle builds, and surfaces a failure. `tokenflow dashboard` binds the CONFIGURED port rather than a hardcoded default the menu bar had never heard of, and a second invocation opens the window instead of dying on EADDRINUSE. Hermes: 37 billion phantom tokens in one day. session_model_usage is keyed on six columns; the adapter's tail key used four, leaving out billing_base_url and billing_mode. Two real rows — one session and model billed with mode "" and again with "chat_completions" — shared one tail, each computed its delta against the other's totals and overwrote it, so every refresh cycle re-emitted the difference with a fresh emission index. Five colliding sessions turned 2026-08-25 into 39.9B tokens at $302, growing with cycle count rather than usage. The key is now the whole primary key, and the tail is a high-water mark so a total that comes back lower can never manufacture usage. `tokenflow reset --source <id>` drops one source's records and clears its cursor for re-ingest, which is the repair path for a store already holding bad data: re-read against the live database it reconciles exactly (1,568 rows, 1,568 records), and 2026-08-25 falls to 2.35B with usage returning to the days it happened on. Regression tests fail without each fix, verified in both directions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first commit captured the watcher's and dashboard's stderr in a Pipe and
read it only in the termination handler. That would have hung the very watcher
the play button starts. A refresh cycle shells out, and those children inherit
fd 2: the git provider alone accounts for 7,791 `fatal: your current branch
'main' does not have any commits yet` lines in this machine's watch.log, a few
per cycle. Nothing drains a Pipe nobody reads, so the buffer (16–64 KB on
macOS) fills within hours, the writing child blocks, and the cycle stops —
reproducing the exact "not running live" symptom this branch set out to fix,
and only on the path a click starts rather than launchd.
Output now appends to watch.log / dashboard.log, the same files the launch
agent writes, so a file that never blocks holds the whole story and there is
one place to look. On a failed start the reason is read back from the log
between the offset recorded at spawn and EOF — only what THIS launch wrote, so
a weeks-old line is never quoted as the cause of a failure that just happened —
preferring the CLI's own error marker over the hint line that follows it.
Also: the ANSI strip used `\u{1B}`, which is Swift's escape syntax, not ICU's.
NSRegularExpression could not compile it, the replace silently did nothing, and
escape codes would have shown up in the popover. It is `\u{001B}` in the
pattern string now, verified against a log carrying real colour codes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The corruption this branch fixes was detected the day it started. On 2026-08-25 the token-spike detector scored 39.90B against a 60-day median of 166.14M — 240×, a modified z-score of 170.9, severity high. Nobody saw it. Severity saturates at "high" around z=6, and the list then broke ties by date, newest first. So a z=170.9 outlier ranked third behind request spikes of z=6.5 and z=11.2 from later that week, and the menu bar shows the top two alerts. For a week the loudest signal the product had was the one thing it did not show. Replayed against the real incident, the ordering is now: before 1. request_spike 08-28 z=6.5 2. request_spike 08-26 z=11.2 after 1. token_spike 08-25 z=170.9 Ordinary alerts still read as a recency feed; only an outlier of a different order (z >= 25, four times the "high" threshold) is floated above it. A z of 171 and a z of 6 are not the same news, whichever happened more recently. Also: `tokenflow reset --source <id>` was added on this branch as the repair path for a store holding bad data, and `tokenflow help` never mentioned it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…GELOG
"Live" needs a resident watcher and nothing installed one. The play button
could start one for the length of a session; that was all. A reboot left stale
numbers behind a paused menu bar, so anyone who wanted live data hand-rolled a
LaunchAgent — and a hand-rolled one is where two real defects came from.
`tokenflow watch --install-agent` installs it, `--uninstall-agent` removes it,
`watch --status` reports its state, and `tokenflow setup` installs it by
default on macOS (announced, with --no-agent to opt out; other platforms get
the systemd/cron hint instead).
KeepAlive is not a boolean here. `KeepAlive: true` restarts the job after ANY
exit, including the clean one a deliberate stop produces — measured on this
machine at about two seconds, which silently defeated the menu bar's stop
button. The agent uses `KeepAlive: { SuccessfulExit: false }`: a crash comes
back, a stop stays stopped until the next login. A watcher that cannot take the
lock exits 1, which under that rule is a restart, so `ThrottleInterval` is 60 —
without it a persistently blocked start is an infinite respawn loop writing to
the log every time, which is precisely what put 2.2 MB of one repeated refusal
in this machine's watch.log.
Installing removes any OTHER agent that runs a watcher, found by reading what
each plist actually runs rather than by matching a label, because a hand-rolled
one can be called anything. It deletes the plist rather than only unloading it:
an unloaded file comes back at the next login and the race with it.
The menu bar's play button now kickstarts the agent when one is installed,
instead of spawning an unsupervised child that no launchd would restart and
that would die at logout. `watch --status` also takes the PID from the lock
rather than the status file, so a restarted watcher is never described with its
predecessor's PID for a cycle.
Verified on a real machine in four states: install removes the conflicting
agent and leaves exactly one loaded; a stop stays stopped; a kickstart starts
it; a SIGKILL is restarted after the throttle, with the new watcher taking over
the lock the killed one never released.
CHANGELOG.md records 1.1.1 as Unreleased, including how to repair a store that
already holds the inflated Hermes records.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A launchd plist always carries POSIX paths, but renderPlist built the log path with `path.join`, which follows the host separator. On Windows that produced `\home\tf\watch.log` and the plist test failed there — the only CI job that could catch it, since the agent itself never runs on Windows. Rendering now uses `path.posix`, so it is identical on every platform and the test means the same thing wherever it runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e one `tokenflow diagnostics` reported `process.env.npm_package_version || '1.1.0'`. That env var is only set when the CLI runs through an npm script, so every direct invocation — which is every real one — read the hardcoded literal. It would have gone on naming 1.1.0 in support reports after this release, and after every release after it. The version now comes from the shipped package.json, which is the only copy that cannot drift. The release workflow validates that the tag matches package.json, so the bump has to land before the tag; it then builds the DMG, the npm tarball, the checksums and the GitHub release from the tag alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three defects, each with a proven root cause and a regression test that fails without the fix.
1. The play button did nothing; the app sat paused
watch.pidheld a bare PID. PID numbers restart and get reused at every boot, so a lock that outlived a reboot kept naming a live process — just not ours. On this machine the lock held810, which after a restart belonged to/usr/libexec/mobilerepaird.kill(810, 0)kept succeeding, soacquireWatchLock()refused to start for days — from theKeepAlivelaunch agent and the menu bar's play button alike. 2.2 MB ofwatch.logis the same refusal over and over. Data went stale behind a phantom.The lock now records the boot its PID was issued by (
src/core/watch-lock.js), so a pidfile from an earlier boot is stale by construction. Legacy bare-number files fall back to asking the OS who owns the number. Both the CLI and the Swift app read that one lock —status.jsonis no longer treated as evidence of liveness, because a watcher block outlives the process that wrote it. A watcher that fails to start now reports why instead of failing silently into/dev/null.2. Dashboard opened a closed port
openDashboard()openedhttp://127.0.0.1:<port>and hoped. With no server running the browser showed a connection error. It now probes/api/ping, starts the server when nothing is serving, showsStarting…while the bundle builds, and surfaces a failure.tokenflow dashboardbinds the configured port instead of a hardcoded default the menu bar had never heard of, and a second invocation opens the window rather than dying onEADDRINUSE.3. 2026-08-25 showed 39.9B tokens
session_model_usageis keyed on six columns. The hermes adapter's tail key used four, omittingbilling_base_urlandbilling_mode. Two real rows — one session and model billed with mode""and again with"chat_completions"— shared one tail, each computed its delta against the other's totals and then overwrote it. Every refresh cycle re-emitted the difference with a fresh emission index, so nothing deduped it:"")chat_completions)The two records that repeated ~3,100 times are exactly
max(0, A-B)andmax(0, B-A): 522,102 and 11,607,768 tokens. Five colliding sessions turned one day into 39.9B tokens at $302 — tokens growing with cycle count, cost with usage.The key is now the whole primary key, and the tail is a high-water mark so a total that comes back lower can never manufacture usage.
Repairing a store that already holds bad data
tokenflow reset --source <id> --yesdrops one source's records and clears its cursor so the next refresh re-reads it from scratch. Every other source is untouched. Re-ingested against the live database it reconciles exactly:* the +75,212 is exactly
SUM(reasoning_tokens) WHERE reasoning_tokens > output_tokens— the adapter's documented additive-reasoning fold.Day totals after the repair (usage also returns to the days it happened on, which the phantom deltas had smeared forward):
Verification
watch.pidand the watcher started anyway.Not changed, both checked and correct: month-to-date reading lower than week-to-date (Sep 1–2 vs Mon Aug 31–Wed Sep 2), and the 2016 records (real git activity, not token usage).
🤖 Generated with Claude Code