v0.7.0
Minor release: new context-saving features (Grep output compression, a session orientation brief, bash loop detection) plus a round of worker, database, and compaction fixes.
Added
-
Grep output compression. Large
grep/rg/ag/ackresults (>30 lines) are compressed to a file-level summary: top 20 files by match count, totals included, full output cached fortoken-goat bash-outputrecall. Typical savings: ~80%. -
Bash loop-detection escalation. The same command run twice triggers a "ran 2×" escalation; three or more repeats produce a "WARNING: ran N×" advisory. Stops runaway loops from burning context unnoticed.
-
Session-wide hint deduplication. Identical hints are suppressed after their first injection within a session. SHA-256 fingerprinting with a JSON-persisted
hints_seenset means the agent never gets nagged twice for the same file. -
Session orientation brief. At session start in a dirty git repository, a compact block (~50 tokens) is injected: current branch, modified/staged/untracked counts, and the five most-recent commits. Disable via
TOKEN_GOAT_SESSION_BRIEF=0or[session_brief] enabled = falsein config.toml. -
Adaptive PreCompact manifest budget. The manifest budget scales from 200 to 600 tokens based on edit count, symbol accesses, and bash activity. Sessions with little activity get a lean manifest; complex ones get the full picture.
-
Git diff --stat in PreCompact manifest. A
git diff --stat HEADsummary (capped at 8 lines / 200 chars) is now included in the compaction manifest. The compaction LLM always sees which files drifted from the last commit, even when the session cache doesn't list them as edited. -
Symbol names in re-read hints. Re-read hints now include up to three symbol names previously accessed in the flagged file (e.g.,
[symbols: login, get_user, Session]), so the agent can decide whethertoken-goat read file::symbolis sufficient. -
Error-preserving smart truncation. When bash output exceeds the size cap, the trimmed view keeps: first 10 lines + up to 10 error-signal lines with 2-line context + last 10 lines, separated by
--- N lines omitted ---. Errors are never lost to truncation. -
Loaded version in
token-goat stats. The stats report now shows the running token-goat package version: a header line in the ANSI renderer (token-goat v0.6.1), the version in the rich fallback renderer's panel title, and a top-levelversionfield in--jsonoutput. Confirms at a glance which build produced the numbers.
Fixed
-
Git-history indexing batches its writes in one transaction.
_index_history_innerinserted up to 200 commit rows on an autocommit connection (isolation_level=None), so everyINSERTcommitted on its own and the trailingconn.commit()was a no-op: 200 separate fsyncs and 200 writer-lock acquisitions per reindex sweep. The batch now runs inside a singleBEGIN/COMMIT, acquiring the lock and committing once. Thelast_indexed_atstaleness marker is also written only when at least one commit stored, so a batch that wholly failed (for example, a database that stayed locked throughout) no longer stamps itself "indexed" and suppresses the retry for an hour. -
project_writer_lockacquisition is now atomic._try_acquirecheckedlock_path.exists()and thenwrite_text— a check-then-write with a TOCTOU window: two callers that both observed the file absent each wrote the lock and each believed it held it, so twoindex_projectruns could write the same per-project database concurrently. Acquisition is now a singleos.open(O_CREAT | O_EXCL)create — the atomic-mutex pattern the worker slot claim already uses — and_stalefalls back to the lock file's mtime so the brief create-then-write window can't be misread as a dead lock. -
Git-history indexing moved to the background worker. The SessionStart hook spawned
git_history.index_project_historyon adaemon=Truethread inside the hook process, which exits within milliseconds — killing the thread before the indexing finished. Git-history hints are now refreshed by the worker's periodic reindex sweep, which runs in a durable process;index_project_historyis idempotent and staleness-gated (1 h), so the move adds no measurable cost. -
Worker claim-slot no longer wedges on a write failure. If
os.writefailed after_try_claim_worker_slotcreated the claim file, the file descriptor leaked and an empty claim file was left on disk._worker_claim_is_staletreats an empty claim as not-stale (to protect the create-then-write window), so that orphan could never be reclaimed and the single-worker slot stayed blocked. The fd is now closed and the empty file removed on a write failure. Separately,run_daemonwrapped its claim-file cleanup in afinallywhosetrybegan only after_write_pid/_register_autostart/cleanup_on_startup, so an exception in any of those skipped the cleanup — thetrynow covers all startup work. -
Session-start git brief is capped by one shared deadline.
_build_session_briefran three git subprocesses (rev-parse,status,log) sequentially, each with a fixed 2 s timeout, so a slow or pathological repository could stack a ~6 s pause onto session start. The three calls now share a single ~2.5 s wall-clock budget, and a call is skipped once the budget is spent. -
A deferred dirty-queue drain no longer slows re-indexing. On Windows a concurrent
enqueue_dirtycan holddirty.txtopen, makingos.replacefail with a sharing violation;drain_dirty_queueretries and then defers. It returned[]for that case — indistinguishable from a genuinely empty queue — so the worker counted a deferred drain as an idle cycle and let adaptive back-off drift re-indexing toward its 10 s maximum while edits piled up.drain_dirty_queuenow returnsNoneon a deferral, and the worker resets the idle counter instead of incrementing it. -
token-goat doctorno longer integrity-checks the production database. The stats summary openedglobal.dbthrough the read-write path, which runsPRAGMA integrity_checkon connect — multi-second on a largeglobal.db, and it created the database file as a side effect when one did not exist yet. The summary now reads throughopen_global_readonly(), sodoctorstays fast regardless of database size and never mutates the database it is diagnosing. -
token-goat statsbreakdown rows now rank by share. The "By kind", "By day", and "By project" tables emitted rows in byte-sorted order while the share column they display is token-derived, so the share percentage zig-zagged whenever bytes and tokens ranked rows differently (an image-heavy day saves bytes but ~0 tokens). Each section renderer now orders its rows by the same share metric it displays — "By source" already did this. -
Unbounded
global.dbWAL growth. Every hook writes stat rows toglobal.db, and under a heavy multi-agent burst its passive autocheckpoints were perpetually blocked by overlapping readers, so the write-ahead-log file only ever grew — one session reached an 11 GBglobal.db-wal, after which every hook (including the SessionStart hook that runs on/compact) stalled for minutes scanning it. Connections now setPRAGMA journal_size_limitso the WAL file is truncated after each checkpoint, and the worker force-runs awal_checkpoint(TRUNCATE)onglobal.dbevery maintenance cycle. Atests/test_wal_growth_guard.pyregression suite, wired into the pre-commit hook, locks both halves of the fix in place. -
Temp files and automation artifacts excluded from PreCompact manifest. Paths under
/tmp/, Windows%APPDATA%,.improve-state-*.json, andimprove_commit_msg_*are filtered before the manifest renders. Previously they leaked into "Files Edited" and wasted manifest budget on entries the compaction LLM couldn't use.