Skip to content

v0.2.2: the context window week

Choose a tag to compare

@bradleyd bradleyd released this 23 Aug 12:42
· 40 commits to main since this release

Every fix in this release came from one week of pointing worksmith at real work: a book chapter, written by a local Qwen 27B on a 32k context window. The session traces told me what broke. A lot broke.

Compaction now works on real sessions

Compaction assumed a session looks like a chat, with plenty of user turns to cut between. Real agent work is one instruction followed by 50 tool calls. On those sessions compaction either did nothing (33 messages in, 33 out, reported as success) or fired every single step and freed 9% each time. One session logged 441 request-would-not-fit retries in a row, each one buying a single token of progress.

Now it cuts by token budget when turn boundaries don't help, spends that budget in the same units as its trigger, and warns you when a compaction freed almost nothing instead of pretending it worked. When a request still won't fit, the loop believes the server's numbers, shrinks once, compacts once, and stops.

The before and after on the same chapter task: 62 compactions and a dead turn at 78% context, versus 2 compactions and a finished chapter.

Skills stay loaded

The trace showed the model loading the same 4kB skill pack eight times in one session. Not the model's fault: compaction kept eating the skill text, so the model kept asking for it back. Skill instructions now live in the system prompt where compaction can't touch them, and each skill pins a small generated map of its reference files and their headings.

That map is the other half of the fix. skill(name, section) fetches one heading's worth of a reference file for a few hundred tokens instead of reading the whole thing. Ask for a file path by mistake and you get that file's headings back. Big tool results are capped at 8kB, and a capped read lists the headings of what you didn't see, so the next call can be a slice instead of a re-read.

Reference files went from 8 whole-file reads each to 1.

Smaller fixes you'll notice

  • A dropped ssh tunnel used to kill the turn. Transport failures, 429s, and 5xxs now retry with backoff. A rejected API key is asked exactly once.
  • /history during a running turn froze the whole TUI. It doesn't take the session lock anymore.
  • The stuck detector accused the model of looping when it re-read a file that compaction had just deleted. Counting resets after a compaction now.
  • worksmith --version prints the git commit. Two stale-binary hunts this week would have been one-line checks.
  • The footer warns when the model's last reply was cut off at the output limit instead of letting it pass as finished.