Repository navigation
v0.5.2
Added
docs/REFERENCE-TANNERY.md— a study of a
running leather deployment. Every numbered example runs from a fresh clone
with onemaketarget, so none of them shows a whole system.
TGPSKI/catnip/tannery
— six agents, four queues, a 41-minute job, unattended — is read in place and
pinned at2033b0drather than vendored; it needscatnip, a populated
store and a served model, so a copy underexamples/could not run.
Covers stages joined by queues rather than clock arithmetic, per-turn scopes
that replace rather than extend, tool-sourced verdicts, and a three-layer
timeout stack. Carries run records from 2026-08-07 against a 35B MoE (~3B
active, NVFP4, one GPU): the analyst stage went from a monolith at 237s
median that never completed a recorded cycle to 43s median, 26 runs for 26,
and no fix in that day's failure ledger required a bigger model. Also carries
the A/B in which a blind frontier control — single-shot, 64k tokens, 7.5
minutes — produced a better document than the ~750k-token pipeline, and the
split of that gap into tool surface and synthesis ceiling.
Every fix here closes a silent failure: a configuration or routing mistake that
produced no error, only a run that looked successful. A tool reference that
resolved to nothing left an agent whose prompt still named the tools, so it
narrated calls it never made. An ingest that could not route stored a hide and
exited 0. A dead scheduler kept advertising its next fire time. leather now
fails closed on all three.
Fixed
- A tool reference that resolves to nothing is now fatal, not skipped
(#71,
#72).
A failed tool-registry load was logged at WARN and the run continued with no
tools registered at all. Because skill headers put tool names and purposes
into the system prompt, the agent had been told in detail about tools the
tool-calling API never received — and the most probable completion is to
role-play them. The measured example: 72 prompt tokens on a turn declaring 14
tools, followed by 2208 completion tokens of fabricated[TOOL CALL]/
[RESPONSE]pairs with invented field names and a made-up hash. A scheduled
agent doing this at 05:07 writes a confident, entirely invented artifact, and
the only signal is one WARN line.
leather run,leather serve, andleather workflow runnow exit non-zero
when the registry fails to load, and a run fails before the first LLM call
when an agent — or any of its per-turn scopes — names a skill, toolset, or
tool the registry did not load. The message names the near miss, because a
*.skill.yamllisted undertoolsets:is one edit from correct and reads as
correct. leather validateloads the registry the runtime loads
(#71).
Two skills in onetool_dirdeclaring the same tool name broke the whole
registry while every file stayed individually well-formed, so per-file
validation reported no errors on a configuration that could not execute a
single tool. Validate now performs a whole-registry load and resolves each
agent's scope against it, catching duplicate tool names, toolsets naming
tools nobody defines, danglingskills:/toolsets:references, and
type: mcptools whosemcp.serveris not configured inmcp-servers.yaml.- Flow-style
mcp:andhttp:mappings parse
(#74).
mcp: { server: shells, tool: report_write }parsed to an empty server and
tool. The tool registered, was offered to the model, was selected, and died at
dispatch withserver "" not found— as a tool result the model could
narrate around, after it had already committed to the call. Both YAML mapping
forms are now accepted wherever a nested map is, and atype: mcptool
missing either half of its dispatch address fails the load instead of failing
mid-run. Found in a tannery whose fourteen tools all used flow style: every
one undispatchable, validate clean. - A toolset written as tool definitions is rejected
(#73).
docs/GUIDE.mdshowed a toolset containing full tool definitions; the parser
reads bare names. Following the guide registered a tool literally named
name: list-tags, the toolset resolved to zero real tools, and no error was
raised. The GUIDE now shows the name-list form and says explicitly that a
toolset references tools some*.skill.yamldefines, and a-entry
containing a colon — which is always this mistake — fails the load naming the
file and the required form. A toolset that lists no tools is also an error. - The two intake surfaces route by one rule
(#75).
queue=alone enqueued over HTTP but not from the CLI;curing=alone
routed on neither, though the named curing already declares its queue. The
unrouted cases printed success-shaped output whose only tell was two absent
lines. Now, on both surfaces: a named curing routes to its declared queue, a
named queue routes and picks up the curing that consumes it, and naming
neither lets the route table decide. A parameter set that names a destination
which cannot route — an unknown curing, a queue the tannery does not declare —
is an error, and a hide-only ingest saysrouting nonerather than leaving
it to be inferred. Aqueue_patternroute reaching/intakenow expands and
enqueues instead of silently resolving to an empty queue name. - CLI ingest refuses to write queues a live serve owns
(#76).
A running serve holds each queue it polls in memory and rewrites the backing
file from that snapshot, so an item appended byleather ingestwas never
seen and was overwritten by the serve's next write. The empty-queue-plus-idle-
worker state looks identical to "nothing was ever enqueued", which is #75's
failure shape — the two stacked bugs took three reproductions to separate.
leather ingestnow probes the state-dir lock before enqueuing and refuses
with the holder's PID and the supported alternative. Hide-only ingests touch
no queue and are still allowed. leather statusreports whether a scheduler is running
(#77).
Every field it printed came off disk, so a serve killed an hour ago produced
output identical to a healthy idle one — including anext=it would never
honour, which for a daily agent stays plausible for a day. Status now probes
the state-dir lock (acquiring it is proof of absence; releasing it
immediately, because status must never hold it) and reports
serve: running (pid N)orserve: not running.servewrites its PID into
the lock file, which was 0 bytes, so the holder can be named instead of
pattern-matched. Anext=already in the past renders as(stale).
Changed
.subagents/README.mdno longer mirrors the root routing table
(#54). The mirror had drifted
fromAGENTS.md, which is the one failure a duplicated index reliably
produces. The README now points at the canonical table and keeps only what is
its own: the per-guide conventions.14-sig-triage:eval/scripts/tui/is vendored from its canonical public
upstream, pane, now at
v0.2.0 (9e80e94).
The copy is byte-identical to upstream and its provenance header now names a
released tag rather than an untagged mid-history commit, so what the eval
viewers run is pinned to something a reader can fetch by version; pane's
tools/check-vendor.shverifies the byte-identity claim rather than merely
asserting it.
Reaching the eval viewers:bar_chartgains aggregation binning, outlier
clipping and top-row label collision handling;interact.py(with
RowCursor) andgrids.pyjoin the vendor set;grids.diverging_bars()
draws signed values either side of a zero line, for series where the sign is
the finding rather than the magnitude. Fixed along the way: a bar carrying
both a peak marker and a value label had the label written over the marker,
so a chart whose legend promised "▲ marks this day" drew no ▲ at all. The
marker is now kept and the value placed beside it, or dropped when there is
no room.