Skip to content

Releases: ahwurm/localharness

v0.13.3

Choose a tag to compare

@ahwurm ahwurm released this 09 Sep 20:01

The orchestrator's out-of-the-box path, dogfooded on a real multi-file drafting
project with a slow thinking model. Every item here is something that made a
turn look hung, end early, or do more work than it needed: a status row that
said "working" for twelve minutes, a generation that fell into a loop nothing
could see, a 30-minute clock that killed productive work, and an edit tool
the model never reached for.

Added

  • The status row says what the model is doing, not just "working". With the
    reasoning stream off, every phase of a reply looked the same: a spinner and the
    word working, for twelve minutes at a time. The row now shows three things: the
    phase of the last delta, colored, with its icon and live token tally — ⋯ thinking 2.3k, ✎ writing 410, ◆ tool call 120, or … waiting before the
    first delta (queue wait or prefill); the elapsed time, which turns into a red
    silent 14s when no delta has arrived for ten seconds mid-stream; and the
    tok/s figure that was already there. The tally is an estimate from the same
    chunk-to-token ratio the rate uses (chars/4 before a ratio has been measured).
    LLMClient.stream_snapshot() exposes the live picture, poll-cheap.
  • A generation that locks into repetition is stopped mid-stream, not at the cap.
    Observed live (qwen3.8-27b, 2026-09-08): the model's hidden reasoning fell into
    413, 313, 213, and stayed there for the rest of a 16,384-token reply while the
    status row said "working". Nothing could catch it — the repetition guard only
    inspects the final text once the stream ends, a typed nudge lands on the next
    request, and an output cap merely postpones the cut, after which the cap grows.
    The stream consumer now watches the tail of everything it receives, reasoning
    and content alike, and aborts the request the moment a short unit has repeated
    back-to-back across a 1,200-character window; closing the stream ends generation
    server-side. The loop records the abort in the ledger as an llm_response with
    finish_reason="degenerate", re-prompts once with presence_penalty=1.5 for the
    rest of the turn (Qwen's own guidance for thinking-mode repetition, applied only
    after a loop was seen, never as an ambient default), and a second abort ends the
    turn as a failure that says why. LLMClient.complete / stream_complete take a
    per-call presence_penalty, sent only when set.

Changed

  • No turn time limit by default. permissions.budget.max_duration_minutes now
    defaults to null, which means no limit; write a number to pin one, exactly as
    with the output cap. The old 30 minutes could not tell a runaway from slow
    hardware: live (2026-09-09, qwen3.8-27b at ~20 tok/s), a productive
    13-iteration turn resuming interrupted work — 56,000 output tokens, every step
    landing — was ended by the clock between iterations at 39 minutes, and its
    continuation faced the same wall. What actually stops a turn that has gone
    wrong is max_actions, the stuck detector, the mid-stream repetition guard,
    the per-chunk silence timeout, the kill file, and Ctrl-C; those are unchanged.
    Built-in subagents keep their own bounded budgets. A config.yaml written by an
    older init still carries max_duration_minutes: 30.0 and still pins it —
    delete the line or set it to null.
  • edit comes with write, and both tools show their work. A fresh install's
    root agent has always held edit alongside write and bash_exec, but the
    model never reached for it: a yaml-defined subagent that listed write got no
    edit at all, and neither tool said what actually changed. Now a child whose
    tools.add names write gets edit too (same capability class — the
    capability floor and the grant gate treat both as host-dangerous — and an
    explicit deny of edit still wins). edit returns a unified diff of the
    change, cut at 40 lines, so the model can verify an edit without re-reading the
    file. write over an existing file reports the line delta, and when the change
    touched a small slice of a long file it says so and points at edit — the
    write still lands, because refusing it would cost a full round-trip, which is
    minutes on a local model. The ToolConfig docstring finally lists edit among
    the global built-ins.

v0.13.2 — no output cap by default

Choose a tag to compare

@ahwurm ahwurm released this 08 Sep 15:30

[0.13.2] — 2026-09-08

Changed

  • There is no output cap by default any more. The model decides when it is
    done.
    When nothing in your config sets max_tokens or default_max_tokens,
    the harness now leaves the max_tokens parameter out of the request
    altogether — not a large number, not a number derived from your window, no
    number at all. The model generates until it has finished, and the context
    window your server serves is the only thing that ends a reply. That is what a
    long-running task needs: a task that takes twelve thousand tokens to finish
    correctly should take twelve thousand tokens, and nothing in the harness should
    be deciding otherwise on a guess. Write a number in config and it is honored
    exactly, on every request, as it always has been.
  • This replaces the derived cap that 0.13.1 shipped one release ago. 0.13.1
    computed an unset cap as a quarter of the served window with a 4,096 floor —
    32,768 tokens on a 131,072-token window. The arithmetic was defensible and the
    number was still arbitrary: a quarter is a guess about how long your reply
    ought to be, made by us, in a file you never opened. The derivation is deleted
    rather than retuned. The 128,000 bound still exists, but only as the largest
    value the config fields will accept from you — nothing derives a cap for it to
    bound.
  • What keeps a runaway generation in check is not a token count. The
    protections that actually catch a model that will not stop are the
    degenerate-repetition guard (a reply that is one line repeated is stopped and
    called out, not published as an answer), the per-chunk stream timeout (a model
    that has genuinely stopped producing tokens ends the request, and a model that
    is still streaming is left alone), and the kill file (localharness kill, which
    ends the turn whatever it is doing). Those are the safety nets. An arbitrary
    cap was never one of them — it truncated good work as readily as bad, which is
    the failure that started this.
  • Two things to know on upgrade. A config.yaml written by an older init
    carries default_max_tokens: 4096, and that line still pins your cap at 4,096
    — delete it, or set it to null, which is what init writes now, to get the
    uncapped default. And with no cap in the request, a reply that comes back
    marked as cut off can only mean your context window filled up, so the harness
    no longer offers to raise a cap in that case: it says the window is full, and
    compaction is what makes room. Where you have configured a number, the
    fit-to-window and grow-on-cutoff behavior from 0.13.1 is unchanged.
  • Open companion item: the request timeout is still a constant. The read
    timeout is a fixed 600 seconds, per chunk rather than per reply, so a streaming
    model resets it on every token and the constant bounds silence rather than
    length. With no cap left in the request there is no token count to derive a
    timeout from either, which leaves the measured decode rate as the only honest
    input. docs/reference-architectures/gaps.md §1 carries the arithmetic and
    states plainly what has and has not been measured.

v0.13.1 — dogfood fix train

Choose a tag to compare

@ahwurm ahwurm released this 08 Sep 15:09

[0.13.1] — 2026-09-04

A fix train from running 0.13.0 on real work, on Linux and on Windows. A long
task that promised a next step and then stopped, a startup that quietly deleted
its own warnings, a picker that crashed on an agent's name, a Windows upgrade
that reported failure after succeeding, and a drafting session whose every
composing step spent the whole output cap on hidden reasoning, came back empty,
and closed as a success — plus the workspace layer offering itself to projects
that do not have one yet.

Added

  • start offers to make a workspace when your project has none. The layer
    has always been there — almost nobody finds init --workspace before reading
    the docs, and the moment you want one is the moment you start the harness
    inside a project. So a start that finds no .localharness/ anywhere above it
    now asks once: create ./.localharness for this project? Default no. Yes
    scaffolds it and the session continues with that layer already active, so there
    is no "now run it again". This is the only prompt in the harness that writes to
    disk, so it stays silent and creates nothing when there is no terminal, when
    --no-input is passed (start takes the flag now), when --config-dir,
    LOCALHARNESS_DIR or LOCALHARNESS_HOME named a directory, when a workspace
    was already found up-tree, and in $HOME or the global config directory —
    none of which are a project. A closed stdin answers no.
  • A declined offer is remembered, once per directory, ever. The answer goes
    in declined_workspace_offers.yaml next to your global config — never in the
    project — and only an answered prompt records anything, so a session that could
    not ask has not spent your decision. Nothing else reads that file: init --workspace, or simply making the directory yourself, still gives you a
    workspace whatever you once answered. If the file is unreadable you get the
    question again rather than a silently skipped one.
  • A repeated stuck-nudge now says something new. Where you have raised
    agent.baton_gate.max_nudges above its default of 1, the second and later
    nudges quote the model's own announcing words back to it and name the two ways
    out — do it with a tool call, or say plainly that you are done or blocked —
    instead of repeating the first nudge verbatim. At the default bound nothing
    changes: the single nudge is the same message as before.
  • Terminal colors from the architecture diagrams. The entity hues on
    localharness.dev's architecture plates — memory cyan, tool purple, provider
    amber — are now the terminal's, at startup and inside every turn, so a tool
    call in the scrollback is the same color as the tool box in the diagram. Names
    take the color; message bodies stay neutral, and success/error keep green and
    red, because a verdict is not a type. One knock-on you will see: the agent has
    taken the accent green, so your own input is now the site's ink rather than
    green.
  • Live reasoning stream: start --show-reasoning, terminal.show_reasoning,
    /reasoning.
    Thinking models were dead air: the client already assembled
    reasoning_content from the stream but nothing surfaced it, so a 3-minute
    think looked identical to a hang (--verbose is per-component startup detail,
    not this). Reasoning deltas now flow from the stream consumer to the terminal
    channel, which prints them as dim lines — line-buffered, with a long
    unbroken paragraph streamed in pieces and the tail flushed when the reply
    lands. Off by default; the sink is always wired so /reasoning on|off toggles
    it mid-session. Needs the server's reasoning parser (vLLM --reasoning-parser,
    llama.cpp --reasoning-format, Ollama think); without one the thinking is
    inline <think> text the harness strips.
  • A Platform support section in the README (Linux / Windows), and
    Git for Windows named as the Windows requirement bash_exec has always had.

Changed

  • The per-reply output cap is the configured value, fitted to the window, and
    grows when a reply is cut off.
    start sent DEFAULT_MAX_TOKENS (4,096)
    whatever default_max_tokens or the agent's max_tokens said, and the reply
    reserve it was clamped into was the same flat number — so raising the value in
    config.yaml changed nothing (the live session carried default_max_tokens: 8192 and still asked for 4,096; five empty replies of ~170 s each, no draft,
    and the cruncher's extracts cut mid-sentence for the same reason). Now the
    agent's resolved max_tokens (agent yaml → division → org default_max_tokens)
    is what the session and the /model swap refit send; the shared reply
    reserve grows to hold it (bounded at half the window; the small-window curve is
    unchanged); every request is fitted to the window's real headroom
    (window − prompt − window/64, prompt taken from the server's own
    prompt_tokens when it reports usage) so a larger cap can never push
    prompt + max_tokens past the served window; and a reply that ends with
    finish_reason="length" doubles the cap for the retry within that headroom —
    for an empty reply, for a truncated tool call, and for a truncated final
    answer, which used to ship with its tail missing and is now re-prompted once.
    The raised cap is kept for the agent's life. LLMClient.complete /
    stream_complete take a per-call max_tokens, and the llm_response Action
    carries output_cap, so the ledger says what each request asked for.
  • …and when you have not set one, that cap auto-derives from the served context
    window
    instead of being a bigger number picked by us. An unset
    max_tokens/default_max_tokens now means "a quarter of the window this
    session runs in, and never below 4,096 tokens" — the served window, or the
    smaller max_context_tokens you pinned: 32,768 on the 131,072
    the reference setup serves, 8,192 on a 32K one, and unchanged on anything
    small, where the 4,096 floor still meets the same reserve curve as before
    (an 8,192-token window still asks for 1,024). A quarter is not a taste number —
    the reply reserve grows to hold the cap but is bounded at half the window, so a
    quarter is the largest fraction that always fits inside that bound with history
    keeping the other three quarters. It is computed where the served window is
    first known (start's window probe, and again on a /model swap, so moving to
    a roomier model widens the reply instead of carrying the old window's number),
    and it only sets the STARTING cap — the per-request fit and the grow-on-cutoff
    above still apply. It is bounded above by the same 128,000 the max_tokens
    fields validate against, so a derived default can never land somewhere you
    could not have typed by hand. A number you write in config is used exactly as
    written, including 4,096. One consequence worth knowing on slow hardware: the
    default worst-case reply is now 8x longer than the arithmetic in
    docs/reference-architectures/gaps.md §1 assumed, against a request timeout
    that is still a constant — that section says what is and is not known about it.
  • An org-level default_max_tokens in config.yaml reaches your agents at
    all.
    The chain read org.yaml — a legacy standalone file nothing in the
    harness has ever written, since init writes org: inside config.yaml
    so the org rung of "agent → division → org" was dead on every real install and
    the value in the file you have was inert. It is now read from config.yaml
    (both layers, workspace over global), and a division that sets no max_tokens
    of its own no longer shadows it with a schema default. Two consequences worth
    knowing on upgrade: an org cap you had set and assumed was working starts
    working, and a config.yaml written by an older init carries
    default_max_tokens: 4096, which now pins the cap at 4,096 — delete that line
    (or set it to null, which is what init writes now) to get the derived one.
    default_temperature and default_model have the same dead rung and are
    deliberately left alone here; switching on a temperature that has been inert in
    someone's config is a change that needs its own decision.
  • A linked git worktree counts as inside the project it was cut from. git worktree add leaves a .git file, not a directory, and the repository walk
    stopped there — so the main checkout's .localharness/ one level up read as
    config from outside your tree, and the harness asked the one-time trust
    question about your own repository. The file names its parent repository, and a
    workspace at or below that parent is now inside. A submodule's .git file has
    the same shape and reads the same way. SECURITY.md documents the rule.

Fixed

  • A long task that ends on "Let me check the config…" gets nudged instead of
    accepted.
    The gate that catches a reply announcing work it never did was
    missing the plainest form of it. "Now let me confirm…" was caught; a bare "Let
    me confirm…" — the same announcement in different grammar — was not, so the
    promise was delivered as the answer and the turn ended. Both forms now share
    one list of action verbs. Separately, and worse, the detector split the reply
    into sentences on every ., so a closing sentence containing a filename was cut
    at the dot: for "…check the content.json format expectations" it judged the
    fragment "json format expectations" and found nothing to catch. Any filename,
    version or decimal in a final sentence did that. A sentence now ends at
    punctuation followed by a space or the end of the text. Closing courtesies like
    "let me know if…" are still accepted, and a subagent that ends this way is
    reported to its parent as having produced no result rather than passed off as a
    finding.
  • start's startup summary no longer deletes its own warnings (#157). The
    warnings were appended inside square brackets and printed through Rich, which
    read the whol...
Read more

v0.13.0 — Workspace layering: per-project config and memory

Choose a tag to compare

@ahwurm ahwurm released this 04 Sep 13:08

v0.13.0 — Workspace layering: per-project config and memory

What this release does

If you run LocalHarness on one machine for more than one thing, everything shared one
brain. One config file, one set of agents, one memory. A fact learned while drafting a
report got injected into an unrelated coding session weeks later, and a hundred domain
lessons from one project became noise in every other one. The leak ran both directions,
and the only tool for it was --config-dir, which is a full fork: duplicate the provider
block, inherit nothing, and now a model swap has to touch every config you own.

v0.13.0 adds a second layer. A .localharness/ folder in a project sits over the
machine-wide ~/.localharness/, and the harness finds it by walking up from wherever you
started — the nearest one wins. Config deep-merges, agents union by name, and memory,
sessions, history, and the audit log move into the project. The global layer is still
there underneath, so you configure the machine once and the project only says what is
different about it.

Two rules deliberately do not bend. Deny patterns from both layers are enforced together,
so a project can add a restriction and can never remove one you set globally. And nothing
ever writes a provider block into a project — the hardware is a property of the machine,
so a model swap still edits exactly one file.

If you have no .localharness/ folder anywhere above you, nothing changes. Behavior is
identical to 0.12, and --config-dir / LOCALHARNESS_DIR still skip discovery entirely.

How to try it

cd ~/your-project
localharness init --workspace     # scaffolds ./.localharness/ — never a provider block
localharness agent create drafter --project   # an agent that exists only here
localharness start                # workspace agents + workspace memory, global provider

localharness config show          # every effective key, and the file that set it
localharness doctor               # both layer paths, and which layer won each key

Inside the session, /memory promote <id> previews moving one fact up to the global
store; /memory promote <id> confirm actually copies it, with its provenance recorded,
and revert retires the copy.

Caveats, named

The trust prompt does not cover a repo you cloned and then worked inside. Config found
outside the directory you are working in asks for permission once before it loads (stored
globally, so one prompt per directory, ever; with no terminal attached the layer is
skipped with a notice rather than loaded). But — quoting SECURITY.md — "if you clone
someone's repository and run the harness inside it, that repository's .localharness/agents/
loads with no prompt, because you are inside that project." The gate is for config reaching
in from elsewhere. It is not a defense against a repository you chose to run in.

Workspace confinement is a default, not a sandbox. A workspace session defaults its
filesystem root to the project folder. Again from SECURITY.md: "it is a default that
narrows what the tools reach by accident, not a sandbox. A command run through bash_exec
can still leave that folder, and the deny patterns remain the mechanism that stops
specific actions."

recall_scope moves reads, not writes. agent.memory.recall_scope (workspace by
default, or global, or both with each injected line labeled by origin) changes which
stores a session reads. Writes always land in the session's own store — there is no
setting that makes a project session write into your global memory. That asymmetry is on
purpose, and /memory promote is the only bridge across it: one fact, typed by hand,
confirmed, revertible. If you were expecting recall_scope: global to also consolidate
upward, it does not.

Plugins and the org guardrails file are global-only. Both load from your global
config directory and are never taken from a workspace. For guardrails that is the
mechanism, not an oversight: a project must not be able to silence the org's safety
context by shipping its own copy, or blank it by having none. The cost is real and worth
naming — a project cannot ship its own plugin, and everything localharness components set writes is machine-wide, autoresearch adoptions included.

Security defaults migrate on first start. The first start after upgrading folds any
missing shipped deny patterns into your config.yaml. It is additive only — it never
removes or reorders an entry you wrote and touches no other key — it is gated on a
defaults_revision stamp, it writes a timestamped config.yaml.bak-<stamp> first, and it
announces what it did. doctor shows which revision you are on, when it last migrated,
and where the backup went. If you would rather do it deliberately, localharness config migrate runs the same thing on demand.

One rename is only skin-deep. Registry layer names are now global-config,
global-overrides, workspace-config, workspace-overrides (the old project confusingly
meant the global config.yaml). The persisted ComponentMutated.layer audit field keeps
its old value so existing event logs stay readable.

Hardening from the pre-release review

Before shipping this, we ran an internal adversarial review of the release — six reviewers
against the code, the memory layer, the docs, the packaging, the CLI's behavior under
hostile input, and the seams between features. It found and we fixed several classes of
problem, and they are worth naming because most of them were quiet failures rather than
crashes:

  • A guarantee that was asserted rather than enforced. recall_scope: global moved a
    session's reads to the other store, and two enrichment writes followed them there. The
    sentence above — no setting makes a project session write into your global memory — is
    now enforced in the code. The honest residue: opening that second store still creates its
    file and applies schema migrations if it is missing or behind, and a global session
    leaves the global store's access counts un-updated.
  • A control artifact a project could move. A workspace could relocate the kill switch's
    value, detaching a session from the operator's KILL with nothing looking unusual. The
    value now resolves from the global layer alone.
  • Settings that were stored, confirmed, and ignored. components set agent.* never
    reached the running agent, and deny patterns in an overrides.yaml were stored but not
    enforced. Both now work, and an empty deny list on screen says how many shipped defaults
    are still enforced behind it.
  • Verdicts filed under the wrong file. validate re-resolved each file by name and so
    checked the winning layer's copy twice, reporting one file's verdict under another's name.
    Every file is now validated where it lives.
  • A whole shape of project that was invisible. A project with a workspace layer and no
    machine config showed nothing from doctor, config show or agent list.
  • Crashes on ordinary input. A folder named [old] proj, an unreadable directory, a
    looped symlink, a deleted working directory, an alias-amplified YAML value — each of
    these turned a command into a traceback. They are messages now.
  • Docs that described software we do not ship. The CLI reference invented flags for
    agent create and agent list while omitting --project, described a doctor --fix
    that repairs databases, and documented a generic environment-variable override and a
    validate --json contract that never existed. Those sections are gone or rewritten from
    the real --help.

Adoption in the autoresearch loop also changed shape as part of this: it writes your global
overrides.yaml rather than committing a project file to git — which is what its own
readers read, and which fixes it dying outright when that file was git-ignored. Undo is
localharness components set <path> <old value>, or an edit of that file. It is a
machine-wide change, in every project on the box.

Also in this release

  • agent list --json emits plain valid JSON. It used to go through the Rich console, which
    wrapped it at the terminal width — at width 80 the output would not parse, and wider it
    parsed but Rich had eaten markup-looking substrings out of the data.
  • org: deny patterns written inside config.yaml now actually reach tool-call
    enforcement. Enforcement previously read only a standalone org.yaml, which init has
    never written, so the org policy people actually had was never enforced.
  • Config errors name the file and line that set the offending key, in either layer.
  • A malformed overrides.yaml reports a config error instead of a traceback.

Pre-1.0: interfaces and config schema may still change without notice.

v0.12.10 — doctor tells the truth

Choose a tag to compare

@ahwurm ahwurm released this 03 Sep 02:39

The bug

doctor's token-counting check branched on the configured provider_type and ran its own per-runtime probe. TokenCounter treats that same field as a hint — it probes both exact shapes and self-heals when the config has drifted from the running server. doctor did not.

So on a box whose config said llamacpp while the server actually spoke vLLM, doctor sent a llama.cpp-shaped body, got a rejection, and reported:

✗ /tokenize returned 404 — token accounting falls back to tiktoken cl100k

That was false. Counting was exact the whole time. The tool you run for reassurance was the one that was wrong — the worst place for a bug to live.

The fix

Two independent sources of truth about one fact, quietly disagreeing. Fixed by deleting the duplicate rather than repairing it: doctor now constructs the same counter the runtime uses and reports its resolved mode, via a new public TokenCounter.mode. They can no longer disagree.

  • A stale provider_type is surfaced as an INFO naming the drift, not a failure — counting already adapted, but anything else reading that config will be wrong, so it still gets said.
  • Message-level checks (vLLM /tokenize messages-mode, llama.cpp /apply-template) are kept, now gated on the resolved mode.
  • Whether approximate counting is a fault or expected also comes from the endpoint resolver rather than stored config: Ollama/LM Studio still report INFO; a vLLM or llama.cpp server that should serve /tokenize but doesn't still fails.
  • No more cascading: the tokenizer probe reuses default_model, so a stale model name failed it for the same reason and invented a second issue. Dependent checks are now skipped and labelled — one root cause reports as one problem.
  • The approximate warning quantifies itself. Measured against a real Qwen tokenizer: ~0% off on English and JSON, ~4% on code, >100% on CJK. The error is content-shaped, so "approximate" alone was not actionable.

Why this one matters beyond the fix

It's the second bug in three releases of exactly one shape: a number or verdict derived from a proxy, while a second component already knew better. v0.12.9 was the tok/s meter counting SSE deltas as tokens. This is doctor trusting config over the server.

Both were in covered code with passing tests — the tests encoded the same wrong belief. Coverage cannot catch that; only an external oracle can.

2719 tests passing. Pre-1.0: interfaces and config schema may still change.

v0.12.9 — the speed readout tells the truth

Choose a tag to compare

@ahwurm ahwurm released this 02 Sep 22:29

The bug

The live tok/s number on the status line counted streamed deltas and treated each as one token. Its own docstring stated the assumption — "chunks≈tokens on local runtimes" — which is true without speculative decoding and false with it: vLLM MTP emits every accepted draft token in a single delta.

Measured on qwen3.8-27b + qwen3_5_mtp: 214 tokens delivered in 63 deltas over 9.08s. Real rate 23.6 tok/s, displayed 6.9.

So enabling MTP made the model ~3.4x faster and the meter ~3.4x slower. That reads exactly like a performance regression, which is the worst possible failure for a number people use to judge their setup.

The fix

  • The live rate is scaled by a tokens-per-delta ratio learned from the exact usage count of each finished stream.
  • That ratio is persisted on the model's existing speed-ledger entry, so a new session's first turn is honest — otherwise every session's opening turn, the one you form your impression from, reads low.
  • When the ratio has never been measured, the live rate is suppressed rather than assuming 1.0. No number beats a wrong number. The blind window is the first stream ever for a model, not one per session.
  • The ratio is only learned from a sample that survived every existing rejection gate, so a run dropped as degenerate still leaves the ledger untouched.

End-of-stream verified rates were always correct and are unchanged — they use the exact token count from usage. Only the in-flight estimate was wrong.

Also fixed: record_tps replaced its whole ledger entry rather than updating it, discarding sibling fields.

If you run llama.cpp or vLLM without speculative decoding, nothing changes — one token per delta, ratio 1.0.

Pre-1.0: interfaces and config schema may still change. Full detail in CHANGELOG.md.

v0.12.8 — start just works against a live endpoint

Choose a tag to compare

@ahwurm ahwurm released this 02 Sep 22:08

Why this release

Two pieces of friction, both hit hardest when the harness and the model server are on different machines (a laptop driving a box elsewhere on the network).

localharness start now picks up the model your endpoint is actually serving. A single-model runtime (vLLM / llama.cpp) serves exactly one checkpoint, so a configured name that no longer matches has exactly one sensible meaning — but start used to hard-error, meaning any server-side model swap 404'd every client still pinned to the old name. In attach mode with no --model, start now reconciles against the live endpoint and adopts the sole served model, printing one line naming the substitution. This makes start consistent with init, which has always discovered models from the endpoint.

Guardrails: an explicit --model still fails loud (naming a model is a deliberate act); two or more served models is a real choice and is never guessed at; an unreachable endpoint still surfaces as "unreachable" rather than being masked by a bogus adoption. The reconciliation runs before the capability probe, so a stale name no longer burns three probe retries — and a session can't start in the wrong tool-call mode.

localharness update upgrades an installed copy to the latest PyPI release. It detects how the copy was installed and shells out to that installer (uv tool upgrade, else pip install --upgrade); --check reports without changing anything. A source/editable checkout is detected and refused with a pointer to git pull — pip'ing over a checkout would shadow your working tree with a published wheel.

Also in this release

  • Security: memory-recall output now enters the ContentStore with untrusted origin (#140), closing the "known gaps" note shipped with 0.12.7.
  • Fixed: context.compaction_threshold_pct is now actually wired into compaction (#147); ReadTool no longer dumps raw binary content into context.
  • Added: session-only start --model/-m, start --list-models, model --download for standalone Hugging Face pulls, and a doctor warning when an AMD GPU is paired with a Vulkan-linked llama.cpp binary (#148).

Thanks to @mjdufresne for #144, #147 and #148 — the AMD/Vulkan work was verified on real AMD hardware.

Pre-1.0: interfaces and config schema may still change. Full detail in CHANGELOG.md.

v0.12.7 — one reply reserve: context budgets now measure the window your server actually serves

Choose a tag to compare

@ahwurm ahwurm released this 31 Aug 17:48

Fix release for the context-budget spine, driven by field report #145 (thanks @mjdufresne).

What was wrong

  • init wrote served_window − 4,096 into new configs while the runtime subtracted another 4,096 internally — every init-written config ran two reserves short of the window it named.
  • Compaction's 0.80/0.95 triggers measured the raw window while the emergency floor measured window − 4,096. On every window under ~82K (that is every llama.cpp/Ollama/LM Studio default) the last-resort hard truncation could become eligible before the 0.95 compaction stage — the designed rescue never got its turn.
  • Fixing the double reserve exposed a latent gap the over-reservation had been masking: requested output tokens were never fitted to the window, so small-window servers could reject requests mid-session (vLLM validates prompt + max_tokens ≤ max_model_len and 400s).

What changed

  • max_context_tokens now means the full served window. The reply reserve is subtracted internally in one shared function (response_reserve), and every stage — eviction, 0.80, 0.95, the emergency floor — measures the same effective limit. The ordering inversion is structurally impossible now, at every window size.
  • Output tokens are clamped to that same reserve wherever the window is known (session start, /model swap, bench num_ctx pins): input + output fit the window by construction.
  • A served window too small to hold any reply (≤ 1,024 tokens) is refused loudly at startup instead of starting and failing on the first turn; init declines to adopt one and says why. Practical minimum: 1,025 tokens.
  • Repeated emergency-floor fires within one turn escalate to a distinct log line with the fire count, the window, the overshoot, and what to change — instead of N identical ERROR lines.
  • Context percentages (the REPL ctx reading, heartbeats, compaction events) now read against the usable budget — a few points higher for the same history.

Honest scope

This fixes the sizing/ordering half of #145. The deeper re-fire mechanism — per-turn compaction output is not persisted, so a long tool-heavy turn can grow back toward the floor — is an architecture item and stays open on #145.

Known wart, documented rather than hidden: the reserve curve steps at ~12K, so windows in 12,288–14,847 get slightly less usable budget than 12,287 does. Safe on both sides (more conservative, never an error); smoothing it changes real 16K-window behavior, so it is a deliberate follow-up, not part of this fix.

Migration

Nothing breaks. A config an older init wrote (served − 4,096) still works — it is just conservative now. To reclaim those tokens, set context.max_context_tokens to your full served window, or re-run localharness init.

v0.12.6 — router-mode token counting + Ollama reasoning visibility

Choose a tag to compare

@ahwurm ahwurm released this 19 Aug 14:02

Two community-surfaced fixes, shipped same-day.

  • llama.cpp router mode works now (#141, reported by @bgtmanuel with a complete repro — thank you): one llama-server hosting several models rejects unnamed /tokenize requests, so exact token counting refused to start. Every counting hop — /tokenize, /apply-template, and doctor's probes — now names the session's model. Single-model servers ignore the field (live-verified, including with mismatched names). Bonus: a tokenizer endpoint that answers with an error is now reported as "rejected the request (with the server's own message)" instead of "unreachable". Honest limit: router mode itself is unit-mocked against the reported 400 shape — the maintainer box serves a single model, so the end-to-end router path wasn't live-tested here.
  • Ollama reasoning deltas are read now (#142, surfaced by an external fork's commit trail — credit Ruivalim): Ollama spells the thinking field reasoning, not reasoning_content, so on reasoning models the decode-speed window opened only when the answer began (a 21-token generation with 1.5 s of thinking recorded 40 tok/s where the honest figure is 10) and reasoning text never surfaced. Both spellings are read in both streaming and non-streaming paths, an empty-text thinking delta counts as the start of generation, and the streaming path now exposes reasoning text on the assembled message at all — it previously did for no provider. Honest limit: no live Ollama on the maintainer box — the wire shape is mocked from the documented form; the SDK-object mechanics were verified against the installed client library.

v0.12.5 — seven shipped bugs found by live use, fixed, and re-verified live

Choose a tag to compare

@ahwurm ahwurm released this 19 Aug 13:44

A day of scripted live-use sessions across the four documented reference serving configurations surfaced seven shipped bugs — one of them live since the earliest releases. All seven are fixed here; every fix was adversarially reviewed, and every fix was re-verified against a real server after landing.

The bugs (full detail in the CHANGELOG):

  • The tool-result audit trail capped every logged result at 200 characters and stamped it truncated: false — the model always saw full results, but every audit surface lied about what tools returned (#133)
  • Large-read eviction could trap a turn in a read→evict→restore→re-read loop for 24+ minutes at small context budgets; restored content is now pinned for the rest of the turn, and the same ask completes unaided in ~6 minutes (#134)
  • Bare /model stranded text in the input box, so the next command was appended and misparsed — /quit silently became a no-op (#135)
  • The speed ledger recorded physically impossible tok/s on vLLM from chunk-arrival timing artifacts; samples now need real substance (#136)
  • doctor ignored per-model context pins and could judge a pin against the wrong model's served window (#137)
  • A dead endpoint made startup repeat its identical probe cycle three times (#138)
  • Trace packs emitted empty tool messages — live since trace packs shipped in v0.9.25 (#139)

Also in this release: the context-management spec is reconciled to the implementation (it taught three false specifics; the previously undocumented eviction layer is now written up), per-model context pins are documented, and the reference-architecture docs carry the measured decode rates with provenance labels — controlled figures as headlines, live-session spread labeled as such, and the one config the pass didn't cover saying so.

Known gap, deliberately unfixed and documented: memory-recall output is not yet marked untrusted in the content-store taint model (#140 tracks the hardening).