Skip to content

v2.2.3

Choose a tag to compare

@kinncj kinncj released this 20 Oct 17:35
· 34 commits to main since this release

Maple Harness v2.2.3

Highlights

  • GitHub Copilot Auto's cache reads are valued, and stay valued. Copilot bills a request through Copilot Auto
    below the same model asked for by name; the two are now priced separately, so a model used both ways (for
    example gpt-5.3-codex) no longer shows its cache reads as "not valued yet" after its first request by name. See
    Copilot Auto.
  • The Cache card and the Savings card agree, and the cache hit rate has one meaning everywhere: the share of
    input tokens read from the cache, a model's first, cold request included (see Upgrading).
  • The Savings card's rows line up: each value sits on the line of its own label, at every window width.
  • The harness's cost preview works on Copilot Auto, at Copilot Auto's prices, and says so.
  • An unknown slash command never reaches the model, in any mode (see Upgrading for --prompt).

Added

  • /model <name> in the line mode (--repl), as in the full-screen view; a bare /model lists the models.

Fixed

The setup page and the proxy's numbers

  • Copilot Auto requests are valued at their own price. Copilot Auto requests, which Copilot bills below the
    same model by name, are now valued at their own price, shown as "via Copilot Auto" in the Savings card's prices.
    A model used both ways (for example gpt-5.3-codex) showed its cache reads as "not valued yet" for the rest of the
    day after its first request by name; it is valued again. Cache reads left unvalued earlier the same day are
    shown under the price they were answered at. See
    Copilot Auto.
  • The Cache card values Copilot Auto as the Savings card does. What a Copilot Auto request's cache reads cost
    is estimated at Copilot Auto's prices, not at the model's price by name.
  • A reused answer for a Copilot Auto request is valued. An answer from Maple's response cache for a Copilot Auto
    request was recorded under "auto", which has no prices, and was never valued. It is now recorded under the model
    Copilot chose for the original, as the original was, and valued at Copilot Auto's prices on the Savings card.
  • A model that costs nothing on your account (such as gpt-4.1 on GitHub Copilot) is listed as "free on this
    account: nothing to value", on the setup page and in maple-proxy savings, instead of keeping its cache reads
    under "not valued yet" forever.
  • One cache hit rate. A model's first, cold request is now counted as a miss when its provider states its cache
    figures (GitHub Copilot and OpenAI-compatible providers do, as 0 tokens read). The Overview and the Cache card
    left those requests out, so the hit rate read higher than the share of input tokens actually read from the cache.
    See What the numbers mean.
  • An answer with no usage is not a lost cache. GitHub Copilot sometimes answers with no usage at all (when a
    very small output limit is reached first); the usage log and the Cache card now say "the provider answered with
    no usage" instead of counting a lost cache.
  • The Cache card's last hour agrees with today. Its estimate of what cache reads and writes cost uses the prices
    each request was estimated with, as today's does; it could read more than the whole day's. Its request count,
    and that of maple-proxy cache, now says it is the requests sent to a provider, with the answers from Maple's
    response cache given apart (the Overview's Requests today counts both).
  • The Savings card says why a figure is not valued. The prompt-cache note says which reads were answered before
    their model's prices were known (and stay unvalued) and which models are still waiting for prices, naming only
    those. The response-cache line says why a reused answer is not valued ("1 answer reused", in the right number).
    A model's prices read "read … AI units per token; cache writes not reported by the provider", with the unit
    right after the prices.
  • The Savings card's rows line up. On the Savings card and the filtered Savings card, each value now sits on the
    line of its own label, at every window width and in both themes, however long the explanation under the label.
    On a wide window, a value could look as if it belonged to the row above.
    See Reading the numbers.

Maple Harness

  • The cost preview prices Copilot Auto. On Copilot Auto it gave no cost estimate, even when the proxy had
    already priced many of Auto's requests. The preview now uses what that model costs through Auto (the
    model Auto answered the last request with), and says so in its basis; a model reached only through Copilot Auto
    so far says its prices are Copilot Auto's. It needs the proxy from this release; until Auto has answered once in
    a session, it says why there is no estimate yet. See the cost
    preview
    .
  • An unknown slash command is never sent to the model, in the terminal, the line mode, --prompt, remote
    devices or --rpc: /model gpt-6-sol in --repl was sent as a message and answered at the cost of a request.
    The session says there is no such command and sends nothing; begin a message with // to send it starting with
    /. See
    Slash commands.
  • The context meter of a resumed session read "context: 0%" until the next request; it now shows the
    conversation's size at once (the harness's own estimate until the next request's real figure).

Upgrading

  • --prompt with an unknown slash command (/word) now exits with status 1 and sends nothing, instead of
    sending it to the model; start the prompt with // to send a literal /.
  • Copilot Auto's prices are worked out again at the first start after the upgrade, from the usage log, or,
    without enough of it, over the next few requests. Until then, the cost preview and the Savings card use the prices
    they had before the upgrade.
  • Hit rates may read lower than before, because a model's first, cold request now counts as a miss.
  • Going back to 2.2.2 keeps working, but drops the separate Copilot Auto prices and the counts of free models
    from the saved account; on returning to this release the Copilot Auto prices are worked out again, and free
    models are counted again.

Known limitations

  • The checks against real GitHub Copilot were run on Linux. macOS and Windows were not run for this release.
  • GitHub Copilot reports no cache writes for some GPT models (for example gpt-5.3-codex), so "tokens written to the
    cache" is 0 for them, as Copilot states it.
  • Printing the setup page was tested in Chromium only.

Install

curl -fsSL https://raw.githubusercontent.com/kinncj/maple-harness-dist/main/install.sh | bash
irm https://raw.githubusercontent.com/kinncj/maple-harness-dist/main/install.ps1 | iex      # Windows

Update an existing install with maple-harness update. The installers and update verify every download against SHA256SUMS
(and its signature, when the project signs releases). Third-party licences: THIRD_PARTY_NOTICES.md, or maple-harness licenses.

Free for everyone to use, commercial use included. Closed source, provided as is, without warranty — see the attached LICENSE.

macOS: these binaries are not notarised, so macOS will warn if you download them by hand. The installer and maple-harness update are unaffected.
Windows: these binaries are not Authenticode-signed, so SmartScreen may warn.