Skip to content

v2.3.0

Choose a tag to compare

@kinncj kinncj released this 21 Oct 07:23
· 31 commits to main since this release

Maple Harness v2.3.0

Highlights

  • Prices for each model. Set a model's price in config.yaml, or in the model's ⋯ row on the setup page,
    and the Savings card values it from its first request, labelled as your price. Prices learned from your own
    requests always win, and the card tells you when they disagree with the one you set. See
    Prices for each model.
  • Copilot Auto and the same model by name value each other at once. Copilot bills a request through Copilot Auto
    at 0.9 times the model asked for by name; once either one's prices are known, the other is valued straight away,
    and the card says so.
  • One price for a token everywhere. The Cache card's estimates, cache warming, the model list, the cache-misses
    report, the harness's cost preview and its /savings now price a token as the Savings card does.
  • Several replicas without PostgreSQL. storage.mode: redis keeps usage, budgets, settings and tokens in one
    Redis (or Valkey) that keeps its data. PostgreSQL with Redis stays the recommendation for several replicas. See
    Kubernetes: without PostgreSQL.

Added

  • price for a model (providers[].models[].price: unit, input, cached_input, cache_write, output, per
    million tokens), in config.yaml and on the setup page. In AI units it values the model's savings; in dollars it
    gives the cost preview (before the provider's published list price).
  • storage.mode: redis (and MAPLE_STORAGE_MODE=redis): replicas share one Redis or Valkey, with no
    PostgreSQL. The proxy starts only on a Redis that keeps its data (append-only file on, maxmemory-policy noeviction), and a replica stops taking requests (not ready, and says why on the setup page) when Redis stops
    keeping it or loses it; budgets then follow storage.budgets_when_down. Docker (compose.ha-redis.yaml) and
    Kubernetes (config/ha-redis) samples with a restricted Redis user. In this mode the usage lake and SQL over the
    shared usage (--source postgres) are not available; maple-proxy usage commands and exports work as before.
  • Where each figure's prices came from. The usage log records it on each valued request (price_source,
    schema version 7: learned, copilot_auto_discount or configured); maple-proxy usage query --metrics priced_stated counts the requests valued without learned prices of their own, and the filtered Savings card shows
    it.

Fixed

  • A model is no longer valued at prices its requests cannot tell apart. When every request of a model had the
    same shape (the same prompt, the same share of it cached), its prices could come out wrong while still matching
    every bill; the Cache card's estimate of what cache reads cost was then off. Such a model now waits for requests
    that differ, and the card says why. Found on real GitHub Copilot requests.

  • Cache warming for a Copilot Auto conversation decides with Copilot Auto's prices, not the model's price by name.

  • The cost preview of a model reached only through Copilot Auto so far is no longer 10% low when you ask for it
    by name: it uses the Auto prices without the discount, and says so.

  • The model list's price (input_ai_units_per_1k, which the harness uses to pick a model for side jobs) comes
    from the same prices as the Savings card, not from a separate rough estimate.

  • The cache-misses report (maple-proxy usage cache-misses and the setup page) prices a missed token as the
    Savings card does, a Copilot Auto request at Copilot Auto's prices, and says which basis it used.

  • storage.mode in config.yaml with its key in MAPLE_STORAGE_KEY (or its addresses in MAPLE_POSTGRES_DSN
    and MAPLE_REDIS_URL) no longer stops the proxy at start with "needs storage.encryption_key": the environment
    is taken into account when the configuration is checked, as it already was when the proxy runs.

  • A session started for remote control could lose its socket when the remote gateway looked at it in the
    instant between the socket being made and it listening: the gateway took it for a crashed session and removed it,
    and the start failed ("announcing this session"). A session whose process is alive and whose announcement is new
    is now treated as still starting.

  • The harness's /savings learns each model Copilot Auto answers with on its own (their prices differ), and
    values models whose provider reports no cache writes (GPT models on GitHub Copilot) from their reads, as the proxy
    does.

  • Built with Go 1.26.10, which fixes a certificate-checking flaw in Go's standard library (GO-2026-6809).

Upgrading

  • The usage log moves to schema version 7 (one column added at the end) and the query index rebuilds itself once.
    Earlier lines read as before.
  • Today's savings may read higher than before the upgrade: Copilot Auto requests (and requests by name) are valued
    from the first request once the other kind's prices are known, where they used to wait for their own.

Known limitations

  • Redis mode was run against Valkey 9.1 built from source, not yet against managed Redis services (which may
    hide INFO or offer no append-only file: the proxy then refuses to start), Redis failover, or a real cluster.
    Anyone who can write to that Redis can change budgets and usage figures (settings and tokens stay signed and
    encrypted): give the proxy its own restricted Redis user, as the samples do.
  • GitHub Copilot sometimes answers a request with no usage at all (every token count and its price stated as 0),
    seen when a reply is cut off by a small output limit while the model is still reasoning. Maple records exactly
    that, as "the provider answered with no usage"; whether GitHub bills such a request cannot be seen from it.

Install

curl -fsSL https://raw.githubusercontent.com/kinncj/maple-harness-dist/main/install.sh | bash
irm https://raw.githubusercontent.com/kinncj/maple-harness-dist/main/install.ps1 | iex      # Windows

Update an existing install with maple-harness update. The installers and update verify every download against SHA256SUMS
(and its signature, when the project signs releases). Third-party licences: THIRD_PARTY_NOTICES.md, or maple-harness licenses.

Free for everyone to use, commercial use included. Closed source, provided as is, without warranty — see the attached LICENSE.

macOS: these binaries are not notarised, so macOS will warn if you download them by hand. The installer and maple-harness update are unaffected.
Windows: these binaries are not Authenticode-signed, so SmartScreen may warn.