perf(model): cache all ollama completions in one bounded client middleware (JEF-362)

[thejefflarson]

Closes JEF-362

What

Moves completion caching from the per-consumer verdict cache (JEF-350) to the model-client boundary in engine/src/engine/model.rs, so every model consumer is covered by ONE bounded mechanism instead of a cache per consumer. chat() now checks a shared cache before the HTTP call and stores only successful completions.

Middleware shape + key

• Key = DefaultHasher (SipHash) over endpoint + the request body, canonicalized with sorted object keys (array/message order preserved). The body carries everything that determines the response: model name, messages/prompt, temperature, max_tokens. So only a byte-identical request hits.
• Hit → return the cached completion, NO HTTP. Miss → call ollama, store, return.

Bounding + env

• Bounded in-memory Mutex<LruCache<u64, String>>, capacity default 512, overridable via PROTECTOR_MODEL_CACHE_ENTRIES (unset / unparseable / 0 → default).
• The lock is a tiny sync critical section in cache::get/cache::put; it is never held across the HTTP await, so the cache sits in front of the JEF-337 concurrent dispatch without reintroducing serialization.

Cache successes only

Transport errors, non-success statuses (500/502/503), over-cap bodies, and unparseable/shapeless replies all short-circuit to None before the store — they are never cached and retry next pass.

keep_warm bypass

keep_warm does not route through the cache (it never calls chat), so a keep-alive ping always hits the wire — caching it would let the model unload. Covered by a test.

Kept intact

JEF-350's deterministic verdict prompt and the decisive-verdict journal persistence (restart survival) are untouched; this is the in-memory general cache, not a replacement for the durable journal.

Scope note (JEF-363)

Per coordinator direction, the model-hypothesis stage is being removed by JEF-363, leaving adjudication as the only ollama consumer. This middleware is intentionally consumer-agnostic — it now primarily serves adjudication and backstops any future consumer. All edits here are confined to model.rs + model/cache.rs; no changes to hypothesis.rs, run_loop.rs, mod.rs, or reason/proof.

Tests

New tests (12): request-canonicalization key stability + sensitivity (prompt/model/temperature/endpoint/message-order), env parsing, LRU eviction at cap, and end-to-end through chat against a connection-counting localhost server — identical request served from cache with zero HTTP, distinct request misses, error not cached (retries), keep_warm bypass (always hits), and concurrent identical requests don't deadlock.

cargo fmt / cargo clippy --all-targets (clean) / cargo nextest run → 739 passed, 1 skipped (incl. the file-size guard; no file over 1000 lines).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP