Skip to content

v2.20.0 — a validated endpoint is the one connected to

Choose a tag to compare

@acidkill acidkill released this 01 Aug 09:31
· 130 commits to main since this release
0f7ffb4

Reasoning-trace embedding was configured correctly and used anyway by nobody: _get_embedder() probed the environment instead of reading [embedding], and an unrelated cloud API key was enough to shadow a working local provider. Classification silently fell back to keywords. Because that path never ran, everything downstream of it had never been exercised either — which is where the rest of this release comes from.

A validated endpoint was not the endpoint that got connected to

The loopback gate exists so reasoning traces — private user data — never leave the machine. It was not holding.

The host test accepted anything starting with 127., so 127.0.0.1.attacker.example passed. It now parses the host and asks the address itself whether it is loopback.

Worse, for two providers the value that cleared the gate had no causal connection to the URL the HTTP client actually opened. The gate delegated to the provider factory, which re-resolves base_url on its own: openrouter carries a hardcoded remote default, and openai reads only the environment variable. A loopback endpoint set in config.toml — exactly what the new [embedding] endpoint field is for — passed the check while requests went to a hosted API. No attacker required, just the documented configuration surface. ollama skipped the check entirely through short-circuit evaluation, and its base URL comes from OLLAMA_BASE_URL, which nothing validated.

The endpoint that clears the gate is now passed explicitly as the client's base URL, so "the check passed" means the request went there.

Pattern clustering had been calibrated against nothing

The cosine threshold that decides whether two traces are the same pattern was a module constant which, because of the defect above, had never executed against a real embedding model.

Embedding models do not share a similarity scale. Measured against a real corpus, that constant sat above the 99th percentile of pairwise trace similarity in every category: the two largest categories produced no clusters at all, and the embedding path yielded roughly a fifth of what its own move-set fallback produced over the same traces. Mining reported patterns learned while whole categories stayed empty — which reads as "not enough material" rather than as a threshold defect.

It is now reasoning_training.cluster_cosine (default 0.75, clamped to 0.05–1.0), so it travels with the configured embedder. The floor matters: single-linkage clustering collapses into one component as the threshold approaches the corpus baseline, so a very low value yields fewer patterns, not more. A non-finite value falls back rather than clamping into the floor.

If a brain's category coverage looks stuck, check this value before raising pattern_targets — a higher target cannot help when nothing clusters.

[reasoning_training]
cluster_cosine = 0.75

Also in this release

[embedding] endpoint joins the reranker's equivalent knob, so a local OpenAI-compatible embedding server can be pinned in config.toml instead of only through an environment variable.

Documentation reconciled against the code: memory-type expiry tables, a documented strategy with no enum member, five MCP tools and two API surfaces that do not exist, stale maturation advice, and every count in README/docs/promo re-derived from source. The reference scanner gained the forms it used to walk past.

Upgrade notes

No migration required. Existing patterns are untouched; distillation skips signatures it has already materialised, so re-running it adds rather than duplicates.

If your coverage was suppressed by the threshold defect, re-open the backlog and re-distil:

smem reasoning mine --reprocess