v0.1.32

Upgrade notes

• Daemon file logging is now hardened and rotating (#626): the detached
  daemon's stm-daemon.log moves onto the shared private handler, gaining
  0o600/0o700 permissions and 2 MiB × 3 rotation it lacked; a file-open
  failure there now propagates instead of degrading (stdio is DEVNULL, so
  the file is the only crash trace). A malformed MEMTOMEM_STM_LOG_LEVEL now
  raises loudly at server startup instead of silently falling back to
  WARNING.
• Per-upstream circuit breaker is on by default (#620): after 3
  consecutive failed calls an upstream's tools fail fast with circuit_open
  for up to 60s instead of each call burning the full retry/deadline budget.
  Set circuit_max_failures: 0 on an upstream to restore the old
  always-retry behavior.
• Embedding-scorer compression moved off the event loop (#628): with an
  embedding relevance scorer configured, compression now runs on a worker
  thread so one slow endpoint no longer stalls concurrent calls. The default
  BM25 path is byte-identical; per-call semantics are unchanged.
• mms status no longer prints per-server rows (#645): status is now a
  config summary; per-server detail (including the surfacing toggle) lives in
  mms list, which gains a SURFACING column. Scripts scraping status's human
  compression= / surfacing= lines must switch to mms list or
  mms status --json — the JSON shape is unchanged apart from two additive
  keys (server_count, pruned_count).

Security

• CI supply-chain hardening (#627, issue #609) — the release pipeline was already hardened (PyPI Trusted Publishers OIDC, least-privilege id-token: write), but CI itself had no supply-chain defenses. Every workflow uses: ref is now pinned to a full commit SHA with the version tag in a trailing comment (previously mutable tags — @v4, @release/v1 — which a compromised upstream repo can move silently); ci.yml and release.yml gain a top-level permissions: contents: read baseline (jobs with elevated needs — bench_qa_pr_comment, publish — already carry job-level blocks, which fully replace the top level); a new .github/dependabot.yml covers the github-actions and uv ecosystems weekly with minor/patch updates grouped (Dependabot maintains SHA pins natively, and the existing dependabot[bot] CLA allowlist entry becomes functional); and a new advisory audit CI job runs pip-audit against the exported lockfile (continue-on-error: true, mirroring the mypy convention — the current baseline has known CVEs in transitive dev deps, so it starts advisory; promote to required once Dependabot cleans the baseline). CI/infra only — no runtime code touched.

Added

• --json result summaries for the mutating CLI commands (#644, issue #614) — mms add / remove / prune / eject only spoke human text, so scripting them meant scraping colorized output; the read-only commands (status/list/stats/health/…) have taken --json for a long time. Each now emits a single JSON result document on stdout with a shared envelope: action + ok on every payload, error (stable snake_case code) + message on failures, all keys always present. Following the mms host sync --json precedent, --json is an output format, not consent: remove/prune/eject without --yes (and not --dry-run) refuse with exit 2 and a confirmation_required envelope instead of prompting — even on a TTY — and the eject secret gate fails the entry at plan time rather than confirming. stdout stays pure JSON (human stdout lines move into the payload or warnings; stderr diagnostics unchanged); operational failures keep exit 1. Secrets stay out: add's server block is redacted like list --json, and eject never serializes the captured host original into plan rows (failed[].hint embeds the restore payload exactly like the existing stderr fallback hint — treat failed-eject --json output like the config file). init deliberately stays without --json — it would need a fully non-interactive wizard first. add --json --from-clients is a usage error (interactive selection flow).
• mms tune — preview and apply per-tool tuning recommendations (#642, issue #615) — stm_tuning_recommendations computes per-tool max_result_chars / compression suggestions but ended with "apply manually to stm_proxy.json", leaving the user to hand-edit the nested per-tool override block — exactly the error-prone step the recommendations were meant to remove. mms tune runs the same CompressionTuner analysis offline against the on-disk metrics/feedback stores (no running server needed; a preview never creates the DB files) and renders the tool_overrides diff it would write; --apply writes the accepted overrides through the raw-dict load/save path (unknown keys survive) under the config write lock, after snapshotting the config to a timestamped stm_proxy.json.bak-<UTC> slot (mode 0600, non-clobbering) — a running proxy hot-reloads the file, so the echoed one-command cp restore is the safety net. Selection is per (server, tool): --yes applies all, a TTY gets the enter-to-toggle picker with everything pre-selected (MMS_NO_TUI degrades to per-tool confirms), and non-TTY without --yes errors loudly. Colliding tuner actions are merged before writing (compression last-wins — agent-reported feedback beats the latency pin; max_result_chars takes the numeric max), the mutated dict is schema-validated before anything is written (never produce a config a running server silently ignores, #611), and recommendations for upstreams present only in metrics (env-defined / renamed) are reported as skipped rather than written. --since-hours / --tool mirror the MCP tool's parameters; --json emits the preview for scripting.
• Observability discoverability hints — data source, hidden tools, inert INDEX (#641, issue #613) — three gaps in an otherwise strong observability surface let a user reach a wrong conclusion or miss tools entirely; all pure presentation, no counter semantics change. (1) mms stats reads only the on-disk stores while stm_proxy_stats reports live in-memory counters (cache hits/misses, latency percentiles, reconnects, RPS, progressive and hint events — never persisted), so comparing the two silently diverged with the caveat buried in the command docstring; the human output now carries a Data : line and --json a data_source key naming the on-disk source and pointing at the live MCP tool. The divergence is structural (those counters are never written to disk), not a flush lag, so there is nothing to reconcile. (2) 9 of the 13 observability tools are hidden behind advertise_observability_tools (default off) with no runtime hint they exist; mms health now appends "9 observability tools hidden; set MEMTOMEM_STM_ADVERTISE_OBSERVABILITY_TOOLS=true to expose them" (plus obs_tools_hidden/obs_tools_hint JSON keys), driven off the same _should_advertise_obs_tools() signal that gates registration and counting a new _OBSERVABILITY_TOOL_NAMES source-of-truth tuple so the number can't drift. The hint lives on mms health (a CLI command, always available) rather than stm_proxy_health because that MCP tool is itself gated — it is unreachable over MCP in the exact flag-off state where the hint would apply. (3) stm_index_stats returned the generic "No INDEX activity recorded" message even when the INDEX stage is structurally unwired (the bundled mms server passes no index engine, #288) — indistinguishable from wired-but-idle; a new public ProxyManager.index_engine accessor (parity with selection_log) lets the tool render a distinct "INDEX stage inactive in this server (no index engine wired) — see #288." instead. This only improves the message — it does not pre-empt the #288 retire-vs-wire decision.
• Per-upstream circuit breaker on the proxied call_tool path (#620, issue #608) — SECURITY.md has claimed "per-upstream circuit breaker isolates failures" since the resilience section was written, but the upstream fetch path had none: a persistently-failing upstream was retried on every call, each caller paying the full retry/deadline budget (default up to 180s) indefinitely, with no fast-fail and no isolation beyond the shared timeouts. Each upstream connection now carries its own CircuitBreaker (the same 3-state pure-read primitive the surfacing engine and LLM compressor/extractor already use, #600). Failure semantics: one count per call, not per attempt — only a call that exhausts its retry/deadline budget on a transport fault or timeout counts (the three terminal exits: retries-exhausted/replay-unsafe, overall-deadline exceeded, and a mid-loop reconnect failure); any completed round-trip records success — a tool isError result and a JSON-RPC protocol error (-326xx) both prove the upstream replied, so both reset the failure streak — while a programming/internal exception (no round-trip) does neither. After circuit_max_failures (default 3) consecutive failed calls the breaker opens and calls to that upstream fast-fail with a new circuit_open error category naming the upstream and the time until the next probe; after circuit_reset_seconds (default 60) the next call probes and its outcome commits the state. Cached responses keep serving while the breaker is open (the fast-fail sits after the cache fast-path), other upstreams are unaffected, and stm_proxy_health renders a per-upstream circuit breaker: line from the same pure-read state labels as the surfacing breaker line. circuit_open rows count as upstream-attributable in the tool-exposure health filter so fast-fails don't dilute per-tool error rates during an outage. New per-upstream config circuit_max_failures / circuit_reset_seconds (0 disables; connect-time snapshot like max_retries). Behavior change: after 3 consecutive failed calls an upstream's tools fail fast with circuit_open for up to 60s instead of each call burning the full retry/deadline budget; set circuit_max_failures: 0 on an upstream to restore the old always-retry behavior.
• mms config validate, unknown-key warnings, and visible parse failures (#625, issue #611) — the proxy config models keep pydantic's default extra="ignore" (so an older binary tolerates fields a newer CLI wrote), which means a typo'd key silently vanishes across the 100+-field surface, and load_from_file collapsed "file present but broken" into the same None as "file missing" behind only a stderr warning — invisible for an MCP-launched stdio server whose stderr the client captures or drops. Three additions close the gap: (1) an annotation-driven find_unknown_keys walker names every dropped key as a dotted path (descending into dict[str, Model] fields like upstream_servers / tool_overrides but never into free-form dict leaves — env, headers, server_name_map, origin.original — classified off model_fields annotations, not a name allowlist, so it tracks model evolution), and unknown keys found in the raw file dict (before the env merge, so env-injected keys are never misattributed) are logged as one aggregated warning per load; (2) load_from_file_with_status returns ConfigLoadResult(config, error, unknown_keys) with error set iff the file exists but failed to parse — load_from_file delegates with unchanged signature/behavior — and stm_proxy_health now surfaces a parse failure as its cause instead of only the downstream "No upstream servers configured." symptom; (3) a new mms config validate command reports both the parse status and the unknown-key list for a config without starting the server.
• Opt-in rotating file log under ~/.memtomem/ (#626, issue #612) — the MCP stdio server logged to stderr only, which the launching client captures or drops, so diagnosing "why did my proxy do nothing" (the dark-failure modes in #611 are only visible there) meant hunting per-client log locations. A new logging_setup.py adds PrivateRotatingFileHandler (0o600 file via os.open+fchmod, parent 0o700, 2 MiB × 3 backups) wired through configure_server_logging (stderr always on; opt-in file via MEMTOMEM_STM_LOG_FILE, degrading to stderr-only on OSError); server.main() now folds log_level + log_file into one STMConfig() env surface, so a bad MEMTOMEM_STM_* value fails loudly at startup with a logged ValidationError instead of the lifespan raising it later unlogged, and mms health prints the active log destination (text + logging JSON key). The daemon's own stm-daemon.log converges onto the same hardened handler (see Upgrade notes). Redaction is untouched — it happens at message-construction time (manager.py:_redacted_error), so the file handler inherits it for free.
• Startup warning when an LLM path runs with the privacy scan disabled (#640, issue #610) — with privacy_scan_enabled=false, ProxyManager passes None for privacy_patterns, so raw upstream responses reach the external LLM provider without the #289 credential redaction; the scan is default-on, but an operator who flipped it off got no signal anywhere that credentials now leave the machine unscanned. Mirroring the #288 "config enabled but inert" startup warnings, ProxyManager.start() now scans the resolved compression and extraction paths and logs a WARNING naming each enabled site and the destination provider/endpoint when the scan is off. Scoped to external destinations via a new LLMCompressorConfig.is_external_destination() — OpenAI/Anthropic always qualify, Ollama only on a non-loopback base_url — so the common local-Ollama scan-off setup (text never leaves the box) stays silent; extraction's warning is further gated on index_engine is not None (the bundled mms server wires none, #288), and only an explicit compression: llm_summary is flagged, not auto (which resolves at runtime and would produce static false positives). Part 1 of #610; the entropy heuristic (part 2) is out of scope.

Changed

• mms status is a config summary; per-server detail moved to mms list (#645, issue #614) — the two commands printed near-identical output (both enumerated every server's prefix/transport/compression/command), so neither had a clear job. status now answers "is the proxy set up and pointed at the right config": path, enabled flag, schema-validation warning, and Servers: N (P host-pruned) — the pruned count reuses _origin_fully_pruned, the same predicate behind list's * marker and remove's orphaning hint, so the three surfaces cannot disagree — plus pointer lines at mms list / mms health (or an mms add hint when empty). list stays the per-server view and gains a SURFACING column (the visible home of the per-server mms surfacing toggle, since status no longer shows it); deliberately no MAX_CHARS column — the effective value is per-tool once mms tune --apply writes tool_overrides, so read it via --json or the config file. Behavior change: mms status human output no longer prints per-server rows — scripts scraping its compression= / surfacing= lines must switch to mms list or status --json; status --json itself is unchanged apart from two additive keys (server_count, pruned_count), keeping the full redacted servers map that scripted consumers and the #476/GHSA redaction pins depend on.
• mypy is now a required CI gate on both platforms (#638, issue #617) — typecheck had been advisory (continue-on-error: true on the mypy step) since the Windows matrix landed, which let real errors accumulate unseen. The #617 burn-down fixed all 9 baseline errors and re-enabled the globally-disabled attr-defined error code (#635, 19 more errors fixed — including typing the pending-store params with the existing PendingStore Protocol and a FeedbackDbStatus TypedDict for inspect_feedback_db); this change then fixes the 9 remaining Windows-only errors the advisory step had been masking (POSIX-only stdlib usage — fcntl in mms/state.py, os.getpgid/os.killpg/signal.SIGKILL in daemon/server.py — now behind sys.platform guards mypy can narrow; all three sites were already runtime-unreachable on Windows, so the guards are runtime-equivalent) and drops the continue-on-error flag, so typecheck (ubuntu-latest) / typecheck (windows-latest) fail red on any new mypy error. The two typecheck contexts are added to branch protection's required checks alongside lint/test. CLAUDE.md, README, and CONTRIBUTING updated to match (mypy moves from "advisory" to must-pass). The audit job's pip-audit step is now the only advisory step in CI, on the same promote-when-baseline-clean trajectory (#609).

Fixed

• Tuner feedback no longer pools across servers sharing a tool name (#643) — get_tool_feedback_summary aggregated compression-feedback rows with GROUP BY tool, kind, dropping the server column that record() persists, and the tuner joined feedback onto profiles by raw tool name — so with two upstreams exposing the same tool name, one server's feedback reports surfaced on the other server's profile and the feedback heuristic recommended tuning the wrong server's tool_overrides. Display-only wrong attribution in stm_tuning_recommendations so far; caught by review of the mms tune --apply PR (#642), which would have turned it into an automatic wrong config write. The aggregate is now keyed by (server, tool) — the identity every other per-tool store uses — and the tuner joins on the pair; the summary's only consumer is the tuner, so the shape change has a single call site.
• A slow embedding endpoint no longer stalls the whole proxy (#628, issue #618) — EmbeddingScorer makes a synchronous httpx call inside score_sections while every consumer runs on the asyncio event loop, so with an embedding relevance scorer configured (opt-in; the default BM25 scorer is pure CPU) one slow or unresponsive Ollama/OpenAI endpoint froze every concurrent proxied call, surfacing pass, and health probe for up to embedding_timeout (default 10s) per compression — measured end-to-end, a concurrent 1-line tool call took ~3.5s behind a 4s embedding block. Scorers now declare uses_blocking_io (read via getattr with a False default, so custom scorers keep the inline path), and a new ProxyManager._compress_maybe_offthread helper routes all eight scorer-carrying sync compress() sites through asyncio.to_thread only when the flag is set; fallback_count increments under a lock so worker-thread failures can't lose a count against the loop-thread metrics read. Behavior change: with an embedding scorer configured, compression runs on a worker thread (the loop stays responsive; per-call wall time and the BM25-fallback-on-error semantics are unchanged); the default BM25 path takes no thread hop and is byte-identical. Follow-ups deliberately out of scope: embedding response cache, circuit breaker on the embedding path, full async conversion of compress().
• The mid-loop reconnect-failure log no longer leaks the credentialed upstream URL (#623, issue #622) — the #605/#606 credential-redaction sweep scrubbed every reconnect/cleanup log site in ProxyManager except one: the mid-loop reconnect failure on _fetch_upstream's retry-continue path logged the raw exception at logger.error. _reconnect_server re-raises the underlying connect error, and for an HTTP/SSE upstream that httpx error embeds cfg.url — which can carry userinfo credentials — so a reconnect failure during a retry storm wrote the token to the logs, at the default ERROR level (broader than the DEBUG siblings). The site now renders through the existing _redacted_error choke point (redact_url_userinfo + MAX_ERROR_MESSAGE_CHARS cap), matching the three sibling reconnect-failure sites in the same loop. Pure hardening — no behavior change beyond the log string; a per-site redaction regression test now covers the mid-loop path the original sweep missed.