Skip to content

v2.17.0

Choose a tag to compare

@tcconnally tcconnally released this 04 Jul 13:59
9e472e6

Security / Hardening

  • Multimodal ingest is now bounded against decompression bombs. A .docx is a
    DEFLATE zip, so a tiny on-disk file (within MIMIR_MAX_INGEST_BYTES) could
    decompress word/document.xml to many GB — the on-disk cap couldn't bound it,
    and the read was unbounded (OOM). The decompressed read is now capped at
    MIMIR_MAX_DECOMPRESSED_BYTES (default 256 MiB) and rejected past it. PDF
    extraction is bounded by the on-disk cap only (pdf_extract owns decompression
    with no limit API — documented; lower MIMIR_MAX_INGEST_BYTES for untrusted
    PDFs).
  • Network transport & gRPC hardening (audit phases 1–3):
    • Secure-bind guard: binding an HTTP surface (MCP transport or web
      dashboard) to a non-loopback address with no auth token now refuses to
      start instead of coming up wide open. Override with
      MIMIR_ALLOW_INSECURE_BIND=1 for a trusted network / auth-terminating proxy.
    • Constant-time token comparison for Bearer auth on both HTTP surfaces
      (was a byte-wise ==, a timing side-channel on the secret).
    • Request-body cap (MIMIR_MAX_HTTP_BODY_BYTES, default 8 MiB) and a
      global token-bucket rate limit (MIMIR_HTTP_RATE_PER_SEC /
      MIMIR_HTTP_RATE_BURST, default 50 req/s + burst 100 → 429).
    • Tightened transport CORS — explicit methods/headers instead of Any,
      with an optional MIMIR_CORS_ALLOWED_ORIGINS allowlist.
    • gRPC security model: serve now supports a Bearer-token auth interceptor
      (MIMIR_GRPC_AUTH_TOKEN), TLS and mutual-TLS (MIMIR_GRPC_TLS_CERT/KEY,
      MIMIR_GRPC_TLS_CLIENT_CA), a message-size cap (MIMIR_GRPC_MAX_MSG_BYTES),
      and the same secure-bind guard. See docs/GRPC-SECURITY.md.
  • Encryption canary (fail-fast wrong-key detection). set_encryption now
    verifies the configured key against a dedicated canary row at startup and
    aborts loudly ("the provided key is incorrect or the database is corrupt")
    instead of letting a wrong/rotated key silently AuthFailed on every later
    read. The canary is established on first encrypted setup (or when encryption is
    enabled on a legacy-plaintext DB); a canary-less store with pre-existing
    encrypted data is validated by scanning for authentic ciphertext, so a wrong
    key can never "bless" itself by writing a canary under it. Stored in its own
    encryption_canary table — invisible to recall/FTS/stats and caller-facing
    state tools.
  • Build-time model fetch is now supply-chain pinned (build.rs): the bundled
    all-MiniLM-L6-v2 ONNX model + tokenizer are fetched from an immutable commit
    revision
    (was the mutable main ref) and SHA-256 verified before being
    baked into the binary via include_bytes!. A compromised or updated upstream
    repo can no longer silently change the embedded model — a mismatch fails the
    build. Operator-supplied files (MIMIR_BUNDLED_MODEL_DIR, air-gapped builds)
    are verified against the same hashes.
  • Windows key-file ACLs: keygen now restricts the new key file to the current
    user via icacls (Windows has no 0600-at-creation equivalent), warning
    loudly if that fails; enabling encryption on Windows also emits a one-line
    runtime reminder that key-file ACLs are operator-owned.
  • Bumped anyhow 1.0.102 → 1.0.103 to clear RUSTSEC-2026-0190 (unsoundness in
    Error::downcast_mut()).

Performance

  • Empty-query browse recall no longer degrades on large stores. The browse path
    orders by retrieval_count DESC, last_accessed_unix_ms DESC, id ASC, but
    idx_entities_recall covered only the first two keys — so a large tie-group on
    the leading keys (a cold or bulk-imported store with uniform last_accessed)
    forced SQLite to sort the whole group by id to satisfy LIMIT k
    (O(tie-group)). The index now includes the id tie-break, making browse a pure
    k-row range scan. Measured p50 at 1,000,000 rows: 29.7 ms → 0.046 ms
    (~645×); FTS and point-lookup latencies were already flat and are unchanged.
    Ships a v13 schema migration that rebuilds the index on existing databases.

Fixed

  • De-flaked concurrent_writer_not_starved_during_cohere: the #400 lock-hold
    gate asserted exactly zero SQLITE_BUSY, which spuriously failed on loaded CI
    runners when OS scheduler jitter delayed a single (correctly chunked) cohere
    commit past the writer's ~250ms budget. Now asserts a low busy rate (<10%) —
    the #400 regression it guards produces ~100%, jitter ~0.5%, so detection is
    preserved with wide margin. No production-code change.