v0.18.0 — the pack substrate consumers can build on

v0.18.0 — the pack substrate consumers can build on

Since v0.17.1. No schema change (still v50). Additive API; no removals.

Packs: provenance, signed settings, allowlist recall, mount-ordered context (#203)

Four things a consumer (an expertise lease, the Hermes plugin, the MCP server) could only work around before, now on the engine so every consumer gets the same answer:

• Every recall hit carries structured pack provenance — RecallResult.pack: Option<PackProvenance { pack_id, name, version, trust, content_digest }>; hit["pack"] in Python, None for host rows. The pack:{name} prose stamp is name-only and stays for one release.
• mounted_packs() reports the signed facts — namespace (dropped at the Python boundary until now), content_digest, coverage, recommended_top_k, recommended_min_similarity, publisher_pubkey, signed. No consumer needs the unsigned pack.toml.
• pack_context_for(pack_ids) — the context block for only those packs, in mount order, duplicates collapsed; unknown id → PackNotMounted (new typed exception, subclasses RuntimeError); [] → None.
• recall_from_packs_for(pack_ids, …) — pack-only recall. Allowlist validated before anything is searched or embedded; only the named packs are searched (host rows are never candidates); the fetch widens until top_k rows are admitted or the index is exhausted, so filters cannot hide a valid row; each pack's signed floor is a wall on raw similarity — effective_pack_floor(declared, host_min) = max(valid declared, valid host_min), the host may raise, never lower; host supersedes corrections still remove pack rows; exact-score ties break by mount order then rid.

The ordinary recall merge seam is unchanged (no floor): applying the signed floor to default ranking is a benchmark-gated decision.

Review trail: eight defects found and fixed in review (saturating fetch width; widening fetch with no approximate-search early stop; validate_embedding after the allowlist; validation before embedding a query string; contained name/pack_id in the context header; chunked SQL IN lists at 500; no-op calls spend nothing; exact tie-break pinned). Tests: 16 Rust (pack_allowlist.rs) + 14 Python (test_pack_substrate_0_18.py).

Fixed: every pack published before 0.16 was unmountable (#205)

If you tried to mount a pack built by an engine at or below 0.15.x on 0.16.0, 0.17.0 or
0.17.1, it failed with no such column: synthesis_state. That was every pack this project
has published — measured across the whole line: 40 of 40 failed before this release, 40 of
40 mount again now.

load_scoring_cache projected the v41→v42 synthesis columns unconditionally while serving two
databases with different guarantees: the host's, which is always migrated before it runs, and
a mounted pack's, which is opened read-only from a file another engine sealed and can never be
migrated. The host path could not surface the break, so it shipped. The projection is now built
from the columns that exist; absent ones read as NULL, which is exactly what v42 migrates
existing host rows to. Your pack files are never rewritten.

If you publish packs: nothing to do, and no need to re-seal. If you consume them: upgrade and
your existing .ydbpack files mount again.

Release engineering

• Publish smoke on the Python floor (#202) — pypi.yml installs the built wheel on 3.10, asserts version = metadata = tag and a record/get round-trip, and gates PyPI and crates.io on it. Closes the blind spot that shipped 0.16.0/0.17.0 with a 3.11-only import.

Measured before release

Co-iteration gate v1.4 (yantrikdb-hermes-plugin, one checkpointed seed, baseline 0.17.1 vs
candidate 0.18.0, 2 rounds × 7 repeats, alternating process order, 6 cold opens per arm per
round): comparator exit 0, no identity refusal. precision@5 0.5 = 0.5, possessive top-1
0.9167 = 0.9167, Jaccard 0.9444 = 0.9444, direction separation 0.025–0.0625 → 0.0347–0.0625,
role share on ambiguous queries 0.5625–0.625 both. Ordering signatures: candidate-only ∅,
baseline-only ∅. Plugin suite on the candidate engine: 488 passed. No category regressed.

Also verified on a real store: the engine was installed in a throwaway venv against a
6003-row copy of a live production database (schema 50 → 50, no migration), exercised
write + recall + every new pack surface, with the live store untouched.

Compatibility

• Additive: new #[serde(default)] field on RecallResult, new PackInfo fields, new methods, new exception. Rust callers building RecallResult/PackInfo by struct literal need the new fields (both are engine-constructed types).
• Dependents pin <0.18.0 (hermes-plugin, mcp, langchain): each admits 0.18.0 through its own gate before its pin lifts.