Skip to content

v1.1.0 — Anonymization: prompt-safe pseudonymization

Choose a tag to compare

@sathish-mg sathish-mg released this 16 Aug 17:16
· 25 commits to main since this release
1cc0fed

Added

  • Anonymization: prompt-safe pseudonymization (areev anonymize,
    cookbook recipe 16). Declare one anon:<ns> policy — a file-truth that
    replicates write-if-absent and fails reads closed when unreadable — and
    every model-facing read (recall/search/CAL/MCP/graph reads) returns typed
    placeholders ([PERSON_1]) instead of identities:
    • Detection is layered: built-in Tier-0 (structural known-identity
      propagation, regex + Luhn/mod-97 validators, secrets, keyword cues,
      dictionaries), a pluggable NER command seam (--anonymize-cmd), and a
      grounded LLM detector (--anonymize-llm-cmd) — a policy demanding an
      uninstalled detector fails closed. Actions: pseudonym, mask,
      redact, generalize:month|year|decade, allow.
    • The round trip: mappings stay in process custody
      (anon_mappings(), rehydrate_text(); payloads carry an anonymized
      report with mapping ids only). PseudonymizingBackend wraps any
      LlmBackend so extraction requests leave pseudonymized and responses
      return rehydrated.
    • Ingress mode + memory scope (encrypted memories): value-derived
      tokens transform before the content address commits; FORGET SUBJECT/REPORT SUBJECT recompute the stored pseudonym from the real
      identity, so pseudonymized-at-rest never means erasure-proof.
    • The sealed vault (vault: rows under an HKDF subkey of the page
      key; never replicated; erased with the subject; TTL-swept): tokens
      continue across processes, and areev anonymize reveal /
      reveal_tokens() is admin-gated and Tier-2 audited by fingerprint.
    • Surfaces: CLI verb family + --anonymize-egress host floor, Python and
      Node methods in lockstep, the console's Anonymization card + per-grain
      "Model view" (GET /api/anon/preview, POST /api/anon/config),
      /api/config observability, conformance cases on both backends
      (Postgres: egress/audit work; value-derived features refuse loudly —
      no page cipher there).
    • Explicit text APIs ship too: scan_text / anonymize_text /
      rehydrate_text and the store-free areev anonymize scan.
    • Honest scope, by design: this is pseudonymization of the egress
      channel, not anonymity — see docs/security-model.md and
      ARCHITECTURE.md §10 for the threat model and named decision.
  • min_reader_version stamping on anonymization policies so older
    builds warn loudly at open; anon: joins the replicable meta prefixes,
    vault: is reserved and never replicates.

Changed

  • One rendering stack. Per-grain
    rendering now has a single implementation — areev_cal::render — shared
    by CAL's FORMAT arms and areev-context, with byte parity pinned by a
    cross-surface golden. Output changes that follow:
    • FORMAT sml emits semantic per-type elements
      (<fact confidence="0.95" date="2026-01-13">john prefers window seat</fact>) instead of generic <grain type=…> field dumps; event
      elements carry the speaker as role="…".
    • FORMAT markdown gains dedicated arms for state / workflow / reasoning /
      consensus / consent / recommendation grains (topology and labels instead
      of a raw field-pair dump); fact/event/tool lines are byte-identical to
      before.
    • recall --render (markdown/json/toon/plain) converges on the CAL
      shapes: markdown carries the documented - bullet and the
      confidence-below-1.0 rule, json is the {hash, grain_type, fields}
      envelope, toon rows come from the registry columns.
    • FORMAT toon's state rows read the OMS §8.3 context key (previously
      context_data, which never matched — rows always fell back to
      state,state).
    • One chars/4 token estimator (render::estimate_tokens) serves
      ASSEMBLE … BUDGET and the areev-context allocators, so a budget means
      the same thing on every path.
  • Progressive disclosure is real. The context allocators emit
    Full→Summary→Omit (70%/95% thresholds); budgeted FORMAT TEMPLATE renders
    pick their disclosure tier from tokens-per-grain, so ELEMENT_SUMMARY
    fires under pressure and ELEMENT_OMIT accounts for dropped grains —
    behavior the reference already promised. JSON and TOON stay whole-entry
    (a prose summary inside a structured dump would corrupt it).
  • The registry replicates. Bundles/segments carry saved queries,
    templates and retention policies in a v2 MGB2 meta segment (emitted only
    when the file has registry rows — registry-free bundles stay MGB1 and
    readable by older builds; older builds refuse an MGB2 bundle loudly).
    Import merges latest-wins on updated_at; last_run_at never replicates
    and survives locally; retention rows apply only when locally absent; a
    point-in-time restore skips the segment. New conformance cases cover both
    backends; ImportStats gains meta_applied/meta_skipped.

Removed

  • The six whole-result builtin templates (triples, progressive,
    llm_system_prompt, llm_chat, weekly_standup, toon) — unused, and
    toon/triples shadowed the same-named FORMAT arms with different
    output. Builtins are now exactly the three §10.1 sectioned presets
    (structured/readable/compact), and a builtin can never take a
    FORMAT arm name. FORMAT TEMPLATE toon now returns TemplateNotFound
    — use FORMAT toon.
  • The never-wired CalExecutorConfig::max_cal_queries/max_cal_templates
    caps (no host set them, and their Some(-1) = unlimited convention was
    implemented backwards). The registry-level limits (100 queries/namespace,
    50 templates, body-size caps) remain the enforcement.
  • Dead areev-context dependency declarations in areev-py, areev-js,
    and areev-server.

Docs

  • Saved queries and templates are now discoverable where agents look:
    the cal-for-llms.md grammar card gains a SAVED block, the MCP reference
    documents the DESCRIBE QUERIESRUN pattern under areev_cal, and
    cookbook recipe 15 walks the ship-assembly-logic-in-the-file pattern
    (the Hermes provider's override). llms.txt's MCP tool count corrected
    (14 → 23); docs/facts/context-assembly.md re-verified.