v0.12.0 — anti-hallucination

The anti-hallucination release. The synthesis LLM is the one component that
can fabricate; this release rings it with a seven-layer deterministic defense
(A–G) — every shield is no-LLM and structural, so the safety gate never rests on
a fallible judge. A–F detect fabrication (numeric, quote, citation-attribution,
card-source, cache, drift) and escalate to needs_review; G turns that
detection into prevention with a single blockOnNeedsReview refusal switch. The
non-functional GraphRAG path — proven to return 0 graph neighbours on every query
— is removed (BREAKING). Each layer also gains a standardised way to prove it
fires and to localise failures. Still drop-in (DB + OPENAI_KEY); every new guard
is off or neutral by default.

Added

• blockOnNeedsReview answer-policy switch (anti-hallucination defense G). A
  single KNOWLEDGE_ANSWER_POLICY field that hard-refuses any answer the engine
  flagged needs_review — the one switch over all deterministic hallucination
  signals (fabricated figure/quote, mis-attributed citation, cited-but-unentailed
  claim, stale-only sources). Turns the detection that A–F produce into prevention:
  the answer is replaced by a refusal and policy: { action: "block", reasons } is
  attached to the trace. Off by default (review-only behaviour unchanged). A
  zero-hallucination strict-posture example is documented in docs/knobs.md.
• Quoted-text verbatim check (anti-hallucination defense F). A deterministic,
  no-LLM Tier-A check (ungroundedQuotes): every substantial double-quoted phrase
  (≥3 words) the answer presents as a direct quotation must appear verbatim
  (whitespace/case-normalised) in a cited snippet, else the answer escalates to
  needs_review. A fabricated quotation is a distinct, high-trust-looking
  hallucination class that the numeric (B) and attribution (E) checks miss. Surfaced
  on the answer + wired into diagnose.
• Citation-attribution check (anti-hallucination defense E). A deterministic,
  no-LLM Tier-A check (weaklyAttributedClaims): a substantial cited claim (≥5
  salient content words) whose own snippet shares zero salient words is almost
  certainly mis-attributed — the citation marker points at an unrelated passage.
  Tier-A only checked the marker resolves; B checks numbers; this catches a
  fabricated non-numeric claim wearing a decorative/wrong citation. Escalates to
  needs_review, surfaced on the answer, wired into diagnose. Conservative
  (only a total miss on a substantial claim flags) so paraphrase doesn't false-fire.
• Fleet health / drift monitor (anti-hallucination detection D). ask_telemetry
  now persists each answer's status (migration 0010); summarizeAskHealth (pure,
  unit-tested) turns a window's per-status counts into rates + threshold concerns,
  and scripts/health.ts prints them and exits non-zero on a breach (cron/alert
  friendly). Catches systemic degradation — model swap, corpus drift, regression —
  that the per-answer guards can't see. DATABASE_URL only, no LLM.
• Card-source grounding at ingest (anti-hallucination defense C). A
  deterministic, no-LLM filter (dropUngroundedCards): an LLM-extracted card that
  asserts a numeric figure absent from the source text it was extracted from is
  dropped before storage — otherwise a fabricated/garbled number would be persisted
  as a card and later surface as a confident card_synth answer. Tabular (no-LLM)
  cards skip the filter; reuses the Tier-A numeric check, so no extra LLM call.
• Numeric verbatim check (anti-hallucination defense B). A deterministic,
  no-LLM Tier-A check (ungroundedFigures): every substantial numeric figure the
  answer asserts must appear in a cited snippet, else the answer escalates to
  status = needs_review and the figures are surfaced on the answer. Tier-A only
  checked citation presence, never the number itself, so a fabricated figure in a
  cited sentence slipped through — the highest-impact hallucination class. Catches
  it without relying on another (fallible) LLM judge. Conservative (commas
  normalised; lone single digits ignored) to avoid false flags. Wired into the
  diagnose classifier too.
• Layer verification & diagnostics framework. A standardised way to test each
  layer and localise failures, so it isn't re-invented per incident (see
  docs/verification.md):
  • lib/eval/layers.ts — one declarative manifest of every optional layer (knob,
    default, purpose, and a pure "did it fire?" predicate). Single source of truth.
  • smokeCheckableLayers() — the "enabled-but-dead" guard: every enabled layer
    must demonstrably fire. This is exactly the regression the cache (0% hit) and
    GraphRAG (0 graph rows) outages would have tripped.
  • lib/eval/diagnose.ts (classifyAnswer) — codifies the symptom→stage→knob
    runbook as a pure, unit-tested classifier; scripts/diagnose.ts feeds it a live
    answer's why-trace and prints the ranked "where to fix it".
• scripts/cache-bench.ts — integration benchmark for the cache's actual
  purpose (consistency / hit-rate / latency / near-dup) — the test that would have
  caught the bugs above; the pass-rate ablation was blind to it.
• Full-pipeline entrenchment eval. scripts/entrenchment-eval-full.ts proves
  the whole compounding lifecycle against a live Postgres + a real LLM judge: a
  grounded answer is promoted through the judge gate to an approved card, then
  retracted to deprecated once its cluster turns net-negative. Complements the
  no-LLM entrenchment-eval.ts (brake only). Self-cleaning.
• Eval harness measures the env-level components. scripts/seed-eval-corpus.ts
  honours KNOWLEDGE_ACCEPTANCE_EVALUATOR at ingest, so the reranker and the
  acceptance evaluator can be ablated by flipping their flag across two runs — not
  just the AskProps components.

Changed

• Cache only stores supported answers (anti-hallucination defense A). The
  cache write moved to after faithfulness/status are computed and is now gated on
  status === "supported" — previously any cited answer was cached before
  groundedness was even known, so a weakly-grounded/conflicted/hallucinated answer
  could be cached and then served back confidently + consistently to many agents
  (the cache amplifies whatever it holds). Deterministic; no extra cost.

Removed

• BREAKING: GraphRAG removed. The graph-augmented retrieval path is gone —
  the kl_graph_neighbors query, the KNOWLEDGE_GRAPHRAG_ENABLED / …_QAPLAN
  flags, and the graphContext provider. Targeted verification showed it
  non-functional: it returned 0 graph neighbours on every query, so it
  added cost, config surface, and failure surface while contributing nothing to
  retrieval (it was never the source of an answer). Reranking, by contrast, was
  kept — measured neutral on the small fixture but a genuine large-corpus
  feature, not dead weight. Removing GraphRAG shrinks the trust surface that the
  rest of this release is hardening. Deployments that set
  KNOWLEDGE_GRAPHRAG_ENABLED should drop the flag; no data migration is needed.

Fixed

• Answer cache never stored or hit (two bugs). With KNOWLEDGE_CACHE_ENABLED
  on, the cache silently did nothing:
  1. storeAnswer interpolated the material-id JS array as drizzle's ($n)
     value-list against the uuid[] column → Postgres malformed array literal →
     every write failed. Now bound as a {a,b}::uuid[] brace literal
     (pgUuidArrayLiteral, unit-tested).
  2. lookupAnswer read tx.execute(...) as a bare array, but it resolves to a
     { rows } QueryResult → the row was always undefined → every lookup
     missed even with a matching row.
     Verified end-to-end (scripts/cache-bench.ts): cache went from 0% → 75% hit
     rate (−75% LLM calls on a repeated-query workload) and answer consistency from
     3.5 distinct/question to 1.00 (byte-identical hits). The cache is off by
     default, so deployments running the default were unaffected.

Docs

• Definition of a bad answer. OPERATIONS.md §6 now opens by defining the
  category the runbook debugs (the dual of "what counts as knowledge"): AgenticMind
  promises faithful + honestly-labeled + safe, not true. A bad answer is a
  breach of that promise, judged on correctness × calibration — Tier 1 (the label
  lied: hallucination-as-grounded, confident-wrong, unsafe, stale-as-fresh),
  Tier 2 (quality, contract intact), Tier 3 (NOT bad — a wrong answer honestly
  marked unsupported kept the promise). Debug Tier 1 first.
• Debugging runbook — "where do I fix it?" OPERATIONS.md gains a
  symptom→signal→stage→knob table answering the operability worry of a dozen
  toggleable components: every stage stamps an attributable signal on the answer
  (status, servedBy, groundedness, contested, staleSourcesOnly,
  rerankUsed, phases[], citation trust/lifecycle), so a single bad answer
  localises to one stage and one knob. Start from status + servedBy.
• docs/evals.md records the live numbers. Full-suite baseline 224/234
  (95.7%, gate passed, citation precision/recall 100%, every safety bucket green);
  the promote→demote lifecycle measured green; and an ablation table showing
  contested-sources and Tier-B faithfulness each contribute +1.6 pts while cards
  and cache are correctness-neutral (latency/efficiency) on the fixture corpus.
  The reranker also measured +0.0 on the fixture (a scale feature the small
  corpus can't exercise — kept, not cut), and the acceptance evaluator held 29%
  of cards as candidate (a governance control, retrieval-neutral). Verdict: only the
  two LLM-judge correctness features move the pass rate here; nothing is cut without
  a corpus that exercises it.
• Abstention posture documented. docs/evals.md explains that out-of-corpus
  queries are surfaced as unsupported / groundedness = 0 (gate-able) by
  default, and hard abstained decline is opt-in (no sources, refusal phrasing,
  or a KNOWLEDGE_ANSWER_POLICY minGroundedness) — a deliberate, unit-tested
  surface-not-decide boundary, not an overconfidence defect.