feat(ai): retrieval evals, in-app tools, and the AI quality gaps 0394 found

[crs48]

Implements exploration 0394 — a retrospective map of how AI is integrated in xNet, and the gaps it found. 9/17 checklist items; 7 of 10 implementation items.

What this fixes

The repo measured agent interfaces but never AI outputs. The token benchmark was the only eval. So every retrieval improvement shipped on reasoning alone.

• Golden-set retrieval eval (packages/brain/src/__evals__/) — pinned corpus, 13 queries, recall@k + MRR over the real retrieve() pipeline, gated on a committed baseline that ratchets. Scored at k=5: at k=12 every case recalls 1.00 and the gate could never fail. Carries its own mutation tests, so a broken graph or entry stage reddens it.
• HOP_DECAY swept for real — and the honest result is a null one: everything in [0.35, 0.85] scores identically, and the nominal winner (0.20) leads by 0.04, which is one golden item on a 13-query set, under the eval's own resolution. 0.55 stays — not vindicated, just unfalsified. The gate fails if anything ever beats it by more than one item.
• RRF_K deliberately not swept. It needs a genuinely semantic second ranker; a mock model measures the mock, and real MiniLM weights mean a network download in a lane that must not flake.
• Groundedness screen for meeting notes — contract rule 2 is "never invent", and nothing checked it. screenGroundedness() flags numbers and name-positioned words absent from the transcript and the user's notes. It's a screen, not a proof, and says so.

Scoped AI search silently under-returned. schemaId was filtered after loading a cross-schema BM25 window, so a scoped query came back near-empty whenever that schema's matches ranked below the window. Now pushed into SQL (verified against real FTS5).

A search that fell back to a scan looked identical to an exhaustive one — so the model would state "no such node" with unearned confidence. Responses now carry index, degraded, and a notice.

On-device AI was dead in the packaged desktop app. The Electron CSP denied model-weight hosts, WebAssembly, blob workers, and the 127.0.0.1 loopback the connector ladder actually probes (it allowed localhost only, which doesn't cover it). Both shells are now pinned by a contract test.

The in-app assistant had zero tools — the 28 xnet_* tools reached MCP clients, the CLI and the bridge, but not the chat. AiAgentRuntime now runs a bounded tool loop, and the panel hands it a read-only subset on tiers that call tools reliably. Weak tiers keep the context-pack path, since tools there produce invention rather than grounding. Tool activity is visible while it runs.

Phase 1 is read-only by construction: the allow-list requires both an explicit name and a low risk rating, so neither adding a low-risk tool to the registry nor reclassifying an existing one can widen what the chat reaches.

Fixes a settle bug the loop exposed: model.completed fires once per round trip, so treating it as end-of-turn truncated the reply at the first tool call.

What I did NOT do, on purpose

The web connect-src wildcard stays. The checklist said remove it; the inventory says you can't. https://* was added in c8e2e96ad (0341) so crash-report ingest reaches a user-configured custom hub, and wss://* carries its sync socket. Those origins are typed at runtime — no static meta-tag policy can name them. Removing it drops custom-hub support from the web build. That's a product decision with a migration story, not a cleanup. Documented in place and pinned by test. (Link previews are not affected — they proxy through the hub's SSRF-guarded /unfurl.)

Phase 2 (in-chat approval) and the /v1/agent/stream client are deferred. Phase 2 is the gate that makes writes safe; shipping it half-built is worse than not shipping it — a flow that looks trustworthy but mishandles the medium-risk nonce trains users to click through. The stream client is deferred with it, because its payoff is the permission_request frame Phase 2 renders. Rationale and the seams they extend are in the doc.

Verification

• Full suite: 11133 passed, 1 pre-existing timing flake in data-bridge/binary-state (a package this branch doesn't touch; passes in isolation at 30ms vs a 490ms threshold).
• Typecheck clean across apps/web, plugins, brain, meetings.
• Agent-surface benchmark unchanged: files-cli 2728 / mcp-slim 18260 / mcp-legacy 24346 tokens, 15/15 each — in-app tools are app-side and never touch the MCP surface.
• Retrieval eval baseline: recall@5 all=0.81 keyword=1.00 graph=0.50 mrr=0.69. The graph=0.50 is real headroom, recorded rather than hidden.

🤖 Generated with Claude Code

[github-actions]

🖼️ UI changes in this PR

Screens

✏️ Home _(SSIM 0.981)_

before	after	diff

_{Auto-captured by CI · run. Informational — not a blocking check.}

[github-actions]

Preview removed for PR #629.