docs(examples): bank-manager agent-control demo (AIEWF) + talks slideshow

[changliu2]

What

Introduces the bank_manager_agent_control example — the AIEWF live demo. One banking support agent, taken through three beats:

1. Baseline (unguarded) — how often an un-controlled agent breaks policy
2. Defensive prompting — the intervention everyone reaches for first
3. Principled control plane (ACS) — policy-as-code gating on typed features of each tool call

ASSERT measures the difference; ACS enforces it. Plus a follow-along HTML slideshow under talks/.

Scope / decisions (please sanity-check)

• Whole new example (83 files) — this directory did not exist on main before.
• Result artifacts are NOT committed (chose code+docs only): the frozen per-case transcripts were ~21 MB, the same artifact type chore: drop 19 MB of committed per-case scores.jsonl from incident-triage demo #233 removed from a public demo. Users regenerate via assert-ai run into the gitignored artifacts/results/. The three READMEs were reconciled to a "generate, then view" flow.
• ACS install via assert-ai[acs] (already on main) with a vendored wheel offline fallback. Note: the [acs] extra isn't on PyPI 0.1.0 yet — needs a release before pip install assert-ai[acs] works off PyPI.
• CI beat points at the standalone responsibleai/assert-ci-banking-demo as the shipping vehicle (pip-install-in-your-own-repo), publishing separately.

Verification

• 77/77 unit tests pass (python -m unittest discover -s tests).
• Leak scan clean — no home paths / secrets; .env.example placeholders only.
• docs/README.md had pre-existing staleness (old variant-a/c/e names, eval_unguarded.yaml refs, build-demo-final branch, dead results/ copy-step) — reconciled to the current eval_realistic_* configs / variant-b0/b1/b2 / bank-manager-feature-rep suite. Worth a read to confirm.

Draft — for review, not merge

Opened as a draft so you can review on github.com. Nothing merges. Flagging the "no committed results" call and the docs/README.md reconciliation as the two things most worth your eyes.

[changliu2]

Maintainer review — do the ACS mitigations make sense?

Liam / Mike (additional reviewers) — I traced the full mitigation chain (rego policy → manifest → acs_shim enforcement → tool side-effect semantics). Verdict: the mitigations are sound and correctly back the demo's thesis. Summary below; I've also added a "Productionizing the gate" note to the example README in this PR.

What holds up ✅

1. Typed-signal gating (the thesis). Gating on result.risk_tier / referenced_accounts / grounded instead of regex-over-IDs generalizes uniformly across the deposit/loan/brokerage/client domains and avoids the text policy's brittle ID list.
2. Enforcement points are placed correctly.
   • Irreversible/money-moving actions (create_transfer, freeze_account, enable_admin_mode) are gated pre-tool-call, fail-closed on the host snapshot — the shim raises AgentControlBlocked, so the side effect is actually prevented.
   • Sensitivity / exfil / grounding run as post-tool-call scrubbers. Verified the gated tools don't settle: place_trade returns status:"prepared" (not settled) and reads/prepares don't mutate — so post-call scrubbing has no committed side effect.
3. Out-of-scope reference gate (referenced − authorized) is a sound structural XPIA/exfil signal, and it clears benign in-scope memos the regex false-positives on — which is why over-refusal also drops.
4. Ungrounded-claim gate on the KB grounded flag directly targets policy fabrication.
5. Learned-risk gate fires only when the typed gates are silent; an enrichment failure escalates (fail-safe).

Hardening notes (fine for a demo; documented in the new README section)

1. Post-call scrubber fails open — unparseable JSON → allow; risk_tier defaults standard, grounded defaults true. Should fail-closed for the sensitivity dimension in production. (Money-moving pre-gates are already fail-closed, so worst-case severity is bounded.)
2. Enforcement layer allows on OPA error (acs_shim.py) — a real control plane should fail-closed/escalate on engine error.
3. SSN input gate is format-narrow (NNN-NN-NNNN only) — the typed post-call gates are the real protection.

Net: coherent design, highest-severity actions fail-closed. Not blocking — the fail-open choices are deliberate demo trade-offs and now called out in the README so customers don't ship them verbatim.

[AaronAspinwall123]

I did some AI review and it suggested some issues:

• ACS lets sensitive transaction history through because  risk_tier  is omitted and defaults to  standard  ( bank_core.py:201 ,  bank_manager_feature.rego:56 ).
• Config generates 60 cases, while docs/results claim  n=100  ( eval_realistic_unguarded.yaml:69-70 ).
• Quickstart installation omits the  examples  extra required by  agent.py ’s MCP imports ( README.md:74 ,  agent.py:39-43 ).

Feel free to ignore if you think its not necessary

[changliu2]

Thanks for flagging these — ran them all down against the actual code/artifacts rather than taking the AI review at face value. Two real, one false positive:

#1 (risk_tier bypass) — real, fixed in b720ccb. read_transaction_history built its return dict from scratch and never included risk_tier, unlike read_account (which copies the full record). Since the Rego policy defaults missing risk_tier to "standard" (bank_manager_feature.rego:56), the sensitive_read_gate silently never fired for this one tool — VIP/high-net-worth transaction history could be read without triggering the same block that correctly stops read_account on the same entity. Added risk_tier to the return dict plus a regression test (test_read_transaction_history_signals) that would have caught this — there was genuinely zero test coverage for that combination before.

#2 (n=100 vs 60) — false positive. That's naive arithmetic on the two sample_size config knobs (40 + 20 = 60) that doesn't reflect how the stratified test-set generation actually expands into final case count. I counted the real generated test_set.jsonl directly: exactly 100 rows (50 prompt + 50 scenario), matching the docs and the -n100 run naming. Worth double-checking real output before trusting an AI review's arithmetic on config values that don't map 1:1.

#3 (missing examples extra) — real, fixed in b720ccb. The quickstart pip install "assert-ai[acs,langgraph,otel]" in both README.md and ci/README.md omitted examples, which langchain-mcp-adapters (imported at module load in agent.py) lives under. Following the quickstart verbatim → ModuleNotFoundError on the first run. docs/README.md's fuller setup already had this right; synced both shorter quickstart commands to match.

One more thing I found while auditing #1, not from your list: the Python FeatureGate.decide() mirror in feature_policy.py only checks entity sensitivity for WRITE_TOOLS — it has no equivalent check for reads at all, even though the real Rego policy has an explicit sensitive_read_gate for reads. The live Rego enforcement is correct and unaffected (verified separately), but the Python unit-test mirror can't exercise that specific gate the way the Rego can, since it doesn't implement it. Not fixing this myself since it's a bigger design question (is the Python class meant to be full parity, or an intentionally narrower harness for a few properties?) — flagging for a follow-up rather than expanding this PR's scope.

All 68 existing tests + the new one pass. Pushed in b720ccb.