feat(audit-log): mirror decision-changing guardrail evaluations into AuditEntry (P2) - #681
Merged
Merged
Conversation
…AuditEntry (P2) Land audit-log PRD Phase 2 as a selective-write: guardrail evaluations that change a call's outcome (route_to_approval / blocked / tripwire) are mirrored into the shared AuditEntry substrate as detail.kind = "guardrail_evaluation", while plain `execute` evaluations stay solely in the dedicated guardrail_evaluations operational table. The mirror is a fire-and-forget write on the audit queue from the single choke point (persistGuardrailEvaluations), so all three dispatch paths — agent tool gate, orchestration tool guardrail, orchestration engine — are covered, and the dry-run preview (which never persists) produces no entries. Entries are platform-originated: null principal columns, action `guardrails:Evaluate`, with the full evaluation record as detail; a route_to_approval entry also carries the filed approval_id. A shared schema fixture (guardrailEvaluationDetail.ts) is asserted on both the write side (camelCase, via the resolver gate test) and the read side (snake_case, via the audit-log REST test) so the guardrails kind and the audit PRD cannot drift. Docs, PRD, and roadmap updated to reflect P2 shipped and the selective-write decision; the product-feed-ownership question is left to approvals Phase 4. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vf1q3tun1okV7hpnXpvB1o
Deploy Outputs
|
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Implements Audit-log PRD Phase 2 — the next item on the delivery roadmap's substrate tier. Decision-changing guardrail evaluations now surface on the shared
AuditEntryaudit substrate, the same substrate the activity feed (approvals P4) and SIEM export build on.The selective-write decision
The PRD's original framing ("land it in
AuditEntryrather than a parallel table") predated theguardrail_evaluationstable actually shipping. That table is now the full operational log — one row per guardrail per call, including plainexecute. Migrating or dual-writing that high-volume firehose into the mutation-focused audit log would flood it and break existing consumers (approval cross-links, dry-run reuse, generation stamping).So instead of migrate/dual-write, this selectively mirrors only the evaluations that changed the call's outcome:
execute(identity)route_to_approvalblockedtripwireThis gives up nothing on any axis: acceptance criteria met, existing consumers untouched,
AuditEntry's mutation-only/low-volume design respected, no data duplication.How
persistGuardrailEvaluations, so all three dispatch paths (agent tool gate, orchestration tool guardrail, orchestration engine) are covered; the dry-run preview (never persists) produces no entries.action: guardrails:Evaluate, guardrail as the resource, full evaluation record asdetail(detail.kind = "guardrail_evaluation"). Aroute_to_approvalentry also carries the filedapproval_id.AuditEntry.detailvs a dedicatedActivityEntrymodel) is intentionally out of scope — it's an approvals-P4 concern and P2 doesn't need it settled.Tests (red/green TDD)
agentToolGuardrailGate.test.ts): class D/C/B (blocked / route_to_approval / tripwire) each write aguardrail_evaluationaudit entry; class A (execute) writes none.auditLog.test.ts): a decision-changing evaluation surfaces snake-cased through the API;executewrites nothing.guardrailEvaluationDetail.ts) is asserted on both sides — camelCase (write) and snake_case (read) — so the guardrails kind and the audit PRD cannot drift (acceptance criterion Feat files #2).Docs & housekeeping
docs/prd-audit-log.md— P2 marked shipped, selective-write decision recorded, stale "blocked on guardrails P3" removed (P3 is shipped).docs/roadmap.md— P2 marked done across backlog, dependency graph, build order, and the activity-feed reconciliation note.packages/website/docs/modules/audit-log.md+guardrails.md— document the newguardrail_evaluationsystem-originated entry.detaildescription mentions the new kind (no SDK/CLI diff — cosmetic).Checks
pnpm typecheck✅ ·pnpm eslint✅ · noas any/as unknownpnpm docs-lint✅🤖 Generated with Claude Code
https://claude.ai/code/session_01Vf1q3tun1okV7hpnXpvB1o
Generated by Claude Code