applied-guardrails telemetry: capture which guardrails governed a request (#379 A)

[moonming]

✅ COMPLETE — all three phases merged

• A1 (DP) ai-gateway#526 (deacb20) · A2 (cp-api) AISIX-Cloud#702 (260ae765) · A3 (dashboard) AISIX-Cloud#704 (e969a7b5)
• Full round-trip verified end-to-end against a live DP (CI e2e-playwright green). Follow-up test-gap tracked in AISIX-Cloud#703.

Goal

Record which guardrails governed each request (kind + hook) and surface it in the dashboard request logs, so operators see which guardrail acted — not just the boolean guardrail_blocked. Cross-repo: A1 (DP, this repo) → A2 (cp-api) → A3 (dashboard), both A2/A3 in api7/AISIX-Cloud.

Settled design (decided with the maintainer)

• Granularity (v1): the attached set per request = {kind, hook} list. Per-guardrail verdicts are V2.
• Ordering: cp-api's /dp/telemetry uses gin ShouldBindJSON (lenient — ignores unknown fields), so the DP can add the new UsageEvent field FIRST with zero CP breakage. Sequence A1 → A2 → A3.
• Capture point: at chain-build time (the builder has each row's kind + hook_point) — no wide trait change.
• Shared type: AppliedGuardrail { kind, hook } in aisix-core (aisix-obs already depends on aisix-core).

A1 (DP — this repo) — ✅ DONE — merged in #526 (squash deacb20)

All seven steps landed and the worktree/branch are cleaned up:

• Re-export AppliedGuardrail from aisix-core (lib.rs + models/mod.rs).
• GuardrailChain: applied field + applied() accessor + new_with_applied (new keeps applied empty).
• Populate at both build points — build_chain_from_snapshot and the per-request GuardrailIndex::resolve (IndexEntry carries each member's descriptor; resolve collects from the deduplicated entries, so applied mirrors the chain 1:1). Push on Ok(Some) only.
• UsageEvent.applied_guardrails (skip_serializing_if = "Vec::is_empty") + wire-shape tests.
• Proxy capture on every emit path of chat.rs + messages.rs (non-stream success, guardrail-block, streaming) — captured once after resolve() via a &mut Vec<_> out-param so the input-block path surfaces it too; closes the anthropic gap in guardrail metrics: cover the /v1/messages (anthropic) path #519 for this field.
• Unit tests (5 new): both build points, dedup-mirror, empty-on-no-match, chain accessor, UsageEvent wire shape.
• cargo fmt/test/clippy clean; independent audit (APPROVE, no HIGH/MEDIUM); CI green incl. e2e; squash-merged; worktree removed.

Audit LOW-1 (no unit coverage of the proxy out-param plumbing — dispatch needs a live AppState) folds into the A2/A3 e2e: once cp-api ingests the field, add an e2e that fires a guardrailed (incl. input-blocked) /v1/chat/completions + /v1/messages and asserts the telemetry row's applied_guardrails.

A2 (cp-api — AISIX-Cloud) — ✅ DONE — merged in AISIX-Cloud#702 (squash 260ae765)

Mirrored the existing routing_attempts JSONB-array column across all 8 layers:

• telemetryEvent.applied_guardrails + parseTelemetryEvent validates JSON-array & copies verbatim (rejects non-array).
• dpmgr_usage_events.applied_guardrails JSONB column — GORM AutoMigrate adds it (this repo uses AutoMigrate, not SQL migration files; additive + nullable). Threaded through dpstore.UsageEvent + InsertBatch/nullableJSONB + usageEventScanRow + toUsageEvent + models.UsageEvent.
• usage read API (/usage_events): usageEventView.applied_guardrails passthrough, omitempty.
• Go unit tests (ingest validate/reject + read-view preserve/omit); build/vet/gofmt clean; CI green incl. e2e-go + e2e-playwright; independent audit APPROVE (no HIGH/MEDIUM code defects).
• Audit MEDIUM (PR body overstated DB round-trip coverage) resolved; pre-existing InsertBatch→ListByEnv round-trip test gap filed as AISIX-Cloud#703.

A3 (dashboard — AISIX-Cloud) — ✅ DONE — merged in AISIX-Cloud#704 (squash e969a7b5)

• /logs row expansion: "Guardrails that ran" panel — a kind · hook badge per applied guardrail, gated on a non-empty set. Threads applied_guardrails through the dashboard's UsageEvent wire→view mapping (mirrors routing_attempts: AppliedGuardrail type + isAppliedGuardrail guard + filter).
• Live-DP e2e closing the round-trip — extended guardrails-live.spec.ts: the keyword-block test expands the blocked row and asserts the panel names the guardrail (the DP records the applied set even on the request it rejects). Verified locally green against cp-api/dpm built from current main (A2) + a post-A1 DP image, and CI e2e-playwright green.
• Values rendered as auto-escaped React text (no dangerouslySetInnerHTML) — A2 audit LOW addressed.
• Independent audit APPROVE (no HIGH/MEDIUM); CI green incl. e2e-go + e2e-playwright.

Related

• Epic: api7/AISIX-Cloud#379 (the applied-guardrails line).
• Folds in guardrail metrics: cover the /v1/messages (anthropic) path #519 (anthropic guardrail telemetry gap — A1 step 5 covers the applied list). Separate: guardrail metrics: a distinct fail-closed counter (error-block vs policy-block) #520 (fail_close counter).

[coderabbitai]

🔗 Related PRs

#112 - feat(guardrail): apply to request by policy [closed]
api7/AISIX-Cloud#461 - feat(logs): show routing target attempts [merged]
api7/AISIX-Cloud#571 - test: Add DeepSeek observability CI coverage [merged]
#506 - feat(guardrails): add Aliyun text-moderation guardrail kind (#603) [merged]
api7/AISIX-Cloud#681 - feat(logs): surface client source IP, client type, and cache tokens in request logs (#492) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[moonming]

✅ A1 (DP) merged — #526 (squash deacb20). Captures {kind, hook} at both build points and emits UsageEvent.applied_guardrails across /v1/chat/completions + /v1/messages (success, guardrail-block, streaming). Additive + omitted-when-empty, so cp-api ingests it whenever A2 lands — no DP/CP ordering hazard. Independent audit: APPROVE, no HIGH/MEDIUM. CI green incl. e2e (vitest). Branch + worktree cleaned up.

Next: A2 (cp-api migration) → A3 (dashboard) in api7/AISIX-Cloud, per the sections above.

[moonming]

✅ A2 (cp-api) merged — AISIX-Cloud#702 (squash 260ae765). cp-api now ingests applied_guardrails on /dp/telemetry, persists it as a JSONB column on dpmgr_usage_events (GORM AutoMigrate), and returns it from /usage_events. Mirrors routing_attempts exactly; additive + nullable + omitempty. Independent audit APPROVE; CI green incl. e2e-go + e2e-playwright. Round-trip test gap filed as AISIX-Cloud#703.

Remaining: A3 (dashboard) — render "guardrails that ran" (kind+hook) in the /logs row expansion + the live-DP e2e closing the round-trip.

[moonming]

🎉 #379-A complete — all three phases merged.

• A1 (DP) — ai-gateway#526 deacb20: DP emits applied_guardrails {kind,hook} on every chat/messages emit path.
• A2 (cp-api) — AISIX-Cloud#702 260ae765: ingest on /dp/telemetry → JSONB column (AutoMigrate) → /usage_events read API.
• A3 (dashboard) — AISIX-Cloud#704 e969a7b5: /logs row expansion renders 'Guardrails that ran' (kind · hook).

Each phase: independent audit APPROVE (no HIGH/MEDIUM), CI green. The full round-trip is verified end-to-end against a live DP — CI e2e-playwright green on A3, and locally: keyword guardrail blocks → DP emits the applied set (even on the rejected request) → cp-api serves it → dashboard renders it.

Follow-up (non-blocking): AISIX-Cloud#703 — add a real-PG InsertBatch→ListByEnv round-trip test for the usage-events JSONB columns. Closing this tracker.