feat(guardrails): capture applied guardrails (kind+hook) into usage telemetry (#379 A1) - #526
Conversation
…elemetry (#379) Record which guardrails governed each request — the {kind, hook} set — and surface it on the telemetry UsageEvent so operators see *which* guardrails ran, not just the boolean `guardrail_blocked`. This is A1 (DP) of the cross-repo #379-A series; cp-api (A2) and the dashboard (A3) follow. cp-api's /dp/telemetry binds JSON leniently, so the DP ships this additive field ahead of the CP with zero breakage. - aisix-core: `AppliedGuardrail { kind, hook }` + `GuardrailKind::kind_str` / `GuardrailHookPoint::as_str`, re-exported from the crate root. - aisix-guardrails: `GuardrailChain::{new_with_applied, applied}`; capture the set at BOTH build points — `build_chain_from_snapshot` and the per-request `GuardrailIndex::resolve` (IndexEntry now carries each member's descriptor; resolve collects from the deduplicated entries so `applied` mirrors the chain 1:1). Pushed only for materialised rows (Ok(Some)), so inert / disabled / feature-pruned rows never claim to have governed a request. - aisix-obs: `UsageEvent.applied_guardrails` (skip_serializing_if empty). - aisix-proxy: thread the resolved set into telemetry on every emit path of /v1/chat/completions and /v1/messages (non-stream success, guardrail-block, and streaming end-of-stream), captured once after resolve via a small out-param. The /v1/messages coverage also closes the anthropic applied-telemetry gap for this field (#519). Refs #379. Folds in #519 (applied list).
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (9)
📝 WalkthroughWalkthroughThis PR adds telemetry tracking for applied guardrails by introducing the ChangesApplied Guardrail Telemetry Tracking
🎯 3 (Moderate) | ⏱️ ~20 minutes Note 🎁 Summarized by CodeRabbit FreeYour organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above. Comment |
Independent pre-merge audit (CLAUDE.md §8)A fresh general-purpose agent with no shared context reviewed the diff against Security / leakage: clean. Breaking changes: none — additive, LOW findings (no code change required; justified)
Merge gate satisfied (no HIGH/MEDIUM). Proceeding once CI is green. |
What
Record which guardrails governed each request — the
{kind, hook}set — and surface it on the telemetryUsageEventso operators see which guardrails ran, not just the booleanguardrail_blocked.This is A1 (DP, this repo) of the cross-repo applied-guardrails line (
#379epic, tracked in #525):applied_guardrails.dpmgr_usage_eventsJSONB column + telemetry ingest + usage read API.Why DP-first is safe
cp-api's
/dp/telemetrybinds JSON with ginShouldBindJSON(lenient — ignores unknown fields), so the DP can ship this additiveUsageEventfield ahead of the CP with zero breakage. Sequence is A1 → A2 → A3.Design (settled in #525)
{kind, hook}list. Per-guardrail verdicts are V2.kind+hook_pointare in scope — no wide trait change.AppliedGuardrail { kind, hook }lives inaisix-core(aisix-obs already depends on it).Changes
aisix-coreAppliedGuardrail { kind, hook }+GuardrailKind::kind_str/GuardrailHookPoint::as_str; re-exported from crate root.aisix-guardrailsGuardrailChain::{new_with_applied, applied}. Capture at both build points:build_chain_from_snapshotand the per-requestGuardrailIndex::resolve(IndexEntrynow carries each member's descriptor;resolvecollects from the deduplicated entries soappliedmirrors the chain 1:1). Pushed only onOk(Some), so inert / disabled / feature-pruned rows never claim to have governed a request.aisix-obsUsageEvent.applied_guardrails: Vec<AppliedGuardrail>withskip_serializing_if = "Vec::is_empty"(absent == empty set on the wire).aisix-proxy/v1/chat/completionsand/v1/messages(non-stream success, guardrail-block, streaming end-of-stream). Captured once right afterresolve()and returned via a small&mut Vec<_>out-param, so the failure (input-block) path surfaces it too without threading a field through everySuccess/DispatchFailure/DispatchOutcomesite. The/v1/messagescoverage also closes the anthropic applied-telemetry gap for this field (#519).guardrail_index.resolveis used only bychat.rs+messages.rs; the other endpoints (responses/embeddings/audio/images/rerank) don't run guardrails, so emptyappliedthere is correct, not a silent gap.Test plan
cargo fmt— only the 8 edited files touched.cargo test -p aisix-core -p aisix-guardrails -p aisix-obs -p aisix-proxy --all-features— 743 pass.cargo clippy --all-features --all-targets -- -D warnings— clean.cargo check --workspace --all-features— clean (incl. aisix-server telemetry worker consuming the new field).New unit tests:
build_chain_reports_applied_kind_and_hook— build-point 1 captures kind+hook.applied_excludes_inert_and_disabled_rows—applied== what actually governs (Ok(Some) only).resolved_chain_reports_applied_and_mirrors_dedup— per-request path;appliedmirrors the deduplicated chain 1:1.resolved_chain_applied_empty_when_no_attachment_matches— no match → empty set.new_has_empty_applied_and_new_with_applied_reports_it— chain accessor contract.applied_guardrails_serialise_when_set+ omit-when-empty assertion —UsageEventwire shape.End-to-end observation of
applied_guardrailsreaching the dashboard is gated on A2/A3 (the field is additive and the CP currently ignores it).Contract this PR pins
A request whose resolved guardrail chain is non-empty emits a
UsageEventwhoseapplied_guardrailslists the{kind, hook}of each materialised, deduplicated chain member — on success, on guardrail-block, and on the streaming path, for both/v1/chat/completionsand/v1/messages. Empty/absent for guardrail-free requests and requests rejected before guardrail resolution.Refs #379. Closes the applied-list portion of #519.
Summary by CodeRabbit