Skip to content

feat(guardrails): capture applied guardrails (kind+hook) into usage telemetry (#379 A1) - #526

Merged
moonming merged 1 commit into
mainfrom
feat/guardrail-applied-capture
Jun 5, 2026
Merged

feat(guardrails): capture applied guardrails (kind+hook) into usage telemetry (#379 A1)#526
moonming merged 1 commit into
mainfrom
feat/guardrail-applied-capture

Conversation

@moonming

@moonming moonming commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

What

Record which guardrails governed each request — the {kind, hook} set — and surface it on the telemetry UsageEvent so operators see which guardrails ran, not just the boolean guardrail_blocked.

This is A1 (DP, this repo) of the cross-repo applied-guardrails line (#379 epic, tracked in #525):

  • A1 (this PR): DP captures + emits applied_guardrails.
  • A2 (cp-api): dpmgr_usage_events JSONB column + telemetry ingest + usage read API.
  • A3 (dashboard): render "guardrails that ran" in the logs row expansion.

Why DP-first is safe

cp-api's /dp/telemetry binds JSON with gin ShouldBindJSON (lenient — ignores unknown fields), so the DP can ship this additive UsageEvent field ahead of the CP with zero breakage. Sequence is A1 → A2 → A3.

Design (settled in #525)

  • Granularity (v1): the attached set per request = {kind, hook} list. Per-guardrail verdicts are V2.
  • Capture point: at chain-build time, where each row's kind + hook_point are in scope — no wide trait change.
  • Shared type: AppliedGuardrail { kind, hook } lives in aisix-core (aisix-obs already depends on it).

Changes

Crate Change
aisix-core AppliedGuardrail { kind, hook } + GuardrailKind::kind_str / GuardrailHookPoint::as_str; re-exported from crate root.
aisix-guardrails GuardrailChain::{new_with_applied, applied}. Capture at both build points: build_chain_from_snapshot and the per-request GuardrailIndex::resolve (IndexEntry now carries each member's descriptor; resolve collects from the deduplicated entries so applied mirrors the chain 1:1). Pushed only on Ok(Some), so inert / disabled / feature-pruned rows never claim to have governed a request.
aisix-obs UsageEvent.applied_guardrails: Vec<AppliedGuardrail> with skip_serializing_if = "Vec::is_empty" (absent == empty set on the wire).
aisix-proxy Thread the resolved set into telemetry on every emit path of /v1/chat/completions and /v1/messages (non-stream success, guardrail-block, streaming end-of-stream). Captured once right after resolve() and returned via a small &mut Vec<_> out-param, so the failure (input-block) path surfaces it too without threading a field through every Success/DispatchFailure/DispatchOutcome site. The /v1/messages coverage also closes the anthropic applied-telemetry gap for this field (#519).

guardrail_index.resolve is used only by chat.rs + messages.rs; the other endpoints (responses/embeddings/audio/images/rerank) don't run guardrails, so empty applied there is correct, not a silent gap.

Test plan

  • cargo fmt — only the 8 edited files touched.
  • cargo test -p aisix-core -p aisix-guardrails -p aisix-obs -p aisix-proxy --all-features — 743 pass.
  • cargo clippy --all-features --all-targets -- -D warnings — clean.
  • cargo check --workspace --all-features — clean (incl. aisix-server telemetry worker consuming the new field).

New unit tests:

  • build_chain_reports_applied_kind_and_hook — build-point 1 captures kind+hook.
  • applied_excludes_inert_and_disabled_rowsapplied == what actually governs (Ok(Some) only).
  • resolved_chain_reports_applied_and_mirrors_dedup — per-request path; applied mirrors the deduplicated chain 1:1.
  • resolved_chain_applied_empty_when_no_attachment_matches — no match → empty set.
  • new_has_empty_applied_and_new_with_applied_reports_it — chain accessor contract.
  • applied_guardrails_serialise_when_set + omit-when-empty assertion — UsageEvent wire shape.

End-to-end observation of applied_guardrails reaching the dashboard is gated on A2/A3 (the field is additive and the CP currently ignores it).

Contract this PR pins

A request whose resolved guardrail chain is non-empty emits a UsageEvent whose applied_guardrails lists the {kind, hook} of each materialised, deduplicated chain member — on success, on guardrail-block, and on the streaming path, for both /v1/chat/completions and /v1/messages. Empty/absent for guardrail-free requests and requests rejected before guardrail resolution.

Refs #379. Closes the applied-list portion of #519.

Summary by CodeRabbit

  • New Features
    • Enhanced observability for guardrails: Usage events now track which guardrails were applied to requests, including their type and execution point (input, output, or both).
    • Improved telemetry capture across chat and message endpoints to provide better visibility into guardrail application.

…elemetry (#379)

Record which guardrails governed each request — the {kind, hook} set — and
surface it on the telemetry UsageEvent so operators see *which* guardrails
ran, not just the boolean `guardrail_blocked`. This is A1 (DP) of the
cross-repo #379-A series; cp-api (A2) and the dashboard (A3) follow.
cp-api's /dp/telemetry binds JSON leniently, so the DP ships this additive
field ahead of the CP with zero breakage.

- aisix-core: `AppliedGuardrail { kind, hook }` + `GuardrailKind::kind_str`
  / `GuardrailHookPoint::as_str`, re-exported from the crate root.
- aisix-guardrails: `GuardrailChain::{new_with_applied, applied}`; capture
  the set at BOTH build points — `build_chain_from_snapshot` and the
  per-request `GuardrailIndex::resolve` (IndexEntry now carries each
  member's descriptor; resolve collects from the deduplicated entries so
  `applied` mirrors the chain 1:1). Pushed only for materialised rows
  (Ok(Some)), so inert / disabled / feature-pruned rows never claim to
  have governed a request.
- aisix-obs: `UsageEvent.applied_guardrails` (skip_serializing_if empty).
- aisix-proxy: thread the resolved set into telemetry on every emit path
  of /v1/chat/completions and /v1/messages (non-stream success,
  guardrail-block, and streaming end-of-stream), captured once after
  resolve via a small out-param. The /v1/messages coverage also closes
  the anthropic applied-telemetry gap for this field (#519).

Refs #379. Folds in #519 (applied list).
@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown

Review Change Stack

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: 50033944-ac99-4270-9351-967634efa384

📥 Commits

Reviewing files that changed from the base of the PR and between 6fcc246 and acef39a.

📒 Files selected for processing (9)
  • crates/aisix-core/src/lib.rs
  • crates/aisix-core/src/models/guardrail.rs
  • crates/aisix-core/src/models/mod.rs
  • crates/aisix-guardrails/src/build.rs
  • crates/aisix-guardrails/src/chain.rs
  • crates/aisix-guardrails/src/index.rs
  • crates/aisix-obs/src/usage.rs
  • crates/aisix-proxy/src/chat.rs
  • crates/aisix-proxy/src/messages.rs

📝 Walkthrough

Walkthrough

This PR adds telemetry tracking for applied guardrails by introducing the AppliedGuardrail type, storing applied metadata throughout the guardrail resolution pipeline, and propagating {kind, hook} pairs into usage events for observability across chat and messages endpoints.

Changes

Applied Guardrail Telemetry Tracking

Layer / File(s) Summary
Core telemetry types and helper methods
crates/aisix-core/src/models/guardrail.rs, crates/aisix-core/src/models/mod.rs, crates/aisix-core/src/lib.rs
AppliedGuardrail struct and helper methods GuardrailKind::kind_str() and GuardrailHookPoint::as_str() provide canonical string discriminators for wire format and telemetry.
GuardrailChain storage and accessors
crates/aisix-guardrails/src/chain.rs
GuardrailChain stores applied guardrails via new applied field; new_with_applied constructor and applied() accessor enable telemetry propagation; unit tests validate both initialization paths.
Index-level applied guardrail tracking
crates/aisix-guardrails/src/index.rs
IndexEntry stores AppliedGuardrail metadata; GuardrailIndex::resolve accumulates and passes applied descriptors in lockstep with deduplicated chain; push_entry and test helpers updated accordingly.
Build-time guardrail telemetry capture
crates/aisix-guardrails/src/build.rs
build_chain_from_snapshot and build_index_from_snapshot populate AppliedGuardrail metadata for materialized guardrails; comprehensive tests validate correct reporting, exclusion of inert/disabled rows, and deduplication consistency.
UsageEvent observability field
crates/aisix-obs/src/usage.rs
UsageEvent adds applied_guardrails field with conditional serialization (omitted when empty); serialization tests verify presence when populated and absence when empty.
Chat endpoint applied guardrail propagation
crates/aisix-proxy/src/chat.rs
/v1/chat/completions captures resolved metadata in dispatch via out-parameter; metadata flows through success and error paths and into both streaming and non-streaming usage telemetry via updated UsageExtras.
Messages endpoint applied guardrail propagation
crates/aisix-proxy/src/messages.rs
/v1/messages and Anthropic passthrough capture resolved metadata in dispatch; emit_anthropic_usage_event receives and includes applied guardrails; streaming paths capture telemetry at stream completion for consistent reporting.

🎯 3 (Moderate) | ⏱️ ~20 minutes


Note

🎁 Summarized by CodeRabbit Free

Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.

Comment @coderabbitai help to get the list of available commands and usage tips.

@moonming

moonming commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator Author

Independent pre-merge audit (CLAUDE.md §8)

A fresh general-purpose agent with no shared context reviewed the diff against origin/main (6fcc246). Result: APPROVE — no HIGH, no MEDIUM findings. It independently re-verified all five contract claims (additive AppliedGuardrail; capture at both build points only on Ok(Some); resolve mirrors the deduplicated chain 1:1; skip_serializing_if empty; threaded into every emit path of both endpoints, with the out-param set before every early return Err incl. the input-block on both endpoints).

Security / leakage: clean. applied_guardrails flows only into UsageEvent (one-way telemetry sink → /dp/telemetry); it never reaches a client response. Only the kind discriminator + hook are captured — never config/secret values (the api_key/access_key-bearing GuardrailKind variants are matched by discriminator only).

Breaking changes: none — additive, Default-able, omitted-when-empty; GuardrailChain::new preserved; push_entry is pub(crate) with both callers updated; the dispatch out-param is on a private fn.

LOW findings (no code change required; justified)

  • LOW-1 — proxy emit-path plumbing has no unit coverage. dispatch needs a live AppState (snapshot/budget I/O), so the out-param threading is only observable via e2e. End-to-end assertion that applied_guardrails reaches cp-api is correctly gated on A2 (cp-api ingest) + A3 — folded into that work in applied-guardrails telemetry: capture which guardrails governed a request (#379 A) #525. Core logic is well-covered (5 new tests with full assert_eq! on the {kind,hook} vec + dedup content, not .len()).
  • LOW-2 — hook derives from hook_point, not direction. Deliberate and correct: direction-based filtering in resolve is "not yet implemented" per the field's own docs, and hook_point is the operative per-hook control for v1.

Merge gate satisfied (no HIGH/MEDIUM). Proceeding once CI is green.

@moonming
moonming merged commit deacb20 into main Jun 5, 2026
8 checks passed
@moonming
moonming deleted the feat/guardrail-applied-capture branch June 5, 2026 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant