Skip to content

Add ApplyGuardrailScope for security guardrail evaluations#173

Merged
nikhilNava merged 3 commits into
mainfrom
feature/apply-guardrail-scope
May 28, 2026
Merged

Add ApplyGuardrailScope for security guardrail evaluations#173
nikhilNava merged 3 commits into
mainfrom
feature/apply-guardrail-scope

Conversation

@nikhilNava
Copy link
Copy Markdown
Collaborator

Summary

Implements the guardrail span (\ApplyGuardrailScope) to make security guardian evaluations observable as OTel spans. This is a Python port of the .NET implementation from microsoft/opentelemetry-distro-dotnet#109.

What's included

New scope: \ApplyGuardrailScope\

  • Factory method \start(details, agent_details, request?, user_details?, span_details?)\

  • ecord_decision(decision_type, reason?)\ — update decision mid-flight

  • ecord_content_output(output_value)\ — opt-in sanitized content

  • ecord_content_input(input_value)\ — opt-in input content

  • ecord_finding(finding)\ — add \microsoft.security.finding\ span events
  • Span kind defaults to \INTERNAL\
  • Span name: \�pply_guardrail {guardian_name} {target_type}\

Data contracts

  • \GuardrailDetails\ — input contract with target type, decision, guardian info, policy info
  • \GuardrailFinding\ — per-finding event (risk_category, severity, score, metadata)
  • \GuardrailDecisionType\ — allow/audit/deny/modify/warn
  • \GuardrailRiskSeverity\ — none/low/medium/high/critical
  • \GuardrailTargetType\ — llm_input/llm_output/tool_call/tool_definition/memory_store/etc.

Exporter integration

  • Added \�pply_guardrail\ to \GEN_AI_OPERATION_NAMES\ allowlist so guardrail spans survive export filtering

Tests

  • 23 unit tests covering span creation, attributes, findings, exporter eligibility, and disabled state

Notes

  • ETW is not applicable for Python so that path was skipped (as expected).
  • Content input/output values are explicit opt-in methods (not set automatically from request).

Implement the guardrail span (ApplyGuardrailScope) to make security guardian
evaluations observable as OTel spans. This is a port of the .NET implementation
from microsoft/opentelemetry-distro-dotnet#109.

New files:
- apply_guardrail_scope.py: Main scope with start(), record_decision(),
  record_content_output(), record_content_input(), record_finding()
- guardrail_details.py: Input contract dataclass
- guardrail_finding.py: Per-finding event dataclass
- guardrail_decision_type.py: Constants (allow/audit/deny/modify/warn)
- guardrail_risk_severity.py: Constants (none/low/medium/high/critical)
- guardrail_target_type.py: Constants (llm_input/llm_output/tool_call/etc.)
- test_apply_guardrail_scope.py: 23 unit tests

Key integration:
- Added 'apply_guardrail' to GEN_AI_OPERATION_NAMES exporter allowlist
- Added microsoft.guardian.* and microsoft.security.* attribute constants
- Exported all new classes from a365.core.__init__

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new ApplyGuardrailScope to record OpenTelemetry spans for security guardrail (guardian) evaluations, ported from the .NET distro. Introduces supporting data classes/constants (GuardrailDetails, GuardrailFinding, GuardrailDecisionType, GuardrailRiskSeverity, GuardrailTargetType), new semantic-convention attribute keys under microsoft.security.* / microsoft.guardian.*, and registers apply_guardrail in the exporter's allowed operation names. Spans default to INTERNAL kind and are named apply_guardrail {guardian_name} {target_type}. Comprehensive unit tests cover attribute mapping, findings as span events, exporter eligibility, and disabled-state no-op behavior.

Changes:

  • New ApplyGuardrailScope with record_decision, record_content_input/output, and record_finding (emits microsoft.security.finding events).
  • New guardrail data classes/enums and ~22 new constants added to both a365/constants.py and a365/core/constants.py; new operation name added to exporter allowlist.
  • 23 unit tests in tests/a365/test_apply_guardrail_scope.py; public API exposed via core/__init__.py.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/microsoft/opentelemetry/a365/core/apply_guardrail_scope.py New scope class mirroring ExecuteToolScope pattern with INTERNAL span kind and guardrail-specific attribute/event recording.
src/microsoft/opentelemetry/a365/core/guardrail_details.py Dataclass holding required + optional guardrail evaluation inputs.
src/microsoft/opentelemetry/a365/core/guardrail_finding.py Dataclass for per-finding event payload (risk, severity, score, metadata, policy info).
src/microsoft/opentelemetry/a365/core/guardrail_decision_type.py String constants for decision types (allow/audit/deny/modify/warn).
src/microsoft/opentelemetry/a365/core/guardrail_risk_severity.py String constants for severity levels.
src/microsoft/opentelemetry/a365/core/guardrail_target_type.py String constants for target types.
src/microsoft/opentelemetry/a365/core/constants.py Adds APPLY_GUARDRAIL_OPERATION_NAME and microsoft.security.*/microsoft.guardian.* attribute keys.
src/microsoft/opentelemetry/a365/constants.py Mirrors the same new constants in the top-level constants module.
src/microsoft/opentelemetry/a365/core/exporters/utils.py Adds apply_guardrail to GEN_AI_OPERATION_NAMES so spans pass export filtering.
src/microsoft/opentelemetry/a365/core/init.py Re-exports ApplyGuardrailScope and guardrail data classes/enums.
tests/a365/test_apply_guardrail_scope.py 23 unit tests covering naming, kind, attributes, findings, exporter eligibility, and disabled state.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 28, 2026

Performance comparison

Threshold: regressions >15.0% on gating scenarios fail the build. Higher ops/s is better; positive Δ means the PR is slower.

Scenario Gating Baseline (ops/s) Candidate (ops/s) Δ % Status
azure_monitor_log yes 26,716.5 25,685.2 +4.02%
azure_monitor_span yes 154,990.7 154,297.2 +0.45%
otel_log no 32,650.9 31,726.9 +2.91%
otel_span no 34,089.0 35,058.2 -2.76%

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Member

@rads-1996 rads-1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

request.content can be an InputMessages object which the OTel SDK cannot
accept as an attribute value (only primitives/sequences). Now
record_content_input accepts both str and InputMessagesParam, normalizing
and serializing structured messages to JSON before setting the attribute,
matching InferenceScope's approach for input messages.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilNava nikhilNava enabled auto-merge (squash) May 28, 2026 20:07
@nikhilNava nikhilNava merged commit b7b34c0 into main May 28, 2026
10 checks passed
@nikhilNava nikhilNava deleted the feature/apply-guardrail-scope branch May 28, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants