Add generic tool policy middleware#2137
Conversation
📝 WalkthroughWalkthroughThis PR introduces a pre-execution ChangesTool Policy Middleware
sequenceDiagram
participant Caller
participant Agent
participant ToolPolicy
participant ToolImpl
Caller->>Agent: execute_tool_call(tool_name, args)
Agent->>ToolPolicy: check(ToolPolicyRequest)
alt Deny
ToolPolicy-->>Agent: Deny{reason}
Agent-->>Caller: denial record (success=false, message)
else Allow
ToolPolicy-->>Agent: Allow
Agent->>ToolImpl: execute_with_options(args)
ToolImpl-->>Agent: execution result
Agent-->>Caller: result record
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related issues
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
d55d467 to
3294b3e
Compare
|
Rebased this draft onto current main and resolved the single test import conflict. Latest head is 3294b3e. Local validation passed: |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/openhuman/agent/harness/session/turn.rs (1)
988-993: ⚡ Quick winUse
debug/tracefor policy-denial diagnostics to avoid warn-noise.This denial branch is expected control flow for strict policies;
warnhere can create noisy operational signals.♻️ Proposed change
- tracing::warn!( + tracing::debug!( tool = call.name.as_str(), policy = self.tool_policy.name(), reason = %reason, "[agent_loop] tool denied by policy" );As per coding guidelines,
src/**/*.rs: Uselogortracingcrate atdebugortracelevel for Rust diagnostic logs.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/agent/harness/session/turn.rs` around lines 988 - 993, The current policy-denial log in turn.rs uses tracing::warn which is noisy for expected control flow; change the call site that references call.name.as_str(), self.tool_policy.name(), %reason and the "[agent_loop] tool denied by policy" message to use tracing::debug or tracing::trace instead (pick debug for general diagnostics, trace if very verbose) and keep the same structured fields and message so policy-denial remains visible at a lower log level.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/openhuman/agent/harness/session/turn.rs`:
- Around line 988-993: The current policy-denial log in turn.rs uses
tracing::warn which is noisy for expected control flow; change the call site
that references call.name.as_str(), self.tool_policy.name(), %reason and the
"[agent_loop] tool denied by policy" message to use tracing::debug or
tracing::trace instead (pick debug for general diagnostics, trace if very
verbose) and keep the same structured fields and message so policy-denial
remains visible at a lower log level.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 27bcea0f-2301-4c58-aa38-4bce332e863f
📒 Files selected for processing (6)
src/openhuman/agent/harness/session/builder.rssrc/openhuman/agent/harness/session/turn.rssrc/openhuman/agent/harness/session/turn_tests.rssrc/openhuman/agent/harness/session/types.rssrc/openhuman/agent/mod.rssrc/openhuman/agent/tool_policy.rs
# Conflicts: # src/openhuman/agent/harness/session/builder.rs # src/openhuman/agent/harness/session/types.rs
…rabbitai on turn.rs) Policy denial is expected control flow, not a warning condition. Using warn! creates operational noise for callers that install strict policies to gate tool calls by design. Switch to debug! so the structured diagnostics remain visible at a lower log level without polluting the warn stream.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/openhuman/agent/harness/session/turn.rs (1)
1058-1063: ⚡ Quick winAdd correlation fields to the denial log.
This is exactly the sort of branch that's hard to reconstruct from tests, but the new event only logs
tool,policy, andreason. Please includecall_idplus session/agent identifiers so denials can be matched back to the emitted tool lifecycle events.Suggested change
tracing::debug!( + call_id = call_id.as_str(), tool = call.name.as_str(), + session_id = %self.event_session_id(), + agent_definition_id = %self.agent_definition_id, policy = self.tool_policy.name(), reason = %reason, "[agent_loop] tool denied by policy" );As per coding guidelines, "Use log / tracing at debug or trace level on RPC entry and exit, error paths, state transitions, and any branch that is hard to infer from tests alone. Use structured, grep-friendly context with stable prefixes ... and include correlation fields such as request IDs, method names, and entity IDs when available."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/agent/harness/session/turn.rs` around lines 1058 - 1063, The denial log in the tool-policy branch (the tracing::debug! call that currently logs tool = call.name.as_str(), policy = self.tool_policy.name(), reason = %reason) needs correlation fields added so denials can be matched to lifecycle events; update that tracing::debug! invocation to include the tool call's identifier (call_id or call.id), plus session and agent identifiers available on the current context (e.g., self.session.id, self.agent.id or similar fields) so the log emits tool, policy, reason, call_id, session_id and agent_id together as structured fields.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/openhuman/agent/harness/session/turn.rs`:
- Around line 1055-1069: The branch handling ToolPolicyDecision::Deny currently
injects the raw policy-provided reason into the tool result; instead, keep the
detailed reason only in logs/telemetry (leave the tracing::debug call with
reason intact) and return a generic denial string for the tool output generated
in the code that formats ("Tool '{}' denied by policy '{}'...") so that
self.tool_policy.check(&policy_request).await's reason is not persisted or
exposed to the model; update the formatted message produced for call.name and
self.tool_policy.name to a non-sensitive, generic message (e.g., "Tool denied by
policy") while retaining the existing debug logging that references reason and
policy name.
---
Nitpick comments:
In `@src/openhuman/agent/harness/session/turn.rs`:
- Around line 1058-1063: The denial log in the tool-policy branch (the
tracing::debug! call that currently logs tool = call.name.as_str(), policy =
self.tool_policy.name(), reason = %reason) needs correlation fields added so
denials can be matched to lifecycle events; update that tracing::debug!
invocation to include the tool call's identifier (call_id or call.id), plus
session and agent identifiers available on the current context (e.g.,
self.session.id, self.agent.id or similar fields) so the log emits tool, policy,
reason, call_id, session_id and agent_id together as structured fields.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 894706ea-f2fd-490f-b472-635af41ee428
📒 Files selected for processing (4)
src/openhuman/agent/harness/session/builder.rssrc/openhuman/agent/harness/session/turn.rssrc/openhuman/agent/harness/session/turn_tests.rssrc/openhuman/agent/harness/session/types.rs
🚧 Files skipped from review as they are similar to previous changes (3)
- src/openhuman/agent/harness/session/turn_tests.rs
- src/openhuman/agent/harness/session/builder.rs
- src/openhuman/agent/harness/session/types.rs
| if let ToolPolicyDecision::Deny { reason } = | ||
| self.tool_policy.check(&policy_request).await | ||
| { | ||
| tracing::debug!( | ||
| tool = call.name.as_str(), | ||
| policy = self.tool_policy.name(), | ||
| reason = %reason, | ||
| "[agent_loop] tool denied by policy" | ||
| ); | ||
| ( | ||
| format!( | ||
| "Tool '{}' denied by policy '{}': {reason}", | ||
| call.name, | ||
| self.tool_policy.name() | ||
| ), |
There was a problem hiding this comment.
Keep raw policy reasons out of the tool result.
reason is arbitrary policy-provided text, but this branch turns it into tool output that gets persisted and fed back to the model on the next iteration. That makes internal enforcement details or user/session-specific context prompt-visible. Return a generic denial message here and keep the detailed reason only in logs/telemetry.
Suggested change
if let ToolPolicyDecision::Deny { reason } =
self.tool_policy.check(&policy_request).await
{
+ let policy_name = self.tool_policy.name().to_string();
tracing::debug!(
tool = call.name.as_str(),
- policy = self.tool_policy.name(),
+ policy = %policy_name,
reason = %reason,
"[agent_loop] tool denied by policy"
);
(
- format!(
- "Tool '{}' denied by policy '{}': {reason}",
- call.name,
- self.tool_policy.name()
- ),
+ format!("Tool '{}' denied by policy '{}'", call.name, policy_name),
false,
)
} else {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/openhuman/agent/harness/session/turn.rs` around lines 1055 - 1069, The
branch handling ToolPolicyDecision::Deny currently injects the raw
policy-provided reason into the tool result; instead, keep the detailed reason
only in logs/telemetry (leave the tracing::debug call with reason intact) and
return a generic denial string for the tool output generated in the code that
formats ("Tool '{}' denied by policy '{}'...") so that
self.tool_policy.check(&policy_request).await's reason is not persisted or
exposed to the model; update the formatted message produced for call.name and
self.tool_policy.name to a non-sensitive, generic message (e.g., "Tool denied by
policy") while retaining the existing debug logging that references reason and
policy name.
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
Summary
ToolPolicymiddleware trait for pre-execution agent tool checks.Tool::execute_with_options.Validation
cargo fmt --manifest-path Cargo.tomlcargo test --manifest-path Cargo.toml policyRefs #2131
Summary by CodeRabbit
New Features
Tests