Deterministic authorization + learned sequence evidence + risk-vs-friction evaluation for agent tool use.
Core question: Does the safeguard reduce risky agent behavior enough to justify the friction it introduces?
AI-agent security changes once a model can act. Individually reasonable calls can combine into a dangerous trajectory: read sensitive data, follow untrusted context, invoke another tool, then send data externally.
AgentShield places an independent runtime security layer between an AI agent and downstream MCP-style tools, APIs, SaaS services, databases, or infrastructure actions.
It separates three concerns:
Deterministic runtime policy
+
Learned trajectory evidence
+
Security-control measurement
↓
Auditable decision + behavioral context + risk/friction tradeoff
The deterministic policy remains authoritative. The learned model can explain that a sequence is unusual, but it cannot silently grant privilege or override a hard security rule.
The policy engine evaluates declared user intent, tool identity, read/write/destructive behavior, external destinations, sensitive labels, untrusted context, delegation state, and recent same-agent call history.
It returns one of four explicit actions:
| Decision | Meaning |
|---|---|
ALLOW |
action remains within intent and policy |
ALLOW_WITH_REDACTION |
action may proceed after sensitive fields are removed |
REQUIRE_APPROVAL |
execution pauses for an authorized human |
BLOCK |
hard policy or dangerous trajectory prevents execution |
| Measure | Baseline |
|---|---|
| Expected policy decisions matched | 6 / 6 |
| High-impact cases controlled | 2 / 2 |
| Benign tasks preserved | 4 / 4 |
| False blocks on benign cases | 0 / 4 |
| Blocked | 1 |
| Human approval required | 1 |
| Allowed with redaction | 1 |
| Allowed | 3 |
These are synthetic regression results, not production efficacy claims.
Blocking more actions is not automatically better security. A useful safeguard should reduce risky outcomes without unnecessarily breaking legitimate agent work.
AgentShield therefore measures the control as a security data-science system, not only as a policy engine.
Representative metrics include:
- prevented-risk rate — risky synthetic outcomes intercepted;
- benign false-positive rate — legitimate tasks unnecessarily blocked or escalated;
- legitimate task completion — whether safe work still succeeds;
- approval rate and approval latency — operational friction from human checkpoints;
- detection latency — how quickly risky behavior is surfaced;
- recovery success — whether interrupted workflows recover safely;
- net security utility — transparent risk reduction minus illustrative friction costs.
The repository includes a deterministic strict-vs-adaptive safeguard experiment plus a dependency-free bootstrap interval for the candidate-minus-baseline utility difference.
Strict control Adaptive control
│ │
├─ risk prevented ├─ risk prevented
├─ false positives ├─ false positives
├─ task completion ├─ task completion
└─ approval latency └─ approval latency
\ /
\ /
→ security utility ←
The point is not to claim a universal utility formula. The point is to make the tradeoff explicit, inspectable, and sensitivity-testable.
See docs/security-measurement.md for the measurement contract and production caveats.
AgentShield includes a Laplace-smoothed Markov trajectory model trained from synthetic normal-reference sequences.
Tool calls are converted into behavioral states such as:
external_read
internal_read
sensitive_read
external_write
internal_write
destructive_write
unknown
The model learns transition frequencies from normal synthetic workflows and calculates trajectory surprisal—average negative log likelihood—for the current sequence.
For a proposed call it returns:
sequence— recent behavior states plus the current action;surprisal— how improbable the sequence is under the reference model;anomaly_percentile— unusualness relative to normal-reference trajectories;unusual_transition— the final low-probability state transition, when present.
Example:
sensitive_read → external_write
↑
unusual transition
A static classifier sees one call. Agent security often depends on order, so the sequence model complements—but does not replace—hard controls.
flowchart LR
U[User intent] --> A[Agent proposes tool call]
A --> P[Deterministic policy]
A --> S[Behavior tokenization]
H[Recent call history] --> S
S --> M[Trajectory model]
M --> E[Surprisal + anomaly percentile]
P --> D{Runtime decision}
D --> O[Allow / Redact / Approval / Block]
E --> O
O --> X[Outcome + friction events]
X --> Q[Security measurement]
Q --> R[Risk reduction · task success · latency · utility]
This portfolio implementation is MCP-aware, not a live production MCP proxy. It models the security properties needed for tool governance—tool identity, destinations, sensitive data, delegation, destructive capabilities, and multi-step history—without connecting to real MCP servers or credentials.
A production version would add authenticated server identity, signed/versioned tool manifests, transport integrity, live tool registry state, cross-agent delegation graphs, and monitored tool-definition changes.
python -m venv .venv
source .venv/bin/activate
pip install -e '.[api]'
agentshield
python -m unittest discover -s tests -v
uvicorn agentshield.api:app --reloadThe CLI report includes policy outcomes, trajectory ML evidence, and the control-measurement experiment.
GitHub Actions validates Python 3.10, 3.11 and 3.12 across runtime policy tests, trajectory ML tests, security-measurement tests, CLI report generation, FastAPI smoke tests, benchmarking, and module compilation.
- Policy is authoritative. ML never grants privilege or bypasses a block.
- Model independence. The control plane does not need to trust the model proposing the action.
- Human control. High-impact actions can require approval.
- Explainability. Policy reasons and learned evidence remain inspectable.
- Measure friction. A control is not considered successful merely because it blocks more.
- Least privilege. Unknown tools fail closed.
- Synthetic evaluation. No production credentials, tool servers, or customer data are used.
A production implementation could train sequence models on authorized agent traces, segment peers by workflow, calibrate thresholds against analyst dispositions, run assignment-aware staged experiments, monitor control-treatment interference, sensitivity-test utility weights, add rare-severity guardrails, and track security outcomes over longer horizons.
Any learned model should remain advisory unless an independently governed policy explicitly authorizes automated enforcement.
All tool calls, identities, destinations, experiments, and training sequences are synthetic. The project demonstrates evaluation mechanics and runtime integration; it does not establish real-world attack recall, false-positive rate, or MCP compromise prediction.
AgentShield does not execute destructive actions, connect to live MCP servers, collect credentials, or modify production systems.
Policy decides what an agent may do. Measurement tells us whether the policy is actually helping.