feat(evaluators): add Cisco AI Defense evaluator and examples#60
Merged
abhinav-galileo merged 6 commits intomainfrom Mar 11, 2026
Merged
feat(evaluators): add Cisco AI Defense evaluator and examples#60abhinav-galileo merged 6 commits intomainfrom
abhinav-galileo merged 6 commits intomainfrom
Conversation
|
The author of this PR, rucpande, is not an activated member of this organization on Codecov. |
rucpande
changed the title
rucpande
changed the title
lan17
reviewed
Mar 5, 2026
evaluators/contrib/cisco/src/agent_control_evaluator_cisco/ai_defense/evaluator.py
Outdated
Show resolved
Hide resolved
evaluators/contrib/ai_defense/src/agent_control_evaluator_ai_defense/chat_inspect/evaluator.py
Outdated
Show resolved
Hide resolved
evaluators/contrib/ai_defense/tests/test_chat_inspect_evaluator.py
Outdated
Show resolved
Hide resolved
evaluators/contrib/ai_defense/src/agent_control_evaluator_ai_defense/chat_inspect/client.py
Outdated
Show resolved
Hide resolved
Collaborator
Author
|
Thanks for the quick and thorough review! Will address the last comment tomorrow. |
Collaborator
Author
|
@abhinav-galileo please let me know if anything else is required. |
abhinav-galileo
requested changes
Mar 10, 2026
evaluators/contrib/cisco/src/agent_control_evaluator_cisco/ai_defense/evaluator.py
Show resolved
Hide resolved
evaluators/contrib/cisco/src/agent_control_evaluator_cisco/ai_defense/evaluator.py
Outdated
Show resolved
Hide resolved
evaluators/contrib/cisco/src/agent_control_evaluator_cisco/ai_defense/client.py
Outdated
Show resolved
Hide resolved
evaluators/contrib/cisco/src/agent_control_evaluator_cisco/ai_defense/config.py
Outdated
Show resolved
Hide resolved
evaluators/contrib/cisco/src/agent_control_evaluator_cisco/ai_defense/evaluator.py
Outdated
Show resolved
Hide resolved
- Add contrib package at evaluators/contrib/ai_defense - Add examples under examples/ai_defense - Update Makefile with AI Defense targets - Update evaluators/builtin/pyproject.toml for discovery - Update .gitignore
- Honor on_error on missing API key and add metadata - Add tests: missing key deny/allow, api_url override, safe/unsafe mapping, client reuse - Recreate client on key/endpoint/timeout change; assert non-null client - Narrow client import exception to ImportError - Fix examples to use server_url kwarg - Remove duplicate nested register_agent.py - Remove AI Defense targets from root Makefile (keep contrib out)
abhinav-galileo
force-pushed
the
feature/cisco-ai-defense-integration
branch
from
f455de5 to
7018215
Compare
abhinav-galileo
approved these changes
Mar 11, 2026
galileo-automation
pushed a commit
that referenced
this pull request
Mar 18, 2026
## [2.0.0](ts-sdk-v1.1.0...ts-sdk-v2.0.0) (2026-03-18) ### ⚠ BREAKING CHANGES * **server:** implement recursive control condition trees (#115) ### Features * **docs:** simplify main README quick start v2 ([#100](#100)) ([7701c65](7701c65)) * **docs:** Update README.md to accurately mention Strands Agents SDK ([#105](#105)) ([84dfa57](84dfa57)) * **evaluators:** add Cisco AI Defense evaluator and examples ([#60](#60)) ([52e2416](52e2416)) * **sdk:** add Google ADK framework integration ([#83](#83)) ([f2b53ee](f2b53ee)) * **server:** implement recursive control condition trees ([#115](#115)) ([2ff4c12](2ff4c12)) ### Bug Fixes * **docs:** clarify README quickstart flow ([#119](#119)) ([317c626](317c626)) * **docs:** cleanup readme ([#66](#66)) ([eca0d6b](eca0d6b)) * **docs:** promote Google ADK plugin in README ([#99](#99)) ([f038aa4](f038aa4)) * **evaluators:** address Cisco AI Defense review issues ([#103](#103)) ([81da81b](81da81b)) * **examples:** shut down SDK in short-lived demos ([#85](#85)) ([df43264](df43264)) * **sdk:** fail closed on post-hook evaluation errors ([#114](#114)) ([67f8fc1](67f8fc1)), closes [#112](#112) * **sdk:** skip evaluation when no controls apply ([#124](#124)) ([d2cd9f9](d2cd9f9)) * **server:** escape ILIKE pattern in evaluator-configs list endpoint ([#108](#108)) ([cb6e89f](cb6e89f)) * **server:** reject empty string list evaluator values ([#121](#121)) ([2c55d27](2c55d27)) * **server:** revert unwanted changes in server config ([#117](#117)) ([d92e168](d92e168)) * **ui:** create flow - name trimming issue ([#93](#93)) ([1a9759d](1a9759d))
Collaborator
|
🎉 This PR is included in version 2.0.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Scope
Risk and Rollout
Testing
make check(or explained why not)Checklist