feat(evaluators): add required_column_values for multi-tenant SQL validation#30
Merged
abhinav-galileo merged 1 commit intomainfrom Feb 19, 2026
Merged
Conversation
…idation [sc-52949] Adds value-aware multi-tenant enforcement to the SQL evaluator. The existing required_columns check only verifies column presence in WHERE - it doesn't validate the operator or value, so `WHERE user_id != 'user_001'` passes. New required_column_values config maps columns to runtime context keys and enforces: strict equality (=) with literal values matching context, AND-only conjunctive paths (no OR bypasses), top-level WHERE only (no subquery-only predicates), per-statement enforcement, and fail-closed on non-literals, missing context, or ambiguous unqualified columns in JOINs.
|
The author of this PR, abhinav-galileo, is not an activated member of this organization on Codecov. |
namrataghadi-galileo
approved these changes
Feb 18, 2026
galileo-automation
pushed a commit
that referenced
this pull request
Mar 4, 2026
## 1.0.0 (2026-03-04) ### ⚠ BREAKING CHANGES * **server:** Feature/56688 fix image bug (#48) * **sdk:** a bug in docker file (#46) * **server:** Feature/56688 fix docker and create bash (#45) * **evaluators:** Evaluator reorganization with new package structure Package Structure: - agent-control-evaluators (v3.0.0): core + regex, list, json, sql - agent-control-evaluator-galileo (v3.0.0): Luna2 evaluator Key Changes: - Entry points for evaluator discovery (agent_control.evaluators) - Dot notation for external evaluators (galileo.luna2 not galileo/luna2) - Dynamic __version__ via importlib.metadata - Server uses evaluators as runtime dep (no longer vendored) - Release workflow publishes both packages to PyPI Bug Fixes: - JSON evaluator: field_constraints/field_patterns in extra-fields allow-list - SQL evaluator: LIMIT/OFFSET bypass fix Migration: - Import: agent_control_evaluator_galileo.luna2 (not agent_control_evaluators.galileo_luna2) - DB: UPDATE controls SET evaluator.name replace('/', '.') * **server:** add time-series stats and split API endpoints (#6) * **evaluators:** rename plugin to evaluator throughout (#81) * **models:** simplify step model and schema (#70) ### Features * Add plugin auto-discovery via Python entry points ([#49](#49)) ([1521182](1521182)) * **docs:** add GitHub badges and CI coverage reporting ([#90](#90)) ([be1fa14](be1fa14)) * **evaluators:** add required_column_values for multi-tenant SQL validation ([#30](#30)) ([532386c](532386c)) * **sdk-ts:** automate semantic-release for npm publishing ([#52](#52)) ([2b43958](2b43958)) * **sdk:** Add PyPI packaging with semantic release ([#52](#52)) ([7c24f7f](7c24f7f)) * **sdk:** Auto-populate init() steps from [@control](https://github.com/control)() decorators ([#23](#23)) ([dc0f2a4](dc0f2a4)) * **sdk:** export ControlScope, ControlMatch, and EvaluatorResult models ([#18](#18)) ([0d49cad](0d49cad)) * **sdk:** Get Agent Controls from SDK Init ([#15](#15)) ([a485f93](a485f93)) * **sdk:** Refresh controls in a background loop ([#43](#43)) ([03f826d](03f826d)) * **sdk:** ship TypeScript SDK with deterministic method naming ([#32](#32)) ([a76e9b0](a76e9b0)) * **server:** add evaluator config store ([#78](#78)) ([cc14aa6](cc14aa6)) * **server:** add initAgent conflict_mode overwrite mode with SDK defaults ([#40](#40)) ([f3ed2b8](f3ed2b8)) * **server:** Add observability system for control execution tracking ([#44](#44)) ([fd0bddc](fd0bddc)) * **server:** add prometheus metrics for endpoints ([#68](#68)) ([775612c](775612c)) * **server:** add time-series stats and split API endpoints ([#6](#6)) ([a0fa597](a0fa597)) * **server:** hard-cut migrate to remove agent UUID ([#44](#44)) ([ee322c9](ee322c9)) * **server:** Optional Policy and many to many relationships ([#41](#41)) ([1a62746](1a62746)) * **ui:** add sql, luna2, json control forms and restructure the code ([#54](#54)) ([c4c1d4a](c4c1d4a)) * **ui:** allow to delete control ([#39](#39)) ([7dc4ca3](7dc4ca3)) * **ui:** Control Store Flow Updated ([#4](#4)) ([dda9f70](dda9f70)) * **ui:** stats dashboard ([#80](#80)) ([4cbb7fe](4cbb7fe)) * **ui:** Steps dropdown rendered based on api return values ([#36](#36)) ([a2aca43](a2aca43)) * **ui:** tests added and some minor ui changes, added error boundaries ([#61](#61)) ([009852b](009852b)) * **ui:** update agent control icon and favicon ([#42](#42)) ([19af8fa](19af8fa)) ### Bug Fixes * **ci:** Add ui scope to PR title validation ([#59](#59)) ([e0fdb52](e0fdb52)) * **ci:** correct galileo contrib path in release build script ([#51](#51)) ([2de6013](2de6013)) * **ci:** Enable pr title on prs ([#56](#56)) ([3d8b5fe](3d8b5fe)) * **ci:** Fix release ([#11](#11)) ([9dd3dd7](9dd3dd7)) * **ci:** Use galileo-automation bot for releases ([#57](#57)) ([bc8eea0](bc8eea0)) * **docs:** Add Example for Evaluator Extension ([#3](#3)) ([c2a70b3](c2a70b3)) * **docs:** add setup script ([#49](#49)) ([7a212c3](7a212c3)) * **docs:** Clean up Protect ([#76](#76)) ([99c16fd](99c16fd)) * **docs:** Fix Examples for LangGraph ([#64](#64)) ([23b30ae](23b30ae)) * **docs:** Improve documentation for open source release ([#47](#47)) ([9018fb3](9018fb3)) * **docs:** Remove old/unused examples ([#66](#66)) ([f417781](f417781)) * **docs:** Update Contributing Guide ([#8](#8)) ([10b34c8](10b34c8)) * **docs:** Update readme ([#37](#37)) ([7531d83](7531d83)) * **docs:** Update README ([#2](#2)) ([379bb15](379bb15)) * **examples:** Control sets cleanup with signed ([#65](#65)) ([af7b5fb](af7b5fb)) * **examples:** Update crew ai example to use evaluator ([#93](#93)) ([1c65084](1c65084)) * **infra:** Add plugins directory to Dockerfile ([#58](#58)) ([171d459](171d459)) * **infra:** install engine/evaluators in server image ([#14](#14)) ([d5ae157](d5ae157)) * **models:** use StrEnum for error enums ([#12](#12)) ([3f41c9f](3f41c9f)) * **sdk-ts:** add conventional commits preset dependency ([#55](#55)) ([540fe9d](540fe9d)) * **sdk-ts:** export npm token for semantic-release npm auth ([#54](#54)) ([1b6b993](1b6b993)) * **sdk:** 54253 add steer action and example ([#38](#38)) ([bf2380a](bf2380a)) * **sdk:** a bug in docker file ([#46](#46)) ([12d1794](12d1794)) * **sdk:** Add step_name as parameter to control ([#25](#25)) ([19ade9d](19ade9d)) * **sdk:** emit observability events for SDK-evaluated controls and fix non_matches propagation ([#24](#24)) ([6a9da69](6a9da69)) * **sdk:** enforce UUID agent IDs ([#9](#9)) ([5ccdbd0](5ccdbd0)) * **sdk:** Fix logging ([#77](#77)) ([b1f078c](b1f078c)) * **sdk:** plugin to evaluator.. agent_protect to agent_control ([#88](#88)) ([fc9b088](fc9b088)) * **server:** enforce public-safe API error responses ([#20](#20)) ([e50d817](e50d817)) * **server:** Feature/56688 fix docker and create bash ([#45](#45)) ([7277e27](7277e27)) * **server:** Feature/56688 fix image bug ([#48](#48)) ([71e6b44](71e6b44)) * **server:** fix alembic migrations ([#47](#47)) ([c19c17c](c19c17c)) * **server:** reject initAgent UUID/name mismatch ([#13](#13)) ([19d61ff](19d61ff)) * tighten evaluation error handling and preserve control data ([52a1ef8](52a1ef8)) * **ui:** Fix UI and clients for simplified step schema ([#75](#75)) ([be2aaf0](be2aaf0)) * **ui:** json validation ([#10](#10)) ([a0cd5af](a0cd5af)) * **ui:** selector subpaths issue ([#34](#34)) ([79cb776](79cb776)) * **ui:** UI feedback fixes ([#27](#27)) ([6004761](6004761)) ### Code Refactoring * **evaluators:** rename plugin to evaluator throughout ([#81](#81)) ([0134682](0134682)) * **evaluators:** split into builtin + extra packages for PyPI ([#5](#5)) ([0e0a78a](0e0a78a)) * **models:** simplify step model and schema ([#70](#70)) ([4c1d637](4c1d637))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes sc-52949
The SQL evaluator's
required_columnscheck verifies column presence in WHERE but doesn't validate the operator or value. This meansWHERE user_id != 'user_001'passes validation becauseuser_idis technically present - even though it negates the tenant filter.required_column_valuesconfig option that maps column references to runtime context keys=) with a literal value matching the context value$1, function calls), missing/non-scalar context, and ambiguous unqualified columns in JOINsTest plan
TestRequiredColumnValuescovering all valid/blocked scenarios