agents: sandbox guardrails — engine caps, typed resource errors, network watchdog (contract 0.3.0)#613
Merged
Conversation
…ork watchdog (contract 0.3.0) Three changes for running LLM-generated SQL in small sandboxes: - Engine-side row bound (max_result_rows=cap+1, break) so an unbounded SELECT can't OOM the process before the client-side trim; new optional max_memory_usage / max_result_bytes; per-call max_rows clamped to the constructor cap. - Resource errors (158/159/241/307/396) classify as ChDBResourceError and carry a model-facing hint; truncated envelope results carry one too. ACCESS_DENIED renamed ALLOWLIST_DENIED (collided with engine error 497). - network_timeout (default 60s): fast-fail engine baseline + a deadline on queries referencing url()/s3()/remote()/...; expiry raises NETWORK_TIMEOUT with a hint. Needed because a black-holed endpoint hangs the native call past every engine-side timeout (verified on a firewalled sandbox). The abandoned call can't be cancelled, so the tool is poisoned afterwards and the session intentionally leaks — documented in CONTRACT P5. Contract 0.3.0; capabilities gains resource_caps + network_watchdog; 6 new conformance cases. 52 tests green (6 skipped: no smolagents).
…test A stuck network call on an executor's non-daemon worker blocks interpreter exit at the atexit join. Switched to a daemon thread. The new test hangs a real url() call against a silent local endpoint, checks the watchdog fires, and confirms a fresh ChDBTool keeps working in the same process.
… send/receive, not connection_timeout Verified against a chdb-core main build (042dd51): the handshake is bounded by max(send, receive) socket timeouts and one attempt costs ~4-5x the setting. Baseline now sets all three timeouts to min(network_timeout, 10). An engine-side Poco::TimeoutException on a network-source query gets the same hint as a watchdog expiry.
Parity with the TS binding, where one-active-data-directory-per-process makes releasing it a requirement: when the watchdog-abandoned engine call eventually returns, the session leaves _ABANDONED_SESSIONS and is closed if tool-owned. Leaks for process lifetime only if the call never settles. Also recovers the result instead of poisoning when the call settles in the window between wait timeout and the abandonment decision.
Contributor
Author
|
@chibugai, please review it |
There was a problem hiding this comment.
Pull request overview
Adds contract 0.3.0 guardrails to the chdb.agents tool surface to make LLM-driven queries safer in memory-tight / firewalled sandboxes, while keeping cross-binding behavior consistent via the CONTRACT + conformance fixtures.
Changes:
- Enforces engine-side result caps (rows + optional bytes/memory) and adds truncation hints to model-visible results.
- Introduces typed “resource limit” errors with standardized model-facing hints; renames allowlist failures to
ALLOWLIST_DENIED. - Adds a binding-side network watchdog for queries that reference network table functions, poisoning the tool after an abandoned call.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_agents_conformance.py | Extends conformance assertions to verify presence/absence of hint in result/error envelopes. |
| tests/test_agents_chdbtool.py | Updates allowlist error type expectations and adds network-watchdog contract tests. |
| chdb/agents/tool.py | Implements engine caps, truncation hints, network watchdog + poisoning behavior, and ALLOWLIST_DENIED typing. |
| chdb/agents/safety.py | Defines NETWORK_TABLE_FUNCTIONS used to decide when the watchdog applies. |
| chdb/agents/errors.py | Adds ChDBResourceError, standardized hints (RESOURCE_HINT, NETWORK_HINT), and resource-code classification. |
| chdb/agents/descriptors.py | Bumps CONTRACT_VERSION to 0.3.0 and adds new capability flags. |
| chdb/agents/descriptors.json | Bumps contract_version to 0.3.0 (model-visible schema source-of-truth). |
| chdb/agents/CONTRACT.md | Updates normative contract text for 0.3.0 (engine caps, resource typing, watchdog semantics). |
| chdb/agents/conformance/cases.jsonl | Bumps contract header and adds/updates conformance cases for new behaviors and renamed error types. |
| chdb/agents/init.py | Exports ChDBResourceError from the package surface. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…t network_timeout validation Only None/0 disable the watchdog; other values (including '') validate as INVALID_ARGUMENT instead of silently disabling a guardrail. Graveyard mutations now go through a module lock rather than relying on GIL atomicity. The test file's __main__ block moved below the last class.
Contributor
Author
|
@wudidapaopao please review this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contract 0.3.0: guardrails for running LLM-generated SQL in memory-tight sandboxes.
max_result_rows = max_rows + 1withresult_overflow_mode='break', so an unbounded SELECT stops at block granularity instead of materializing everything before the client-side trim (in a 512MB sandbox that's an OOM, not a truncation). The+1keeps thetruncatedflag exact; a per-callmax_rowsabove the constructor cap is clamped. New optionalmax_memory_usage/max_result_bytes.ChDBResourceErrorand carry a model-facing hint (the SQL was valid — narrow it, don't abandon, don't retry unchanged). Truncated envelope results carry a hint too.ACCESS_DENIEDrenamedALLOWLIST_DENIED: it collided with engine error 497 of the same name.network_timeout(default 60s) adds a fast-fail HTTP baseline plus a client-side deadline on queries referencingurl()/s3()/remote()/... Verified on a firewalled sandbox (E2B with egress off): a black-holed HTTPS endpoint hangs the engine call indefinitely — no engine-side timeout fires — so the binding imposes the deadline, envelopesNETWORK_TIMEOUTwith a hint, and poisons the tool afterwards (the abandoned native call can't be cancelled; CONTRACT P5 documents the tradeoff).Contract bumped to 0.3.0 (descriptors.json, constant, fixture header);
capabilities()gainsresource_capsandnetwork_watchdog; 6 new conformance cases. Tests: 52 passed.Mirror PR: chdb-io/chdb-node#74 (vendored files byte-identical).
🤖 Generated with Claude Code
Note
Add engine row caps, typed resource errors, and network watchdog to
ChDBTool(contract 0.3.0)ChDBResourceError, a new error subclass for engine resource limit hits (rows/bytes/time/memory), with a hint advising callers to narrow their query; error codes 158, 159, 241, 307, and 396 are mapped to it._run_with_network_deadline) toChDBToolthat runs queries referencing network table functions in a daemon thread and raisesNETWORK_TIMEOUTwith an actionable hint if the deadline is exceeded; the tool instance is poisoned and rejects further queries after a timeout.ChDBTool.__init__now acceptsnetwork_timeout(default 60s),max_memory_usage, andmax_result_bytesparameters; engine session settings enforce a row cap (max_result_rows) and result overflow mode.ACCESS_DENIEDerror type toALLOWLIST_DENIEDfor allowlist violations in_enforce_allowlistand_create_file_view.resource_capsandnetwork_watchdogcapabilities.ALLOWLIST_DENIEDis a breaking rename ofACCESS_DENIED; callers checking error type strings will need to update their handling.Macroscope summarized c763846.