You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We run DSH 24/7 on a small VPS and wrap several CLIs as LLM providers via custom cordis plugins. Sharing what we learned and asking for the blessed pattern.
Context. When a CLI coding agent (agy, codex, claude) is spawned headlessly as a provider, its internal agent loop can start doing its own tool calls behind the harness's back — we observed agy auto-approving its own ViewFile/edit steps because our first adapter passed --dangerously-skip-permissions. That surfaced as empty assistant turns (the CLI burned thousands of tokens internally and returned nothing).
What we verified locally (agy 1.1.14):
--mode plan + --sandbox + --disable-slash-commands = pure-LLM posture (no edit/apply tools, no slash expansion). Text answers work normally.
--dangerously-skip-permissions = full autonomy — never for a provider adapter.
--json-schema gives structured final results; --input-format stream-json allows multi-turn NDJSON.
Our adapter hardening (open to review): process-group kill (SIGTERM→SIGKILL), stderr ring drain, TTFT/idle/wall timers, exit-vs-close handling incl. grandchildren holding pipes, error taxonomy mapped to RATE_LIMIT(+providerRetryAfterMs)/AUTH/INVALID_REQUEST/TIMEOUT/SERVER, never fabricating content on empty results, and a fake-CLI test harness (17 integration scenarios + unit tests, all hermetic).
Questions for maintainers/community:
Is there a blessed hook for cross-provider fallback in dsh-llm? We found providerRetryPolicy (same-provider retry) but no route chain — is the intended pattern a custom plugin on agent/request-error, or something like LiteLLM-style fallbacks on the roadmap?
Does dsh-fs-sandbox compose with CLI-provider adapters (i.e., constrain what a CLI child can touch beyond its own flags), or is bubblewrap the expected OS layer?
Any plans for a first-party "CLI agent adapter" package in the harness (shared spawn/parse/safety layer)?
Happy to share our adapter + test suite as a plugin if useful. Thanks for the great harness!
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
We run DSH 24/7 on a small VPS and wrap several CLIs as LLM providers via custom cordis plugins. Sharing what we learned and asking for the blessed pattern.
Context. When a CLI coding agent (
agy,codex,claude) is spawned headlessly as a provider, its internal agent loop can start doing its own tool calls behind the harness's back — we observedagyauto-approving its ownViewFile/edit steps because our first adapter passed--dangerously-skip-permissions. That surfaced as empty assistant turns (the CLI burned thousands of tokens internally and returned nothing).What we verified locally (agy 1.1.14):
--mode plan+--sandbox+--disable-slash-commands= pure-LLM posture (no edit/apply tools, no slash expansion). Text answers work normally.--dangerously-skip-permissions= full autonomy — never for a provider adapter.--json-schemagives structured final results;--input-format stream-jsonallows multi-turn NDJSON.codex exec --sandbox read-only --json. Claude Code:-p --permission-mode dontAsk --disallowedTools "*" --output-format stream-json.Our adapter hardening (open to review): process-group kill (SIGTERM→SIGKILL), stderr ring drain, TTFT/idle/wall timers,
exit-vs-closehandling incl. grandchildren holding pipes, error taxonomy mapped toRATE_LIMIT(+providerRetryAfterMs)/AUTH/INVALID_REQUEST/TIMEOUT/SERVER, never fabricating content on empty results, and a fake-CLI test harness (17 integration scenarios + unit tests, all hermetic).Questions for maintainers/community:
providerRetryPolicy(same-provider retry) but no route chain — is the intended pattern a custom plugin onagent/request-error, or something like LiteLLM-style fallbacks on the roadmap?dsh-fs-sandboxcompose with CLI-provider adapters (i.e., constrain what a CLI child can touch beyond its own flags), or is bubblewrap the expected OS layer?Happy to share our adapter + test suite as a plugin if useful. Thanks for the great harness!
All reactions