Agent write/edit tools bypass settings.yaml schema validation — malformed configs crash DSH on restart

[idoitss]

Problem

Agent (the in-harness AI) can modify DSH configuration files (settings.yaml, cordis.patch.yml, etc.) through the write/edit filesystem tools. However, these writes bypass the schema validation that the Settings Web UI enforces.

The result: Agent writes a malformed config (wrong YAML structure, invalid provider nesting, duplicate loader entry ids, etc.), then restarts dsh web, and DSH fails to start — with no clear error guidance in the browser. The user has to manually fix the file and restart.

Root Cause

The settings layer (dsh-settings + dsh-settings-file) has a complete schema + validate pipeline:

• Each namespace registers a z.object schema (via schemastery)
• A validate callback (e.g., assertServiceable for llm-pi-ai) rejects invalid configs at write time
• publish() gracefully degrades: if a stored section is invalid, it keeps the last good value and logs a warning

But this validation only runs on the settings.mutate / settings.replace write path (i.e., when the user edits through the Web UI Settings page). Agent's write/edit tools go through fs.writeFile directly, completely bypassing the settings layer.

Similarly, cordis.patch.yml (the loader patch layer) has no schema validation at all — it's a raw YAML file parsed by the loader at boot time. Agent writes to it are blind.

Reproduction Steps

1. Ask Agent to add a new provider to settings.yaml (e.g., a new llm-pi-ai.providers.xxx entry)
2. Agent writes the file with incorrect YAML nesting — DSH Web UI settings page would have rejected it, but the write tool has no such check
3. Agent executes dsh --profile web (or similar restart command)
4. DSH fails to start — browser tab stays blank, no error in the GUI
5. User must manually inspect settings.yaml, fix the structure, and restart

Evidence

Config diff that triggered this issue

Old (working) settings.yaml:

agent-default-model:
  provider: deepseek-official
  model: deepseek-v4-flash
llm-deepseek:
  apiKeyEnv: DEEPSEEK_API_KEY
  baseURL: https://api.deepseek.com
  models:
    - id: deepseek-v4-pro
    - id: deepseek-v4-flash
llm-pi-ai:
  providers:
    my-provider:
      displayName: my-provider
      apiKeyEnv: MY_PROVIDER_API_KEY
      api: openai-completions
      baseURL: https://internal-proxy.example.com/personal/openai/api/v1
      models:
        - id: deepseek-v4-flash
        - id: deepseek-v4-pro

After Agent edit — the llm-deepseek section was removed, provider reference changed to my-provider, but the YAML structure was wrong (indentation/nesting issue in the provider config). DSH failed to start on next restart.

Source code evidence

In dsh-settings/lib/index.js:

• register() (line 311): stores schema + validate callback
• write() (line 439): calls resolve() with validate, catches errors before persist
• publish() (line 476): on external file change, gracefully degrades — keeps last good value if section is invalid (line 489-491)

In dsh-llm-pi-ai/lib/index.js:

• Config schema (line 1394): z.object({ providers: z.dict(profile).default({}) })
• assertServiceable() (line 1407): calls resolveProfiles() which checks provider names, baseURL, model ids, etc.
• installSettingsSection() (line 1846): registers the schema + validate callback with the settings service

The gap: all this validation is bypassed when Agent uses write/edit filesystem tools instead of the settings API.

Proposed Solutions

Option A: Intercept config file writes

Add a file-write hook or a dsh-fs middleware that intercepts writes to known config files (settings.yaml, cordis.patch.yml) and runs a pre-write validation:

1. Parse the YAML
2. Run the relevant settings namespace schema against the modified section
3. If validation fails, reject the write with a clear error message naming the offending field

Option B: Pre-restart config check

Add a dsh --check-config command that validates all config files against their registered schemas, and make it a required step before any dsh web restart:

1. Agent runs dsh --check-config before restart
2. If check fails, Agent gets a structured error report and can fix it before restarting
3. The CLI could also be called automatically by the dsh-cordis-host-runner or dsh-app-boot plugin

Option C: Tool-level protected paths

Add a dsh-tool-fs configuration option that lists "protected file paths" — files whose writes are validated against a registered schema or a YAML syntax check, and rejected if invalid.

Related

• dsh-settings-file README: validates at settings API level, not filesystem level
• dsh-plugin-guard (community): protects against bad plugin installs, but not config file writes
• dsh-doctor (community): post-hoc diagnostics, not pre-write validation

Discussion

This is a structural gap: DSH's "everything is a plugin" philosophy means Agent is a first-class citizen with filesystem access, but the validation layer that protects config files from human editors is not extended to Agent's own write tools. The fix should be in the framework layer, not pushed to individual AGENTS.md rules.

[argszero]

Verified against rc.7 (99f6f02fec) — the structural gap is real, and your source references are accurate. Adding the verified layer, one precision correction, and a fix-placement recommendation that matches the existing architecture.

1. Confirmed at source:

• settings write path validates: write() (packages/settings/settings/src/index.ts:578+) → resolve(registration.schema, ..., registration.validate) (:633) — schema + validate callback run before persist.
• write/edit tools bypass it entirely: packages/fs/tool-fs/src/write.ts:114 → ctx.fs.writeText(target, input.content, ...) — no settings/patch awareness anywhere in the fs-tool path (grep-verified zero references to settings or cordis.patch in tool-fs).
• cordis.patch.yml: entryListSchema validation exists (vendor/include/src/index.ts:23-25) but runs only at boot/load (:262 "must be a top-level array"); no pre-write check.

2. Precision correction — settings.yaml is partially self-healing, cordis.patch.yml is not. The settings-file provider hot-publishes external edits with graceful degradation (settings-file/src/index.ts:300-310: "reload failed at %s; keeping the last good document") — so a malformed settings.yaml section is caught at publish (keeps last good, warns). The two failure modes that actually kill boot:

• settings.yaml parse failure (wrong YAML, not just wrong schema) → boot fails loud (:233-234 "must fail loud, never be" silently skipped);
• cordis.patch.yml any malformation → boot fails with a loader error, and the GUI gives no clear recovery path (your scenario 4 exactly).

So the highest-value target is cordis.patch.yml pre-write validation, then settings.yaml syntax (not schema) checking.

3. Fix placement — where it belongs architecturally. Your Option A (intercept config writes) is the right shape but the intercept shouldn't be a new middleware — the existing seam is the fs write policy (sandboxPolicy, already threaded through write.ts:108-114). The fs-observation-policy layer already exists (packages/fs/fs-observation-policy/src/) for exactly this kind of write-side governance; a "config-file guard" policy that:

1. detects writes targeting settings.yaml / cordis.patch.yml (resolved via resolveDshHome),
2. parses the new content (YAML),
3. for settings.yaml: runs the registered namespace schemas (or at minimum a YAML-parse check),
4. for cordis.patch.yml: runs entryListSchema (vendor/include/src/index.ts:23) — the same parser boot uses,
5. rejects with a structured error naming the offending field/line.
   This composes with the existing policy slot instead of adding a parallel path, and it's testable in isolation.

4. Option B (--check-config) is the natural complement — and there's precedent: renderConfigDump (packages/boot/app-boot/src/index.ts:404) already composes the config offline with the same applyEntryPatches boot uses. A --check-config that runs entryListSchema + settings schemas over the composed result would catch everything pre-restart and give the agent (and human) a structured error report. This also closes the #1404-family "dump-vs-runtime drift" gap (dump shows one thing, boot does another).

5. Family note. This is the config-file twin of the settings-seam security family I've been tracking (#1688 settings prototype pollution, #2445 redact-fail-open, #1559 .env credential redirection): the settings layer has validation and redaction, but the filesystem path around it is ungoverned. The agent having write is by-design ("everything is a plugin"); the gap is that the config-protection layer wasn't extended to the agent's own write surface. Worth a consolidated "config-plane write governance" workstream rather than one-off guards.

Good writeup — the three options are all viable; Option A in the policy slot + Option B as complement is the cleanest pairing, and both are small, testable, framework-layer changes.