Production case study: using DSH for a read-only disk investigation with human approval gates

[yangbobo2021]

Disclosure: I’m affiliated with KeySync, which provided the remote access path used in this incident. This discussion is about the DSH operational workflow and safety boundary; it contains no product download link.

I used DSH on a production Ubuntu host to investigate a release blocked by low disk space. The server had 6.5 GiB free, while the CI safety gate required 17.4 GiB.

The initial instruction was deliberately restrictive:

Perform a read-only investigation only. Do not delete anything, restart services, or modify configuration. Identify what is consuming the disk, distinguish active data from recoverable duplicates, and report which actions require approval.

DSH kept the investigation on the affected host and followed the evidence across several layers:

1. df confirmed the root filesystem was at 98%.
2. du narrowed the scope to application data and Docker.
3. Container and volume inspection showed all 37 containers were running, so blind Docker cleanup was ruled out.
4. A release upload-staging directory accounted for 77.85 GiB.
5. DSH followed that directory to the retention script and found a production naming mismatch.

Production directories used names such as:

client-0.1.113
client-0.1.114
client-0.1.115

The cleanup rule matched only:

client-v*

The script exited normally but never matched the production directories. Tests passed because their fixtures also used client-v....

DSH remained read-only after locating the bug. It hash-verified 5,487 files before proposing the removal of three duplicate staging directories. Only after explicit human approval did the deletion run. Free space increased from 6.41 GiB to 21.84 GiB, allowing the release to continue.

The same session then reproduced the retention failure using real production names, added focused tests, identified unique files requiring recoverable backups, and proposed a second narrowly scoped cleanup. After another approval, eleven historical upload sets were removed while the newest two and approximately 7.89 GiB of unique data were preserved. The operation recovered 48.46 GiB, followed by verification of downloads, update feeds, web files, and protected business data.

The useful pattern was:

detect -> investigate read-only -> rule out unsafe shortcuts ->
identify root cause -> propose exact mutation -> wait for approval ->
execute -> verify

I expect remote-host agent workflows to become common in operations, but production use needs stronger boundaries than a prompt alone.

I’d be interested in how DSH users and maintainers think these controls should evolve:

• Should read-only and mutating tool sets be separate profiles?
• Should destructive shell commands require a structured approval receipt?
• Would command allowlists or policy hooks be useful at the harness level?
• What evidence should an agent present before requesting approval?
• Should post-action verification be a first-class part of an approved operation?

Would you allow DSH to investigate or maintain production hosts? What technical boundary would make that acceptable in your environment?