Skip to content

docs(prompts,skills): bound recon output for shared-disk hygiene#783

Merged
0xallam merged 3 commits into
mainfrom
devin/1784201376-bound-recon-disk
Jul 16, 2026
Merged

docs(prompts,skills): bound recon output for shared-disk hygiene#783
0xallam merged 3 commits into
mainfrom
devin/1784201376-bound-recon-disk

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Agents on a deep black-box scan can write very large, uncapped recon artifacts (e.g. katana JSONL crawl output) into the single shared /workspace disk, with no self-limiting. Nothing in the prompt or the katana skill currently nudges them to bound crawl output or clean it up, so a broad -jsl/-kf all crawl on a large site can grow until the sandbox disk saturates. This adds lightweight, best-effort guidance (no code/runtime changes) so agents keep recon output bounded and tidy.

Layered so the rule is seen regardless of which skills are loaded:

  • agents/prompts/system_prompt.jinja — always-on DISK & SCRATCH HYGIENE note in the shared-workspace (AGENT ISOLATION & SANDBOXING) block: /workspace is shared/finite, prefer bounded recon over "collect everything", send large output to /workspace/scratch/ and remove raw output once distilled, and — when disk is tight — clean up only files from your own task (don't delete another agent's in-use artifacts on the shared disk). The PHASE 1 "CRAWL thoroughly" bullet also gains a bound-and-tidy reminder.
  • skills/tooling/katana.md — document the flags that actually bound crawl output size (from projectdiscovery's katana docs) and a reduce-then-delete workflow:
    • -mdp (max-domain-pages — katana's default is unlimited), -fsu (collapse near-identical URLs), -fs fqdn (default rdn crawls all subdomains), -f url (plain URL list vs verbose JSONL), -or/-ob (omit raw/body), -mrs (cap per-response bytes).
    • Baseline command now ships -ct 10m -mdp 2000 -fsu; deep-JS example is time-bounded.
    • New "Keeping output small" section: bound scope/volume, shrink each record, reserve -jsl/-kf all for narrowed targets, then distil (-f url / short path list) and delete the raw crawl; sanity-check with du -sh.

Intentionally soft in tone (guidance, not hard mandates) and free of specific size thresholds; a model told to "go very deep" can still ignore it, so this is best paired with a hard inner-sandbox disk guard as the real backstop. This PR is the prompt/skill layer only.

Link to Devin session: https://app.devin.ai/sessions/dad023e379e942f287bcf6822463b7a4
Requested by: @0xallam

Add lightweight, always-on disk-hygiene guidance so agents keep recon
artifacts bounded on the shared /workspace instead of writing very large
uncapped crawl output.

- system_prompt.jinja: DISK & SCRATCH HYGIENE note in the shared-workspace
  block; recon PHASE 1 crawl bullet asks to bound each crawl and tidy up.
- skills/tooling/katana.md: bound the baseline/deep examples with -ct,
  add a Keeping-output-manageable note (bound by -ct/-d, reserve -jsl/-kf
  all for narrowed targets, check du -sh, dedupe and remove raw .jsonl).
@0xallam 0xallam self-assigned this Jul 16, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds guidance for keeping reconnaissance output bounded on the shared workspace. The main changes are:

  • Adds shared-disk and scratch-file hygiene to the system prompt.
  • Bounds crawls by target, depth, duration, and output volume.
  • Documents Katana flags for smaller, scoped output.
  • Recommends reducing crawl results before deleting raw artifacts.

Confidence Score: 5/5

This looks safe to merge.

  • Cleanup guidance now protects files belonging to other active tasks.
  • URL deduplication guidance is limited to an extracted URL list.
  • No blocking issues remain in the updated documentation.

Important Files Changed

Filename Overview
strix/agents/prompts/system_prompt.jinja Adds bounded reconnaissance and ownership-aware cleanup guidance for the shared workspace.
strix/skills/tooling/katana.md Adds scoped Katana examples and guidance for limiting, reducing, and cleaning crawl output.

Reviews (3): Last reviewed commit: "docs(skills): add research-backed katana..." | Re-trigger Greptile

Comment thread strix/agents/prompts/system_prompt.jinja Outdated
Comment thread strix/skills/tooling/katana.md Outdated
Address Greptile review:
- system_prompt: only clean up your own task's files; don't delete
  another agent's files in the shared workspace unless confirmed unused.
- katana.md: extract+dedupe URLs with jq before removing raw .jsonl
  (sort -u on JSONL compares whole records, not URLs).
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptile

Per projectdiscovery katana docs, add the flags that actually bound
crawl output size and a reduce-then-delete workflow:
- -mdp (max-domain-pages; default is unlimited), -fsu (filter-similar),
  -fs scope, -f url (URL-only), -or/-ob (omit raw/body), -mrs.
- Baseline now includes -mdp 2000 -fsu; new 'Keeping output small'
  section: bound scope/volume, shrink records, distil then delete raw
  crawls.
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptile

@0xallam
0xallam merged commit 89ee7b9 into main Jul 16, 2026
1 check passed
@0xallam
0xallam deleted the devin/1784201376-bound-recon-disk branch July 16, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant