Skip to content

fix(cascade-tools): harden multiline text and large diff I/O for shell agents#1381

Merged
zbigniewsobiecki merged 3 commits into
devfrom
fix/cascade-tools-shell-multiline-hardening
May 19, 2026
Merged

fix(cascade-tools): harden multiline text and large diff I/O for shell agents#1381
zbigniewsobiecki merged 3 commits into
devfrom
fix/cascade-tools-shell-multiline-hardening

Conversation

@aaight
Copy link
Copy Markdown
Collaborator

@aaight aaight commented May 18, 2026

Summary

Closes MNG-1059.

Hardens cascade-tools against three friction clusters that wasted agent budget:

  1. Shell-quoting of markdown / multiline / backticks (MNG-908, MNG-910, MNG-1046) — agents typed inline --body '...markdown...' and lost backticks / $(...) to shell expansion.
  2. Multiple stdin consumers (MNG-917) — --body-file - --comments-file - silently truncated one payload because readFileSync(0, ...) can only drain stdin once.
  3. Large PR diff truncation (MNG-1045) — single one-line JSON diffs (>100KB) truncated stdout, losing review context.

Changes

1. Shared multiple-stdin guard (src/gadgets/shared/cli/params.ts)

  • New rejectMultipleStdinConsumers() helper scans fileInputAlternatives for - values and emits a structured flag-parse error envelope (error.flag: "body-file,comments-file", hint pointing at temp files) before any readFileSync(0, ...) call.
  • Called from cliCommandFactory.ts after oclif parsing but before resolveDirectParams().
  • Direct file paths remain pairwise-compatible: --body-file - --comments-file /tmp/c.json and --body-file /tmp/b.md --comments-file - both work.

2. Shell-safe multiline guidance in native tool prompts (src/backends/shared/nativeToolPrompts.ts)

  • New "cascade-tools shell-safety rules" section in the system prompt rendered by buildSystemPrompt() — documents the one-stdin-consumer invariant and shows safe heredoc / temp-file patterns for one and two payloads.
  • Manifest threads fileInputAlternative (on direct text params) and fileInputFor (on --*-file synth flags) so the prompt renderer can semantically link the pair.
  • isShellSensitiveExample() detects examples with backticks, code fences, $(...), or newlines. When a file-input companion exists, the renderer suppresses the inline --body '...' example and emits --body-file <path> # write the markdown/multiline content to a temp file (shell-sensitive: ...) instead.

3. Refreshed file-input descriptions (src/gadgets/{github,pm}/definitions.ts)

  • --body-file, --text-file, --description-file, --details-file, --comments-file descriptions now explicitly call out "markdown / multiline content with backticks, code fences, $(...) or newlines."
  • The --comments-file description also documents the single-stdin rule.

4. get-pr-diff --outputFile escape hatch (src/gadgets/github/core/getPRDiff.ts)

  • New cliOnly: true flag support on ParameterDefinition — symmetric with gadgetOnly. Included in CLI + manifest (so prompts show it) but EXCLUDED from the SDK Gadget Zod schema. Gadget factory skips cliOnly params in buildZodSchema.
  • New outputFile parameter on getPRDiffDef (cliOnly). When set, getPRDiff() writes the full multiline Markdown payload to disk and returns a compact {outputFile, fileCount, bytes, pathFilter} summary instead of the raw text. Default behavior preserved.
  • Function overloads give callers the correct return type: Promise<string> without outputFile, Promise<PRDiffFileOutputSummary> with it.
  • Extracted pure helpers: filterByPath, formatPRDiffFile, formatPRDiffPayload — testable in isolation.

5. Review-agent context hints + docs

  • src/agents/shared/prFormatting.ts:formatSkippedFilesInjection — skipped-files guidance now suggests --outputFile /tmp/diff-<path>.md for large or one-line JSON diffs.
  • src/agents/prompts/templates/review.eta — review prompt mentions --outputFile, the one-stdin-consumer rule, and the safe --body-file <path> pattern for markdown bodies with backticks / code fences.
  • src/gadgets/README.md — documents the one-stdin-consumer rule, cliOnly parameter flag, and the get-pr-diff --outputFile reference.
  • docs/architecture/07-gadgets.md — new "Shell-safety contract (MNG-1059)" subsection covers all three changes.
  • CLAUDE.md/AGENTS.md — references the new contract from the review-context section.
  • CHANGELOG.md — Unreleased > Fixed entries for both clusters.

6. Regression coverage

  • Dual stdin rejection — tests in tests/unit/gadgets/shared/cli/params.test.ts and tests/unit/cli/file-input-flags.test.ts (real CreatePRReview) assert --body-file - --comments-file - emits the envelope and does NOT call the core function.
  • Single-stdin success--body-file - --comments-file /tmp/c.json and --body-file /tmp/b.md --comments-file - round-trip through the factory.
  • Shell-sensitive content preserved — backticks / $(...) / multi-line markdown reaches the core function byte-for-byte (CreatePR --body-file).
  • Prompt safetytests/unit/backends/shared-nativeToolPrompts.test.ts covers the new shell-safety rules section and the suppression of unsafe inline examples when a file companion exists.
  • Large diff outputtests/unit/gadgets/github/core/getPRDiff.test.ts exercises a synthesized ~100KB one-line JSON diff and asserts stdout summary stays <500 bytes while the file holds the full payload.
  • Manifest threadingtests/unit/gadgets/shared/factories.test.ts pins fileInputFor / fileInputAlternative flow through the manifest generator.
  • CLI wiringtests/unit/cli/scm/scm-commands.test.ts covers --outputFile via real GetPRDiff command.

All 256 focused tests pass. Full npm test passes (one unrelated flaky test in async-resolver.test.ts — passes on retry). npm run typecheck and npm run lint are clean.

Test plan

  • Run focused unit tests
  • Run prompt tests
  • npm run typecheck
  • npm run lint
  • npm test (full unit suite)

🕵️ claude-code · claude-opus-4-7 · run details

@codecov
Copy link
Copy Markdown

codecov Bot commented May 18, 2026

Codecov Report

❌ Patch coverage is 97.26027% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/agents/shared/prFormatting.ts 33.33% 2 Missing ⚠️
src/backends/shared/nativeToolPrompts.ts 86.66% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown
Collaborator

@nhopeatall nhopeatall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Requesting changes for two agent-facing contract issues: the new diff flag is documented with an invalid spelling in one prompt-visible description, and the CLI-only outputFile example still leaks into the SDK gadget examples.

Code Issues

Should Fix

  • src/gadgets/github/definitions.ts:307 — The GetPRDiff description tells agents to use --output-file, but the generated oclif flag is --outputFile. I verified the parser accepts --outputFile and rejects --output-file as a nonexistent flag, so this prompt/help text can send shell agents down the failure path for the exact escape hatch this PR adds.
  • src/gadgets/github/definitions.ts:364 — The new outputFile example is attached to the shared ToolDefinition, so createGadgetClass() copies it into the in-process GetPRDiff gadget examples even though buildZodSchema() strips cliOnly params. That means SDK gadget users can be shown an example with outputFile, but the schema removes it and the gadget returns the raw diff string instead of writing a file summary.

🕵️ codex · gpt-5.5 · run details

Comment thread src/gadgets/github/definitions.ts Outdated
name: 'GetPRDiff',
description:
'Get the unified diff of all file changes in a GitHub pull request. Shows each file with additions, deletions, and the patch content.',
'Get the unified diff of all file changes in a GitHub pull request. Shows each file with additions, deletions, and the patch content. For large diffs or one-line JSON patches that would truncate stdout, use --output-file <path> to write the full diff to disk and return a compact summary.',
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description is prompt/help-visible, but --output-file is not the flag this definition actually generates. The parameter key is outputFile, and the current parser accepts --outputFile while rejecting --output-file as a nonexistent flag. Either document --outputFile here or add output-file as an alias; otherwise agents following this guidance will hit the error path for the new large-diff escape hatch.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ddbf4d0. The prompt/help description now points at the canonical --outputFile spelling, and --output-file is accepted as an alias so either spelling works. I also added CLI coverage for the kebab-case alias.

repo: 'myapp',
prNumber: 42,
path: 'src/big-generated.json',
outputFile: '/tmp/pr-42-diff.md',
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this example lives on the shared ToolDefinition, it is also copied into the SDK Gadget examples by createGadgetClass(), but outputFile is marked cliOnly and is stripped from the gadget schema. A non-native gadget agent can therefore see and follow an outputFile example even though that parameter is removed before GetPRDiff.execute() runs, so it will get the raw diff string instead of the advertised file-output summary. The CLI-only example needs to be filtered out of gadget examples or represented in CLI-only metadata.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ddbf4d0. createGadgetClass() now filters examples that reference cliOnly params out of SDK gadget examples, so the outputFile example stays available to CLI/native-tool surfaces without leaking into the in-process gadget surface.

@zbigniewsobiecki
Copy link
Copy Markdown
Member

@aaight address code review concerns

@aaight
Copy link
Copy Markdown
Collaborator Author

aaight commented May 19, 2026

Addressed the code review concerns in ddbf4d0:

  • Corrected the GetPRDiff large-diff guidance to use --outputFile and added --output-file as a supported alias.
  • Filtered SDK gadget examples that reference cliOnly params so the outputFile example no longer leaks into GetPRDiff gadget examples.
  • Added regression tests for the alias and example filtering.

Verification: focused regression tests, npm run lint, npm run typecheck, full npm test, and pre-push test:fast all passed.

🕵️ codex · gpt-5.5 · run details

Copy link
Copy Markdown
Collaborator

@nhopeatall nhopeatall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Requesting changes for one prompt-visible recovery path: the new skipped-file command suggests an output file path that fails for typical changed files under directories.

Code Issues

Should Fix

  • src/agents/shared/prFormatting.ts:262 — The /tmp/diff-<path>.md template turns a skipped path like src/big.json into /tmp/diff-src/big.json.md. getPRDiff() writes with writeFileSync(outputFile, ...) and does not create parent directories, so an agent following this guidance gets an ENOENT runtime envelope instead of the full diff. Use a slash-safe filename template everywhere this guidance appears, or create parent directories before writing.

Focused tests passed locally for the new stdin guard, prompt rendering, manifest threading, and get-pr-diff output mode.

🕵️ codex · gpt-5.5 · run details

Comment thread src/agents/shared/prFormatting.ts Outdated
// the agent about the file-output escape hatch so it does not lose
// content to terminal truncation.
prNumber !== undefined
? ` • \`cascade-tools scm get-pr-diff --prNumber ${prNumber} --path <path> --outputFile /tmp/diff-<path>.md\` for large or one-line JSON diffs, then \`Read /tmp/diff-<path>.md\``
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This output path template breaks for the common case where the skipped file path contains a slash. Following it for src/big.json produces /tmp/diff-src/big.json.md; getPRDiff writes with writeFileSync and does not create /tmp/diff-src, so the recovery command fails with ENOENT instead of producing the diff. Please use a slash-safe template such as /tmp/pr-diff.md, or create parent directories before writing.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 185ecb0. I replaced the slash-derived /tmp/diff-<path>.md guidance with the slash-safe /tmp/pr-diff.md path in the skipped-files injection and review prompt/docs, and added unit coverage to keep that template from regressing.

@zbigniewsobiecki
Copy link
Copy Markdown
Member

@aaight address code review concerns

@aaight
Copy link
Copy Markdown
Collaborator Author

aaight commented May 19, 2026

Addressed the latest code review concern in 185ecb0:

  • Replaced the slash-unsafe /tmp/diff-<path>.md skipped-file recovery template with /tmp/pr-diff.md in the runtime injection, review prompt, CLAUDE/AGENTS guidance, and changelog.
  • Added a regression test for formatSkippedFilesInjection() to assert the slash-safe output path.

Verification: focused unit tests, npm run lint, npm run typecheck, full npm test, and pre-push test:fast all passed.

🕵️ codex · gpt-5.5 · run details

Copy link
Copy Markdown
Collaborator

@nhopeatall nhopeatall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I reviewed the updated MNG-1059 changes and the follow-up fixes. The stdin guard runs before file reads, the diff output escape hatch is kept to the CLI surface with SDK examples filtered, the skipped-file guidance now uses a stable slash-safe temp path, and live CI is green.

🕵️ codex · gpt-5.5 · run details

@zbigniewsobiecki zbigniewsobiecki merged commit 58fa89f into dev May 19, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants