Add regression tests for Copilot SDK shell-permission parsing on issue command shapes#37574
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot add logging in parser to explain rejection |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Addressed in commits 57a19c5 and 9b1a931. I reviewed issue #37538 and added regression coverage for the exact denied command shapes both for jq-only denial and for approval when the corresponding shell permissions are granted, using the SDK |
There was a problem hiding this comment.
Pull request overview
Adds focused regression coverage for the Copilot SDK permission handler’s fullCommandText fallback parsing when commands: [] is provided, ensuring real-world “issue command shapes” are correctly rejected under a narrow shell allowlist and accepted when corresponding shell permissions are granted.
Changes:
- Added a regression test that denies non-
jqshell commands when policy is restricted toshell(jq:*). - Added a regression test that allows the same command shapes when the workflow grants the needed shell permissions.
Show a summary per file
| File | Description |
|---|---|
| actions/setup/js/copilot_sdk_driver.test.cjs | Adds regression tests covering deny/allow behavior for specific complex shell command shapes when SDK identifiers are omitted. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 2
| expect( | ||
| handler({ | ||
| kind: "shell", | ||
| // Intentional: exercise fullCommandText fallback when SDK omits identifiers. | ||
| commands: [], | ||
| fullCommandText: command, | ||
| }) | ||
| ).toEqual({ kind: "reject", feedback: "Tool invocation is not allowed by workflow tool permissions." }); |
| expect( | ||
| handler({ | ||
| kind: "shell", | ||
| // Intentional: exercise fullCommandText fallback when SDK omits identifiers. | ||
| commands: [], | ||
| fullCommandText: command, | ||
| }) | ||
| ).toEqual({ kind: "approve-once" }); |
The Copilot SDK permission parser was reviewed against the denied command set from the failing draft-pr-cleanup run, including repo-style path tokens (
github/gh-aw) inside shell arguments. This change adds focused regression coverage to ensure those commands are denied under a restricted policy and allowed when corresponding permissions are granted.Problem scope
shell(jq:*)), so non-jqcommands should be rejected regardless of argument structure.Change
actions/setup/js/copilot_sdk_driver.test.cjs:denies issue-37538 commands when workflow only allows jq shell usageallows issue-37538 commands when corresponding shell permissions are grantedgh ...,safeoutputs ... | head ...,git ... && gh ...,echo "test"), withcommands: []to intentionally exercise the parser fallback path overfullCommandTextwhen SDK identifiers are omitted.Behavior locked in