Skip to content

[q] Enforce safe-output fallback for Q when tool permissions are blocked#36915

Merged
pelikhan merged 5 commits into
mainfrom
copilot/debug-q-workflow-failure
Jun 4, 2026
Merged

[q] Enforce safe-output fallback for Q when tool permissions are blocked#36915
pelikhan merged 5 commits into
mainfrom
copilot/debug-q-workflow-failure

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Jun 4, 2026

The Q workflow was succeeding at the job level but producing no safe outputs, which triggered [aw] Q failed incidents. In the failing run, the agent encountered repeated non-interactive permission denials and exited with plain text instead of a safe-output action.

  • Prompt contract hardening (.github/workflows/q.md)

    • Added a Safe Output Reliability section that explicitly requires:
      • stop retrying when repeated Permission denied and could not request permission from user errors occur
      • emit report_incomplete with blocked tool/command + exact error
      • always finish with a safe-output tool call (create-pull-request, add-comment, add-labels, noop, or report_incomplete)
  • Compiled workflow update (.github/workflows/q.lock.yml)

    • Regenerated lockfile to carry the source prompt contract changes into runtime behavior.
### Safe Output Reliability
- If repeated tool calls fail with `Permission denied and could not request permission from user`, stop retrying blocked tools and call `report_incomplete` with the blocked tool/command and exact error text.
- Never finish with plain text only. Every run must end with at least one safe-output call (`create-pull-request`, `add-comment`, `add-labels`, `noop`, or `report_incomplete`).

Copilot AI and others added 2 commits June 4, 2026 14:03
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Debug Q workflow failure to generate safe outputs [q] Enforce safe-output fallback for Q when tool permissions are blocked Jun 4, 2026
Copilot AI requested a review from pelikhan June 4, 2026 14:08
Comment thread pkg/cli/q_workflow_contract_test.go Outdated
@@ -0,0 +1,34 @@
//go:build !integration
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot delete test

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in dfbaf1f: I removed pkg/cli/q_workflow_contract_test.go from the PR.

@pelikhan pelikhan marked this pull request as ready for review June 4, 2026 14:17
Copilot AI review requested due to automatic review settings June 4, 2026 14:17
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI requested a review from pelikhan June 4, 2026 14:20
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens the Q workflow’s prompt contract to ensure it always emits a safe-output action, even when tool permissions are repeatedly denied, and adds a regression test to prevent the contract from drifting.

Changes:

  • Added “Safe Output Reliability” rules to .github/workflows/q.md, including a report_incomplete fallback on repeated permission denials and a “must end with safe output” requirement.
  • Added a workflow contract test (pkg/cli/q_workflow_contract_test.go) asserting the presence of the new reliability language and safe-output expectations.
  • Regenerated .github/workflows/q.lock.yml to propagate the updated prompt contract into the compiled workflow.
Show a summary per file
File Description
pkg/cli/q_workflow_contract_test.go Adds a contract test to guard Q’s prompt against missing safe-output fallback requirements.
.github/workflows/q.md Documents explicit handling for repeated permission denials and mandates ending with a safe-output call.
.github/workflows/q.lock.yml Updates compiled metadata/hash to reflect the prompt contract change at runtime.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 0

@pelikhan pelikhan merged commit 454875f into main Jun 4, 2026
14 of 17 checks passed
@pelikhan pelikhan deleted the copilot/debug-q-workflow-failure branch June 4, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Q failed

3 participants