Skip to content

Require noop call in Multi-Device Docs Tester when no issues found#14553

Merged
pelikhan merged 2 commits intomainfrom
copilot/debug-multi-device-tester
Feb 8, 2026
Merged

Require noop call in Multi-Device Docs Tester when no issues found#14553
pelikhan merged 2 commits intomainfrom
copilot/debug-multi-device-tester

Conversation

Copy link
Contributor

Copilot AI commented Feb 8, 2026

Workflow run #21801448336 succeeded but triggered a failure issue because the agent produced no safe outputs. The prompt instructed to "provide summary" and create an issue "if issues are detected" but didn't specify what to do when no issues exist.

Changes

  • Added explicit noop requirement when no issues are detected:
    ### If NO Issues Found
    
    **YOU MUST CALL** the `noop` tool to log completion:
    
    ```json
    {
      "noop": {
        "message": "Multi-device documentation testing complete. All {device_count} devices tested successfully with no issues found."
      }
    }
    
    
  • Restructured reporting section to separate "no issues" from "issues found" paths
  • Updated summary to require safe output in all cases (issue creation OR noop)

Pattern Alignment

Matches existing workflows that explicitly require noop:

  • breaking-change-checker.md - "YOU MUST CALL the noop tool"
  • daily-malicious-code-scan.md - noop or alert required
  • daily-cli-tools-tester.md - "use noop if all tests pass"

The workflow now guarantees outputs.jsonl is written, preventing "agent succeeded but produced no safe outputs" failures.

Original prompt

This section details on the original issue you should resolve

<issue_title>[agentics] Multi-Device Docs Tester failed</issue_title>
<issue_description>### Workflow Failure

Workflow: Multi-Device Docs Tester
Branch: main
Run URL: https://github.com/github/gh-aw/actions/runs/21801448336

⚠️ No Safe Outputs Generated: The agent job succeeded but did not produce any safe outputs. This typically indicates:

  • The safe output server failed to run
  • The prompt failed to generate any meaningful result
  • The agent should have called noop to explicitly indicate no action was taken

Action Required

Debug this workflow failure using the agentic-workflows agent:

/agent agentic-workflows

When prompted, instruct the agent to debug this workflow failure.

Generated from Multi-Device Docs Tester

  • expires on Feb 15, 2026, 4:32 PM UTC

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Debug workflow failure for multi-device docs tester Require noop call in Multi-Device Docs Tester when no issues found Feb 8, 2026
Copilot AI requested a review from pelikhan February 8, 2026 19:36
@pelikhan pelikhan marked this pull request as ready for review February 8, 2026 19:38
Copilot AI review requested due to automatic review settings February 8, 2026 19:38
@pelikhan pelikhan merged commit 9cfb43f into main Feb 8, 2026
3 checks passed
@pelikhan pelikhan deleted the copilot/debug-multi-device-tester branch February 8, 2026 19:38
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Multi-Device Docs Tester agentic workflow prompt to ensure the run always produces a safe output (avoiding “no safe outputs” failures), especially when no issues are found.

Changes:

  • Adds an explicit requirement to call the noop tool when no issues are detected.
  • Splits reporting guidance into “no issues” vs “issues found” paths and updates the summary to emphasize safe outputs.
Comments suppressed due to low confidence (1)

.github/workflows/daily-multi-device-docs-tester.md:179

  • This instruction is now unconditional (“Create a GitHub issue…”). Given the new “If NO Issues Found” branch, this should remain explicitly conditional (e.g., “If issues are found, create…”) and ideally also state not to create an issue when clean. Otherwise the agent may create a report issue even on successful/no-issue runs, defeating the purpose of the noop path.
Create a GitHub issue titled "🔍 Multi-Device Docs Testing Report - [Date]" with:


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

```json
{
"noop": {
"message": "Multi-device documentation testing complete. All {device_count} devices tested successfully with no issues found."
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The noop message uses a {device_count} placeholder that isn’t defined anywhere else in this workflow (the issue template uses {count}) and may end up being logged literally. Prefer using a single placeholder consistently (or instruct the agent to substitute the actual numeric count) so the noop output is always clear and accurate.

Suggested change
"message": "Multi-device documentation testing complete. All {device_count} devices tested successfully with no issues found."
"message": "Multi-device documentation testing complete. All {count} devices tested successfully with no issues found."

Copilot uses AI. Check for mistakes.

### If Issues ARE Found

## 📝 Report Formatting Guidelines
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The heading structure is now inconsistent: ### If Issues ARE Found is immediately followed by a higher-level ## 📝 Report Formatting Guidelines. This makes the “issues found” branch look like it ends early and can confuse the agent about which instructions are conditional. Consider demoting the guidelines heading (e.g., to ####) or moving it so it’s clearly nested under the “issues found” path.

This issue also appears on line 178 of the same file.

Suggested change
## 📝 Report Formatting Guidelines
#### 📝 Report Formatting Guidelines

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[agentics] Multi-Device Docs Tester failed

2 participants