Skip to content

Auto-close stale smoke-test issues when a newer run succeeds #22724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 2 commits into from
Mar 24, 2026
Merged

Auto-close stale smoke-test issues when a newer run succeeds #22724

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3e70e7d
Initial plan
Copilot Mar 24, 2026
f0a1c34
feat: add close-older-key to smoke workflows and fix smoke-codex to a…
Copilot Mar 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/smoke-claude.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions .github/workflows/smoke-claude.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ safe-outputs:
expires: 2h
group: true
close-older-issues: true
close-older-key: "smoke-claude"
labels: [automation, testing]
add-labels:
allowed: [smoke-claude]
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/smoke-codex.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 10 additions & 5 deletions .github/workflows/smoke-codex.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ safe-outputs:
create-issue:
expires: 2h
close-older-issues: true
close-older-key: "smoke-codex"
labels: [automation, testing]
add-labels:
allowed: [smoke-codex]
Expand Down Expand Up @@ -103,22 +104,26 @@ checkout:

## Output

**Check `${{ github.event_name }}` to determine the correct output action:**
**ALWAYS create an issue** with a summary of the smoke test run:
- Title: "Smoke Test: Codex - ${{ github.run_id }}"
- Body should include:
- Test results (✅ or ❌ for each test)
- Overall status: PASS or FAIL
- Run URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
- Timestamp
Comment on lines +107 to +113
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The updated Output instructions aim to ensure close-older-issues runs every time, but this section doesn’t explicitly say to use the create_issue safe-output tool (vs. describing it generically as “create an issue”). To reduce the chance the agent chooses a different output path, consider explicitly naming the create_issue tool as mandatory on every run and updating the example output in this section to demonstrate the create_issue invocation rather than noop.

This issue also appears on line 126 of the same file.

Copilot uses AI. Check for mistakes.

- **If `${{ github.event_name }}` is `pull_request`**: Use the `add_comment` safe-output tool to add a **very brief** comment (max 5-10 lines) to the triggering pull request, specifying `item_number: ${{ github.event.pull_request.number }}` (use this exact number — do NOT search GitHub for a PR):
**Only if this workflow was triggered by a `pull_request` event**: Additionally use the `add_comment` safe-output tool to add a **very brief** comment (max 5-10 lines) to the triggering pull request, specifying `item_number: ${{ github.event.pull_request.number }}` (use this exact number — do NOT search GitHub for a PR):
- PR titles only (no descriptions)
- ✅ or ❌ for each test result
- Overall status: PASS or FAIL

- **If `${{ github.event_name }}` is `schedule` or `workflow_dispatch`**: Call the `noop` safe-output tool with a brief summary of the test results (there is no pull request to comment on for scheduled or manually triggered runs).

If all tests pass and this workflow was triggered by a `pull_request` event:
- Use the `add_labels` safe-output tool to add the label `smoke-codex` to the pull request (use `item_number: ${{ github.event.pull_request.number }}`)
- Use the `remove_labels` safe-output tool to remove the label `smoke` from the pull request (use `item_number: ${{ github.event.pull_request.number }}`)
- Use the `unassign_from_user` safe-output tool to unassign the user `githubactionagent` from the pull request (this is a fictitious user used for testing; use `item_number: ${{ github.event.pull_request.number }}`)
- Use the `add_smoked_label` safe-output action tool to add the label `smoked` to the pull request (call it with `{"labels": "smoked", "number": "${{ github.event.pull_request.number }}"}`)

**Important**: You **MUST** always call exactly one safe-output tool. Failing to call any safe-output tool is the most common cause of safe-output workflow failures.
**Important**: If no action is needed after completing your analysis, you **MUST** call the `noop` safe-output tool with a brief explanation. Failing to call any safe-output tool is the most common cause of safe-output workflow failures.

```json
{"noop": {"message": "No action needed: [brief explanation of what was analyzed and why]"}}
Expand Down
Loading
Loading