Skip to content

Upgrade test workflow: release-created trigger, prerelease coverage, and failure conclusion/issue flow#27372

Merged
pelikhan merged 2 commits intomainfrom
copilot/add-conclusion-job-for-tests
Apr 20, 2026
Merged

Upgrade test workflow: release-created trigger, prerelease coverage, and failure conclusion/issue flow#27372
pelikhan merged 2 commits intomainfrom
copilot/add-conclusion-job-for-tests

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 20, 2026

The extension upgrade workflow only validated stable/latest paths and had no centralized post-matrix conclusion handling for release runs. This update adds release-created coverage (including prereleases), aggregates matrix outcomes, and opens a failure issue when release validation fails.

  • Trigger + scope

    • Add release: { types: [created] } to run upgrade validation on newly created releases.
    • Keep existing workflow_dispatch and PR-path trigger for iterative development.
  • Release selection logic (includes prereleases)

    • Replace stable-only /releases/latest behavior with release-list selection that includes non-draft prereleases.
    • On release events, use github.event.release.tag_name as the upgrade target and select the next most recent non-draft release as n-1.
  • Conclusion job with failure escalation

    • Add conclusion job (if: always()) to collect per-OS matrix outcomes from the run.
    • Publish a compact failure summary to job summary output.
    • For release-triggered runs with failures, create a GitHub issue containing failed jobs and run URL.
  • Permission minimization

    • Set workflow-level permissions to empty (permissions: {}).
    • Scope permissions per job:
      • test job: contents: read
      • conclusion job: actions: read, contents: read, issues: write
on:
  workflow_dispatch: {}
  release:
    types: [created]

permissions: {}

jobs:
  conclusion:
    if: always()
    permissions:
      actions: read
      contents: read
      issues: write

Copilot AI and others added 2 commits April 20, 2026 14:04
@pelikhan pelikhan marked this pull request as ready for review April 20, 2026 15:11
Copilot AI review requested due to automatic review settings April 20, 2026 15:11
@pelikhan pelikhan merged commit 8ba1171 into main Apr 20, 2026
@pelikhan pelikhan deleted the copilot/add-conclusion-job-for-tests branch April 20, 2026 15:11
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Extension Upgrade Test workflow to validate upgrade paths for newly created releases (including prereleases) and to centralize post-matrix conclusion handling with failure escalation.

Changes:

  • Add release: { types: [created] } trigger and broaden release selection to include non-draft prereleases.
  • Add a conclusion job that aggregates matrix job outcomes and writes a compact summary.
  • Create a GitHub issue on release-triggered failures, with scoped per-job permissions under workflow-level permissions: {}.
Show a summary per file
File Description
.github/workflows/upgrade-test.yml Adds release-created trigger, prerelease-aware release selection, and a conclusion/issue-escalation flow with minimized permissions.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 1/1 changed files
  • Comments generated: 2

on:
workflow_dispatch: {}
release:
types: [created]
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on: release: types: [created] will also run for draft releases. Since the workflow later assumes the target tag is installable, draft releases are likely to fail/spam issues. Consider switching the trigger to types: [published] (still includes prereleases) or add a job-level guard like if: github.event.release.draft == false for the release path.

Suggested change
types: [created]
types: [published]

Copilot uses AI. Check for mistakes.
EOF
)

gh issue create --title "$TITLE" --body "$BODY" --label "bug"
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gh issue create is executed in the conclusion job without checking out the repo or specifying a repository, so gh may not know which repo to operate on and can fail. Pass --repo "${{ github.repository }}" (or set GH_REPO) or add a checkout step before creating the issue.

Suggested change
gh issue create --title "$TITLE" --body "$BODY" --label "bug"
gh issue create --repo "${{ github.repository }}" --title "$TITLE" --body "$BODY" --label "bug"

Copilot uses AI. Check for mistakes.
@github-actions github-actions bot mentioned this pull request Apr 20, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

i️ Scope note: This PR modifies only .github/workflows/upgrade-test.yml — a GitHub Actions workflow file. The Test Quality Sentinel analyzes Go (*_test.go) and JavaScript (*.test.cjs, *.test.js) test files. No files matching those patterns were added or modified in this PR, so no Test Quality Score is computed.


Workflow-as-Test Observations

The changed file (.github/workflows/upgrade-test.yml) functions as an end-to-end integration test for the gh aw extension upgrade path. While outside the sentinel's scoring scope, the changes are qualitatively positive:

Change Behavioral Impact
Added release: types: [created] trigger ✅ Test now runs automatically on every release — closes the gap where the workflow only ran on workflow_dispatch or PR changes
Prerelease support (select(.draft == false) instead of select(.prerelease == false and .draft == false)) ✅ Broadens coverage to include prerelease tags
Event-aware target selection (github.event_name == 'release' branch) ✅ On release events, uses the exact released tag as target, improving determinism
VERSION_REGEX env var for prerelease semver matching ✅ Handles vX.Y.Z-rc.1 and similar pre-release formats correctly
New conclusion job ✅ Aggregates per-matrix outcomes into a single observable conclusion; provides step summary
Create issue for release failures step ✅ Auto-files a bug issue when a release-triggered run fails — improves observability/alerting

Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 0 tests
  • 🟨 JavaScript (*.test.cjs, *.test.js): 0 tests

i️ The only changed file (.github/workflows/upgrade-test.yml) is a GitHub Actions workflow and is outside the current analysis scope.


Verdict

No traditional test files were modified. Test Quality Sentinel skipped scoring. The workflow changes appear well-structured and improve both coverage (prerelease support, automatic release trigger) and observability (conclusion job + auto-issue creation).


References:

🧪 Test quality analysis by Test Quality Sentinel · ● 428.3K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: No Go or JavaScript test files were modified in this PR. Analysis scope (Go *_test.go, JavaScript *.test.cjs/*.test.js) did not apply. The workflow changes look well-structured — see comment for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants