Skip to content

fix(ci): fix commit message check and summary job name keys#3

Merged
LucienSong merged 2 commits intocopilot/add-github-actions-pr-review-workflowfrom
copilot/fix-commit-message-check
Mar 23, 2026
Merged

fix(ci): fix commit message check and summary job name keys#3
LucienSong merged 2 commits intocopilot/add-github-actions-pr-review-workflowfrom
copilot/fix-commit-message-check

Conversation

Copy link
Contributor

Copilot AI commented Mar 23, 2026

Two bugs in .github/workflows/pr-review.yml cause the PR Review workflow to fail on PRs created by the Copilot coding agent.

Changes

  • commit-message-check: Add a skip for commits whose subject is exactly "Initial plan" or "Initial commit" — auto-generated by the Copilot agent and not subject to conventional commit enforcement.
# Skip known bot/agent auto-generated commits
if echo "$msg" | grep -qE "^Initial (plan|commit)$"; then
  continue
fi
  • summary job: Fix job name keys in the for loop — they used underscores (security_audit, commit_message_check, pr_description_check) but the needs context keys mirror the YAML job IDs which use hyphens. The mismatch caused those jobs to always resolve as unknown, silently masking real failures.
# Before
for job in format lint test docs security_audit build commit_message_check pr_description_check

# After
for job in format lint test docs security-audit build commit-message-check pr-description-check

Type of Change

  • Bug Fix

Checklist

  • Code follows the project's style guidelines (cargo fmt passes)
  • No Clippy warnings (cargo clippy -- -D warnings passes)
  • Unit tests have been added or modified (cargo test passes)
  • Documentation has been updated (cargo doc --no-deps passes)
  • No sensitive information (keys, secrets, PII) is exposed
  • Commit messages follow the conventional commit format (e.g., feat:, fix:, chore:)

Related Issues

Relates to #2

Additional Notes

No source code changes — workflow YAML only.

Original prompt

Problem

The PR Review workflow in PR #2 (branch copilot/add-github-actions-pr-review-workflow) is failing. The workflow run at https://github.com/ShellDAO/shell-chain/actions/runs/23417589365 shows conclusion: failure.

Root Cause Analysis

There are two bugs in .github/workflows/pr-review.yml that need to be fixed:

Bug 1: Commit Message Check fails on Copilot agent's "Initial plan" commit

The commit-message-check job validates ALL commits between base.sha and head.sha using git log --format="%s" "${BASE_SHA}..${HEAD_SHA}".

The PR has 2 commits:

  1. 45aaaa5 — message: "Initial plan" (created automatically by Copilot coding agent)
  2. cd98c9e — message: "ci: add automated PR review workflow, PR template, and script header" (actual work commit)

The first commit "Initial plan" does NOT follow conventional commit format and fails the check.

Fix: The commit message validation script should also skip commits that are auto-generated by bots/agents (e.g., messages like "Initial plan" from Copilot coding agent). The simplest robust fix is to make the regex pattern also skip known bot-generated commit messages. A better approach: also skip commits where the message is exactly "Initial plan" (a known Copilot agent auto-commit). Alternatively and more robustly, add a pattern to also skip commits containing "Agent-Logs-Url" in the full message body OR whose subject is "Initial plan".

The best approach: In the commit-message-check job, modify the validation script to:

  1. Use git log --format="%s" "${BASE_SHA}..${HEAD_SHA}" as before but also skip commits whose subject line matches known automated bot patterns like "Initial plan", "Initial commit", or any message starting with "Merge " (already handled).
  2. Add a skip for the pattern: messages that are exactly "Initial plan" or "Initial commit".

Here's the specific change needed in the validation script (the while loop):

# Skip merge commits
if echo "$msg" | grep -qE "^Merge "; then
  continue
fi
# Skip known bot/agent auto-generated commits
if echo "$msg" | grep -qE "^Initial (plan|commit)$"; then
  continue
fi

Bug 2: Summary job uses wrong job name keys

In the summary job's "Check all jobs passed" step, the Python script iterates over job names using underscores:

for job in format lint test docs security_audit build commit_message_check pr_description_check; do

But the needs context from GitHub Actions uses the YAML job IDs which contain hyphens (not underscores):

  • security-audit (not security_audit)
  • commit-message-check (not commit_message_check)
  • pr-description-check (not pr_description_check)

So the Python lookup data['security_audit'] will never find the key security-audit, causing those jobs to report unknown status instead of detecting failures.

Fix: Change the for loop to use the correct hyphenated job names:

for job in format lint test docs security-audit build commit-message-check pr-description-check; do

And update the Python script to handle hyphenated keys properly (they need to be passed as strings, not bare shell variables with hyphens):

key = '$job'
# Handle the key as-is since it may contain hyphens
if key in data:
    print(data[key]['result'])
else:
    print('unknown')

Actually, since $job contains hyphens and is substituted as a string literal inside the Python code, it should work fine as '$job' already wraps it in quotes. The issue is just the underscore vs hyphen mismatch in the bash for loop.

Files to modify

Only one file needs to be changed: .github/workflows/pr-review.yml

Summary of changes needed

  1. In the commit-message-check job's "Validate conventional commit messages" step, add a skip for "Initial plan" and "Initial commit" messages (right after the existing merge commit skip).

  2. In the summary job's "Check all jobs passed" step, change the for loop from:

    for job in format lint test docs security_audit build commit_message_check pr_description_check; do
    

    to:

    for job in format lint test docs security-audit build commit-message-check pr-description-check; do
    

Make sure the commit message for this fix follows conventional commit format, e.g.: fix(ci): fix commit message check and summary job name keys

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: @ShellDAO/shell-chain/pull/2 PR review 没有通过,你去修复

Assistant:

Let me first look at the PR details and understand what failed in the review.

Assistant: 我已经找到了失败的原因。让我查看具体的 workflow 文件中 summary job 的逻辑,确认所有需要修复的问题。

Assistant: 我已经找到了 CI 失败的根本原因。让我来分析并创建修复 PR。

🔍 失败原因分析

PR Review workflow run 失败在 Commit Message Check job。

错误日志:

##[error]Commit message does not follow conventional format: "Initial plan"

PR #2 有 2 个 c...

This pull request was created from Copilot chat.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

@LucienSong LucienSong marked this pull request as ready for review March 23, 2026 01:49
@LucienSong LucienSong self-requested a review as a code owner March 23, 2026 01:49
Copilot AI changed the title [WIP] Fix commit message validation for auto-generated commits fix(ci): fix commit message check and summary job name keys Mar 23, 2026
@LucienSong LucienSong merged commit ad34238 into copilot/add-github-actions-pr-review-workflow Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants