Skip to content

Add default gh aw fix codemod to lowercase discussion trigger categories in frontmatter#31872

Merged
pelikhan merged 8 commits into
mainfrom
copilot/add-lowercase-discussion-category-codemod
May 13, 2026
Merged

Add default gh aw fix codemod to lowercase discussion trigger categories in frontmatter#31872
pelikhan merged 8 commits into
mainfrom
copilot/add-lowercase-discussion-category-codemod

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 13, 2026

✨ Enhancement

gh-aw already normalizes discussion category/type strings to lowercase at compile time, but workflow source could remain mixed-case and drift from runtime behavior. This adds a default-on codemod so gh aw fix --write rewrites source to the same lowercase form the compiler uses.

  • What does this improve?

    • Aligns source frontmatter with runtime-normalized discussion trigger values.
    • Eliminates confusing mismatches like Agentic Workflows in source vs agentic workflows at runtime.
  • Why is this valuable?

    • Reduces config drift and makes trigger behavior directly readable from source.
    • Applies automatically in the existing gh aw fix codemod pipeline (no opt-in flag).
  • Implementation approach:

    • Added new codemod: discussion-trigger-categories-lowercase.
    • Targets mixed-case string values under:
      • on.discussion.types
      • on.discussion_comment.types
    • Performs in-place lowercasing while preserving YAML structure/formatting.
    • No-op when values are already lowercase.
    • Registered codemod in the default GetAllCodemods() sequence.
    • Added focused tests for:
      • mixed-case rewrite
      • already-lowercase no-op
      • codemod metadata/registration expectations
# before
on:
  discussion:
    types:
      - Agentic Workflows
  discussion_comment:
    types: [General]

# after (gh aw fix --write)
on:
  discussion:
    types:
      - agentic workflows
  discussion_comment:
    types: [general]

Copilot AI and others added 2 commits May 13, 2026 05:50
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add codemod to lowercase discussion-category values in workflow source Add default gh aw fix codemod to lowercase discussion trigger categories in frontmatter May 13, 2026
Copilot AI requested a review from pelikhan May 13, 2026 05:53
@pelikhan pelikhan marked this pull request as ready for review May 13, 2026 05:56
Copilot AI review requested due to automatic review settings May 13, 2026 05:56
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new default-on gh aw fix codemod that rewrites discussion trigger types values in workflow frontmatter to lowercase so the source matches the compiler’s runtime-normalized behavior.

Changes:

  • Registered a new codemod (discussion-trigger-categories-lowercase) in the default GetAllCodemods() pipeline.
  • Implemented a frontmatter line transformer to lowercase values under on.discussion.types and on.discussion_comment.types (block and inline list forms).
  • Added unit tests covering mixed-case rewrites, no-op behavior, and registry/order expectations.
Show a summary per file
File Description
pkg/cli/fix_codemods.go Registers the new codemod in the default codemod sequence.
pkg/cli/fix_codemods_test.go Updates codemod ID/order expectations to include the new codemod.
pkg/cli/codemod_discussion_trigger_categories.go Implements the new codemod and the YAML-frontmatter line transformation logic.
pkg/cli/codemod_discussion_trigger_categories_test.go Adds focused tests for codemod metadata and rewrite/no-op behavior.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

pkg/cli/codemod_discussion_trigger_categories.go:143

  • The trigger/type detection is overly strict: it requires trimmed == "discussion:"/"discussion_comment:" and treats types: as a block list only when the remainder after types: is empty. This means common YAML like discussion: # comment or types: # comment followed by - ... items will not be rewritten. Consider matching these keys with HasPrefix("discussion:") / HasPrefix("types:") and treating a remainder that begins with # as a block-list start.
		if len(indent) > len(onIndent) && (trimmed == "discussion:" || trimmed == "discussion_comment:") {
			currentTrigger = strings.TrimSuffix(trimmed, ":")
			triggerIndent = indent
			inTypes = false
			continue
		}

		if currentTrigger == "" {
			continue
		}

		if inTypes && len(indent) <= len(typesIndent) {
			inTypes = false
		}

		if strings.HasPrefix(trimmed, "types:") {
			typesIndent = indent
			afterColon := strings.TrimSpace(strings.TrimPrefix(trimmed, "types:"))
			if strings.HasPrefix(afterColon, "[") && strings.HasSuffix(afterColon, "]") {
				updatedLine, changed := lowercaseInlineTypesArrayLine(line)
				if changed {
					result[i] = updatedLine
					modified = true
				}
				inTypes = false
			} else if afterColon == "" {
				inTypes = true
			}
			continue
  • Files reviewed: 4/4 changed files
  • Comments generated: 1

Comment on lines +92 to +97
if isTopLevelKey(line) && strings.HasPrefix(trimmed, "on:") {
inOn = true
onIndent = indent
currentTrigger = ""
inTypes = false
continue
@github-actions github-actions Bot mentioned this pull request May 13, 2026
Generated by Design Decision Gate workflow. Documents the decision to
add a default-on codemod that lowercases discussion trigger category
values so source frontmatter matches compile-time normalized form.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

Commit pushed: 7a83339

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (354 new lines under pkg/cli/) but does not have a linked Architecture Decision Record (ADR).

AI has analyzed the PR diff and generated a draft ADR to help you get started:

📄 Draft ADR: docs/adr/31872-lowercase-discussion-trigger-categories-codemod.md

What to do next

  1. Review the draft ADR committed to your branch — it was generated from the PR diff.
  2. Complete the missing sections — add context the AI couldn't infer, refine the decision rationale, and list any additional alternatives you actually considered.
  3. Commit the finalized ADR to docs/adr/ on your branch.
  4. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-31872: Lowercase Discussion Trigger Categories via Default Codemod

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

Why ADRs Matter

"AI made me procrastinate on key design decisions. Because refactoring was cheap, I could always say 'I'll deal with this later.' Deferring decisions corroded my ability to think clearly."

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you.

📋 What's in the draft ADR

The draft captures the decision to add discussion-trigger-categories-lowercase as a default-on codemod in GetAllCodemods(), including:

  • Context: source-vs-runtime case drift for on.discussion.types and on.discussion_comment.types.
  • Decision: in-place line-based lowercasing in the existing gh aw fix --write pipeline.
  • Alternatives Considered: compile-time error only; structural YAML round-trip; opt-in flag.
  • Consequences: matches runtime behavior and preserves formatting, but is more brittle than a full YAML parse and rewrites source silently.
  • Normative Spec (RFC 2119): registration, transformation scope (block and inline lists, expression skip, quoting/comment preservation), idempotence/no-op behavior, and error handling.

Please verify each section reflects your actual reasoning — especially the alternatives you weighed and any consequences I may have missed.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context — What is the problem? What forces are at play?
  • Decision — What did you decide? Why?
  • Alternatives Considered — What else could have been done?
  • Consequences — What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number (e.g., 31872-...md for PR #31872).

🔒 This PR cannot merge until the ADR is reviewed, finalized, and linked in the PR body.

References:

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · ● 4.9M ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /tdd — this is a new feature addition with accompanying tests, so test quality and behavioural coverage are the right lens.

Key Themes

  • Expression-guard inconsistency (correctness bug): lowercaseYAMLListItemLine correctly skips values containing ${{, but lowercaseInlineTypesArrayLine does not. Any inline-array entry that is a GitHub Actions expression would be silently lowercased. This is the most important issue to fix before merge.
  • Comment-detection fragility: Using strings.Index(line, "#") to find inline comments will misfire on a # inside a quoted value. Using # (space + hash) is a safer heuristic that matches standard YAML convention.
  • Test coverage gaps: The two existing tests cover the happy path and the no-op path, but the inline-array specific code paths (quoted values, expression guards) have no test coverage. The expression-guard bug above would have been caught with such a test.

Positive Highlights

  • ✅ Clean two-phase design: frontmatter parse to decide whether to apply, then line transform to do the work — consistent with every other codemod in the package.
  • ✅ Correct build-tag (//go:build !integration) and use of require vs assert in tests.
  • ✅ No-op guard in hasMixedCaseDiscussionTypeValues avoids unnecessary line-by-line scanning for files that need no change.
  • ✅ Registration in GetAllCodemods() and the order test updated together — no drift.

Verdict

Requesting changes on the ${{ expression guard and a couple of targeted tests before merge.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · ● 6.2M

for i, part := range parts {
trimmed := strings.TrimSpace(part)
unquoted := strings.Trim(trimmed, `"'`)
lower := strings.ToLower(unquoted)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] lowercaseInlineTypesArrayLine does not guard against ${{ expressions, but its sibling lowercaseYAMLListItemLine does (line 240). A category value like types: [${{ env.CATEGORY }}] would be silently mangled.

Apply the same guard used in the list-item path:

for i, part := range parts {
    trimmed := strings.TrimSpace(part)
    if strings.Contains(trimmed, "${{") {
        continue  // leave expressions untouched
    }
    // ...
}

A matching test case (types: [${{ env.CATEGORY }}] → no-op) would confirm the fix is regression-proof.

}

func lowercaseInlineTypesArrayLine(line string) (string, bool) {
commentIndex := strings.Index(line, "#")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] Comment detection via strings.Index(line, "#") will false-positive on a # inside a quoted value — e.g. types: ["#general"] would treat #general"] as a comment, producing a garbled replacement.

The same issue affects the list-item path implicitly, but is less likely there. Consider searching for # (space+hash) instead, which is the conventional inline-comment separator in YAML:

commentIndex := strings.Index(line, " #")

Or alternatively, only search for # outside any quoted segment.

result, applied, err := codemod.Apply(content, frontmatter)

require.NoError(t, err)
assert.True(t, applied)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The tests cover the two main paths (mixed-case rewrite, already-lowercase no-op) but miss the inline-array edge cases that the new code explicitly handles:

  1. Quoted valuestypes: ["General", 'Special'] should become types: ["general", 'special']
  2. Expression guardtypes: [${{ env.CATEGORY }}] should be left unchanged

These scenarios exercise distinct branches in lowercaseInlineTypesArrayLine and would have caught the missing ${{ guard mentioned in the companion comment.

@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 77/100

⚠️ Acceptable, but check failed on implementation-test ratio

Metric Value
New/modified tests analyzed 3
✅ Design tests (behavioral contracts) 2 (66.7%)
⚠️ Implementation tests (low value) 1 (33.3%)
Tests with error/edge cases 2 (66.7%)
Duplicate test clusters 0
Test inflation detected No (87 test lines / 264 production lines = 0.33)
🚨 Coding-guideline violations ⚠️ Missing assertion messages in new test file

Test Classification Details

View per-test classification table
Test File Classification Issues Detected
TestGetDiscussionTriggerCategoriesLowercaseCodemod codemod_discussion_trigger_categories_test.go:12 ⚠️ Implementation Only checks struct field values; no edge/error case; missing assertion messages
TestDiscussionTriggerCategoriesCodemod_LowercasesMixedCaseValues codemod_discussion_trigger_categories_test.go:20 ✅ Design Verifies observable output transformation; includes require.NoError; missing assertion messages
TestDiscussionTriggerCategoriesCodemod_NoOpWhenAlreadyLowercase codemod_discussion_trigger_categories_test.go:55 ✅ Design Verifies idempotency/no-op edge case; missing assertion messages

Flagged Tests — Requires Review

⚠️ TestGetDiscussionTriggerCategoriesLowercaseCodemod (codemod_discussion_trigger_categories_test.go:12)

Classification: Implementation test

Issues: Only asserts on struct field values (ID, Name, Description, IntroducedIn) without calling Apply at all. This verifies that string constants were not mistyped, but exercises no behavioral logic.

What design invariant does this test enforce? Minimal — it checks that metadata fields are set, but the codemod's actual contract (lowercasing discussion category strings) is entirely untested here.

What would break if deleted? Only if someone mistyped a string constant. Both behavioral tests below already verify the codemod works end-to-end.

Suggested improvement: Either remove this test (the behavioral tests below already instantiate the codemod) or expand it to verify a specific invariant, e.g., that the codemod ID is stable across calls (important for migration tooling). At minimum, add descriptive assertion messages.


⚠️ Missing assertion messages in codemod_discussion_trigger_categories_test.go

The project guideline requires a descriptive message argument on every testify assertion. All assertions in this new file use the bare form:

// ❌ Missing message
assert.Equal(t, "discussion-trigger-categories-lowercase", codemod.ID)
assert.Contains(t, result, "- agentic workflows")

Fix by adding descriptive context strings:

// ✅ With message
assert.Equal(t, "discussion-trigger-categories-lowercase", codemod.ID, "codemod ID must be stable")
assert.Contains(t, result, "- agentic workflows", "mixed-case type should be lowercased in output")

Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 3 tests — unit (//go:build !integration) ✅
  • 🟨 JavaScript (*.test.cjs, *.test.js): 0 tests

Verdict

Check failed. 33.3% of new tests are classified as low-value implementation tests, exceeding the 30% threshold. The TestGetDiscussionTriggerCategoriesLowercaseCodemod test checks only struct metadata without exercising any behavioral logic. Additionally, all assertions in the new test file are missing the required descriptive message argument.

Note: The two behavioral tests (LowercasesMixedCaseValues and NoOpWhenAlreadyLowercase) are well-designed and cover the primary contract and an important idempotency edge case — the fixes needed are minor.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does — observable outputs, state changes, error handling. They remain valid after internal refactoring.

Implementation Tests (Low Value) verify how the system does it — struct field values, internal function calls, constants. They give false assurance: they pass even when the system is wrong.

Goal: Shift toward tests that describe the system's behavioral contract.

References: §25781271674

🧪 Test quality analysis by Test Quality Sentinel · ● 7.1M ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Test Quality Sentinel: 77/100. 33.3% of new tests are classified as low-value implementation tests, exceeding the 30% threshold. The TestGetDiscussionTriggerCategoriesLowercaseCodemod test checks only struct metadata without exercising any behavioral logic. Additionally, all assertions in the new test file are missing the required descriptive message arguments. Please review the flagged tests in the comment above.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review all comments

Copilot AI and others added 2 commits May 13, 2026 09:40
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 13, 2026

@copilot review all comments

Addressed the actionable review feedback in commit a3ffb9a. The codemod now handles quoted "on" keys and quoted discussion trigger keys with inline comments/spaces, and tests were added to cover these cases.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot merge main and recompile

Copilot AI and others added 2 commits May 13, 2026 10:11
…add-lowercase-discussion-category-codemod

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan merged commit 3abf9ce into main May 13, 2026
@pelikhan pelikhan deleted the copilot/add-lowercase-discussion-category-codemod branch May 13, 2026 10:14
Copilot stopped work on behalf of pelikhan due to an error May 13, 2026 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[plan] Codemod: lowercase discussion-category values in workflow source

3 participants