Skip to content

Allow configuring conclusion failure issue expiration via aw.json#26688

Merged
pelikhan merged 10 commits intomainfrom
copilot/configure-expiration-time-action-failures
Apr 16, 2026
Merged

Allow configuring conclusion failure issue expiration via aw.json#26688
pelikhan merged 10 commits intomainfrom
copilot/configure-expiration-time-action-failures

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 16, 2026

Summary

  • add repository-level aw.json support for configuring expiration of action failure issues created by the conclusion job
  • propagate the configured value into the conclusion job as GH_AW_ACTION_FAILURE_ISSUE_EXPIRES_HOURS
  • update handle_agent_failure.cjs to use the configured expiration for both grouped parent issues and per-run failure issues
  • set this repository’s .github/workflows/aw.json value to 12 hours per feedback

Changes

  • extend .github/workflows/aw.json schema with maintenance.action_failure_issue_expires (integer hours, minimum 1)
  • extend typed repo config parsing/accessors in pkg/workflow/repo_config.go
  • update conclusion job generation in pkg/workflow/notify_comment.go to load repo config and pass expiration env var with default fallback (168h)
  • add/extend tests in:
    • pkg/workflow/repo_config_test.go
    • pkg/workflow/notify_comment_test.go
    • actions/setup/js/handle_agent_failure.test.cjs
  • update docs in docs/src/content/docs/guides/ephemerals.md
  • add repository config file .github/workflows/aw.json with:
    • maintenance.action_failure_issue_expires: 12

Validation

  • make fmt
  • go test -v -run "TestConclusionJobActionFailureIssueExpiration|TestLoadRepoConfig_ActionFailureIssueExpires|TestActionFailureIssueExpiresHours_Default" ./pkg/workflow/
  • cd actions/setup/js && npm run test -- handle_agent_failure.test.cjs
  • make build
  • make agent-finish (fails due to pre-existing unrelated testifylint issues in pkg/stats/spec_test.go and pkg/testutil/spec_test.go)
  • parallel_validation (Code Review + CodeQL) run twice; no security findings
  • go test -v -run "TestLoadRepoConfig_ActionFailureIssueExpires|TestLoadRepoConfig_MaintenanceEmptyObject|TestLoadRepoConfig_FileNotFound" ./pkg/workflow/

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot set the value to 12h in this repository in aw.json

@pelikhan pelikhan marked this pull request as ready for review April 16, 2026 16:45
Copilot AI review requested due to automatic review settings April 16, 2026 16:45
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds repository-level configuration (via .github/workflows/aw.json) for the expiration window used when the conclusion job opens “action failure” issues, and wires that value through to the JS failure handler.

Changes:

  • Extends the repo config schema and Go config loader/accessors to support maintenance.action_failure_issue_expires (hours, min 1).
  • Updates conclusion job generation to pass GH_AW_ACTION_FAILURE_ISSUE_EXPIRES_HOURS (defaulting to 168h).
  • Updates handle_agent_failure.cjs to apply the configured expiration to both grouped parent issues and per-run issues, with added tests/docs.
Show a summary per file
File Description
pkg/workflow/repo_config.go Adds default constant + accessor for configured expiration hours.
pkg/parser/schemas/repo_config_schema.json Adds maintenance.action_failure_issue_expires with integer/minimum validation.
pkg/workflow/notify_comment.go Loads repo config and injects GH_AW_ACTION_FAILURE_ISSUE_EXPIRES_HOURS into conclusion job env.
actions/setup/js/handle_agent_failure.cjs Reads expiration hours from env and uses it when creating parent/per-run failure issues.
pkg/workflow/repo_config_test.go Adds parsing + validation + default accessor tests.
pkg/workflow/notify_comment_test.go Adds tests for default/configured env var propagation into conclusion job.
actions/setup/js/handle_agent_failure.test.cjs Adds unit tests for env parsing helper.
docs/src/content/docs/guides/ephemerals.md Documents the new action_failure_issue_expires config field.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 8/8 changed files
  • Comments generated: 3

Comment on lines +222 to +234
actionFailureIssueExpiresHours := DefaultActionFailureIssueExpiresHours
repoConfig, repoConfigErr := LoadRepoConfig(c.gitRoot)
if repoConfigErr != nil {
notifyCommentLog.Printf(
"Warning: failed to load repo config for action failure issue expiration (using default %d hours): %v. Check that %s is valid JSON",
DefaultActionFailureIssueExpiresHours,
repoConfigErr,
RepoConfigFileName,
)
} else {
actionFailureIssueExpiresHours = repoConfig.ActionFailureIssueExpiresHours()
}
agentFailureEnvVars = append(agentFailureEnvVars, fmt.Sprintf(" GH_AW_ACTION_FAILURE_ISSUE_EXPIRES_HOURS: %q\n", strconv.Itoa(actionFailureIssueExpiresHours)))
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildConclusionJob loads and validates aw.json every time the conclusion job is generated. When compiling many workflows in one run, this repeatedly hits the filesystem and schema validation for a repo-level setting that is effectively constant. Consider loading repo config once (e.g., cache it on Compiler or pass it into buildConclusionJob) and reusing the parsed value.

Copilot uses AI. Check for mistakes.
Comment on lines +225 to +230
notifyCommentLog.Printf(
"Warning: failed to load repo config for action failure issue expiration (using default %d hours): %v. Check that %s is valid JSON",
DefaultActionFailureIssueExpiresHours,
repoConfigErr,
RepoConfigFileName,
)
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The warning text says “Check that … is valid JSON”, but LoadRepoConfig can fail for reasons other than JSON syntax (e.g., schema validation errors or read failures). Updating the message to mention schema validation (or to just surface the underlying error without implying JSON-only) would make debugging more accurate.

Copilot uses AI. Check for mistakes.
Comment on lines +569 to +578
func TestConclusionJobActionFailureIssueExpiration_DefaultFromRepoConfig(t *testing.T) {
compiler := NewCompiler()
workflowData := &WorkflowData{
Name: "Test Workflow",
SafeOutputs: &SafeOutputsConfig{
NoOp: &NoOpConfig{},
},
}

job, err := compiler.buildConclusionJob(workflowData, string(constants.AgentJobName), []string{})
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test uses NewCompiler() without overriding compiler.gitRoot, so it will read the real repository’s .github/workflows/aw.json if one is present. That makes the test outcome dependent on repo state (or a developer’s local file) and can become flaky if aw.json is added/changed later. Set compiler.gitRoot to a temp dir without aw.json for this default-case assertion.

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 79/100

⚠️ Acceptable, with suggestions

Metric Value
New/modified tests analyzed 8
✅ Design tests (behavioral contracts) 8 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 5 (62%)
Duplicate test clusters 0
Test inflation detected Yes — notify_comment_test.go (+58 lines) vs notify_comment.go (+15 lines), ratio ≈ 3.9:1
🚨 Coding-guideline violations None

Test Classification Details

View all 8 tests
Test File Classification Issues Detected
TestConclusionJobActionFailureIssueExpiration_DefaultFromRepoConfig pkg/workflow/notify_comment_test.go ✅ Design Edge case: no config present; asserts YAML env var value
TestConclusionJobActionFailureIssueExpiration_UsesAWJSONConfig pkg/workflow/notify_comment_test.go ✅ Design Verifies end-to-end config propagation from aw.json → conclusion job YAML
TestLoadRepoConfig_ActionFailureIssueExpires pkg/workflow/repo_config_test.go ✅ Design Happy path; covers both raw field and accessor
TestLoadRepoConfig_InvalidActionFailureIssueExpires pkg/workflow/repo_config_test.go ✅ Design Error case: action_failure_issue_expires: 0 must fail schema validation
TestActionFailureIssueExpiresHours_Default pkg/workflow/repo_config_test.go ✅ Design Edge case: empty RepoConfig{} returns default constant
returns default when env var is missing actions/setup/js/handle_agent_failure.test.cjs ✅ Design Edge case: absent env var → default 168
returns configured value when env var is a positive integer actions/setup/js/handle_agent_failure.test.cjs ✅ Design Happy path: valid int env var is parsed and returned
returns default for invalid values actions/setup/js/handle_agent_failure.test.cjs ✅ Design Edge cases: 0 and "invalid" both fall back to default

Observations

⚠️ Test Inflation — notify_comment_test.go

notify_comment_test.go grew by 58 lines while the production file notify_comment.go grew by only 15 lines (ratio ≈ 3.9:1, threshold 2:1). This triggered the 10-point inflation deduction, but the tests themselves are high-quality — they exercise two distinct integration paths and contain meaningful behavioral assertions. The inflation is a metric artefact, not a quality concern.


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 5 tests — unit (//go:build !integration) ✅ build tags present in both files
  • 🟨 JavaScript (*.test.cjs): 3 tests (vitest)

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). All 8 tests verify observable behavioral contracts: env-var defaults and overrides, config parsing, schema validation, and YAML output content. The only deduction is a mechanical inflation ratio on notify_comment_test.go, which is justified by the thorough end-to-end coverage it provides.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.


References: §24522564949

🧪 Test quality analysis by Test Quality Sentinel · ● 452.8K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 79/100. Test quality is acceptable — 0% of new tests are implementation tests (threshold: 30%). All 8 new/modified tests verify observable behavioral contracts (env-var defaults, config parsing, schema validation, YAML output content). Slight test inflation on notify_comment_test.go (3.9:1 ratio) is a metric artefact — the extra coverage is justified.

…expiration

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

Commit pushed: feb0198

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (122 new lines in pkg/) but does not have a linked Architecture Decision Record (ADR).

AI has analyzed the PR diff and generated a draft ADR to help you get started:

📄 Draft ADR: docs/adr/26688-configurable-expiration-for-action-failure-issues.md

What to do next

  1. Review the draft ADR committed to your branch — it was generated from the PR diff
  2. Complete the missing sections — add context the AI couldn't infer, refine the decision rationale, and list real alternatives you considered
  3. Commit the finalized ADR to docs/adr/ on your branch
  4. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-26688: Configurable Expiration for Action Failure Issues via aw.json

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

Why ADRs Matter

"AI made me procrastinate on key design decisions. Because refactoring was cheap, I could always say 'I'll deal with this later.' Deferring decisions corroded my ability to think clearly."

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you.


📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context — What is the problem? What forces are at play?
  • Decision — What did you decide? Why?
  • Alternatives Considered — What else could have been done?
  • Consequences — What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number (e.g., 26688-configurable-expiration-for-action-failure-issues.md for PR #26688).

🔒 This PR cannot merge until an ADR is linked in the PR body.

References:

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · ● 120.3K ·

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review all comments

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 16, 2026

@copilot set the value to 12h in this repository in aw.json

Done in 659e88b — added /home/runner/work/gh-aw/gh-aw/.github/workflows/aw.json with maintenance.action_failure_issue_expires set to 12 (12h).

Copilot AI and others added 2 commits April 16, 2026 17:11
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 16, 2026

@copilot review all comments

Done in 39d28f8 (with prior change efa4739): I reviewed and addressed all outstanding review comments by caching repo config loading in the compiler, updating the warning message to cover schema/read failures, isolating the default expiration test from repo state, and adding cache-behavior tests.

@github-actions
Copy link
Copy Markdown
Contributor

Hey @Copilot 👋 — great work on adding configurable expiration for action-failure issues! This is a clean, well-scoped addition that follows the project's agentic development process perfectly.

The PR is focused on a single concern, includes thorough test coverage across Go and JS layers (compiler_repo_config_test.go, notify_comment_test.go, repo_config_test.go, handle_agent_failure.test.cjs), ships an ADR, updates docs, and extends the JSON schema — this looks ready for maintainer review. 🚀

Generated by Contribution Check · ● 1.9M ·

@github-actions github-actions bot added the lgtm label Apr 16, 2026
@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot recompile

@pelikhan pelikhan merged commit 8dbc64e into main Apr 16, 2026
51 checks passed
@pelikhan pelikhan deleted the copilot/configure-expiration-time-action-failures branch April 16, 2026 18:33
Copilot stopped work on behalf of pelikhan due to an error April 16, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants