Skip to content

[ci-coach] ci: add unit test job and remove permanently-disabled marketplace job #27359

@github-actions

Description

@github-actions

Summary

Analysis of the CI pipeline revealed that 5,222 unit tests were never run in CI. The integration matrix runs with -tags 'integration', which excludes all files tagged //go:build !integration. A separate test-unit job is required to cover these tests.

Additionally, the integration-marketplace-compile job was permanently disabled via if: $\{\{ ... && (false) }} — dead code that was removed.

Optimizations

1. Add test-unit CI Job

Type: Test Coverage
Impact: Adds coverage for ~5,200 previously-uncovered unit tests
Risk: Low

Root cause: The integration matrix uses go test -tags 'integration' ..., which compiles only files with //go:build integration. All 853 unit test files tagged //go:build !integration (in pkg/cli, pkg/workflow, pkg/parser, and 21 other packages) were never executed in CI.

Fix: Added a test-unit job that runs go test -v -parallel=4 -timeout=10m -run='^Test' ./... -short (no -tags flag, matching make test-unit).

Packages now covered by test-unit
Package Unit Tests
pkg/cli 1,523
pkg/workflow 3,008
pkg/parser 275
pkg/console 113
pkg/agentdrain 38
pkg/constants 32
cmd/gh-aw 8
17 others 225
Total 5,222

2. Remove Permanently-Disabled integration-marketplace-compile Job

Type: Dead Code Cleanup
Impact: Reduces YAML noise, no functional impact
Risk: None

The job had if: $\{\{ needs.changes.outputs.has_changes == 'true' && (false) }} — the (false) expression unconditionally disables it. It was never executing and could never execute without a code change.

Validation Results

✅ YAML syntax validated (python3 -c "import yaml; yaml.safe_load(...)")
✅ Unit tests (make test-unit) passed — verified by CI coach workflow pre-run
✅ Build already validated by CI coach workflow

Metrics Baseline

  • CI runs analyzed: 100
  • Main branch success rate: 63% (21 failures, 41 successes of 65 main runs)
  • Unit test functions previously uncovered: 5,222

Proposed by CI Coach workflow run #136


Warning

Protected Files — Push Permission Denied

This was originally intended as a pull request, but the patch modifies protected files. A human must create the pull request manually.

Protected files

The push was rejected because GitHub Actions does not have workflows permission to push these changes, and is never allowed to make such changes, or other authorization being used does not have this permission.

Create the pull request manually
# Download the patch from the workflow run
gh run download 24669880229 -n agent -D /tmp/agent-24669880229

# Create a new branch
git checkout -b ci-coach/add-unit-test-job-573c91e140e8fde0 main

# Apply the patch (--3way handles cross-repo patches)
git am --3way /tmp/agent-24669880229/aw-ci-coach-add-unit-test-job.patch

# Push the branch and create the pull request
git push origin ci-coach/add-unit-test-job-573c91e140e8fde0
gh pr create --title '[ci-coach] ci: add unit test job and remove permanently-disabled marketplace job' --base main --head ci-coach/add-unit-test-job-573c91e140e8fde0 --repo github/gh-aw

Generated by CI Optimization Coach · ● 3.1M ·

  • expires on Apr 22, 2026, 1:57 PM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions