[repository-quality] Repository Quality Report: Test Infrastructure & Build Tag Compliance (2026-05-25) #34656
Replies: 2 comments
-
|
💥 WHOOSH! 🦸♀️ The Smoke Test Agent rockets onto the scene in a blaze of green CI checks! ⚡
🎺 Brought to you by Run §26407618325 — back to your regularly scheduled discussion! 🎺 Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
Beta Was this translation helpful? Give feedback.
-
|
🧪 Smoke test agent zipped through, tapped the gauges, and left a tiny confetti trail. Copilot was here! Warning Firewall blocked 6 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "android.clients.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
🎯 Repository Quality Improvement Report - Test Infrastructure & Build Tag Compliance
Analysis Date: 2026-05-25
Focus Area: Test Infrastructure & Build Tag Compliance
Strategy Type: Custom
Custom Area: Yes — the repository has a mandatory requirement for build tags on all test files (per AGENTS.md), and static analysis found 13 non-testdata test files missing this tag. The test suite also shows mixed assertion styles and large test file maintainability gaps.
Executive Summary
The repository enforces a mandatory policy (AGENTS.md) requiring every
*_test.gofile to carry a//go:builddirective at the top. A scan of 1,206 test files found 13 production test files (excluding testdata) that violate this policy — including shared test helpers, utility package tests, and CLI tests. These files are excluded from the unit-test build tag (!integration) by accident, which can cause unpredictable test run behaviour when filtering by build tags.Beyond the build tag gap, the largest test files show inconsistent assertion styles:
pkg/workflow/copilot_engine_test.go(2,257 lines) bypasses testify entirely and uses rawt.Errorfcalls, whilepkg/workflow/compiler_safe_outputs_config_test.go(3,126 lines) has 35 assertions that lack descriptive messages. Both patterns reduce debuggability when a test fails in CI.The
pkg/workflow/compiler_jobs_test.goat 3,700 lines is the single largest test file in the repository — nearly 3× the recommended 100–200 line validator guideline — and is a strong candidate for decomposition into focused sub-files to improve discoverability and reduce merge conflicts.Full Analysis Report
Focus Area: Test Infrastructure & Build Tag Compliance
Current State Assessment
Metrics Collected:
//go:buildt.Errorfinstead of testifyt.Skipcalls (conditional environment skips)Findings
Strengths
//go:builddirectivest.Run()patterns in many areasAreas for Improvement
//go:buildtag — violates AGENTS.md policy; may cause tests to be excluded or double-run depending on build constraintst.Errorf— 268 manual assertions instead of testify, reducing consistency and debuggabilityDetailed Analysis
Files missing
//go:buildtag:All 13 files are unit tests (no "integration" in filename), so they should receive
//go:build !integration. The shared helpers (test_helpers_shared_test.go,compiler_test_helpers_test.go,test_helpers_git_test.go) are particularly risky because they are compiled conditionally based on whatever tag is active when another test file in the package uses them.copilot_engine_test.go assertion style:
The file uses
if engine.GetID() != "copilot" { t.Errorf(...) }patterns throughout its 2,257 lines — none use testify. This means failures show raw Go error strings without the rich diff output that testify provides.compiler_safe_outputs_config_test.go:
Of 267 assertion calls, 35 provide no descriptive message string. Per AGENTS.md testifylint guidelines, all assertions should include a message.
🤖 Tasks for Copilot Agent
NOTE TO PLANNER AGENT: Split the following tasks into individual work items.
Improvement Tasks
Task 1: Add Missing Build Tags to 13 Test Files
Priority: High
Estimated Effort: Small
Focus Area: Test Infrastructure & Build Tag Compliance
Description: 13 non-testdata
*_test.gofiles are missing the mandatory//go:build !integrationtag required by AGENTS.md. Each file should have//go:build !integrationas its first line, followed by a blank line, then the package declaration.Acceptance Criteria:
//go:build !integrationas the first linepackagedeclarationmake build && make fmtpasses cleanlymake test-unitpasses with no regressionsCode Region:
pkg/cli/compile_workflow_processor_test.go,pkg/cli/status_dependency_tree_test.go,pkg/cli/test_helpers_git_test.go,pkg/cli/mcp_inspect_tree_test.go,pkg/cli/codemod_activation_outputs_test.go,pkg/workflow/github_context_prompt_test.go,pkg/workflow/docker_cli_proxy_test.go,pkg/workflow/compiler_test_helpers_test.go,pkg/workflow/test_helpers_shared_test.go,pkg/syncutil/onceloader_test.go,pkg/errorutil/errors_test.go,pkg/parser/import_bfs_test.go,pkg/constants/permissions_policy_test.go(go/redacted):build !integration
Task 2: Migrate copilot_engine_test.go to Testify Assertions
Priority: Medium
Estimated Effort: Large
Focus Area: Test Infrastructure & Build Tag Compliance
Description:
pkg/workflow/copilot_engine_test.go(2,257 lines, 268+ manual assertion patterns) uses rawt.Errorf/t.Fatalfinstead of the testify library used throughout the rest of the repository. Migrating torequire.*/assert.*improves error readability, consistency, and enables testifylint linter checks.Acceptance Criteria:
if x != y { t.Errorf(...) }patterns replaced withassert.Equal(t, y, x, "message")if err != nil { t.Fatalf(...) }patterns replaced withrequire.NoError(t, err, "message")make lintreports zero testifylint violationsmake test-unitpasses with no regressionsCode Region:
pkg/workflow/copilot_engine_test.goAfter migration, run
make lintto catch any remaining testifylint violations, thenmake test-unitto verify no regressions.For each bare assertion, add a meaningful final string argument describing what the test checks. Examples:
assert.Equal(t, expected, actual)→assert.Equal(t, expected, actual, "output name should match workflow step ID")assert.NoError(t, err)→assert.NoError(t, err, "safe output config should compile without error")assert.NotNil(t, result)→assert.NotNil(t, result, "compilation result should not be nil")After updating, run
make lintto verify zero testifylint violations, thenmake test-unitto confirm no regressions.Verify the same number of test functions pass.
Beta Was this translation helpful? Give feedback.
All reactions