[CI Failure Doctor] CI Failure Investigation - Run #37508

[github-actions]

Summary

Permission validation tests fail because the validator now prints Missing required permissions for GitHub toolsets: while every test still asserts on the lowercase Missing required permissions for github toolsets:. The string mismatch causes the suites to fail before emitting recorded failures.

Failure Details

• Run: 22313328227
• Commit: 727b147
• Trigger: push

Root Cause Analysis

permissions_validation.go builds the warning text starting with "Missing required permissions for GitHub toolsets:". The permissions_* tests still look for the lowercase phrase; their strings.Contains checks never find the message, so they panic before emitting failure reports. The same assertion failure shows up across both the unit and integration suites.

Failed Jobs and Errors

• test failure – The unit suite aborted with ❌ FAILURES FOUND in: test-result-unit.json while running TestFormatValidationMessage (including Missing_permissions_message) and the TestValidatePermissions_ComplexScenarios subtests because the expected warning text was missing.
• Integration: Workflow Infra – The integration suite aborts when TestPermissionsWithoutGitHubTool/permissions_with_github_tool_-_validates_permissions and the related warning tests cannot find the lowercase warning string (Expected warning message not found).

Investigation Findings

• The actual warning text from permissions_validation.go begins with Missing required permissions for GitHub toolsets: followed by the Option 1/Option 2 suggestions; that matches the literal used everywhere else but not in the tests.
• Once the assertion fails, the test harness raises assert.NotContains/assert.Contains failures and the package exits with FAIL before reporting individual test failures.
• Running go test ./pkg/workflow -run 'TestPermissions|TestFormatValidationMessage|TestValidatePermissions' here tries to download Go 1.25.0 via golang.org/toolchain, and that request is blocked (Forbidden), so we cannot re-run the suite until a Go 1.25 toolchain is available locally.

Recommended Actions

• Align every test that locks on this warning string with the actual message produced by permissions_validation.go (share the literal or lower-case the comparison) so capitalization-only tweaks do not break the suite again.
• Re-run go test ./pkg/workflow -run 'TestPermissions|TestFormatValidationMessage|TestValidatePermissions' (or make agent-finish) once Go 1.25 is available in this environment to verify no other regressions remain.

Prevention Strategies

Avoid case-sensitive literal matches against user-facing warning text; either reuse the helper that formats it or assert with case-insensitive substring checks so capitalization-only fixes do not introduce new failures.

AI Team Self-Improvement

When writing tests for formatted CLI warnings, pull the expected text from the same helper that emits it or compare case-insensitively so future copy edits (like capitalizing “GitHub”) remain safe.

Historical Context

• See #17912 for the same regression introduced when the message first switched to "GitHub" in run 37498.

🩺 Diagnosis provided by CI Failure Doctor

To install this workflow, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. View source at https://github.com/githubnext/agentics/tree/ea350161ad5dcc9624cf510f134c6a9e39a6f94d/workflows/ci-doctor.md.

• expires on Feb 24, 2026, 3:58 PM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #17993

View newer issue

---

This action was performed automatically by the CI Failure Doctor workflow.