Skip to content

[CI Failure Doctor] CI Failure Investigation - Plugin validation rejects Claude/Codex #14534

@github-actions

Description

@github-actions

🏥 CI Failure Investigation - Run #34433

Summary

The new plugin validation now rejects Claude and Codex workflows that install plugins because the engines still report that they do not support plugins, so any unit or integration test that exercises plugin compilation immediately fails.

Failure Details

Root Cause Analysis

ClaudeEngine and CodexEngine never set supportsPlugins: true in their BaseEngine, so Compiler.validatePluginSupport concludes that neither engine supports plugins even though their workflow steps explicitly install them. The guardrail now runs before the jobs emit plugin commands, so it drives every plugin-related test (both unit and integrations) into the engine 'claude/codex' does not support plugins failure.

Failed Jobs and Errors

  • testTestCompileWorkflowWithPluginImportsClaudeEngine fails with "engine 'claude' does not support plugins..." from validatePluginSupport
  • Integration: Workflow String & Sanitization – plugin compilation tests for Claude and Codex hit the same message and short-circuit (multiple FAIL cases listed)
  • Integration: Workflow Misc Part 2 – plugin token precedence and plugin compilation regressions across Claude/Codex all fail with the identical validation error

Investigation Findings

  • The guardrail accurately detects plugin installation requests but misrepresents the supported engines because the metadata flags were never updated when those engines gained plugin installation steps.
  • As soon as the guardrail fires, the compiler refuses to even emit plugin commands, so nothing downstream can succeed.

Recommended Actions

  • Flip supportsPlugins: true for ClaudeEngine and CodexEngine so the guardrail recognizes that they are plugin-capable.
  • Refresh validatePluginSupport’s error text so it no longer claims only Copilot supports plugins (list the actual engines or generically note “engines with plugin support”).
  • Re-run the plugin compilation tests locally or via CI to ensure Claude/Codex plugin scenarios now build successfully.

Prevention Strategies

Sync every engine’s supportsPlugins boolean with the real plugin behavior and derive the validation messaging from that metadata instead of hardcoding single-engine claims so future additions can’t regresses silently.

AI Team Self-Improvement

When you add plugin-installation steps for an engine, remember to set supportsPlugins: true and revisit validatePluginSupport so the guardrail and its messaging stay accurate.

Historical Context

These plugin compilation tests previously passed on main; after the new validation we now see the exact same "does not support plugins" error in the test, Integration: Workflow String & Sanitization, and Integration: Workflow Misc Part 2 jobs on run #34433.

AI generated by CI Failure Doctor

To add this workflow in your repository, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.

  • expires on Feb 9, 2026, 3:39 PM UTC

Metadata

Metadata

Labels

cookieIssue Monster Loves Cookies!

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions