Skip to content

[WIP] Fix plugin validation error for Claude and Codex#14557

Closed
Copilot wants to merge 2 commits intomainfrom
copilot/fix-plugin-validation-error
Closed

[WIP] Fix plugin validation error for Claude and Codex#14557
Copilot wants to merge 2 commits intomainfrom
copilot/fix-plugin-validation-error

Conversation

Copy link
Contributor

Copilot AI commented Feb 8, 2026

Fix Plugin Validation for Claude and Codex Engines

Plan

  • Enable supportsPlugins: true for Claude engine in pkg/workflow/claude_engine.go
  • Enable supportsPlugins: true for Codex engine in pkg/workflow/codex_engine.go
  • Add plugin installation steps to Claude GetInstallationSteps()
  • Add plugin installation steps to Codex GetInstallationSteps()
  • Update plugin import test to expect Claude/Codex to support plugins
  • Add Codex plugin import test
  • Update engine validation tests to expect Claude/Codex support
  • Add regression test for validatePluginSupport error message listing all supported engines
  • Update TestOtherEnginesNoPluginSupport to TestEnginesWithPluginSupport
  • Run make agent-finish before final commit

Context

The issue #34439 identified that Claude and Codex engines have plugin installation code (GeneratePluginInstallationSteps) but their SupportsPlugins() method returns false, causing validation to reject plugins. This PR:

  1. Sets supportsPlugins: true for Claude and Codex engines
  2. Adds plugin installation steps to both engines' GetInstallationSteps() methods
  3. Updates tests to reflect that these engines now support plugins
  4. Adds a regression test to ensure validation messages stay in sync with engine capabilities

Changes Made

  • pkg/workflow/claude_engine.go: Set supportsPlugins: true, add plugin installation in GetInstallationSteps()
  • pkg/workflow/codex_engine.go: Set supportsPlugins: true, add plugin installation in GetInstallationSteps()
  • pkg/workflow/plugin_import_test.go: Updated Claude test to expect success, added Codex test
  • pkg/workflow/engine_validation_test.go: Updated tests to expect Claude/Codex support, added regression test
  • pkg/workflow/copilot_engine_test.go: Renamed and updated test to verify all three engines support plugins
Original prompt

This section details on the original issue you should resolve

<issue_title>[CI Failure Doctor] 🏥 CI Failure Investigation - Run #34439</issue_title>
<issue_description># 🏥 CI Failure Investigation - Run github/gh-aw#34439

Summary

test and Integration: Workflow String & Sanitization jobs fail because plugin validation only allows Copilot even though some tests compile workflows that import plugins for Claude.

Failure Details

Root Cause Analysis

validatePluginSupport rejects any engine whose SupportsPlugins() returns false. Claude and Codex embed BaseEngine with supportsPlugins false, so all plugin-related tests for those engines hit the error path even though their installation steps call GeneratePluginInstallationSteps. The error message still hard-codes the Copilot-only sentence, so the advice pointed users to switching engines even though Claude and Codex were supposed to work.

Failed Jobs and Errors

  • test: pkg/workflow/plugin_import_test.go:296error: engine 'claude' does not support plugins... Only the 'copilot' engine currently supports plugin installation.
  • Integration: Workflow String & Sanitization: pkg/workflow/plugin_compilation_test.go:95error: engine 'claude' does not support plugins... Only the 'copilot' engine currently supports plugin installation.

Investigation Findings

  • Plugin compilation and import tests for Claude require SupportsPlugins() to return true so validatePluginSupport allows plugin installation.
  • The supportsPlugins flag was only set for Copilot, so Claude/Codex were treated as unsupported even though their installation steps already install plugins.
  • validatePluginSupport also hard-coded the Copilot-only message, so the error sounded like the only fix was to switch engines even though Claude/Codex were intended to support plugins.

Recommended Actions

  • Enable plugin capability metadata for Claude and Codex and update validatePluginSupport to list every engine that supports plugins (already done).
  • Add a regression test that asserts the validation message reports the supported engines whenever supportsPlugins is updated so future engine additions stay in sync.

Prevention Strategies

Keep engine capability metadata (supportsPlugins) and validation messaging synchronized with the actual install steps. When a new engine starts installing plugins, make the corresponding capability switch and update validatePluginSupport so the error messaging reflects every engine that can install plugins.

AI Team Self-Improvement

When adding plugin installation steps for a new engine, remember to update its BaseEngine capability flags and re-run validatePluginSupport tests so the user-facing error messages list all supported engines.

Historical Context

The only recent CI Failure Doctor issue was unrelated (displayFileContent logging). This plugin validation regression surfaced when tests started exercising Claude and Codex plugin flows.

AI generated by CI Failure Doctor

To add this workflow in your repository, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.

  • expires on Feb 9, 2026, 6:40 PM UTC

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

- Set supportsPlugins: true for Claude and Codex engines
- Add plugin installation steps to Claude and Codex GetInstallationSteps()
- Update tests to expect plugin support from Claude and Codex
- Add regression test for validation error message listing all supported engines
- Update TestOtherEnginesNoPluginSupport to TestEnginesWithPluginSupport

Fixes #34439 - CI Failure Investigation - Run #34439

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan closed this Feb 8, 2026
Copilot AI requested a review from pelikhan February 8, 2026 19:53
Copilot stopped work on behalf of pelikhan due to an error February 8, 2026 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure Doctor] 🏥 CI Failure Investigation - Run #34439

2 participants