feat(e2e): add /fix-e2e command and AI agent skills#4516
Open
zdrapela wants to merge 22 commits intoredhat-developer:mainfrom
Open
feat(e2e): add /fix-e2e command and AI agent skills#4516zdrapela wants to merge 22 commits intoredhat-developer:mainfrom
zdrapela wants to merge 22 commits intoredhat-developer:mainfrom
Conversation
|
Skipping CI for Draft Pull Request. |
Contributor
|
The container image build workflow finished with status: |
zdrapela
force-pushed
the
skill-agent-e2e-fix
branch
from
e8cd40c to
d237b44
Compare
Contributor
|
The container image build workflow finished with status: |
Contributor
|
The container image build workflow finished with status: |
Contributor
|
The container image build workflow finished with status: |
gustavolira
reviewed
Apr 6, 2026
Contributor
zdrapela
force-pushed
the
skill-agent-e2e-fix
branch
from
d947b22 to
0a20e05
Compare
Member
Author
|
Addressing review comment #4516 (comment):
Fixed. |
Member
Author
|
Addressing review comment #4516 (comment):
Fixed. Replaced |
zdrapela
force-pushed
the
skill-agent-e2e-fix
branch
from
0a20e05 to
def054a
Compare
Code Review by Qodo
1.
|
Review Summary by QodoAdd autonomous E2E CI failure fix workflow with Playwright agents and multi-platform skill distribution
WalkthroughsDescription• Add comprehensive autonomous E2E CI failure investigation and fix workflow with 7 AI agent skills • Implement /fix-e2e command orchestrating full lifecycle: parse failure, create branch, deploy RHDH, reproduce, diagnose/fix with Playwright agents, verify, and submit PR with Qodo review • Initialize Playwright Test Agents (healer/generator/planner) with MCP server configuration for live browser interaction • Distribute all skills and rules across rulesync, Claude, Cursor, and OpenCode platforms for consistent AI agent access • Add seed test for E2E initialization and .env file generation for Playwright Test Agents environment setup • Enable rulesync skills feature and configure MCP server for Playwright test automation Diagramflowchart LR
A["CI Failure"] -->|parse-ci-failure| B["Extract Metadata"]
B -->|setup-fix-branch| C["Create Fix Branch"]
C -->|deploy-rhdh| D["Deploy RHDH"]
D -->|reproduce-failure| E["Classify Failure"]
E -->|diagnose-and-fix| F["Fix with Playwright Agents"]
F -->|verify-fix| G["Stability & Quality Check"]
G -->|submit-and-review| H["Submit PR & Qodo Review"]
H --> I["Monitor CI"]
J["Skills & Rules"] -->|rulesync| K["Claude/Cursor/OpenCode"]
L["MCP Server"] -->|Playwright Agents| F
File Changes1. e2e-tests/playwright.config.ts
|
…mit-and-review Document required vs optional job sections, shortened job names, and update the mapping table with actual /test commands from the bot. Assisted-by: OpenCode
Assisted-by: OpenCode
Assisted-by: OpenCode
Assisted-by: OpenCode
…Qodo polling - Fix duplicate ### 5 numbering in e2e-verify-fix (Review the Diff is now ### 6) - Add Playwright report URL to fix-e2e.md input list - Add scripted polling loop for Qodo review (15s interval, 5 min timeout) - Update command to match new Qodo polling interval Assisted-by: OpenCode
…skills Assisted-by: OpenCode
…e job Present classified failures, ask user which to fix, group by shared root cause when possible. Assisted-by: OpenCode
… optional Assisted-by: OpenCode
The healer agent is only supported in OpenCode and Claude Code. Add notes to all 3 healer-using skills directing Cursor users to direct execution and manual diagnosis instead. Assisted-by: OpenCode
Assisted-by: OpenCode
zdrapela
force-pushed
the
skill-agent-e2e-fix
branch
from
2863dcb to
138c106
Compare
Contributor
|
The container image build workflow finished with status: |
zdrapela
changed the title
gustavolira
reviewed
Apr 14, 2026
| echo "$safe_key=$(env_quote "$value")" | ||
| done < <(printf '%s' "$SECRETS_JSON" | jq -r 'keys[]') | ||
| } > "$ENV_TMP" | ||
| mv -f "$ENV_TMP" "$ENV_FILE" |
Member
There was a problem hiding this comment.
No set -e here, so if mv -f fails the script keeps going and log::success below still prints. Worth wrapping it as if ! mv -f "$ENV_TMP" "$ENV_FILE"; then rm -f "$ENV_TMP"; return 1; fi, plus a trap 'rm -f "$ENV_TMP"' RETURN so the temp file doesn't leak on other failures.
.rulesync/rules/e2e-fix-workflow.md
Outdated
|
|
||
| // BAD — deprecated CSS selectors | ||
| page.locator('.MuiButton-root') | ||
| page.locator('[data-testid="..."]') |
Member
There was a problem hiding this comment.
This line calls data-testid deprecated, but playwright-locators.md in this repo lists getByTestId as a valid option (priority 7). The bad part is using the CSS form (page.locator('[data-testid=...]')), not the attribute itself. As written, the healer will start rewriting legitimate getByTestId calls thinking they're the old pattern.
Member
Author
There was a problem hiding this comment.
Okay, I'll remove that, and it's better to link the existing best practices, instead of repeating them here
.rulesync/rules/e2e-fix-workflow.md
Outdated
| ``` | ||
|
|
||
| **Forbidden patterns**: | ||
| - `page.waitForNetworkIdle()` / `networkidle` |
Member
There was a problem hiding this comment.
page.waitForNetworkIdle() isn't a real Playwright method — the actual one is page.waitForLoadState('networkidle'). The agent won't flag the real anti-pattern if it's looking for a non-existent API name.
.rulesync/rules/e2e-fix-workflow.md
Outdated
| | Pattern in job name | Release branch | | ||
| |---------------------|---------------| | ||
| | `*-rhdh-main-*` | `main` | | ||
| | `*-rhdh-release-1.9-*` | `release-1.9` | |
Member
There was a problem hiding this comment.
This table (and the image tag table below) needs manual edits every release cycle in four places. Since the job name always contains -rhdh-<branch>-, a regex would cover future versions without touching the file:
BRANCH=$(echo "$JOB_NAME" | grep -oE '\-rhdh-(main|release-[0-9]+\.[0-9]+)-' | sed 's/^-rhdh-//;s/-$//')
if [[ "$BRANCH" == "main" ]]; then
REPO="rhdh-community/rhdh"; TAG="next"
else
REPO="rhdh/rhdh-hub-rhel9"; TAG="${BRANCH#release-}"
fi
Member
Author
There was a problem hiding this comment.
True, let's do that for BRANCH
Member
Author
There was a problem hiding this comment.
The image resolution is done later and we also have the next-1.9 etc. tags at rhdh-community/rhdh
|
|
||
| ```bash | ||
| # Get the PR number from the create output, then comment | ||
| gh pr comment <PR-number> --repo redhat-developer/rhdh --body "/agentic_review" |
Member
There was a problem hiding this comment.
The workflow can loop through Phase 5→6→7 multiple times, and each iteration re-triggers /agentic_review. Qodo has quota limits on some plans — worth noting in the skill that this should fire once per PR lifecycle, with /improve or /review used for follow-up passes.
Member
Author
There was a problem hiding this comment.
I added an option to opt out of Qodo for users without the enterprise Qodo account, but I'd rather keep it as the default, as it creates a great feedback loop for the PRs.
Set PLAYWRIGHT_HTML_OPEN=never in generated .env and replace show-report instruction with direct file access. Assisted-by: OpenCode
Replace duplicated coding conventions, debugging modes, and reference files with cross-references to playwright-locators and ci-e2e-testing. Assisted-by: OpenCode
…nches When working on a release branch, check if the failing test was already fixed on main and cherry-pick if possible. Assisted-by: OpenCode
…flicts Assisted-by: OpenCode
…olve issue Assisted-by: OpenCode
Add fallback for determining base branch from feature branches. Assisted-by: OpenCode
Assisted-by: OpenCode
Prevents depleting limited Qodo quota on iterative fixes. Assisted-by: OpenCode
Remove duplicated sections (deployment, reproduction, diagnosis, verification, PR submission) that are maintained in the skills. Keep only the workflow overview, job name mapping tables (single source of truth), and coding convention cross-references. 290 lines → 85 lines. Assisted-by: OpenCode
Reference e2e-fix-workflow rule for derivation logic. Assisted-by: OpenCode
… rule Single source of truth for branch-to-image derivation. Assisted-by: OpenCode
|
Member
|
/lgtm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
How
/fix-e2eworksThe
/fix-e2ecommand accepts a Prow job URL, Playwright report URL, or Jira ticket and runs through 7 phases autonomously:local-run.shCI=trueEach phase loads a dedicated skill with detailed instructions. The agent stops and asks for user input at decision gates (e.g., product bug vs test issue, cannot reproduce, skip deployment).
This agent was already used to create this PR: #4526
Summary
/fix-e2ecommand for autonomous E2E CI failure investigation and fix workflow.envfile vialocal-test-setup.sh --envwith secure file handling and multiline secret supportSkills
e2e-parse-ci-failuree2e-deploy-rhdhe2e-reproduce-failuree2e-diagnose-and-fixe2e-verify-fixe2e-submit-and-review/test ?responseKey design decisions
gh search code/ local clone fallbacks-sdeploy-only mode; K8s jobs (AKS/EKS/GKE) require full executiontest.fixme()requires healer confirmation, user approval, and aRHDHBUGSJira ticket--project=any-testfor individual test runs to skip smoke test dependency/test ?bot responds in seconds (5s poll); Qodo review takes 1-3 min (15s poll).envwritten atomically via temp file withumask 077+chmod 600