TRT-2827: Migrate solver to ai-helpers plugin skills - #82844
Conversation
…ills Replace hand-crafted prompt heredocs with standardized skills from openshift-eng/ai-helpers: - jira-solver: Replace ~60-line solve prompt with `/jira:solve` skill invocation in --ci mode. Pre-fetched issue JSON is passed via system prompt since CI lacks Jira credentials. Repo-specific solve-config.md still appended via --append-system-prompt-file. - review-responder: Replace ~100-line followup prompt + trusted-user filtering + comment formatting with `/openshift-developer:address-review-pr` skill in --ci mode. The skill handles comment fetching, authorization, categorization, and replying. Polling loop retained for periodic checking with lightweight comment-count detection. PR creation, eval mode, timeout/nudge, and artifact handling unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The skill migration dropped guidance that was in the old followup prompt heredoc but isn't covered by the address-review-pr skill: push-back rules (questions vs code changes, contradictions, over-engineering), the critical rule about not re-adding rejected approaches, and the Important section (address all comments, no CI config changes, no new PRs). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@smg247: This pull request references TRT-2827 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci-robot
added
the
jira/valid-reference
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe change enables Claude Jira and OpenShift Developer plugins in agentic images and workflows. The Jira solver now uses structured system prompts and validates branch creation. The review responder now uses system guidance and count-based polling. ChangesAgentic Claude workflows
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant ReviewResponder
participant GitHub
participant Claude
ReviewResponder->>GitHub: Count comments, reviews, issue comments, and failing checks
GitHub-->>ReviewResponder: Return activity counts
ReviewResponder->>Claude: Run the PR command when tracked activity changes
Claude-->>ReviewResponder: Process the pull request with the system prompt
Possibly related PRs
Suggested labels: Suggested reviewers: Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 error, 1 inconclusive)
✅ Passed checks (13 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
openshift-ci
Bot
added
the
approved
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/openshift/agentic/trt/review-responder/openshift-agentic-trt-review-responder-commands.sh`:
- Around line 118-121: Update the GitHub query fallback handling in the
review-responder script, including the gh pr checks assignment and the nearby gh
api fallbacks around the referenced commands, to emit a warning whenever a gh
call fails before substituting "0" or "[]". Preserve the existing fallback
values and downstream processing, but make each suppressed API failure visible
in the logs.
- Around line 128-142: Update the has_work block to assign LAST_FAILING_NAMES
from current_failing_names whenever activity is recorded, alongside
LAST_COMMENT_COUNT. This keeps the failure snapshot current so has_new_failures
only triggers for newly appearing or changed failing checks.
- Around line 113-116: Update the count aggregation before comment_total so
inline_count and review_count sum the per-page values emitted by gh api
--paginate --jq, rather than retaining multiline output; add --paginate to the
issue_comment_count request and sum its page counts as well. Preserve each
existing || echo "0" fallback so API failures continue producing zero and do not
stop the polling loop.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 5e610bd6-eb57-4dc2-804d-9c4bf0574704
📒 Files selected for processing (2)
ci-operator/step-registry/openshift/agentic/trt/jira-solver/openshift-agentic-trt-jira-solver-commands.shci-operator/step-registry/openshift/agentic/trt/review-responder/openshift-agentic-trt-review-responder-commands.sh
… bugs The /jira:solve skill failed in CI with "Unknown command" because the ai-helpers plugins were never installed — known_marketplaces.json only tells Claude where to find them, not to load them. Add explicit plugin install commands after Claude Code installation in all three scripts. Also fix the review-responder polling loop: sum per-page counts from --paginate --jq (previously multiline output broke arithmetic), add --paginate to issue comment fetch, and log warnings on gh API failures instead of silently swallowing them. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When Claude fails to create a feature branch in eval mode, the solver stayed on the eval base branch (e.g. eval/case-001-base). The existing main/master check didn't catch this, so the eval rename code deleted the base branch from the remote. Add a check against the eval-base-branch file from SHARED_DIR to exit early instead of destroying the branch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…dev images The agentic-dev Dockerfiles copied known_marketplaces.json from the ai-helpers image but not settings.json with enabledPlugins. Without enabledPlugins, Claude doesn't auto-load plugins from the marketplace, causing "/jira:solve" and other plugin skills to be unavailable at runtime. Add enabledPlugins for jira and openshift-developer to all three agentic-dev image builds (eval, sippy, origin). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
/pj-rehearse pull-ci-openshift-release-main-jira-solver-eval-jira-solver-eval |
|
@smg247: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
…ations The openshift-developer plugin cannot load in CI due to unsatisfiable dependencies (prodsec-skills, golang requiring gopls-lsp). Instead of invoking skills via /openshift-developer:jira-solve and /openshift-developer:address-review-pr, append the SKILL.md files from the ai-helpers image directly into the system prompt and use plain text prompts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
/pj-rehearse pull-ci-openshift-release-main-jira-solver-eval-jira-solver-eval |
|
@smg247: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ci-operator/config/openshift/origin/openshift-origin-main__agentic.yaml`:
- Around line 36-37: Remove openshift-developer@ai-helpers from the
enabledPlugins configuration in
ci-operator/config/openshift/origin/openshift-origin-main__agentic.yaml lines
36-37,
ci-operator/config/openshift/release/openshift-release-main__jira-solver-eval.yaml
lines 37-38, and
ci-operator/config/openshift/sippy/openshift-sippy-main__agentic.yaml lines
37-38; retain jira@ai-helpers and the existing SKILL.md-based workflow behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 41239ce9-1c77-41b2-a4b9-b3e0734c2006
📒 Files selected for processing (5)
ci-operator/config/openshift/origin/openshift-origin-main__agentic.yamlci-operator/config/openshift/release/openshift-release-main__jira-solver-eval.yamlci-operator/config/openshift/sippy/openshift-sippy-main__agentic.yamlci-operator/step-registry/openshift/agentic/trt/jira-solver/openshift-agentic-trt-jira-solver-commands.shci-operator/step-registry/openshift/agentic/trt/review-responder/openshift-agentic-trt-review-responder-commands.sh
…er prompt Eval cases 002 and 005 failed because Claude didn't create a feature branch or write a PR description. Make the prompt more directive about these critical steps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
/pj-rehearse pull-ci-openshift-release-main-jira-solver-eval-jira-solver-eval |
|
@smg247: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Remove dead `claude plugin install` calls from jira-solver, eval-solve, and review-responder (plugins are loaded via image settings.json, and the openshift-developer plugin can't load anyway due to unsatisfiable deps). Revert sippy/origin Dockerfile changes back to main — the enabledPlugins addition was only needed for the plugin invocation approach we no longer use. Revert review-responder skill migration — no eval framework exists yet to validate those changes. Patch saved to .claude/patches/ for later. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The plugin setup removal was unnecessary — the eval config already exists on upstream main with the known_marketplaces.json copy. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
smg247
changed the title
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
|
/pj-rehearse ack |
|
@smg247: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
openshift-merge-bot
Bot
added
the
rehearsals-ack
mstaeble
left a comment
mstaeble
left a comment
There was a problem hiding this comment.
Overall, this looks good.
I left some things to consider. But given the nature of the work, I am happy to approve and let us iterate. Feel free to cancel the hold as you see fit.
/lgtm
/hold
openshift-ci
Bot
added
the
do-not-merge/hold
- Fail fast if SKILL.md is missing instead of silently skipping - Pre-substitute $1/$2/$3 args via sed so instructions are self-contained - Reorder: append solve-config.md after SKILL.md so repo-specific config takes precedence - Redirect .work/ plans to /tmp/ to avoid polluting PR diffs - Simplify -p prompt now that substituted SKILL.md covers branch/push/CI Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
openshift-merge-bot
Bot
removed
the
rehearsals-ack
|
These are unfortunate effects of not being able to get the plugin itself installed in the environment. My first attempt was to do so, but there were missing deps and it didn't seem like something that would be doable without changing the implementation in |
|
/pj-rehearse skip |
|
@smg247: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
openshift-merge-bot
Bot
added
the
rehearsals-ack
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: mstaeble, smg247 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
/hold cancel |
openshift-ci
Bot
removed
the
do-not-merge/hold
|
/override-sticky ci/prow/jira-solver-eval-jira-solver-eval |
|
@smg247: Overrode contexts on behalf of smg247: ci/prow/jira-solver-eval-jira-solver-eval These overrides will persist across retests on the current HEAD SHA. Pushing a new commit will clear them. Use DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
3 participants
Replace hand-crafted prompt heredocs with standardized skills from openshift-eng/ai-helpers:
/jira:solveskill invocation in --ci mode. Pre-fetched issue JSON is passed via system prompt since CI lacks Jira credentials. Repo-specific solve-config.md still appended via --append-system-prompt-file.PR creation, eval mode, timeout/nudge, and artifact handling unchanged.
Summary by CodeRabbit
This PR updates OpenShift CI agent workflows to use standardized skills from
openshift-eng/ai-helpers./jira:solvewith pre-fetched Jira data and repository-specific configuration./openshift-developer:address-review-prand monitors PR comments and failing checks.