Add periodic CI cluster-updates evaluation jobs for 4.22#82093
Conversation
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
Skipping CI for Draft Pull Request. |
|
/pj-rehearse |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (5)
🚧 Files skipped from review as they are similar to previous changes (5)
WalkthroughAdds OCP 4.22 periodic cluster evaluation jobs for four providers and introduces repository-specific Prow ownership, plugin, trigger trust, Slack reporting, and Tide configuration. ChangesOCP evals repository onboarding
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant PeriodicScheduler
participant GenericClaimWorkflow
participant ClusterUpdateTest
PeriodicScheduler->>GenericClaimWorkflow: Start provider-specific periodic job
GenericClaimWorkflow->>ClusterUpdateTest: Export settings and run test-cluster-updates.sh
ClusterUpdateTest->>ClusterUpdateTest: Use mounted credentials and lightspeed-service-api image
Suggested labels: 🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: fao89 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
|
/pj-rehearse periodic-ci-fao89-cluster-updates-evals-main-4.22-cu-eval-openai-periodics |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/config/fao89/cluster-updates-evals/fao89-cluster-updates-evals-main__4.22.yaml`:
- Around line 25-180: Run make update to regenerate the downstream Prow job
definitions for the four new periodic tests—cu-eval-openai-periodics,
cu-eval-watsonx-periodics, cu-eval-vertex-periodics, and
cu-eval-vertex-anthropic-periodics—and include all generated artifacts in the
change.
- Around line 9-11: Update the Dockerfile fragment after the Git installation in
the main image build to restore the image’s standard non-root runtime user
instead of leaving USER 0 active. Add the appropriate non-root USER directive
immediately after the dnf cleanup, preserving the existing installation steps.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 48f09b52-3001-4666-874e-68ce37e75e35
⛔ Files ignored due to path filters (1)
ci-operator/jobs/fao89/cluster-updates-evals/fao89-cluster-updates-evals-main-periodics.yamlis excluded by!ci-operator/jobs/**
📒 Files selected for processing (5)
ci-operator/config/fao89/cluster-updates-evals/OWNERSci-operator/config/fao89/cluster-updates-evals/fao89-cluster-updates-evals-main__4.22.yamlcore-services/prow/02_config/fao89/cluster-updates-evals/OWNERScore-services/prow/02_config/fao89/cluster-updates-evals/_pluginconfig.yamlcore-services/prow/02_config/fao89/cluster-updates-evals/_prowconfig.yaml
| FROM registry.ci.openshift.org/ocp/ubi-python-311:9 | ||
| USER 0 | ||
| RUN dnf install -y git && dnf clean all |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file='ci-operator/config/fao89/cluster-updates-evals/fao89-cluster-updates-evals-main__4.22.yaml'
echo "== File context =="
wc -l "$file"
sed -n '1,220p' "$file" | nl -ba | sed -n '1,120p'
echo
echo "== Search for related build-root Dockerfiles / USER instructions =="
rg -n --hidden --no-messages 'USER 0|USER 1001|build-root|dnf install -y git|FROM registry\.ci\.openshift\.org/ocp/ubi-python-311:9' ci-operator/config/fao89/cluster-updates-evals -S
echo
echo "== Nearby generated metadata / job config references =="
rg -n --hidden --no-messages 'zz_generated_metadata|periodic|presubmit|postsubmit|build_root|build-root' ci-operator/config/fao89/cluster-updates-evals -SRepository: openshift/release
Length of output: 309
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file='ci-operator/config/fao89/cluster-updates-evals/fao89-cluster-updates-evals-main__4.22.yaml'
echo "== Selected line ranges =="
sed -n '1,80p' "$file"
echo
sed -n '80,160p' "$file"
echo
sed -n '160,220p' "$file"
echo
echo "== Exact build-root / user references =="
rg -n '^( {2,})?(FROM|USER|RUN|build_root|build-root|dockerfile|dockerfile_literal):|USER 0|USER 1001|dnf install -y git' "$file" -n -S
echo
echo "== Other similar configs with USER restore patterns =="
rg -n 'USER 0|USER 1001|dnf install -y git' ci-operator/config -SRepository: openshift/release
Length of output: 41037
Restore a non-root runtime user after installing Git.
ci-operator/config/fao89/cluster-updates-evals/fao89-cluster-updates-evals-main__4.22.yaml:9-11 leaves the build-root image running as root after dnf install. Switch back to the image’s standard non-root UID before it is used in CI.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/config/fao89/cluster-updates-evals/fao89-cluster-updates-evals-main__4.22.yaml`
around lines 9 - 11, Update the Dockerfile fragment after the Git installation
in the main image build to restore the image’s standard non-root runtime user
instead of leaving USER 0 active. Add the appropriate non-root USER directive
immediately after the dnf cleanup, preserving the existing installation steps.
Source: Coding guidelines
|
/pj-rehearse periodic-ci-fao89-cluster-updates-evals-main-4.22-cu-eval-openai-periodics |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-fao89-cluster-updates-evals-main-4.22-cu-eval-openai-periodics |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
0a4383c to
c883924
Compare
|
/pj-rehearse periodic-ci-fao89-cluster-updates-evals-main-4.22-cu-eval-openai-periodics |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-fao89-cluster-updates-evals-main-4.22-cu-eval-openai-periodics |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
c883924 to
f76aba2
Compare
|
/pj-rehearse |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
f76aba2 to
da44fcd
Compare
|
/pj-rehearse |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core-services/prow/02_config/fao89/ocp-evals/_pluginconfig.yaml`:
- Around line 9-17: Update the fao89/ocp-evals plugins list to include lgtm
alongside approve so Tide can satisfy both required labels; if this
repository-level plugin list controls trigger handling, also enable the trigger
plugin.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 6ceffa54-d9db-47f2-9b43-03c2f24f0607
⛔ Files ignored due to path filters (1)
ci-operator/jobs/fao89/ocp-evals/fao89-ocp-evals-main-periodics.yamlis excluded by!ci-operator/jobs/**
📒 Files selected for processing (5)
ci-operator/config/fao89/ocp-evals/OWNERSci-operator/config/fao89/ocp-evals/fao89-ocp-evals-main__4.22.yamlcore-services/prow/02_config/fao89/ocp-evals/OWNERScore-services/prow/02_config/fao89/ocp-evals/_pluginconfig.yamlcore-services/prow/02_config/fao89/ocp-evals/_prowconfig.yaml
Add CI configuration for cluster-updates-evals with periodic evaluation jobs for each LLM provider (OpenAI, WatsonX, Google Vertex, Google Vertex Anthropic) running daily against OCP 4.22 clusters. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Fabricio Aguiar <fabricio.aguiar@gmail.com> rh-pre-commit.version: 2.3.2 rh-pre-commit.check-secrets: ENABLED
da44fcd to
670b664
Compare
|
/pj-rehearse |
1 similar comment
|
/pj-rehearse |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse |
|
@fao89: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@fao89: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
1 participant
Add CI configuration for cluster-updates-evals with periodic evaluation jobs for each LLM provider (OpenAI, WatsonX, Google Vertex, Google Vertex Anthropic) running daily against OCP 4.22 clusters.
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
Summary by CodeRabbit
fao89/ocp-evalsto run on OCP nightly stream 4.22, including resource defaults and thelightspeed-service-apibase dependency.generic-claim) at 14:30, 15:30, 16:30, and 17:30:cu-eval-openai-periodicscu-eval-watsonx-periodicscu-eval-vertex-gemini-periodicscu-eval-vertex-anthropic-periodicsEVAL_TAG=cluster-updates-critical, mounts the relevant provider tokens (plus OpenAI), writes provider-specificconfig/system*.yamltoconfig/system.yamlas needed, and runstests/scripts/test-cluster-updates.sh --artifact-dir "${ARTIFACT_DIR}"(with must-gather collection).fao89/ocp-evals: OWNERS (approver/reviewerfao89), Prowapproveplugin configuration (self-approval required), Tide label gating (approved+lgtm), Slack notifications for periodic failures/errors, and trusted trigger viaopenshift-merge-bot.