Skip to content

OCPBUGS-99229: bootstrap: guard CVO render flags for version-skew compatibility - #10704

Closed
jira-solve-bot wants to merge 1 commit into
openshift:mainfrom
jira-solve-bot:fix-OCPBUGS-99229
Closed

OCPBUGS-99229: bootstrap: guard CVO render flags for version-skew compatibility#10704
jira-solve-bot wants to merge 1 commit into
openshift:mainfrom
jira-solve-bot:fix-OCPBUGS-99229

Conversation

@jira-solve-bot

@jira-solve-bot jira-solve-bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Description

The bootkube.sh script unconditionally passes --cluster-version-manifest-path and --feature-gate-manifest-path to the CVO render command. When a 4.22 installer bootstraps with a 4.21 release image whose CVO binary does not support these flags, the cvo-render container crashes with "unknown flag", causing bootstrap to fail deterministically.

This PR:

  1. Probes the CVO render --help output before invoking render, and only passes flags that the release image's CVO binary actually supports. This follows the existing ADDITIONAL_FLAGS pattern used elsewhere in bootkube.sh (e.g. the config-bootstrap block) and is forward-compatible with future flag additions.

  2. Logs a warning when the CVO render --help probe fails (e.g., due to image-pull errors or podman crashes), so operators have a diagnostic breadcrumb in the journal if flags are unexpectedly skipped.

Jira

https://redhat.atlassian.net/browse/OCPBUGS-99229


Always review AI generated responses prior to use.
Generated with Claude Code via openshift-developer plugin


Note: This PR was auto-generated by the jira-agent periodic CI job in response to OCPBUGS-99229. See the full report for token usage, cost breakdown, and detailed phase output.

Summary by CodeRabbit

  • Bug Fixes
    • Improved bootstrap compatibility across release versions by detecting supported cluster configuration options at runtime.
    • Prevented startup failures when optional configuration arguments are unavailable.

The bootkube.sh script unconditionally passes --cluster-version-manifest-path
and --feature-gate-manifest-path to the CVO render command. When a 4.22
installer bootstraps with a 4.21 release image whose CVO binary does not
support these flags, the cvo-render container crashes with "unknown flag",
causing bootstrap to fail deterministically.

Probe the CVO render --help output before invoking render, and only pass
flags that the release image's CVO binary actually supports. This follows
the existing ADDITIONAL_FLAGS pattern used elsewhere in bootkube.sh (e.g.
the config-bootstrap block) and is forward-compatible with future flag
additions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 22, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@jira-solve-bot: This pull request references Jira Issue OCPBUGS-99229, which is invalid:

  • expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Description

The bootkube.sh script unconditionally passes --cluster-version-manifest-path and --feature-gate-manifest-path to the CVO render command. When a 4.22 installer bootstraps with a 4.21 release image whose CVO binary does not support these flags, the cvo-render container crashes with "unknown flag", causing bootstrap to fail deterministically.

This PR:

  1. Probes the CVO render --help output before invoking render, and only passes flags that the release image's CVO binary actually supports. This follows the existing ADDITIONAL_FLAGS pattern used elsewhere in bootkube.sh (e.g. the config-bootstrap block) and is forward-compatible with future flag additions.

  2. Logs a warning when the CVO render --help probe fails (e.g., due to image-pull errors or podman crashes), so operators have a diagnostic breadcrumb in the journal if flags are unexpectedly skipped.

Jira

https://redhat.atlassian.net/browse/OCPBUGS-99229


Always review AI generated responses prior to use.
Generated with Claude Code via openshift-developer plugin

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Changes

Bootstrap render compatibility

Layer / File(s) Summary
Runtime flag detection and render wiring
data/data/bootstrap/files/usr/local/bin/bootkube.sh.template
The cvo-bootstrap stage probes render --help, adds supported manifest-path flags to ADDITIONAL_FLAGS, and appends them to the render command.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • openshift/installer#10700: Updates the same bootstrap template with runtime detection for optional render manifest-path flags.

Suggested labels: needs-ok-to-test

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: conditionally guarding CVO render flags for version-skew compatibility during bootstrap.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR only changes bootkube.sh.template; no Ginkgo test titles were added or modified.
Test Structure And Quality ✅ Passed PR only changes a shell template; no Ginkgo test code or fixtures were added or modified, so the test-structure check is not applicable.
Microshift Test Compatibility ✅ Passed PR only changes bootkube.sh.template; no new Ginkgo e2e tests or MicroShift-sensitive OpenShift API usage were added.
Single Node Openshift (Sno) Test Compatibility ✅ Passed Only bootkube.sh.template changed; no Ginkgo tests or SNO-relevant test logic were added, so the check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed Not applicable: the change only probes CVO render flags in a shell script and adds no manifests, controllers, or scheduling constraints.
Ote Binary Stdout Contract ✅ Passed PR only changes a shell bootstrap template; it does not add stdout writes in any OTE binary entrypoint, so the JSON stdout contract is unaffected.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed Only the bootkube shell template changed; no new Ginkgo e2e tests were added, so this IPv6/disconnected-network check is not applicable.
No-Weak-Crypto ✅ Passed Only bootkube flag-probing logic changed; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret comparisons were added.
Container-Privileges ✅ Passed Only bootkube.sh.template changed; the diff adds help-probe logic and no privileged/hostPID/hostNetwork/hostIPC/SYS_ADMIN/allowPrivilegeEscalation/root settings.
No-Sensitive-Data-In-Logs ✅ Passed The modified bootkube logs are static status messages; no secrets, tokens, PII, hostnames, or customer data are emitted.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Warning

⚠️ This pull request shows signs of AI-generated slop (defensive_cruft, description_diff_mismatch). It has been flagged by CodeRabbit slop detection and should be reviewed carefully.

@openshift-ci openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 22, 2026
@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Hi @jira-solve-bot. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tthvo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
openshift-ci Bot requested review from pawanpinjarkar and sadasu July 22, 2026 08:19
@enxebre

enxebre commented Jul 22, 2026

Copy link
Copy Markdown
Member

/close

@openshift-ci openshift-ci Bot closed this Jul 22, 2026
@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

@enxebre: Closed this PR.

Details

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@jira-solve-bot: This pull request references Jira Issue OCPBUGS-99229. The bug has been updated to no longer refer to the pull request using the external bug tracker. All external bug links have been closed. The bug has been moved to the NEW state.

Details

In response to this:

Description

The bootkube.sh script unconditionally passes --cluster-version-manifest-path and --feature-gate-manifest-path to the CVO render command. When a 4.22 installer bootstraps with a 4.21 release image whose CVO binary does not support these flags, the cvo-render container crashes with "unknown flag", causing bootstrap to fail deterministically.

This PR:

  1. Probes the CVO render --help output before invoking render, and only passes flags that the release image's CVO binary actually supports. This follows the existing ADDITIONAL_FLAGS pattern used elsewhere in bootkube.sh (e.g. the config-bootstrap block) and is forward-compatible with future flag additions.

  2. Logs a warning when the CVO render --help probe fails (e.g., due to image-pull errors or podman crashes), so operators have a diagnostic breadcrumb in the journal if flags are unexpectedly skipped.

Jira

https://redhat.atlassian.net/browse/OCPBUGS-99229


Always review AI generated responses prior to use.
Generated with Claude Code via openshift-developer plugin


Note: This PR was auto-generated by the jira-agent periodic CI job in response to OCPBUGS-99229. See the full report for token usage, cost breakdown, and detailed phase output.

Summary by CodeRabbit

  • Bug Fixes
  • Improved bootstrap compatibility across release versions by detecting supported cluster configuration options at runtime.
  • Prevented startup failures when optional configuration arguments are unavailable.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@data/data/bootstrap/files/usr/local/bin/bootkube.sh.template`:
- Around line 199-203: Update the CVO_RENDER_HELP compatibility probe around
bootkube_podman_run to preserve its exit status instead of appending “|| true”.
Use an if ! branch to log a warning when the probe fails, clear CVO_RENDER_HELP,
and then continue with the fallback behavior so failed probes cannot be mistaken
for successful unsupported-flag checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 0d7b1be8-28a3-4fed-881c-0696cae87686

📥 Commits

Reviewing files that changed from the base of the PR and between b7140c7 and 6f7a588.

📒 Files selected for processing (1)
  • data/data/bootstrap/files/usr/local/bin/bootkube.sh.template

Comment on lines +199 to +203
CVO_RENDER_HELP=$(bootkube_podman_run \
--name cvo-render-help \
--volume "$PWD:/assets:z" \
"${RELEASE_IMAGE_DIGEST}" \
render --help 2>&1) || true

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Do not discard a failed compatibility probe.

|| true makes a missing or broken release image look identical to a successful CVO that does not support these flags. Bootstrap then silently omits the manifest-path overrides and cannot reliably warn about the failure. Capture the status with an if ! ...; then branch, log the warning, and clear the help output before falling back.

As per the PR objective, failed probes must produce a warning.

Suggested failure handling
- CVO_RENDER_HELP=$(bootkube_podman_run \
+ if ! CVO_RENDER_HELP=$(bootkube_podman_run \
 		--name cvo-render-help \
 		--volume "$PWD:/assets:z" \
 		"${RELEASE_IMAGE_DIGEST}" \
-		render --help 2>&1) || true
+		render --help 2>&1); then
+		echo "WARNING: failed to probe CVO render flags; continuing without overrides" >&2
+		CVO_RENDER_HELP=""
+	fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
CVO_RENDER_HELP=$(bootkube_podman_run \
--name cvo-render-help \
--volume "$PWD:/assets:z" \
"${RELEASE_IMAGE_DIGEST}" \
render --help 2>&1) || true
if ! CVO_RENDER_HELP=$(bootkube_podman_run \
--name cvo-render-help \
--volume "$PWD:/assets:z" \
"${RELEASE_IMAGE_DIGEST}" \
render --help 2>&1); then
echo "WARNING: failed to probe CVO render flags; continuing without overrides" >&2
CVO_RENDER_HELP=""
fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@data/data/bootstrap/files/usr/local/bin/bootkube.sh.template` around lines
199 - 203, Update the CVO_RENDER_HELP compatibility probe around
bootkube_podman_run to preserve its exit status instead of appending “|| true”.
Use an if ! branch to log a warning when the probe fails, clear CVO_RENDER_HELP,
and then continue with the fallback behavior so failed probes cannot be mistaken
for successful unsupported-flag checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants