OPNET-780: Add BGPBasedVIPManagement feature gate and BGP VIP management fields#2923
OPNET-780: Add BGPBasedVIPManagement feature gate and BGP VIP management fields#2923mkowalski wants to merge 2 commits into
Conversation
|
Pipeline controller notification For optional jobs, comment This repository is configured in: LGTM mode |
openshift-ci-robot
added
the
jira/valid-reference
|
@mkowalski: This pull request references OPNET-595 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the epic to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
Hello @mkowalski! Some important instructions when contributing to openshift/api: |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: ⛔ Files ignored due to path filters (19)
📒 Files selected for processing (26)
✅ Files skipped from review due to trivial changes (5)
🚧 Files skipped from review as they are similar to previous changes (15)
📝 WalkthroughWalkthroughThis PR adds the 🚥 Pre-merge checks | ✅ 14 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (14 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)Error: build linters: unable to load custom analyzer "kubeapilinter": tools/_output/bin/kube-api-linter.so, plugin: not implemented Comment |
openshift-ci
Bot
added
the
size/XXL
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
mkowalski
changed the title
|
@mkowalski: This pull request references OPNET-780 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
…ield BGP-based VIP management for on-premise clusters (enhancement 1982, OPNET-595): the gate ships in DevPreviewNoUpgrade; BareMetalPlatformStatus.VIPManagement reports which mechanism (Keepalived/BGP) manages the API and Ingress VIPs. Set at install time by the installer; consumed by MCO and CNO. Safe with no backing implementation: feature-gated (field pruned outside DevPreviewNoUpgrade CRD variants), optional, and nothing in-payload sets or reads it until the implementation lands.
Internal MCO API carrying the installer-generated BGP peer configuration from the bgp-vip-config ConfigMap to the template controller, which renders the frr-k8s static pod peer file on control plane nodes (enhancement 1982, OPNET-595). Feature-gated, optional and bounded; a no-op until the MCO implementation consumes it. The controllerconfigs CRD manifests also pick up the vipManagement enum refresh through the embedded Infrastructure schema.
mkowalski
force-pushed
the
opnet-595-bgp-vip-api
branch
from
661c092 to
00a1277
Compare
|
@mkowalski: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Render a sessions-only FRRConfiguration CR named bgp-vip (namespace openshift-frr-k8s) when BGP-based VIP management is active: BareMetal platform, the BGPBasedVIPManagement feature gate enabled, and the Infrastructure CR reporting vipManagement "BGP". Peer/VIP data is read from the installer's bgp-vip-config ConfigMap (schema = baremetal-runtimecfg's FRRPeerMapping). The CR carries only the BGP sessions (neighbors, BFD profiles). VIP advertisement deliberately does not use CRD prefixes/toAdvertise: frr-k8s renders router-level prefixes as unconditional `network` statements that would defeat kube-vip's health gating, and toAdvertise cannot express "advertise redistributed routes" - without it frr-k8s renders deny-any egress prefix-lists. Advertisement therefore happens exclusively via rawConfig: `ip import-table 198` plus health-gated table-direct redistribution of routing table 198 (populated by kube-vip only for VIPs with healthy backends), filtered through route-maps/prefix-lists permitting exactly the VIP prefixes, with high-sequence per-neighbor <peer>-out permits opening egress only for the VIP prefix-lists. The CR carries no node selector: workers' frr-k8s DaemonSet consumes the same sessions and gated redistribution so router-bearing workers advertise the ingress VIP. The feature gate is defined locally and guarded via KnownFeatures until openshift/api#2923 ships the gate, and the Infrastructure vipManagement field is read unstructured until it lands in the vendored openshift/api - the feature is inert until then. Validated end to end (github.com/mkowalski/bgp-vip-demo).
There was a problem hiding this comment.
Pull request overview
This PR introduces the API surface (behind a new BGPBasedVIPManagement feature gate) needed to support BGP-based management of API/Ingress VIPs on on-premise clusters, along with the corresponding feature-set manifests, CRD/OpenAPI regeneration, and validation tests.
Changes:
- Adds the
BGPBasedVIPManagementfeature gate (enabled only inDevPreviewNoUpgrade) and wires it into payload featuregate manifests. - Adds feature-gated API fields:
config/v1:BareMetalPlatformStatus.VIPManagementto report the active VIP management mechanism (Keepalived/BGP).machineconfiguration/v1:ControllerConfigSpec.BGPVIPPeersJSONto carry installer-provided BGP peer config to MCO internals.
- Updates generated CRDs/OpenAPI artifacts and adds feature-gated API validation tests.
Reviewed changes
Copilot reviewed 43 out of 45 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| payload-manifests/featuregates/featureGate-4-10-SelfManagedHA-TechPreviewNoUpgrade.yaml | Adds BGPBasedVIPManagement to the disabled set for this feature-set/profile. |
| payload-manifests/featuregates/featureGate-4-10-SelfManagedHA-OKD.yaml | Adds BGPBasedVIPManagement to the disabled set for OKD. |
| payload-manifests/featuregates/featureGate-4-10-SelfManagedHA-DevPreviewNoUpgrade.yaml | Ensures BGPBasedVIPManagement is enabled in DevPreviewNoUpgrade. |
| payload-manifests/featuregates/featureGate-4-10-SelfManagedHA-Default.yaml | Adds BGPBasedVIPManagement to the disabled set for Default. |
| payload-manifests/featuregates/featureGate-4-10-Hypershift-TechPreviewNoUpgrade.yaml | Adds BGPBasedVIPManagement to the disabled set for this feature-set/profile. |
| payload-manifests/featuregates/featureGate-4-10-Hypershift-OKD.yaml | Adds BGPBasedVIPManagement to the disabled set for OKD. |
| payload-manifests/featuregates/featureGate-4-10-Hypershift-DevPreviewNoUpgrade.yaml | Ensures BGPBasedVIPManagement is enabled in DevPreviewNoUpgrade. |
| payload-manifests/featuregates/featureGate-4-10-Hypershift-Default.yaml | Adds BGPBasedVIPManagement to the disabled set for Default. |
| payload-manifests/crds/0000_80_machine-config_01_controllerconfigs-TechPreviewNoUpgrade.crd.yaml | Generated CRD update reflecting regrouping/annotations. |
| payload-manifests/crds/0000_80_machine-config_01_controllerconfigs-SelfManagedHA-DevPreviewNoUpgrade.crd.yaml | Generated CRD includes feature-gated bgpVIPPeersJSON and related schema updates. |
| payload-manifests/crds/0000_80_machine-config_01_controllerconfigs-SelfManagedHA-CustomNoUpgrade.crd.yaml | Generated CRD includes feature-gated bgpVIPPeersJSON and related schema updates. |
| payload-manifests/crds/0000_80_machine-config_01_controllerconfigs-Hypershift-DevPreviewNoUpgrade.crd.yaml | Generated CRD includes feature-gated bgpVIPPeersJSON and related schema updates. |
| payload-manifests/crds/0000_80_machine-config_01_controllerconfigs-Hypershift-CustomNoUpgrade.crd.yaml | Generated CRD includes feature-gated bgpVIPPeersJSON and related schema updates. |
| payload-manifests/crds/0000_10_config-operator_01_infrastructures-TechPreviewNoUpgrade.crd.yaml | Generated CRD update reflecting regrouping/annotations. |
| payload-manifests/crds/0000_10_config-operator_01_infrastructures-SelfManagedHA-DevPreviewNoUpgrade.crd.yaml | Generated CRD includes feature-gated vipManagement schema. |
| payload-manifests/crds/0000_10_config-operator_01_infrastructures-SelfManagedHA-CustomNoUpgrade.crd.yaml | Generated CRD includes feature-gated vipManagement schema. |
| payload-manifests/crds/0000_10_config-operator_01_infrastructures-Hypershift-TechPreviewNoUpgrade.crd.yaml | Removes/regroups generated CRD due to manifest-merge grouping changes. |
| payload-manifests/crds/0000_10_config-operator_01_infrastructures-Hypershift-DevPreviewNoUpgrade.crd.yaml | Generated CRD includes feature-gated vipManagement schema. |
| payload-manifests/crds/0000_10_config-operator_01_infrastructures-Hypershift-CustomNoUpgrade.crd.yaml | Generated CRD includes feature-gated vipManagement schema. |
| openapi/openapi.json | Updates generated OpenAPI definitions to include vipManagement field. |
| openapi/generated_openapi/zz_generated.openapi.go | Updates generated Go OpenAPI schema for the new field(s). |
| machineconfiguration/v1/zz_generated.swagger_doc_generated.go | Updates generated swagger docs for bgpVIPPeersJSON. |
| machineconfiguration/v1/zz_generated.featuregated-crd-manifests/controllerconfigs.machineconfiguration.openshift.io/BGPBasedVIPManagement.yaml | Generated feature-gated CRD manifest updated for the new field. |
| machineconfiguration/v1/zz_generated.featuregated-crd-manifests.yaml | Registers the new feature gate for feature-gated CRD generation. |
| machineconfiguration/v1/zz_generated.crd-manifests/0000_80_machine-config_01_controllerconfigs-TechPreviewNoUpgrade.crd.yaml | Generated CRD update reflecting regrouping/annotations. |
| machineconfiguration/v1/zz_generated.crd-manifests/0000_80_machine-config_01_controllerconfigs-SelfManagedHA-DevPreviewNoUpgrade.crd.yaml | Generated CRD includes feature-gated bgpVIPPeersJSON. |
| machineconfiguration/v1/zz_generated.crd-manifests/0000_80_machine-config_01_controllerconfigs-SelfManagedHA-CustomNoUpgrade.crd.yaml | Generated CRD includes feature-gated bgpVIPPeersJSON. |
| machineconfiguration/v1/zz_generated.crd-manifests/0000_80_machine-config_01_controllerconfigs-Hypershift-DevPreviewNoUpgrade.crd.yaml | Generated CRD includes feature-gated bgpVIPPeersJSON. |
| machineconfiguration/v1/zz_generated.crd-manifests/0000_80_machine-config_01_controllerconfigs-Hypershift-CustomNoUpgrade.crd.yaml | Generated CRD includes feature-gated bgpVIPPeersJSON. |
| machineconfiguration/v1/types.go | Adds ControllerConfigSpec.BGPVIPPeersJSON behind BGPBasedVIPManagement. |
| machineconfiguration/v1/tests/controllerconfigs.machineconfiguration.openshift.io/BGPBasedVIPManagement.yaml | Adds feature-gated CRD validation tests for bgpVIPPeersJSON. |
| features/features.go | Registers the new BGPBasedVIPManagement feature gate (DevPreviewNoUpgrade). |
| features.md | Updates the feature gate matrix to include BGPBasedVIPManagement. |
| config/v1/zz_generated.swagger_doc_generated.go | Updates generated swagger docs for vipManagement. |
| config/v1/zz_generated.featuregated-crd-manifests/infrastructures.config.openshift.io/BGPBasedVIPManagement.yaml | Generated feature-gated CRD manifest updated for the new field. |
| config/v1/zz_generated.featuregated-crd-manifests.yaml | Registers the new feature gate for feature-gated CRD generation. |
| config/v1/zz_generated.crd-manifests/0000_10_config-operator_01_infrastructures-TechPreviewNoUpgrade.crd.yaml | Generated CRD update reflecting regrouping/annotations. |
| config/v1/zz_generated.crd-manifests/0000_10_config-operator_01_infrastructures-SelfManagedHA-DevPreviewNoUpgrade.crd.yaml | Generated CRD includes feature-gated vipManagement. |
| config/v1/zz_generated.crd-manifests/0000_10_config-operator_01_infrastructures-SelfManagedHA-CustomNoUpgrade.crd.yaml | Generated CRD includes feature-gated vipManagement. |
| config/v1/zz_generated.crd-manifests/0000_10_config-operator_01_infrastructures-Hypershift-DevPreviewNoUpgrade.crd.yaml | Generated CRD includes feature-gated vipManagement. |
| config/v1/zz_generated.crd-manifests/0000_10_config-operator_01_infrastructures-Hypershift-CustomNoUpgrade.crd.yaml | Generated CRD includes feature-gated vipManagement. |
| config/v1/types_infrastructure.go | Adds BareMetalPlatformStatus.VIPManagement behind BGPBasedVIPManagement. |
| config/v1/types_infrastructure_test.go | Updates the TechPreview CRD filename reference to match regrouped manifests. |
| config/v1/tests/infrastructures.config.openshift.io/BGPBasedVIPManagement.yaml | Adds feature-gated CRD validation tests for vipManagement. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // bgpVIPPeersJSON carries the BGP VIP peer configuration (the config.json | ||
| // payload of the bgp-vip-config ConfigMap) for rendering the frr-k8s | ||
| // static pod peer file on control plane nodes. Only set when BGP-based | ||
| // VIP management is enabled. | ||
| // When set, the value must be between 1 and 65536 characters long. |
| // to "Keepalived", the default keepalived-based VIP management is used. | ||
| // +kubebuilder:validation:Enum=Keepalived;BGP | ||
| // +openshift:enable:FeatureGate=BGPBasedVIPManagement | ||
| // +optional | ||
| VIPManagement string `json:"vipManagement,omitempty"` |
What
API surface for BGP-based VIP management on on-premise clusters — enhancement openshift/enhancements#1982 (OPNET-595). Two commits:
BGPBasedVIPManagementfeature gate (DevPreviewNoUpgrade) andBareMetalPlatformStatus.VIPManagement— reports which mechanism (Keepalived/BGP) manages the API and Ingress VIPs. Set at install time by the installer; consumed by MCO and CNO.ControllerConfigSpec.BGPVIPPeersJSON— internal MCO API carrying the installer-generated BGP peer configuration (thebgp-vip-configConfigMap payload, validated and compacted by the MCO operator) to the template controller, which renders the frr-k8s static pod peer file on control plane nodes. Bounded (1–65536), optional.Safety without backing implementation
Deliberately safe to merge stand-alone:
+openshift:enable:FeatureGate=BGPBasedVIPManagement; the gate is enabled only in DevPreviewNoUpgrade (verified across all 8 payload featuregate manifests). The fields are pruned from Default/TechPreview/OKD CRD variants (verified: zero occurrences outside CustomNoUpgrade/DevPreviewNoUpgrade schemas, including the ControllerConfig CRDs that embed Infrastructure).+optional/omitemptywith no defaults; nothing in-payload sets or reads them until the implementation PRs land.make lintclean,make updateidempotent on the branch.Reviewer note on the large generated diff in commit 1
Enabling a profile-agnostic gate in DevPreviewNoUpgrade changes which feature-set CRD schemas are byte-identical, so the manifest-merge generator regroups files (SelfManagedHA {CustomNoUpgrade,DevPreviewNoUpgrade} pair up; Hypershift TechPreview variants merge across profiles). Verified with a control experiment:
make updatetwice on pristine master produces zero changes — the regrouping is entirely attributable to the new gate, pertools/codegen/pkg/manifestmerge/generator.gogrouping logic.Validation
The consuming implementation exists and was validated end to end on a dev-scripts baremetal cluster (installation over a BGP-advertised API VIP, health-gated ECMP for both VIPs, CRD handover); reference implementation and evidence: https://github.com/mkowalski/bgp-vip-demo. Related PRs in flight: kube-vip/kube-vip#1627, openshift-metal3/dev-scripts#1929.