Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add vsphere ELB config #47676

Merged
merged 1 commit into from
Mar 27, 2024
Merged

add vsphere ELB config #47676

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
26 changes: 26 additions & 0 deletions ...penshift-tests-private/openshift-openshift-tests-private-release-4.16__amd64-nightly.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2658,6 +2658,32 @@ tests:
test:
- chain: openshift-e2e-test-qe-destructive
workflow: cucushift-installer-rehearse-vsphere-ipi-zones-multisubnets-external-lb
- as: vsphere-ipi-multisubnets-elb-usermanaged-f28
cron: 52 14 26 * *
steps:
cluster_profile: vsphere-multizone-2
env:
LB_TYPE: UserManaged
leases:
- count: 2
env: VSPHERE_EXTRA_LEASED_RESOURCE
resource_type: vsphere-multizone-2-quota-slice
test:
- chain: openshift-e2e-test-qe
workflow: cucushift-installer-rehearse-vsphere-ipi-zones-multisubnets-external-lb
- as: vsphere-ipi-multisubnets-elb-usermanaged-f28-destructive
cron: 25 8 30 * *
steps:
cluster_profile: vsphere-multizone-2
env:
LB_TYPE: UserManaged
leases:
- count: 2
env: VSPHERE_EXTRA_LEASED_RESOURCE
resource_type: vsphere-multizone-2-quota-slice
test:
- chain: openshift-e2e-test-qe-destructive
workflow: cucushift-installer-rehearse-vsphere-ipi-zones-multisubnets-external-lb
- as: vsphere-ipi-zones-tp-f28
cron: 35 3 3 * *
steps:
Expand Down
176 changes: 176 additions & 0 deletions ...ift/openshift-tests-private/openshift-openshift-tests-private-release-4.16-periodics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29399,6 +29399,182 @@ periodics:
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: vsphere02
cron: 52 14 26 * *
decorate: true
decoration_config:
skip_cloning: true
extra_refs:
- base_ref: release-4.16
org: openshift
repo: openshift-tests-private
labels:
ci-operator.openshift.io/cloud: vsphere
ci-operator.openshift.io/cloud-cluster-profile: vsphere-multizone-2
ci-operator.openshift.io/variant: amd64-nightly
ci.openshift.io/generator: prowgen
job-release: "4.16"
pj-rehearse.openshift.io/can-be-rehearsed: "true"
name: periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-vsphere-ipi-multisubnets-elb-usermanaged-f28
spec:
containers:
- args:
- --gcs-upload-secret=/secrets/gcs/service-account.json
- --image-import-pull-secret=/etc/pull-secret/.dockerconfigjson
- --lease-server-credentials-file=/etc/boskos/credentials
- --oauth-token-path=/usr/local/github-credentials/oauth
- --report-credentials-file=/etc/report/credentials
- --secret-dir=/secrets/ci-pull-credentials
- --secret-dir=/usr/local/vsphere-ipi-multisubnets-elb-usermanaged-f28-cluster-profile
- --target=vsphere-ipi-multisubnets-elb-usermanaged-f28
- --variant=amd64-nightly
command:
- ci-operator
image: ci-operator:latest
imagePullPolicy: Always
name: ""
resources:
requests:
cpu: 10m
volumeMounts:
- mountPath: /etc/boskos
name: boskos
readOnly: true
- mountPath: /secrets/ci-pull-credentials
name: ci-pull-credentials
readOnly: true
- mountPath: /usr/local/vsphere-ipi-multisubnets-elb-usermanaged-f28-cluster-profile
name: cluster-profile
- mountPath: /secrets/gcs
name: gcs-credentials
readOnly: true
- mountPath: /usr/local/github-credentials
name: github-credentials-openshift-ci-robot-private-git-cloner
readOnly: true
- mountPath: /secrets/manifest-tool
name: manifest-tool-local-pusher
readOnly: true
- mountPath: /etc/pull-secret
name: pull-secret
readOnly: true
- mountPath: /etc/report
name: result-aggregator
readOnly: true
serviceAccountName: ci-operator
volumes:
- name: boskos
secret:
items:
- key: credentials
path: credentials
secretName: boskos-credentials
- name: ci-pull-credentials
secret:
secretName: ci-pull-credentials
- name: cluster-profile
secret:
secretName: cluster-secrets-vsphere
- name: github-credentials-openshift-ci-robot-private-git-cloner
secret:
secretName: github-credentials-openshift-ci-robot-private-git-cloner
- name: manifest-tool-local-pusher
secret:
secretName: manifest-tool-local-pusher
- name: pull-secret
secret:
secretName: registry-pull-credentials
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: vsphere02
cron: 25 8 30 * *
decorate: true
decoration_config:
skip_cloning: true
extra_refs:
- base_ref: release-4.16
org: openshift
repo: openshift-tests-private
labels:
ci-operator.openshift.io/cloud: vsphere
ci-operator.openshift.io/cloud-cluster-profile: vsphere-multizone-2
ci-operator.openshift.io/variant: amd64-nightly
ci.openshift.io/generator: prowgen
job-release: "4.16"
pj-rehearse.openshift.io/can-be-rehearsed: "true"
name: periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-vsphere-ipi-multisubnets-elb-usermanaged-f28-destructive
spec:
containers:
- args:
- --gcs-upload-secret=/secrets/gcs/service-account.json
- --image-import-pull-secret=/etc/pull-secret/.dockerconfigjson
- --lease-server-credentials-file=/etc/boskos/credentials
- --oauth-token-path=/usr/local/github-credentials/oauth
- --report-credentials-file=/etc/report/credentials
- --secret-dir=/secrets/ci-pull-credentials
- --secret-dir=/usr/local/vsphere-ipi-multisubnets-elb-usermanaged-f28-destructive-cluster-profile
- --target=vsphere-ipi-multisubnets-elb-usermanaged-f28-destructive
- --variant=amd64-nightly
command:
- ci-operator
image: ci-operator:latest
imagePullPolicy: Always
name: ""
resources:
requests:
cpu: 10m
volumeMounts:
- mountPath: /etc/boskos
name: boskos
readOnly: true
- mountPath: /secrets/ci-pull-credentials
name: ci-pull-credentials
readOnly: true
- mountPath: /usr/local/vsphere-ipi-multisubnets-elb-usermanaged-f28-destructive-cluster-profile
name: cluster-profile
- mountPath: /secrets/gcs
name: gcs-credentials
readOnly: true
- mountPath: /usr/local/github-credentials
name: github-credentials-openshift-ci-robot-private-git-cloner
readOnly: true
- mountPath: /secrets/manifest-tool
name: manifest-tool-local-pusher
readOnly: true
- mountPath: /etc/pull-secret
name: pull-secret
readOnly: true
- mountPath: /etc/report
name: result-aggregator
readOnly: true
serviceAccountName: ci-operator
volumes:
- name: boskos
secret:
items:
- key: credentials
path: credentials
secretName: boskos-credentials
- name: ci-pull-credentials
secret:
secretName: ci-pull-credentials
- name: cluster-profile
secret:
secretName: cluster-secrets-vsphere
- name: github-credentials-openshift-ci-robot-private-git-cloner
secret:
secretName: github-credentials-openshift-ci-robot-private-git-cloner
- name: manifest-tool-local-pusher
secret:
secretName: manifest-tool-local-pusher
- name: pull-secret
secret:
secretName: registry-pull-credentials
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: vsphere02
cron: 26 0 3 * *
Expand Down
23 changes: 21 additions & 2 deletions ...istry/ipi/conf/vsphere/zones/multisubnets/ipi-conf-vsphere-zones-multisubnets-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,23 @@ declare vsphere_extra_portgroup_2
source "${SHARED_DIR}/vsphere_context.sh"
# shellcheck source=/dev/null
source "${SHARED_DIR}/govc.sh"
declare -a vips
mapfile -t vips < "${SHARED_DIR}"/vips.txt

# if loadbalancer is UserManaged, it's mean using external LB,
# then keepalived and haproxy will not deployed, but coredns still keep
if [[ ${LB_TYPE} == "UserManaged" ]]; then
APIVIPS_DEF="apiVIPs:
- ${vips[0]}"
INGRESSVIPS_DEF="ingressVIPs:
- ${vips[1]}"
LB_TYPE_DEF="loadBalancer:
type: UserManaged"
else
APIVIPS_DEF=""
INGRESSVIPS_DEF=""
LB_TYPE_DEF=""
fi

CONFIG="${SHARED_DIR}/install-config.yaml"
base_domain=$(<"${SHARED_DIR}"/basedomain.txt)
Expand Down Expand Up @@ -51,8 +68,9 @@ compute:
- "us-east-3"
platform:
vsphere:
apiVIP:
ingressVIP:
${LB_TYPE_DEF}
${APIVIPS_DEF}
${INGRESSVIPS_DEF}
vCenter: "${vsphere_url}"
username: "${GOVC_USERNAME}"
password: ${GOVC_PASSWORD}
Expand Down Expand Up @@ -99,6 +117,7 @@ networking:
machineNetwork:
- cidr: "${machine_cidr}"
EOF

# TODO: Add this back in once we have an vsphere
# environment that will support topology storage

Expand Down
5 changes: 5 additions & 0 deletions ...registry/ipi/conf/vsphere/zones/multisubnets/ipi-conf-vsphere-zones-multisubnets-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@ ref:
as: ipi-conf-vsphere-zones-multisubnets
from: upi-installer
commands: ipi-conf-vsphere-zones-multisubnets-commands.sh
env:
- name: LB_TYPE
default: OpenShiftManagedDefault
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remember right, the ipi-conf-vsphere-zones-multisubnets is used for cucushift-installer-rehearse-vsphere-ipi-zones-multisubnets-external-lb, which means it using external lb, and leave apiVIPs and ingressVIPs to be empty on purpose, I feel like your case deserve a separate install workflow, but not using the existing one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there should be 2 ways to setup cluster with ELB for now

  1. set apiVIPS and ingressVIPs empty and do not set lb_type
  2. Set apiVIPS and ingressVIPs with lb_type is usermanaged. for feature https://issues.redhat.com/browse/OPNET-305

using the existing one is easy to implement 2.

documentation: |-
There are two values: OpenShiftManagedDefault and UserManaged, if UserManaged will use external LB.
resources:
requests:
cpu: 10m
Expand Down