Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for specifying Kube API server advertise address/port #356

Merged
merged 1 commit into from Jul 15, 2021

Conversation

csrwng
Copy link
Contributor

@csrwng csrwng commented Jul 14, 2021

Why is this needed? In hypershift clusters, we run an HA proxy server on
each worker. This HA proxy server listens on a local interface and sends
traffic to the external control plane endpoint. To listen on the local
interface, we assign a dummy address to the loopback device and make
that the API server's advertise address. In some cases, it may be
necessary to change what this dummy address is to not interfere with
valid network address ranges in the customer's environment. This change
allows specifying that through the API.

The secure port at which the API server listens is also the same port at
which the HA proxy must listen on worker nodes. In the case of ROKS,
this has historically been different than the default (2040 instead of
6443). It may also be necessary to modify this port to not interfere
with pods listening on ports that are bound to the host network on
worker nodes.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 14, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: csrwng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot requested review from enxebre and sjenning July 14, 2021 15:02
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 14, 2021
@csrwng
Copy link
Contributor Author

csrwng commented Jul 14, 2021

/test e2e-aws

@csrwng
Copy link
Contributor Author

csrwng commented Jul 14, 2021

/cc @relyt0925

@sjenning
Copy link
Contributor

do we need to change these?

params.ExternalAPIAddress = DefaultAPIServerIPAddress

@csrwng
Copy link
Contributor Author

csrwng commented Jul 14, 2021

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 14, 2021
@relyt0925
Copy link
Contributor

relyt0925 commented Jul 14, 2021

this prototype pr might help provide reference:
relyt0925#24

Why is this needed? In hypershift clusters, we run an HA proxy server on
each worker. This HA proxy server listens on a local interface and sends
traffic to the external control plane endpoint. To listen on the local
interface, we assign a dummy address to the loopback device and make
that the API server's advertise address. In some cases, it may be
necessary to change what this dummy address is to not interfere with
valid network address ranges in the customer's environment. This change
allows specifying that through the API.

The secure port at which the API server listens is also the same port at
which the HA proxy must listen on worker nodes. In the case of ROKS,
this has historically been different than the default (2040 instead of
6443). It may also be necessary to modify this port to not interfere
with pods listening on ports that are bound to the host network on
worker nodes.
@csrwng
Copy link
Contributor Author

csrwng commented Jul 15, 2021

/test e2e-aws

@csrwng
Copy link
Contributor Author

csrwng commented Jul 15, 2021

/hold cancel
This is working just as well as the main branch. While testing it, discovered that there's an issue with the communication between the kube apiserver and worker kubelets (oc get logs is not working). However, the issue is present in the main branch as well.

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 15, 2021
@relyt0925
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 15, 2021
@openshift-merge-robot openshift-merge-robot merged commit e1ed769 into openshift:main Jul 15, 2021
@relyt0925
Copy link
Contributor

this is a very exciting milestone :) great work all!

@csrwng csrwng deleted the kas_endpoint_api branch June 14, 2022 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants