OSASINFRA-902: Update OpenStack Custom External LB and DNS Docs #4389

iamemilio · 2020-11-17T21:31:45Z

The underlying network architecture has changed a lot since these docs
were initially written. We want to make sure that these docs are accurate
and up to date so that users with complex networking use cases like workers
on a custom subnet and baremetal workers are able to manage their ingress/egress
traffic as needed. More up to date examples and reference information has been added.

We have chosen to omit sections outlining how to replace the internal lb and dns
services since they were inaccurate. We are targeting an upcoming release to handle
these features better given the complexity of our current networking architecture.

iamemilio · 2020-11-17T21:32:05Z

/label platform/openstack
/assign mandre

iamemilio · 2020-11-17T21:36:23Z

/cc @maxwelldb

maxwelldb · 2020-11-18T20:13:19Z

@iamemilio Would you like suggestions on this, or is the review request more of an FYI only thing?

iamemilio · 2020-11-19T04:19:46Z

Suggestions sound great, but up to you.

EmilienM · 2020-11-23T15:47:55Z

/lgtm
/approve

iamemilio · 2020-11-23T15:50:25Z

/hold lets give people some time to put reviews in

maxwelldb

Made a number of comments. Feel free to accept/trash/interrogate however you like. @iamemilio

maxwelldb · 2020-11-23T18:18:48Z

docs/user/openstack/README.md


-This documents how to shift from the internal load balancer, which is intended for internal networking needs, to an external load balancer.
+This documents how to shift external traffic from the internal load balancer that comes stock with OpenShift on OpenStack to an external load balancer.


Suggested change

This documents how to shift external traffic from the internal load balancer that comes stock with OpenShift on OpenStack to an external load balancer.

You can shift network traffic from the default OpenShift on OpenStack load balancer to a load balancer that you provide.

(Or something like this. What's the importance of external and internal in an instructional sense? Could preserve those terms if it's important.)

maxwelldb · 2020-11-23T18:27:09Z

docs/user/openstack/README.md


-The load balancer must serve ports 6443, 443, and 80 to any users of the system.  Port 22623 is for serving ignition start-up configurations to the OpenShift nodes and should not be reachable outside of the cluster.
+It is essential that the instance your load balancing service is running from can reach all of the nodes in the cluster. One easy way to do this is to create the instance in a subnet that is within the OpenShift network created by your installer, and making sure a router interface is attached to that subnet from the OpenShift-external-router. Another solution is to attach a floating IP to the nodes that you want to add to your external load balancer.


Suggested change

It is essential that the instance your load balancing service is running from can reach all of the nodes in the cluster. One easy way to do this is to create the instance in a subnet that is within the OpenShift network created by your installer, and making sure a router interface is attached to that subnet from the OpenShift-external-router. Another solution is to attach a floating IP to the nodes that you want to add to your external load balancer.

To use your own load balancer, the instance that it runs from must be able to access every machine in your cluster. You might ensure this access by creating the instance on a subnet that is within your cluster's network, and then attaching a router interface to that subnet from the `OpenShift-external-router` [object/instance/whatever]. You can also attach a floating IP address to the machines that you want to add to your load balancer.

maxwelldb · 2020-11-23T18:27:31Z

docs/user/openstack/README.md

-openstack floating ip create --port master-port-1 <public network>
-openstack floating ip create --port master-port-2 <public network>
-```
+The following external facing services should be added to your new load balancer:


Suggested change

The following external facing services should be added to your new load balancer:

Add the following external facing services to your new load balancer:

maxwelldb · 2020-11-23T18:28:22Z

docs/user/openstack/README.md


-The first step is to add floating IPs to all the master nodes:
+#### Key OpenShift Services


"Key" == "Required?" "Important?"

maxwelldb · 2020-11-23T18:45:26Z

docs/user/openstack/README.md


-Once complete you can see your floating IPs using:
+- The master nodes serve the OpenShift API over port 6443 via tcp.


Suggested change

- The master nodes serve the OpenShift API over port 6443 via tcp.

- The master nodes serve the OpenShift API on port 6443 by using TCP.

maxwelldb · 2020-11-23T18:54:53Z

docs/user/openstack/README.md

 ```

-The external load balancer should now be operational along with your own DNS solution. The following curl command is an example of how to check functionality:
+#### Verifying API Reachable


Suggested change

#### Verifying API Reachable

#### Verifying that the API is Reachable

Capitalized however you like.

maxwelldb · 2020-11-23T18:56:36Z

docs/user/openstack/README.md

-The external load balancer should now be operational along with your own DNS solution. The following curl command is an example of how to check functionality:
+#### Verifying API Reachable
+
+One good way to test that you can reach the API is to try executing an `oc` command. If you can't do that easily, you can use this curl command:


Suggested change

One good way to test that you can reach the API is to try executing an `oc` command. If you can't do that easily, you can use this curl command:

One way to test whether or not you can reach the API is to run the `oc` command.

If you can't use `oc`, use the following `curl` command:

Could include sample output for good/bad cases while using oc.

maxwelldb · 2020-11-23T18:57:36Z

docs/user/openstack/README.md

  "compiler": "gc",
  "platform": "linux/amd64"
 }
 ```

-Another useful thing to check is that the ignition configurations are only available from within the deployment. The following command should only succeed from a node in the OpenShift cluster:
+Note: the versions may be different, but as long as you get a json payload response, it worked correctly.


Suggested change

Note: the versions may be different, but as long as you get a json payload response, it worked correctly.

Note: The versions in the sample output may differ from your own. As long as you get a JSON payload response, the API is accessible.

maxwelldb · 2020-11-23T18:59:20Z

docs/user/openstack/README.md

-Another useful thing to check is that the ignition configurations are only available from within the deployment. The following command should only succeed from a node in the OpenShift cluster:
+Note: the versions may be different, but as long as you get a json payload response, it worked correctly.
+
+#### Verifying Apps Reachable


Suggested change

#### Verifying Apps Reachable

#### Verifying that Apps Are Reachable

Or something to that effect.

maxwelldb · 2020-11-23T19:00:27Z

docs/user/openstack/README.md

+
+#### Verifying Apps Reachable
+
+An easy way to check that the workers are load balanced correctly is to try to access the OpenShift console. This can be done from your web browser. If you don't have access to a web browser, then you can try to query it with the following curl command:


Suggested change

An easy way to check that the workers are load balanced correctly is to try to access the OpenShift console. This can be done from your web browser. If you don't have access to a web browser, then you can try to query it with the following curl command:

The simplest way to verify that apps are reachable is to open the OpenShift console in a web browser.

If you don't have access to a web browser, query the console by using the following `curl` command:

openshift-ci-robot · 2020-11-23T19:01:14Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: EmilienM, maxwelldb

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~docs/user/openstack/OWNERS~~ [EmilienM]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-merge-robot · 2020-11-24T20:53:32Z

@iamemilio: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-crc	`2e9b816`	link	`/test e2e-crc`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

mandre · 2020-11-25T08:19:29Z

docs/user/openstack/README.md

-You can verify the operation of the load balancer now if you wish, using the curl commands given below.
-
-Now the DNS entry for `api.<cluster name>.<base domain>` needs to be updated to point to the new load balancer:
+To ensure that your API and apps are accessible through your load balancer, [create or update DNS entries](#Create API and Ingress DNS Records) for them. To use your new load balancing service for external traffic, make sure the IP address for these DNS entries is the IP address your load balancer is reachable at.


The anchor should be #create-api-and-ingress-dns-records, otherwise the link doesn't render. The sentence ends bizarrely too.

Suggested change

To ensure that your API and apps are accessible through your load balancer, [create or update DNS entries](#Create API and Ingress DNS Records) for them. To use your new load balancing service for external traffic, make sure the IP address for these DNS entries is the IP address your load balancer is reachable at.

To ensure that your API and apps are accessible through your load balancer, [create or update DNS entries](#create-api-and-ingress-dns-records) for them. To use your new load balancing service for external traffic, make sure the IP address for these DNS entries is the IP address your load balancer:

The underlying network architecture has changed a lot since these docs were initially written. We want to make sure that these docs are accurate and up to date so that users with complex networking use cases like workers on a custom subnet and baremetal workers are able to manage their ingress/egress traffic as needed. More up to date examples and reference information has been added. We have chosen to omit sections outlining how to replace the internal lb and dns services since they were inaccurate. We are targeting an upcoming release to handle these features better given the complexity of our current networking architecture.

iamemilio · 2020-11-30T20:32:45Z

removing the hold, reviewers can lgtm again when ready

/hold cancel

EmilienM · 2020-11-30T21:26:12Z

/lgtm

openshift-ci-robot assigned mandre Nov 17, 2020

openshift-ci-robot added the platform/openstack label Nov 17, 2020

openshift-ci-robot requested review from adduarte and pierreprinetti November 17, 2020 21:32

openshift-ci-robot requested a review from maxwelldb November 17, 2020 21:36

iamemilio force-pushed the update_openstack_day_2_lb branch from 6e3c398 to 52d7e08 Compare November 19, 2020 04:25

openshift-ci-robot assigned EmilienM Nov 23, 2020

openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Nov 23, 2020

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 23, 2020

maxwelldb approved these changes Nov 23, 2020

View reviewed changes

iamemilio force-pushed the update_openstack_day_2_lb branch from 52d7e08 to 2e9b816 Compare November 24, 2020 19:03

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Nov 24, 2020

mandre reviewed Nov 25, 2020

View reviewed changes

iamemilio force-pushed the update_openstack_day_2_lb branch from 2e9b816 to 30e597f Compare November 30, 2020 20:30

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 30, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 30, 2020

openshift-merge-robot merged commit 36cf196 into openshift:master Nov 30, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OSASINFRA-902: Update OpenStack Custom External LB and DNS Docs #4389

OSASINFRA-902: Update OpenStack Custom External LB and DNS Docs #4389

iamemilio commented Nov 17, 2020

iamemilio commented Nov 17, 2020

iamemilio commented Nov 17, 2020

maxwelldb commented Nov 18, 2020

iamemilio commented Nov 19, 2020 •

edited

EmilienM commented Nov 23, 2020

iamemilio commented Nov 23, 2020

maxwelldb left a comment •

edited

maxwelldb Nov 23, 2020

maxwelldb Nov 23, 2020

maxwelldb Nov 23, 2020

maxwelldb Nov 23, 2020

maxwelldb Nov 23, 2020

maxwelldb Nov 23, 2020

maxwelldb Nov 23, 2020

maxwelldb Nov 23, 2020

maxwelldb Nov 23, 2020

maxwelldb Nov 23, 2020

openshift-ci-robot commented Nov 23, 2020

openshift-merge-robot commented Nov 24, 2020

mandre Nov 25, 2020

iamemilio commented Nov 30, 2020

EmilienM commented Nov 30, 2020


		This documents how to shift from the internal load balancer, which is intended for internal networking needs, to an external load balancer.
		This documents how to shift external traffic from the internal load balancer that comes stock with OpenShift on OpenStack to an external load balancer.

	This documents how to shift external traffic from the internal load balancer that comes stock with OpenShift on OpenStack to an external load balancer.
	You can shift network traffic from the default OpenShift on OpenStack load balancer to a load balancer that you provide.


		The load balancer must serve ports 6443, 443, and 80 to any users of the system. Port 22623 is for serving ignition start-up configurations to the OpenShift nodes and should not be reachable outside of the cluster.
		It is essential that the instance your load balancing service is running from can reach all of the nodes in the cluster. One easy way to do this is to create the instance in a subnet that is within the OpenShift network created by your installer, and making sure a router interface is attached to that subnet from the OpenShift-external-router. Another solution is to attach a floating IP to the nodes that you want to add to your external load balancer.

	The following external facing services should be added to your new load balancer:
	Add the following external facing services to your new load balancer:


		The first step is to add floating IPs to all the master nodes:
		#### Key OpenShift Services


		Once complete you can see your floating IPs using:
		- The master nodes serve the OpenShift API over port 6443 via tcp.

	#### Verifying API Reachable
	#### Verifying that the API is Reachable

-One good way to test that you can reach the API is to try executing an `oc` command. If you can't do that easily, you can use this curl command:
+One way to test whether or not you can reach the API is to run the `oc` command.
+If you can't use `oc`, use the following `curl` command:

	Note: the versions may be different, but as long as you get a json payload response, it worked correctly.
	Note: The versions in the sample output may differ from your own. As long as you get a JSON payload response, the API is accessible.

	#### Verifying Apps Reachable
	#### Verifying that Apps Are Reachable


		#### Verifying Apps Reachable

		An easy way to check that the workers are load balanced correctly is to try to access the OpenShift console. This can be done from your web browser. If you don't have access to a web browser, then you can try to query it with the following curl command:

-An easy way to check that the workers are load balanced correctly is to try to access the OpenShift console. This can be done from your web browser. If you don't have access to a web browser, then you can try to query it with the following curl command:
+The simplest way to verify that apps are reachable is to open the OpenShift console in a web browser.
+If you don't have access to a web browser, query the console by using the following `curl` command:

	To ensure that your API and apps are accessible through your load balancer, [create or update DNS entries](#Create API and Ingress DNS Records) for them. To use your new load balancing service for external traffic, make sure the IP address for these DNS entries is the IP address your load balancer is reachable at.
	To ensure that your API and apps are accessible through your load balancer, [create or update DNS entries](#create-api-and-ingress-dns-records) for them. To use your new load balancing service for external traffic, make sure the IP address for these DNS entries is the IP address your load balancer:

OSASINFRA-902: Update OpenStack Custom External LB and DNS Docs #4389

OSASINFRA-902: Update OpenStack Custom External LB and DNS Docs #4389

Conversation

iamemilio commented Nov 17, 2020

iamemilio commented Nov 17, 2020

iamemilio commented Nov 17, 2020

maxwelldb commented Nov 18, 2020

iamemilio commented Nov 19, 2020 • edited

EmilienM commented Nov 23, 2020

iamemilio commented Nov 23, 2020

maxwelldb left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci-robot commented Nov 23, 2020

openshift-merge-robot commented Nov 24, 2020

Choose a reason for hiding this comment

iamemilio commented Nov 30, 2020

EmilienM commented Nov 30, 2020

iamemilio commented Nov 19, 2020 •

edited

maxwelldb left a comment •

edited