improved static networking configuration #291
Conversation
|
This is probably a bit light on some details, but wanted to get this out for comments and refinements. |
openshift-ci-robot
added
the
approved
miabbott
force-pushed
the
static_networking
branch
from
f561bb8
to
b966878
Compare
| - Providing a fully automated solution for providing static network configuration to a fleet of machines | ||
|
|
||
| ## Proposal | ||
| For the bare metal use case, we are proposing the use of the new [live ISO](https://github.com/openshift/enhancements/pull/210) to provide an interactive environment where users can configure their networking parameters and have them persist to the real root. In this environment, users will be able to invoke a TUI that will allow them to configure networking and Ignition source. This information will be used as part of the install of the host and will persist into the real root. |
There was a problem hiding this comment.
So this is bare metal UPI? Not IPI?
There was a problem hiding this comment.
Correct, this is for UPI case only; I'll clarify this. Thanks!
There was a problem hiding this comment.
It doesn't have to be just UPI though...there's a doc that @markmc is floating around this that we can hopefully expand here into an enhancement.
| ## Motivation | ||
| Many customer environments require static networking and do not allow any type of DHCP server. This is most prominent in VMware environments where they struggle with the OVA images and are resisting the bare metal installer (via ISO) approach. This is also a challenge in bare metal environments due to an unfriendly and time sensitive interactive flow (i.e. catching the `grub` prompt). Due to the intersections of Ignition, Platform ID, and active networking in the `initrd`, many of the current RHCOS images append `ip=dhcp,dhcp6` and catching the bootloader on the console is not a pleasant experience. (Even if a static `ip=::::` is passed it will fail as the platform ID will insert `ip=dhcp` and conflict) | ||
|
|
||
| This is basic networking configuration and users assume basic configurations like this are possible. As RHCOS requires a functional networking config early in the boot process, this eliminates many of the existing mechanisms customers use to handle the assigning of static networking configs. Most of the OpenShift personas we target are not aware of `dracut` arguments and it’s not a great experience to expect them to understand very low-level details of how the OS works. |
There was a problem hiding this comment.
I'm not sure I'd agree with
Most of the OpenShift personas we target are not aware of
dracutarguments
for bare metal at least. VMware though is a different story.
|
|
||
| If these changes are unable to be made to RHCOS, we may need to consider shipping newer versions of certain pieces of software in RHCOS that is not available in RHEL. | ||
|
|
||
| The biggest risk to delivering the enhancement to RHCOS is the ability to generate and deliver the live ISO (see [#210](https://github.com/openshift/enhancements/pull/210)). |
There was a problem hiding this comment.
I think for customers these enhancements are going to be one and the same thing; not saying we need to merge these two, but from an end-user documentation point of view we should.
| - Providing a fully automated solution for providing static network configuration to a fleet of machines | ||
|
|
||
| ## Proposal | ||
| For the bare metal use case, we are proposing the use of the new [live ISO](https://github.com/openshift/enhancements/pull/210) to provide an interactive environment where users can configure their networking parameters and have them persist to the real root. In this environment, users will be able to invoke a TUI that will allow them to configure networking and Ignition source. This information will be used as part of the install of the host and will persist into the real root. |
There was a problem hiding this comment.
It doesn't have to be just UPI though...there's a doc that @markmc is floating around this that we can hopefully expand here into an enhancement.
|
Thanks for all the feedback and updates! Last call for comments/questions! |
miabbott
force-pushed
the
static_networking
branch
from
b966878
to
e08a35e
Compare
| - [ ] User-facing documentation is created in [openshift-docs](https://github.com/openshift/openshift-docs/) | ||
|
|
||
| ## Summary | ||
| This enhancement describes improvements to how static networking is configured on Fedora CoreOS (FCOS) and RHEL CoreOS (RHCOS). It covers improving the configuration of static networking for bare metal and VMWare hosts. |
There was a problem hiding this comment.
I think the scope should be bigger, this seems focused on static networking but the use cases are much broader. Even if they have DHCP, they may still need VLAN's or bonding. There are also use cases to disable entire interfaces to speed up boot times.
See: coreos/ignition#979
There was a problem hiding this comment.
Yep, I am pretty sure you're right. Let's take this sub-thread to coreos/ignition#979 (comment)
| ## Proposal | ||
| For the bare metal use case, we are proposing the use of the new [live ISO](https://github.com/openshift/enhancements/pull/210) to provide an interactive install environment where users can configure their networking parameters and have them persisted into the installed system. In this environment, users will be able to invoke a TUI that will allow them to configure networking and Ignition source. We believe this functionality will initially be most useful for users deploying Openshift Container Platform (OCP) onto bare metal using the user provisioned infrastructure (UPI) workflow. | ||
|
|
||
| Additionally, the new live ISO will provide the ability to embed the networking configuration via an Ignition config, resulting in a custom ISO that can be used to install to a host in an automated fashion. This would be done via the new `coreos-installer embed iso` functionality. |
There was a problem hiding this comment.
Does this have to be tied to the live ISO? If you look at how cloud-init is structured, they have support for preflight actions that happen before networking. I really believe ignition needs something similar.
This is problematic for us in metal-ipi because the stub ignition the installer generates has a network dependency (an append with an HTTP URL), but in some customer environments to even get access to the network you need to do some configuration.
I'd like some kind of solution where ignition could do things both prior and after networking is available, e.g. deferring things like appends with network sources (or merges in V3 of ignition), but allowing us to do some networking configuration before NetworkManager is up.
There was a problem hiding this comment.
Technically...it doesn't and if implemented in Ignition it wouldn't be tied to the live ISO. That said, I think you really want to use the Live ISO because once osmet lands for RHCOS it will be a "one stop" model for booting ephemerally and then installing without requiring an additional metal artifact.
There was a problem hiding this comment.
How do you see this working in the IPI install methods for the environments targeted by this enhancement (metal, vsphere) ?
There was a problem hiding this comment.
Stephen took the networking sub-thread to coreos/ignition#979
There was a problem hiding this comment.
Does this enhancement need to be updated to reflect discussions that have happened in the meantime there?
|
|
||
| Additionally, the new live ISO will provide the ability to embed the networking configuration via an Ignition config, resulting in a custom ISO that can be used to install to a host in an automated fashion. This would be done via the new `coreos-installer embed iso` functionality. | ||
|
|
||
| For the VMware use case, the proposal is to the existing `guestinfo` method to provide the static networking configuration. |
There was a problem hiding this comment.
A reference here would be helpful
|
cc: @patrickdillon |
|
Expecting a few updates on this one before it merges. |
The experience configuring static networking for Fedora CoreOS and Red hat CoreOS has been less than ideal. In this enhancement, we propose improvements to the way we configure static networking targeting bare metal and VMware installs.
miabbott
force-pushed
the
static_networking
branch
from
e08a35e
to
ce19377
Compare
|
Pushed the pending updates ⬆️ I believe this addresses most of the questions/comments from the initial review. In order to allow this to merge this promptly, please raise additional questions or concerns via issues to https://github.com/openshift/os |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ashcrow, cgwalters, miabbott The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
The experience configuring static networking for Fedora CoreOS and Red
hat CoreOS has been less than ideal. In this enhancement, we
propose improvements to the way we configure static networking
targeting bare metal and VMware installs.