Add openshift/4.8.0-exp that produces MachineConfigs by default

[bgilbert]

Rename rhcos/0.2.0-experimental to openshift/4.8.0-experimental. Use openshift as the variant name instead of rhcos because RHCOS is an implementation detail of OpenShift, and also would leave OKD in an awkward position. By reusing OpenShift version numbers, we're setting ourselves up to eventually violate semver, but it doesn't seem reasonable to create a novel set of user-facing version numbers that isn't connected to anything else.

Have the new spec generate MachineConfigs by default, but also accept a new -r/--raw/Raw option to generate an Ignition config. This allows FCCT to be used for sugar (e.g. for boot disk mirroring) and validation (e.g. rejecting users other than core) without modifying the pointer config, and without requiring MCO changes. We'll have a new spec version for each OpenShift release, so we'll also have the ability to change the output format as needed by future MCO releases.

As an implementation detail, ship a simplified copy of the MachineConfig structs so we don't have to pull in a bunch of k8s code as dependencies. (And to avoid a circular dependency with the MCO.)

Sample input.occ:

variant: openshift
version: 4.8.0-experimental
metadata:
  name: create-etc-sample
  labels:
    machineconfiguration.openshift.io/role: worker
storage:
  files:
    - path: /etc/sample
      contents:
        inline: hello world

fcct input.occ produces:

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: create-etc-sample
spec:
  config:
    ignition:
      version: 3.2.0
    storage:
      files:
      - contents:
          source: data:,hello%20world
        path: /etc/sample

And fcct --pretty --raw input.occ produces:

{
  "ignition": {
    "version": "3.2.0"
  },
  "storage": {
    "files": [
      {
        "path": "/etc/sample",
        "contents": {
          "source": "data:,hello%20world"
        }
      }
    ]
  }
}

@yuqi-zhang, @sinnykumari, and/or other folks from the MCO team, wdyt?

[sinnykumari]

I don't have full context here, is the usecase of this is that user can write openshift coreos config (yaml config) and can use fcct tool to generate MachineConfig?

cc @kikisdeliveryservice

[bgilbert]

@sinnykumari Yup, that's right. For example, it would let us convert the mirrored disk instructions to use a MachineConfig manifest. Other workflows in the customization docs could also be switched to preprocess through fcct in cases where its sugar and validation capabilities would be useful.

[yuqi-zhang]

In the MCO today I believe we use FCCT to render our templates into machineconfigs, although we make no explicit reference to the schema type. Is that something we should change? Or is that perhaps unrelated to this change

[sinnykumari]

@sinnykumari Yup, that's right. For example, it would let us convert the mirrored disk instructions to use a MachineConfig manifest.

+1

Other workflows in the customization docs could also be switched to preprocess through fcct in cases where its sugar and validation capabilities would be useful.

agree, for better user experience it would be nice to be able to generate rest of MachineConfig file fields as well.
It makes me think renaming the tool from fcct to more generic would be useful as this will get used by FCOS, OKD and OCP.

And current .occ format looks fine to me.

[LorbusChris]

+1 from my side as well.

Just to xref some previous ideas in this area:

• WIP: Add FCCT support in MC proposal openshift/enhancements#525
• *: Support CoreOS configs in MCs openshift/machine-config-operator#1980
• And also @cgwalters' suggestion of creating a coreosconfig CRD as replacement for MCs-before-rendering: WIP: Add FCCT support in MC proposal openshift/enhancements#525 (comment)

Do I see this correctly and this RFC will be able tp obsolete these approaches?

[bgilbert]

In the MCO today I believe we use FCCT to render our templates into machineconfigs, although we make no explicit reference to the schema type. Is that something we should change? Or is that perhaps unrelated to this change

MCO is using FCCT in a limited way to transpile file and unit sections, but not to transpile entire configs. It should be fine to continue as is for now; there's no harm to passing desugared config through FCCT twice.

for better user experience it would be nice to be able to generate rest of MachineConfig file fields as well.

Yup, we can do that as a followup.

It makes me think renaming the tool from fcct to more generic would be useful as this will get used by FCOS, OKD and OCP.

We're planning to do that before 4.8, currently blocked on finding a good name. If anyone has any ideas, please add them to #167.

Do I see this correctly and this RFC will be able tp obsolete these approaches?

I don't think they're incompatible. This RFC lets us incrementally improve our config handling without MCO changes. In the longer term, we could still reconsider the CRD format, which would allow us to avoid asking users to preprocess their configs with an external tool.

[bgilbert]

No major objections raised so far; dropping RFC!

[bgilbert]

This part of the new FCC syntax is worth some thought:

metadata:
  labels:
    machineconfiguration.openshift.io/role: worker

Nowhere else in the FCC or Ignition specs do we allow user-specified map keys. Normally we'd implement this functionality like so:

metadata:
  labels:
    - key: machineconfiguration.openshift.io/role
      value: worker

I'm concerned that if we use a datatype here that we don't use elsewhere, and don't plan to use elsewhere, that it will cause complications. It already introduces a presentation issue in documentation and requires some additional code in translate/util.go. On the other hand, a more Ignition-like structure would be inconsistent with k8s norms.

Thoughts?

[arithx]

I would lean towards the former example. I think making it clear that we're directly passing those arguments through into the resulting MachineConfig is sufficient (as well giving us the mentioned benefit of matching the k8s norms).

[arithx]

LGTM

[arithx]

Do you think we should mention this is only valid on openshift variants or maybe throw a warning/error if we see it set in other variants? I'm not sure that without any wrapper means anything without context.

[bgilbert]

Changed it to never wrap in a MachineConfig; force Ignition output. For the other variants, we always output raw Ignition, so it feels odd to complain if the user redundantly requests it.

[jlebon]

This part of the new FCC syntax is worth some thought:

Agreed with @arithx. Just matching the k8s semantics would be a nicer UX. Documentation-wise, feels like we can just leave out the awkward arbitrary-key line entirely and enhance the labels line to something like:

• labels (object): string key/value pairs to apply as [Kubernetes labels][k8s-labels] to this MachineConfig.

?

[arithx]

Documentation-wise, feels like we can just leave out the awkward arbitrary-key line entirely and enhance the labels line to something like:

• labels (map[string]string): string key/value pairs to apply as [Kubernetes labels][k8s-labels] to this MachineConfig.

Could also do something like this, describing the type as a map[string]string