✨ Add support for generating bootstrap data in Ignition format to CABPK #4172

invidian · 2021-02-10T18:25:36Z

What this PR does / why we need it:

This PR provides an alternative solution to PR #3437, described in #3430 (comment).

For manual testing, you can use https://github.com/kinvolk/cluster-api/tree/invidian/ignition-support-next based on v1alpha3 together with https://github.com/kinvolk/cluster-api-provider-aws/tree/invidian/flatcar-support-format-and-flatcar-specifics.

Please let me know what you think about it and what is missing to get this PR merged.

I guess the following things are missing:

Test for format field, I couldn't figure out where to put it, so I skipped adding new test for now.
Documentation?

CC @iaguis @vbatts @dongsupark

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #3430

k8s-ci-robot · 2021-02-10T18:25:44Z

Welcome @invidian!

It looks like this is your first PR to kubernetes-sigs/cluster-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2021-02-10T18:25:44Z

Hi @invidian. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bootstrap/kubeadm/internal/ignition/clc/clc.go

neolit123 · 2021-02-10T18:51:00Z

i don't see a CAEP (proposal) for this.
given the multiple solutions / PRs it may have been to better discuss all options / implementation details in a proposal before proceeding with the PRs.

vincepri · 2021-02-10T20:02:17Z

/ok-to-test

invidian · 2021-02-11T11:00:26Z

I'd appreciate some help with CI, I have no idea why is it failing 😞

MarcelMue · 2021-02-11T12:09:25Z

I'd appreciate some help with CI, I have no idea why is it failing disappointed

I believe they are failing here:

cluster-api/bootstrap/kubeadm/api/v1alpha3/conversion_test.go

Lines 29 to 37 in 2dd66d6

    
           func TestFuzzyConversion(t *testing.T) { 
        
           	g := NewWithT(t) 
        
           	scheme := runtime.NewScheme() 
        
           	g.Expect(AddToScheme(scheme)).To(Succeed()) 
        
           	g.Expect(v1alpha4.AddToScheme(scheme)).To(Succeed()) 
        
           	t.Run("for KubeadmConfig", utilconversion.FuzzTestFunc(scheme, &v1alpha4.KubeadmConfig{}, &KubeadmConfig{})) 
        
           	t.Run("for KubeadmConfigTemplate", utilconversion.FuzzTestFunc(scheme, &v1alpha4.KubeadmConfigTemplate{}, &KubeadmConfigTemplate{})) 
        
           }

So basically CRD conversion is broken for kubeadm types.

invidian · 2021-02-11T12:20:46Z

So basically CRD conversion is broken for kubeadm types.

Thanks. I managed to figure that out. I guess some manual conversion code is missing, but I didn't figure out yet where to add it.

Another weird thing is that running make generate locally removes all generated files for v1alpha3 for some reason 🤔

invidian · 2021-02-11T12:55:01Z

i don't see a CAEP (proposal) for this.
given the multiple solutions / PRs it may have been to better discuss all options / implementation details in a proposal before proceeding with the PRs.

Hey @neolit123, have you seen #3430 (comment) ? I think I contained the proposal in this comment. Isn't it providing enough context?

LorbusChris · 2021-02-11T13:26:12Z

As @bgilbert noted over the old PR (#3437 (comment)), there are newer Ignition versions (2.x) with an incompatible spec (3.x) and a different config transpiler yaml format (FCCT).

I'd like to make sure this is taken into account here.

Edit: I just saw #3430 (comment) which seems to take it into account 👍

invidian · 2021-02-12T14:01:15Z

@vincepri @neolit123 @JoelSpeed @justinsb tests are now passing. Please have a look 🙏

bootstrap/kubeadm/api/v1alpha3/conversion.go

bootstrap/kubeadm/api/v1alpha4/kubeadmbootstrapconfig_types.go

bootstrap/kubeadm/controllers/kubeadmconfig_controller.go

bootstrap/kubeadm/internal/ignition/clc/clc.go

go.mod

bootstrap/kubeadm/internal/ignition/clc/clc.go

KubeadmConfigSpec has now more validation rules, so we should trigger them, as KubeadmConfigSpec is embedded in KubeadmControlPlane object. Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io> Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>

sbueringer · 2021-12-08T18:16:08Z

Maybe the linter issues point to the test failure cause

Allow provisioning CAPD machines using Ignition as an alternative to cloud-init. So far, cloud-init has been hardcoded as the only provisioning format supported by CAPD. - Add a package called "provisioning" which contains the interfaces and common logic for all supported provisioning formats. - Move the existing cloud-init code under the new package. - Add a new provisioning implementation for Ignition. Co-authored-by: Suraj Deshmukh <suraj@kinvolk.io> Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>

This commit adds the feature gate KubeadmBootstrapFormatIgnition that will control the usage of field Ignition in KubeadmConfig. If user provides ignition field then the webhook config rejects the request with a validation error. Signed-off-by: Suraj Deshmukh <suraj@kinvolk.io> Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>

Signed-off-by: Suraj Deshmukh <suraj@kinvolk.io> Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>

This file tests functions in kubeadmconfig_webhook_test.go. Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>

Ignition doesn't support the replace_fs filesystem parameter and the partition filesystem parameter supported by cloud-init. Disallow using these parameters with Ignition. Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>

invidian · 2021-12-08T18:38:04Z

Maybe the linter issues point to the test failure cause

For some reason, running golangci-lint run in repo's root directory did not show issues in test directory when I did it locally. Neither did gci find any files not formatted properly. Odd. Fixed now, sorry for the noise, as I was trying to rebase it as soon as possible.

While looking at reported linter issues, I also spotted some complexity which could be reduced, so I've done that. Last batch of changes is available to see here:
https://github.com/kubernetes-sigs/cluster-api/compare/0aea984fb43c79ebd5600f328e0288d9e5980d46..87df81f4e4dd0c0ca78abdaa909da69f1112d4b8

invidian · 2021-12-08T18:43:52Z

/test pull-cluster-api-e2e-full-main

sbueringer · 2021-12-08T18:48:13Z

Thx for providing that link

/lgtm 🎉

But let's please wait for green e2e-full results before merge

sbueringer · 2021-12-08T18:48:39Z

That apparently didn't match the regex :)

/lgtm

CecileRobertMichon · 2021-12-08T18:50:16Z

/lgtm

invidian · 2021-12-08T18:54:38Z

@vincepri @fabriziopandini do you want to approve as well? :)

I'll remove hold once e2e-full-main is green.

fabriziopandini · 2021-12-08T19:15:11Z

/lgtm
/approve

k8s-ci-robot · 2021-12-08T19:15:54Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [fabriziopandini]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

invidian · 2021-12-08T19:32:46Z

/unhold

🎉

sbueringer · 2021-12-08T19:37:39Z

Congrats 🎉

k8s-ci-robot requested review from JoelSpeed and justinsb February 10, 2021 18:25

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Feb 10, 2021

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 10, 2021

invidian changed the title ~~✨ Add support for generating bootstrap data in Ignition format to CABPK~~ ✨ Add support for generating bootstrap data in Ignition format to CABPK Feb 10, 2021

invidian mentioned this pull request Feb 10, 2021

Add ignition support in bootstrap provider #3430

Closed

MarcelMue reviewed Feb 10, 2021

View reviewed changes

bootstrap/kubeadm/internal/ignition/clc/clc.go Show resolved Hide resolved

dongsupark mentioned this pull request Feb 11, 2021

✨ [WIP] Introduce bootstrap provider for ignition #3437

Closed

JoelSpeed reviewed Feb 12, 2021

View reviewed changes

bootstrap/kubeadm/api/v1alpha3/conversion.go Outdated Show resolved Hide resolved

bootstrap/kubeadm/api/v1alpha4/kubeadmbootstrapconfig_types.go Outdated Show resolved Hide resolved

invidian mentioned this pull request Feb 16, 2021

Support userdata privacy beyond cloudinit kubernetes-sigs/cluster-api-provider-aws#1875

Closed

vincepri reviewed Feb 18, 2021

View reviewed changes

bgilbert mentioned this pull request Feb 18, 2021

Rename FCCT coreos/butane#167

Closed

bgilbert reviewed Feb 18, 2021

View reviewed changes

bootstrap/kubeadm/internal/ignition/clc/clc.go Outdated Show resolved Hide resolved

johananl and others added 5 commits December 8, 2021 19:36

Document Ignition feature gate

de8748e

Signed-off-by: Suraj Deshmukh <suraj@kinvolk.io> Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>

Rename kubeadmconfig_types_test.go

1df0796

This file tests functions in kubeadmconfig_webhook_test.go. Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>

Disallow unsupported storage params with Ignition

87df81f

Ignition doesn't support the replace_fs filesystem parameter and the partition filesystem parameter supported by cloud-init. Disallow using these parameters with Ignition. Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>

k8s-ci-robot assigned sbueringer Dec 8, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 8, 2021

k8s-ci-robot assigned CecileRobertMichon Dec 8, 2021

k8s-ci-robot assigned fabriziopandini Dec 8, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 8, 2021

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 8, 2021

k8s-ci-robot merged commit f815cef into kubernetes-sigs:main Dec 8, 2021

invidian deleted the invidian/ignition-support-pr branch December 8, 2021 20:02

AverageMarcus mentioned this pull request Feb 23, 2022

Support Flatcar linux for nodes kubernetes-sigs/cluster-api-provider-gcp#540

Open

CecileRobertMichon mentioned this pull request Jan 4, 2023

Bottlerocket OS support in kubeadm bootstrap #7840

Closed

primeroz mentioned this pull request Mar 7, 2023

Cannot upgrade from ubuntu to flatcar due to some immutable fields in kubeadmcontrolplane spec #8242

Closed

invidian mentioned this pull request Jun 30, 2023

Return CLC conversion warnings to user using warnings of validation webhook #8944

Open

johananl mentioned this pull request Nov 14, 2024

📖 ✨ 🧑‍🤝‍🧑 add proposal for Node Bootstrapping working group #11407

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ Add support for generating bootstrap data in Ignition format to CABPK #4172

✨ Add support for generating bootstrap data in Ignition format to CABPK #4172

invidian commented Feb 10, 2021

k8s-ci-robot commented Feb 10, 2021

k8s-ci-robot commented Feb 10, 2021

neolit123 commented Feb 10, 2021 •

edited

Loading

vincepri commented Feb 10, 2021

invidian commented Feb 11, 2021

MarcelMue commented Feb 11, 2021

invidian commented Feb 11, 2021

invidian commented Feb 11, 2021

LorbusChris commented Feb 11, 2021 •

edited

Loading

invidian commented Feb 12, 2021

sbueringer commented Dec 8, 2021

invidian commented Dec 8, 2021

invidian commented Dec 8, 2021

sbueringer commented Dec 8, 2021

sbueringer commented Dec 8, 2021

CecileRobertMichon commented Dec 8, 2021

invidian commented Dec 8, 2021

fabriziopandini commented Dec 8, 2021

k8s-ci-robot commented Dec 8, 2021

invidian commented Dec 8, 2021 •

edited

Loading

sbueringer commented Dec 8, 2021

✨ Add support for generating bootstrap data in Ignition format to CABPK #4172

✨ Add support for generating bootstrap data in Ignition format to CABPK #4172

Conversation

invidian commented Feb 10, 2021

k8s-ci-robot commented Feb 10, 2021

k8s-ci-robot commented Feb 10, 2021

neolit123 commented Feb 10, 2021 • edited Loading

vincepri commented Feb 10, 2021

invidian commented Feb 11, 2021

MarcelMue commented Feb 11, 2021

invidian commented Feb 11, 2021

invidian commented Feb 11, 2021

LorbusChris commented Feb 11, 2021 • edited Loading

invidian commented Feb 12, 2021

sbueringer commented Dec 8, 2021

invidian commented Dec 8, 2021

invidian commented Dec 8, 2021

sbueringer commented Dec 8, 2021

sbueringer commented Dec 8, 2021

CecileRobertMichon commented Dec 8, 2021

invidian commented Dec 8, 2021

fabriziopandini commented Dec 8, 2021

k8s-ci-robot commented Dec 8, 2021

invidian commented Dec 8, 2021 • edited Loading

sbueringer commented Dec 8, 2021

neolit123 commented Feb 10, 2021 •

edited

Loading

LorbusChris commented Feb 11, 2021 •

edited

Loading

invidian commented Dec 8, 2021 •

edited

Loading