Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubeadm: Use separate YAML docs for the kubelet and kube-proxy ComponentConfigs #65787

Merged
merged 6 commits into from Jul 8, 2018

Conversation

@luxas
Copy link
Member

luxas commented Jul 3, 2018

What this PR does / why we need it:
This PR makes kubeadm load the ComponentConfig for the kubelet and kube-proxy from separate YAML documents in the kubeadm init config file. This is backwards-compatible with v1alpha2. The ComponentConfigs are stored internally in the internal kubeadm MasterConfiguration struct, but when marshalling the componentconfigs are written as separate YAML documents.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
ref: kubernetes/kubeadm#911
Depends on:

Special notes for your reviewer:
Only review the last five commits please. (The last commit is purely autogenerated, so can be skipped)

Release note:

kubeadm: Use separate YAML documents for the kubelet and kube-proxy ComponentConfigs

@kubernetes/sig-cluster-lifecycle-pr-reviews
/assign @timothysc

@luxas luxas changed the title Kubeadm split componentconfig from api kubeadm: Use separate YAML docs for the kubelet and kube-proxy ComponentConfigs Jul 3, 2018

@k8s-ci-robot k8s-ci-robot requested review from chuckha and kad Jul 3, 2018

return allErrs
}

scheme, _, err := kubeletscheme.NewSchemeAndCodecs()
if err != nil {
allErrs = append(allErrs, field.Invalid(fldPath, "kubeletConfiguration", err.Error()))
allErrs = append(allErrs, field.Invalid(fldPath, "", err.Error()))

This comment has been minimized.

@fabriziopandini

fabriziopandini Jul 4, 2018

Contributor

Why removing the field name? I think that it useful for getting more detailed error messages

This comment has been minimized.

@luxas

luxas Jul 6, 2018

Author Member

This is not the field name, fldPath gives us that. The second param is the actual value, which is unnecessary to give for a full struct. More detail exists in the error message already anyway.

// If we find an old API version in this gvk list, error out and tell the user why this doesn't work
for _, gvk := range gvks {
if useKubeadmVersion := oldKnownAPIVersions[gvk.GroupVersion().String()]; len(useKubeadmVersion) != 0 {
return fmt.Errorf("your configuration file uses an old API spec: %q. Please use kubeadm %s instead and run 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.", gvk.GroupVersion().String(), useKubeadmVersion)
}
knownKinds[gvk.Kind] = true
}
// MasterConfiguration and NodeConfiguration are mutually exclusive, error if both are specified

This comment has been minimized.

@fabriziopandini

fabriziopandini Jul 4, 2018

Contributor

I'm torn between implementing this check, that is formally correct, a being a little bit permissive and simply ignore everything is not in scope of the current operation

This comment has been minimized.

@rosti

rosti Jul 5, 2018

Member

Having the complete recipe for creating a K8s cluster in a single YAML file can be good for someone. Yet, this can encourage sloppiness on part of some admins and tools. Furthermore, the code may be more streamlined here and there if we avoid having init and join configs at the same time.

This comment has been minimized.

@luxas

luxas Jul 6, 2018

Author Member

Yeah, if both are given we have no idea on what kubeadm should actually do, these should be mutually exclusive, otherwise we'll find weird bugs.

}
// Set the componentconfigs on the internalcfg object; ONLY if they have been loaded from an external YAML file
if kubeletConfigLoaded {
internalcfg.ComponentConfigs.Kubelet = kubeletConfig

This comment has been minimized.

@fabriziopandini

fabriziopandini Jul 4, 2018

Contributor

What about moving this line above at the end of case "KubeletConfiguration" (~line 166), so we can avoid the kubeletConfigLoaded flag?

This comment has been minimized.

@luxas

luxas Jul 6, 2018

Author Member

This code has been rewritten

return kubeadmutil.MarshalToYamlForCodecs(internalcfg, kubeadmapiv1alpha3.SchemeGroupVersion, kubeadmscheme.Codecs)
switch internalcfg.(type) {
case *kubeadmapi.MasterConfiguration:
return configutil.MarshalMasterConfigurationToBytes(internalcfg.(*kubeadmapi.MasterConfiguration), kubeadmapiv1alpha3.SchemeGroupVersion, true)

This comment has been minimized.

@fabriziopandini

fabriziopandini Jul 4, 2018

Contributor

I appreciate that we strive for cleanness and we skip printing component config if equal to default.
However I think that we should add a flag to allow users to ask explicitly to print also component config defaults

This comment has been minimized.

@luxas

luxas Jul 6, 2018

Author Member

Yes, this is coming later in the kubeadm config print-defaults command.

@luxas luxas referenced this pull request Jul 5, 2018

Merged

kubeadm: Add support for reading multiple YAML documents #65631

3 of 3 tasks complete

@luxas luxas force-pushed the luxas:kubeadm_split_componentconfig_from_api branch from 118b908 to 17d89df Jul 5, 2018

@luxas luxas force-pushed the luxas:kubeadm_split_componentconfig_from_api branch from 17d89df to 474d3bf Jul 5, 2018

@chuckha
Copy link
Member

chuckha left a comment

I'll look at this again now that I've done a first pass.

I know this is annoying feedback, but please document all exported types. Some only have one word, some are not documented at all.

I do not want us to add kubeadm packages to the ignore-verify list.

@luxas luxas force-pushed the luxas:kubeadm_split_componentconfig_from_api branch from 474d3bf to 4b678c3 Jul 6, 2018

@luxas

This comment has been minimized.

Copy link
Member Author

luxas commented Jul 6, 2018

Yeah, sorry, should have marked this [WIP] earlier.

Some only have one word, some are not documented at all.

Should be fixed in most places now at least, please point out if I missed some.

I do not want us to add kubeadm packages to the ignore-verify list.

👍. We have to for the v1alphaX API packages, as the conversion and defaulting code uses the Foo_Bar naming pattern which isn't allowed.

}

const (
// KubeletConfigurationKind TODO

This comment has been minimized.

@fabriziopandini

fabriziopandini Jul 7, 2018

Contributor

Why TODO?

This comment has been minimized.

@luxas

luxas Jul 8, 2018

Author Member

fixed

return nil, err
}

for gvk, fileContent := range gvkmap {
// Try to get the registration for the ComponentConfig based on the kind
registration, found := componentconfigs.Known[gvk.Kind]

This comment has been minimized.

@fabriziopandini

fabriziopandini Jul 7, 2018

Contributor

s/registration/componentConfigRegistration

This comment has been minimized.

@luxas

luxas Jul 8, 2018

Author Member

I think registration is just ok, shorter and says the most

defaultedcfg := defaultedInternalConfig()

for _, registration := range componentconfigs.Known {
realobj := registration.GetFromInternalConfig(cfg)

This comment has been minimized.

@fabriziopandini

fabriziopandini Jul 7, 2018

Contributor

s/realobj/currentobj or actualobj

@fabriziopandini

This comment has been minimized.

Copy link
Contributor

fabriziopandini commented Jul 7, 2018

@luxas, great work! Only few minors but nothing blocking
please fix failing test and then ping me for final lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Jul 8, 2018

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Jul 8, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini, luxas

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Jul 8, 2018

New changes are detected. LGTM label has been removed.

@k8s-ci-robot k8s-ci-robot removed the lgtm label Jul 8, 2018

luxas pushed a commit to luxas/kubernetes that referenced this pull request Jul 8, 2018

Kubernetes Submit Queue
Merge pull request kubernetes#65940 from luxas/kubeadm_internal_compo…
…nentconfigs

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubeadm: Start using internal versions for ComponentConfigs in the internal API

**What this PR does / why we need it**:
This PR changes kubeadm to store the internal API versions of the ComponentConfig APIs, in order to in the future be able to read multiple external versions with conversion. It also moves all the ComponentConfigs to a common struct instead of splitting them out in many. This makes it possible to more easily host more ComponentConfigs in the future. In v1alpha3, the ComponentConfigs will later be removed from the external version of the API (see: kubernetes#65787), but for this PR no changes to the external API are made.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: kubernetes/kubeadm#911
Depends on:
 - [x] kubernetes#65776
 - [x] kubernetes#65628
 - [x] kubernetes#65629
 - [x] kubernetes#65631

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews

luxas added some commits Jul 8, 2018

Remove the ComponentConfig structs from the external v1alpha3 API. Us…
…e the new componentconfigs pkg for validation and conversion

@luxas luxas force-pushed the luxas:kubeadm_split_componentconfig_from_api branch from 20a1b53 to 0be8955 Jul 8, 2018

@luxas

This comment has been minimized.

Copy link
Member Author

luxas commented Jul 8, 2018

Rebased this PR, now good to go 🎉

@luxas luxas added the lgtm label Jul 8, 2018

@luxas

This comment has been minimized.

Copy link
Member Author

luxas commented Jul 8, 2018

/hold cancel

@k8s-github-robot

This comment has been minimized.

Copy link
Contributor

k8s-github-robot commented Jul 8, 2018

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot

This comment has been minimized.

Copy link
Contributor

k8s-github-robot commented Jul 8, 2018

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit ac99da5 into kubernetes:master Jul 8, 2018

16 of 17 checks passed

Submit Queue Required Github CI test is not green: pull-kubernetes-kubemark-e2e-gce-big
Details
cla/linuxfoundation luxas authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details

k8s-github-robot pushed a commit that referenced this pull request Jul 9, 2018

Kubernetes Submit Queue
Merge pull request #65945 from luxas/kubeadm_initconfig
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Rename `MasterConfiguration` to `InitConfiguration` in the kubeadm v1alpha3 Config API

**What this PR does / why we need it**:
In v1alpha3, we have made the design decision that `MasterConfiguration` will be renamed `InitConfiguration`. This PR implements that change.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: kubernetes/kubeadm#911
Depends on:
 - [x] #65776
 - [x] #65628
 - [x] #65629
 - [x] #65631
 - [x] #65940
 - [x] #65787

**Special notes for your reviewer**:
Most of the code is autogenerated, using just find and replace.
Please only review commits:
 - `Automated rename from MasterConfiguration to InitConfiguration`
 - `Rename MasterConfiguration to InitConfiguration in v1alpha3, but support both names for this release of kubeadm`

**Release note**:

```release-note
[action required] The `MasterConfiguration` kind in the kubeadm v1alpha2 API has been renamed `InitConfiguration` in v1alpha3
```
@kubernetes/sig-cluster-lifecycle-pr-reviews

@neolit123 neolit123 referenced this pull request Jul 9, 2018

Closed

Tracking issue for "Config to v1beta1" #963

19 of 28 tasks complete

cdkbot pushed a commit to juju-solutions/kubernetes that referenced this pull request Jul 14, 2018

Kubernetes Submit Queue
Merge pull request kubernetes#65951 from luxas/kubeadm_joinconfig
Automatic merge from submit-queue (batch tested with PRs 66138, 65951). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Rename `NodeConfiguration` to `JoinConfiguration` in the kubeadm v1alpha3 Config API

**What this PR does / why we need it**:
In v1alpha3, we have made the design decision that `NodeConfiguration` will be renamed `JoinConfiguration`. This PR implements that change. 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: kubernetes/kubeadm#911
Depends on:
 - [x] kubernetes#65776
 - [x] kubernetes#65628
 - [x] kubernetes#65629
 - [x] kubernetes#65631
 - [x] kubernetes#65940
 - [x] kubernetes#65787
 - [ ] kubernetes#65945

**Special notes for your reviewer**:

Please only review the last three commits here.

**Release note**:

```release-note
[action required] The `NodeConfiguration` kind in the kubeadm v1alpha2 API has been renamed `JoinConfiguration` in v1alpha3
```
@kubernetes/sig-cluster-lifecycle-pr-reviews

k8s-github-robot pushed a commit that referenced this pull request Jul 17, 2018

Kubernetes Submit Queue
Merge pull request #65952 from luxas/kubeadm_init_join_exclusive
Automatic merge from submit-queue (batch tested with PRs 63877, 64559, 65952). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubeadm: Make the kubeadm config kinds mutually exclusive

**What this PR does / why we need it**:
Right now it would be possible to feed kubeadm with a YAML document with all the four different Config kinds kubeadm at HEAD supports, `MasterConfiguration`, `InitConfiguration`, `JoinConfiguration` and `NodeConfiguration`. This PR makes them mutually exclusive so that kubeadm can know how to handle the config file properly.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: kubernetes/kubeadm#911
Depends on:
 - [x] #65776
 - [x] #65628
 - [x] #65629
 - [x] #65631
 - [x] #65940
 - [x] #65787
 - [x] #65945
 - [x] #65951

**Special notes for your reviewer**:
Please only review the last commit

**Release note**:

```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews

k8s-github-robot pushed a commit that referenced this pull request Aug 22, 2018

Kubernetes Submit Queue
Merge pull request #67441 from rosti/kubeadm_clusterconfig
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[reissue] kubeadm: Split out ClusterConfiguration from InitConfiguration

As @luxas is not able to take care of #66219, I am reissuing the same change here. There are a few minor things added by me:

- The original PR is rebased on latest master.
- Some broken tests were fixed.
- Some TODOs were added.
- Run update-bazel and update-gofmt

Below is the text of the original PR by Lucas.

-----

**What this PR does / why we need it:**

Splits MasterConfiguration to InitConfiguration and ClusterConfiguration as outlined in the kubeadm Config KEP. InitConfiguration holds init-only information, and ClusterConfiguration holds cluster-wide information. In the internal representation InitConfiguration wraps ClusterConfiguration as a field, but in serialized format they're different YAML documents.

**Which issue(s) this PR fixes** (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
ref: kubernetes/kubeadm#911
Depends on:

- [X] #65776
- [X] #65628
- [X] #65629
- [X] #65631
- [X] #65940
- [X] #65787
- [X] #65945
- [X] #65951
- [X] #65952

**Special notes for your reviewer:**

**Release note**:
```release-note
kubeadm: InitConfiguration now consists of two structs: InitConfiguration and ClusterConfiguration
```

@kubernetes/sig-cluster-lifecycle-pr-reviews
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.