Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubeadm: mark-control-plane phase #70625

Closed
wants to merge 3 commits into from
Closed

kubeadm: mark-control-plane phase #70625

wants to merge 3 commits into from

Conversation

RA489
Copy link

@RA489 RA489 commented Nov 4, 2018

What type of PR is this?
This PR implement mark-control-plane phase in the kubeadm init workflow.
/kind feature

What this PR does / why we need it:
Ref kubernetes/kubeadm#1163
/sig cluster-lifecycle
@kubernetes/sig-cluster-lifecycle-pr-reviews
Release note:
NONE

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/feature Categorizes issue or PR as related to a new feature. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Nov 4, 2018
@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Nov 4, 2018
@k8s-ci-robot
Copy link
Contributor

@RA489: Reiterating the mentions to trigger a notification:
@kubernetes/sig-cluster-lifecycle-pr-reviews

In response to this:

What type of PR is this?
This PR implement mark-control-plane phase in the kubeadm init workflow.
/kind feature

What this PR does / why we need it:
Ref kubernetes/kubeadm#1163
/sig cluster-lifecycle
@kubernetes/sig-cluster-lifecycle-pr-reviews
Release note:
NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. area/kubeadm labels Nov 4, 2018
@RA489
Copy link
Author

RA489 commented Nov 4, 2018

/release-note-none

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 4, 2018
@neolit123
Copy link
Member

/priority important-soon

@k8s-ci-robot k8s-ci-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Nov 4, 2018
Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RA489 thank you a lot for finding time to work on this!
added some comments mostly about renames and camelCase, but this is a good start.

will get back to this when we merge the previous phase - kubeconfig.

@@ -169,6 +168,8 @@ func NewCmdInit(out io.Writer) *cobra.Command {
initRunner.AppendPhase(phases.NewControlPlanePhase())
initRunner.AppendPhase(phases.NewEtcdPhase())
initRunner.AppendPhase(phases.NewWaitControlPlanePhase())
initRunner.AppendPhase(phases.NewmarkcontrolplanePhase())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NewmarkcontrolplanePhase -> NewMarkControlPlanePhase

@@ -25,68 +25,65 @@ import (
"k8s.io/kubernetes/cmd/kubeadm/app/cmd/options"
cmdutil "k8s.io/kubernetes/cmd/kubeadm/app/cmd/util"
kubeadmconstants "k8s.io/kubernetes/cmd/kubeadm/app/constants"
markmasterphase "k8s.io/kubernetes/cmd/kubeadm/app/phases/markmaster"
"k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow"
markcontrolplanephase "k8s.io/kubernetes/cmd/kubeadm/app/phases/markmaster"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should rename the file to markcontrolplane to avoid the aliasing.

kubeadm alpha phase mark-master

markcontrolplaneExample = normalizer.Examples(`
# Run master markcontrolplane checks using a config file.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applies control-plane label and taint to the current node, functionally equivalent to what is executed by kubeadm init.


markcontrolplaneExample = normalizer.Examples(`
# Run master markcontrolplane checks using a config file.
kubeadm init phase mark-controlplane --config config.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mark-control-plane


# Applies master label and taint to a specific node
kubeadm alpha phase mark-master --node-name myNode
kubeadm init phase mark-controlplane --node-name myNode
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mark-control-plane

},
fmt.Println("[markcontrolplane] running markcontrolplane checks")
var cfgPath string
kubeConfigFile := kubeadmconstants.GetAdminKubeConfigPath()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please align the indentation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here we define the kubeconfig dir that also respects dry run:
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/cmd/init.go#L366
you can also expose KubeConfigDir() for the data object in this file too.

this line would end up something like:
kubeConfigFile := filepath.Join(data.KubeConfig() , kubeadmconstants.AdminKubeConfigFileName)

err = markmasterphase.MarkMaster(client, internalcfg.NodeRegistration.Name, internalcfg.NodeRegistration.Taints)
kubeadmutil.CheckErr(err)
},
fmt.Println("[markcontrolplane] running markcontrolplane checks")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[markcontrolplane] running markcontrolplane checks
->
[mark-control-plane] Running checks

fmt.Println("[markcontrolplane] running markcontrolplane checks")
var cfgPath string
kubeConfigFile := kubeadmconstants.GetAdminKubeConfigPath()
kubeConfigFile = cmdutil.FindExistingKubeConfig(kubeConfigFile)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note to self: seems to me like we are not doing this in other places, where we should be doing it.
possibly best to move it in init.go

options.AddKubeConfigFlag(cmd.Flags(), &kubeConfigFile)
cmd.Flags().StringVar(&cfgPath, "config", cfgPath, "Path to kubeadm config file. WARNING: Usage of a configuration file is experimental")
cmd.Flags().StringVar(&cfg.NodeRegistration.Name, "node-name", cfg.NodeRegistration.Name, `The node name to which label and taints should apply`)
if err := markmasterphase.MarkMaster(client, internalcfg.NodeRegistration.Name, internalcfg.NodeRegistration.Taints); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it's ok to rename the MarkMaster function to MarkControlPlane as part of this refector.
basically master->controlPlane everywhere.

return cmd
}
return nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add new line at end of file.

@neolit123
Copy link
Member

/assign @fabriziopandini
@kubernetes/sig-cluster-lifecycle-pr-reviews

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 5, 2018
Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RA489 thanks a lot for the update!
after this pass we are going to be close to LGTM

// NewMarkControlPlanePhase creates a kubeadm workflow phase that implements mark-controlplane checks.
func NewMarkControlPlanePhase() workflow.Phase {
return workflow.Phase{
Name: "markcontrolplane",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's name it like so for consistency with other phases:
mark-control-plane

func NewMarkControlPlanePhase() workflow.Phase {
return workflow.Phase{
Name: "markcontrolplane",
Short: "Run master markcontrolplane checks",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mark a node as a control-plane

return workflow.Phase{
Name: "markcontrolplane",
Short: "Run master markcontrolplane checks",
Long: "Run master markcontrolplane checks, applies master label.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's remove Long: here given Short fits on one line.


fmt.Println("[mark-control-plane] Running checks")
var cfgPath string
kubeConfigFile := filepath.Join(data.KubeConfig() , kubeadmconstants.AdminKubeConfigFileName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small whitespace problem.
./hack/update-gofmt.sh might help with these.

return errors.New("mark-control-plane phase invoked with an invalid data struct")
}

fmt.Println("[mark-control-plane] Running checks")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's just remove this line.

// MarkMaster taints the master and sets the master label
func MarkMaster(client clientset.Interface, masterName string, taints []v1.Taint) error {
// MarkControlPlane taints the master and sets the master label
func MarkControlPlane(client clientset.Interface, masterName string, taints []v1.Taint) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

masterName -> controlPlaneName


fmt.Printf("[markmaster] Marking the node %s as master by adding the label \"%s=''\"\n", masterName, constants.LabelNodeRoleMaster)
fmt.Printf("[mark-control-plane] Marking the node %s as master by adding the label \"%s=''\"\n", masterName, constants.LabelNodeRoleMaster)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marking the node %s as control-plane by adding the label


if taints != nil && len(taints) > 0 {
taintStrs := []string{}
for _, taint := range taints {
taintStrs = append(taintStrs, taint.ToString())
}
fmt.Printf("[markmaster] Marking the node %s as master by adding the taints %v\n", masterName, taintStrs)
fmt.Printf("[mark-control-plane] Marking the node %s as master by adding the taints %v\n", masterName, taintStrs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marking the node %s as a control-plane by adding the taints

@neolit123
Copy link
Member

/ok-to-test

@RA489 as a helper, please make sure you have run these:
./hack/update-gofmt.sh
./hack/update-bazel.sh

and also include the ./docs changes after running:
./hack/update-generated-docs.sh

thanks!

@k8s-ci-robot k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Nov 5, 2018
Copy link
Member

@fabriziopandini fabriziopandini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RA489 thanks for this PR! Well done!
Please fix test, address few small nits and then ping me again after removing WIP


var cfgPath string
kubeConfigFile := filepath.Join(data.KubeConfig(), kubeadmconstants.AdminKubeConfigFileName)
kubeConfigFile = cmdutil.FindExistingKubeConfig(kubeConfigFile)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The call to FindExistingKubeConfig should be removed and instead we should use data.Kubeconfig

@neolit123 the call of FindExistingKubeConfig should be removed by all phases, because it can make kubeadm use unexpected kubeconfig files while instead we need admin.conf.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ started a thread on slack.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok we had a discussion about this - @RA489 please remove the FindExistingKubeConfig call and use:
kubeConfigFile := data.KubeConfigPath()

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if err != nil {
return err
}
internalcfg, err := configutil.ConfigFileAndDefaultsToInternalConfig(cfgPath, data.Cfg())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The call to ConfigFileAndDefaultsToInternalConfig is not necessary (it is already called by initData). Please remove
You can pass data.Cfg() directly to MarkControlPlane

@@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/

package markmaster
package markcontrolplane
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please kindly rename also file and folder

@RA489 RA489 changed the title [WIP] kubeadm: mark control-plane phase kubeadm: mark-control-plane phase Nov 6, 2018
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 6, 2018
@fabriziopandini
Copy link
Member

@RA489 I guess build is failing because hack/update-bazel.sh should be run after renaming files

@neolit123
Copy link
Member

the previous phase just merged and init has a small conflict.
this phase is up next in the queue.

# Applies control-plane label and taint to the current node, functionally equivalent to what is executed by kubeadm init.
kubeadm init phase mark-control-plane --config config.yml

# Applies master label and taint to a specific node
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it correct to change "master label" with "control-plane label"?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RA489 Can you fix this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. kind/documentation Categorizes issue or PR as related to documentation. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Nov 8, 2018
@fabriziopandini
Copy link
Member

@RA489 it seems another rebase is necessary to fix bazel-build errors...

@neolit123
Copy link
Member

we need to merge this today, ideally.
the remaining phases are waiting on this PR.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 9, 2018
@k8s-ci-robot k8s-ci-robot added the sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. label Nov 9, 2018
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 9, 2018
@neolit123
Copy link
Member

@RA489
./hack/update-gofmt.sh is required.
make sure you have go 1.11.x installed because they changed how gofmt works.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: RA489
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers: smarterclayton, timothysc

If they are not already assigned, you can assign the PR to them by writing /assign @smarterclayton @timothysc in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@neolit123
Copy link
Member

/close
as discussed with @RA489 on slack, i will close this PR and cherry pick the change in a new PR.

@k8s-ci-robot
Copy link
Contributor

@neolit123: Closed this PR.

In response to this:

/close
as discussed with @RA489 on slack, i will close this PR and cherry pick the change in a new PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot
Copy link
Contributor

@RA489: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-bazel-build bbd09b2 link /test pull-kubernetes-bazel-build
pull-kubernetes-e2e-gce bbd09b2 link /test pull-kubernetes-e2e-gce
pull-kubernetes-e2e-gce-device-plugin-gpu bbd09b2 link /test pull-kubernetes-e2e-gce-device-plugin-gpu
pull-kubernetes-e2e-kops-aws bbd09b2 link /test pull-kubernetes-e2e-kops-aws
pull-kubernetes-bazel-test bbd09b2 link /test pull-kubernetes-bazel-test
pull-kubernetes-integration bbd09b2 link /test pull-kubernetes-integration
pull-kubernetes-typecheck bbd09b2 link /test pull-kubernetes-typecheck
pull-kubernetes-kubemark-e2e-gce-big bbd09b2 link /test pull-kubernetes-kubemark-e2e-gce-big
pull-kubernetes-e2e-gce-100-performance bbd09b2 link /test pull-kubernetes-e2e-gce-100-performance
pull-kubernetes-verify bbd09b2 link /test pull-kubernetes-verify

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/kubeadm cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants