New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set leader-elect for kube-scheduler to true / Adds new configuration types and rework how we set defaults #59732

Merged
merged 1 commit into from Mar 30, 2018

Conversation

@dims
Member

dims commented Feb 11, 2018

What this PR does / why we need it:
Thanks to some great sleuthing by ikruglov!

kube-controller-manager defaults --leader-elect to true. We should
do the same for kube-scheduler. kube-scheduler used to have this
set to true, but it got lost during refactoring in:
efb2bb7

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #59729

Special notes for your reviewer:

Release note:

kube-scheduler has been fixed to use `--leader-elect` option back to true (as it was in previous versions)
@dims

This comment has been minimized.

Member

dims commented Feb 12, 2018

/assign @timothysc

@timothysc

/lgtm
/approve

@@ -150,6 +150,7 @@ func NewOptions() (*Options, error) {
return nil, err
}
o.config.LeaderElection.LeaderElect = true

This comment has been minimized.

@liggitt

liggitt Feb 12, 2018

Member

seems like defaults should be applied in defaults.go, not here (and options round-tripped through those defaults)

This comment has been minimized.

@dims
@liggitt

This comment has been minimized.

Member

liggitt commented Feb 12, 2018

/hold

@liggitt

This comment has been minimized.

Member

liggitt commented Feb 12, 2018

why is this default different than all the other defaults for the scheduler config, applied in the api defaulting path? the setting here won't get applied when reading from file

@timothysc timothysc removed the lgtm label Feb 12, 2018

@k8s-ci-robot k8s-ci-robot removed the approved label Feb 12, 2018

@dims

This comment has been minimized.

Member

dims commented Feb 12, 2018

@liggitt looks like we need to do both ... cmd/kube-scheduler/app/server.go and pkg/apis/componentconfig/v1alpha1/defaults.go if i don't patch up server.go, then the command line default is still false.

@liggitt

This comment has been minimized.

Member

liggitt commented Feb 12, 2018

if i don't patch up server.go, then the command line default is still false.

that means the internal config is not being round tripped and defaulted... that seems like it should be fixed as well

@dims

This comment has been minimized.

Member

dims commented Feb 12, 2018

@liggitt can you point me to an existing pattern/files where this is done please?

@liggitt

This comment has been minimized.

Member

liggitt commented Feb 12, 2018

@liggitt can you point me to an existing pattern/files where this is done please?

for kubeproxy:

func (o *Options) ApplyDefaults(in *kubeproxyconfig.KubeProxyConfiguration) (*kubeproxyconfig.KubeProxyConfiguration, error) {
external, err := o.scheme.ConvertToVersion(in, v1alpha1.SchemeGroupVersion)
if err != nil {
return nil, err
}
o.scheme.Default(external)
internal, err := o.scheme.ConvertToVersion(external, kubeproxyconfig.SchemeGroupVersion)
if err != nil {
return nil, err
}
out := internal.(*kubeproxyconfig.KubeProxyConfiguration)
return out, nil
}
// NewProxyCommand creates a *cobra.Command object with default parameters
func NewProxyCommand() *cobra.Command {
opts := NewOptions()
cmd := &cobra.Command{
Use: "kube-proxy",
Long: `The Kubernetes network proxy runs on each node. This
reflects services as defined in the Kubernetes API on each node and can do simple
TCP and UDP stream forwarding or round robin TCP and UDP forwarding across a set of backends.
Service cluster IPs and ports are currently found through Docker-links-compatible
environment variables specifying ports opened by the service proxy. There is an optional
addon that provides cluster DNS for these cluster IPs. The user must create a service
with the apiserver API to configure the proxy.`,
Run: func(cmd *cobra.Command, args []string) {
verflag.PrintAndExitIfRequested()
cmdutil.CheckErr(opts.Complete())
cmdutil.CheckErr(opts.Validate(args))
cmdutil.CheckErr(opts.Run())
},
}
var err error
opts.config, err = opts.ApplyDefaults(opts.config)

for kubelet:

// NewKubeletConfiguration will create a new KubeletConfiguration with default values
func NewKubeletConfiguration() (*kubeletconfig.KubeletConfiguration, error) {
scheme, _, err := kubeletscheme.NewSchemeAndCodecs()
if err != nil {
return nil, err
}
versioned := &v1alpha1.KubeletConfiguration{}
scheme.Default(versioned)
config := &kubeletconfig.KubeletConfiguration{}
if err := scheme.Convert(versioned, config, nil); err != nil {
return nil, err
}
return config, nil
}

@dims

This comment has been minimized.

Member

dims commented Feb 12, 2018

Thanks @liggitt looks like it was already setup. but there was a mismatch in bool vs *bool for LeaderElect (in the 2 types.go file) and a hard coding of LeaderElect to false in DefaultLeaderElectionConfiguration.

@dims dims changed the title from Set leader-elect for kube-scheduler to true to [WIP] Set leader-elect for kube-scheduler to true Feb 12, 2018

@@ -129,7 +129,7 @@ type LeaderElectionConfiguration struct {
// leaderElect enables a leader election client to gain leadership
// before executing the main loop. Enable this when running replicated
// components for high availability.
LeaderElect bool
LeaderElect *bool

This comment has been minimized.

@liggitt

liggitt Feb 12, 2018

Member

internal bool, external *bool was intentional. Convert_Pointer_bool_To_bool handles the conversion, and external pointer lets you apply defaulting logic if nil.

This comment has been minimized.

@dims

dims Feb 12, 2018

Member

Ack going back to what i had before

@k8s-ci-robot k8s-ci-robot added size/XS and removed size/S labels Feb 12, 2018

@dims

This comment has been minimized.

Member

dims commented Mar 6, 2018

/milestone clear

@k8s-ci-robot k8s-ci-robot removed this from the v1.10 milestone Mar 6, 2018

@dims

This comment has been minimized.

Member

dims commented Mar 6, 2018

we merged a small change and pushing this PR off to v1.11

@timothysc timothysc added the approved label Mar 9, 2018

@bsalamat bsalamat modified the milestones: v1.10, v1.11 Mar 12, 2018

@dims

This comment has been minimized.

Member

dims commented Mar 20, 2018

@mtaufen @liggitt let's get this in please? now that master is open for 1.11. thanks!

@k8s-merge-robot

This comment has been minimized.

Contributor

k8s-merge-robot commented Mar 20, 2018

[MILESTONENOTIFIER] Milestone Removed From Pull Request

@deads2k @dims @mtaufen @timothysc

Important: This pull request was missing labels required for the v1.11 milestone for more than 3 days:

kind: Must specify exactly one of kind/bug, kind/cleanup or kind/feature.
priority: Must specify exactly one of priority/critical-urgent, priority/important-longterm or priority/important-soon.

Help

@k8s-merge-robot k8s-merge-robot removed this from the v1.11 milestone Mar 20, 2018

@dims

This comment has been minimized.

Member

dims commented Mar 27, 2018

@liggitt now that master is open and it's early in the cycle, let's please get this in.

// LeaderElectionConfiguration defines the configuration of leader election
// clients for components that can run with leader election enabled.
type LeaderElectionConfiguration struct {
metav1.TypeMeta `json:",inline"`

This comment has been minimized.

@liggitt

liggitt Mar 28, 2018

Member

Why does this need type info? It's not a top level type, right?

This comment has been minimized.

@dims

dims Mar 28, 2018

Member

@liggitt some of the tests fail without it

This comment has been minimized.

@liggitt

liggitt Mar 28, 2018

Member

unless this is serialized in standalone way (which I'm not aware of), this should not have type info. tests that fail without that should be fixed.

This comment has been minimized.

@dims

dims Mar 30, 2018

Member

ack removing it

@dims

This comment has been minimized.

Member

dims commented Mar 30, 2018

Ready and all green now @liggitt . thanks!

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
type KubeControllerManagerConfiguration struct {

This comment has been minimized.

@liggitt

liggitt Mar 30, 2018

Member

these should probably end up in controllermanager.config.k8s.io under pkg/controller/apis, similar to kubelet. can you move them in a follow up?

This comment has been minimized.

@dims

dims Mar 30, 2018

Member

yes i can

Set leader-elect for kube-scheduler to true
Thanks to some great sleuthing by ikruglov!

kube-controller-manager defaults --leader-elect to true. We should
do the same for kube-scheduler. kube-scheduler used to have this
set to true, but it got lost during refactoring in:
efb2bb7
@liggitt

This comment has been minimized.

Member

liggitt commented Mar 30, 2018

thanks
/lgtm

@k8s-ci-robot

This comment has been minimized.

Contributor

k8s-ci-robot commented Mar 30, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, dims, liggitt, timothysc, wlan0

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-merge-robot

This comment has been minimized.

Contributor

k8s-merge-robot commented Mar 30, 2018

/test all [submit-queue is verifying that this PR is safe to merge]

@bsalamat

This comment has been minimized.

Contributor

bsalamat commented Mar 30, 2018

Could you please update the title of this PR and state that it adds new configuration types?

@k8s-merge-robot

This comment has been minimized.

Contributor

k8s-merge-robot commented Mar 30, 2018

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-merge-robot k8s-merge-robot merged commit a44c6a8 into kubernetes:master Mar 30, 2018

13 of 14 checks passed

Submit Queue Required Github CI test is not green: pull-kubernetes-e2e-gce
Details
cla/linuxfoundation dims authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce Job succeeded.
Details
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details

@dims dims changed the title from Set leader-elect for kube-scheduler to true to Set leader-elect for kube-scheduler to true / Adds new configuration types and rework how we set defaults Mar 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment