Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout value to configure how long kubeadm init takes to timeout #1168

Closed
soggiest opened this issue Oct 12, 2018 · 8 comments
Closed
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature. lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Milestone

Comments

@soggiest
Copy link

soggiest commented Oct 12, 2018

FEATURE REQUEST

Add a KubeADM configuration option or CLI flag that determines how long kubeadm init takes to timeout. Cloud-based installations have a chance to fail if the API Server's load balancer takes too long to recognize an instance is listening on the appropriate port. This can potentially be mitigated by modifying the load balancer's health checks. However, an option to extend the kubeadm init would be helpful.

Versions

kubeadm version (use kubeadm version): 1.11.3

Environment:

What happened?

KubeADM init runs were consistently failing on AWS with the following error: "timed out waiting for condition"
After each KubeADM run the Kubernetes control plane was working as expected. During an investigation into what was causing this error I observed that the AWS ELB would become active after KubeADM had timed out. Modifying the ELB health checks to be quicker alleviated the issue.

From a UX stand point I should be able to set the init timeout from a config or cli flag rather than worry about a race against time with my ELB.

What you expected to happen?

KubeADM init would wait long enough for the control plane to come up.

How to reproduce it (as minimally and precisely as possible)?

Region:
ap-southeast-1 (Singapore)

Configure an AWS EC2 Instance with:
AMI: Ubuntu 16.04 LTS
Size: m5.xlarge
Storage: 50GB
Tag: kubernetes.io/cluster/kubernetes: owned

Configure an ELB with:
Standard Health Checks Intervals, Health check against TCP:6443
TCP Passthrough 443 -> TCP 6443

Run kubeadm init with a kubeadm config that points to the ELB as the controlPlaneEndpoint

@neolit123
Copy link
Member

From a UX stand point I should be able to set the init timeout from a config or cli flag rather than worry about a race against time with my ELB.

i agree with this.

/kind feature
/cc @kubernetes/sig-cluster-lifecycle

@k8s-ci-robot k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Oct 13, 2018
@fabriziopandini
Copy link
Member

@timothysc opinions?

@dixudx
Copy link
Member

dixudx commented Oct 15, 2018

I'd love to add this. +1.

@timothysc
Copy link
Member

timothysc commented Oct 16, 2018

This has been requested a number of times, but the use case was on the other end of the spectrum.

@fabriziopandini could we roll this into the v1beta1 component(apiserver) migration work?

@timothysc timothysc added the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Oct 16, 2018
@timothysc timothysc added this to the v1.13 milestone Oct 16, 2018
@timothysc
Copy link
Member

/assign @soggiest
/assign @fabriziopandini
/assign @timothysc

@fabriziopandini
Copy link
Member

@timothysc ok
Just to set expectations: I'm going to propose a generic solution for defining timeouts in the kubeadm config API (not in flags) and use the new settings for making configurable only the timeout above + eventually the same timeout in the kubeadm --join-workflow (not all the timeouts in kubeadm).

@fabriziopandini fabriziopandini added the lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. label Oct 18, 2018
@timothysc timothysc assigned rosti and unassigned timothysc and soggiest Oct 31, 2018
@neolit123
Copy link
Member

xref kubernetes/kubernetes#70480

@neolit123
Copy link
Member

timeout for the api server was added in the v1beta1 config:
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/apis/kubeadm/v1beta1/types.go#L135

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants