Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable mode for kube proxy #937

Merged
merged 12 commits into from
May 28, 2021

Conversation

jnummelin
Copy link
Collaborator

@jnummelin jnummelin commented May 26, 2021

Issue
Currently kube-proxy mode is not really configurable. There are cases when we want to make the proxy mode something else than the default iptables, e.g.:

  • huge amount of services --> iptables is way too CPU heavy
  • very limited set of kernel modules available --> we need to set it to userspace mode

Partial (the kube-proxy part) fix for #427 as this now add possibility to disable kube-proxy completely.

What this PR Includes

This PR makes kube proxy mode configurable and possible to disable too.

This also includes quite a few fixes for the stack delete logic as that was pretty badly broken in some cases.

Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
- The Applier discovery internals were not properly initialized if stack delete happens before apply
- The "unit" test was not testing the stack delete path at all
- When trying to ignore NotFound and Gone cases in resource deletion, code incorrectly always returned an error --> only one resource per stack was deleted 🤦
- Discovering pruneable resource incorrectly collected the api group/version/kind info as empty strings.

Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
@jnummelin jnummelin requested a review from a team as a code owner May 26, 2021 19:13
@jnummelin jnummelin requested review from kke and trawler May 26, 2021 19:13
…in stack apply

Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
…bject

Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
@jnummelin jnummelin marked this pull request as draft May 26, 2021 22:44
@jnummelin
Copy link
Collaborator Author

Marking as draft as this needs some finetuning...

Now that the API discovery is actually properly working, it turns out that finding prunable resources takes quite a bit of time and resources:

 # grep "finding prunable resources took" /tmp/k0s-controller.log 
time="2021-05-26 21:22:56" level=debug msg="finding prunable resources took 16.771622637s" stack=konnectivity
time="2021-05-26 21:23:42" level=debug msg="finding prunable resources took 1m2.305347895s" stack=coredns
time="2021-05-26 21:23:48" level=debug msg="finding prunable resources took 1m8.106179424s" stack=calico
time="2021-05-26 21:23:54" level=debug msg="finding prunable resources took 1m14.27752149s" stack=metricserver
time="2021-05-26 21:24:00" level=debug msg="finding prunable resources took 1m20.077316951s" stack=kubeproxy
time="2021-05-26 21:24:18" level=debug msg="finding prunable resources took 1m1.994904597s" stack=helm
time="2021-05-26 21:24:40" level=debug msg="finding prunable resources took 19.998879199s" stack=bootstraprbac
time="2021-05-26 21:24:58" level=debug msg="finding prunable resources took 34.998554166s" stack=kubelet
time="2021-05-26 21:25:18" level=debug msg="finding prunable resources took 18.799687039s" stack=defaultpsp
time="2021-05-26 21:25:36" level=debug msg="finding prunable resources took 23.996521471s" stack=coredns
time="2021-05-26 21:25:52" level=debug msg="finding prunable resources took 33.398416512s" stack=kubeproxy
time="2021-05-26 21:26:12" level=debug msg="finding prunable resources took 17.589150279s" stack=metricserver
time="2021-05-26 21:26:28" level=debug msg="finding prunable resources took 32.59850719s" stack=calico
time="2021-05-26 21:26:44" level=debug msg="finding prunable resources took 31.999208489s" stack=coredns
time="2021-05-26 21:27:01" level=debug msg="finding prunable resources took 15.797285968s" stack=calico_init

This is the reason why smokes are red, we're hitting timeouts. 😢

Need to figure out some way(s) to speed things up

Some general refactorings for stack to simplify and add debuggability.

Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
…n creadnetials so we know we have full access to all namespaces anyway.

Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
…When we bootup k0s,

it needs to sync / apply LOT of manifests and resource in the API and thus will make lot of requests.
Thus many of the calls ended up being throttled thus slowing up the speed in which we get everything up-and-running.

Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
ncopa
ncopa previously approved these changes May 28, 2021
Copy link
Collaborator

@ncopa ncopa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great!

Signed-off-by: Jussi Nummelin <jnummelin@mirantis.com>
@jnummelin jnummelin marked this pull request as ready for review May 28, 2021 13:53
@ncopa ncopa merged commit 194311f into k0sproject:main May 28, 2021
@mviitane mviitane added this to the 1.21.x June milestone Jun 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants