Skip to content
This repository has been archived by the owner on Oct 24, 2023. It is now read-only.

refactor: simplify kubelet systemd service #3167

Merged
merged 3 commits into from May 1, 2020

Conversation

jackfrancis
Copy link
Member

@jackfrancis jackfrancis commented Apr 29, 2020

Reason for Change:

This PR dusts off the cobwebs from the kubelet systemd implementation. Specifically:

  • moves bash block statements out from single line systemd ExecStartPre strings into the pre-existing /opt/azure/containers/kubelet.sh script (a ExecStartPre dependency)
  • enables the following containerd kubelet configuration to be overridden via api model:
    • --container-runtime
    • --runtime-request-timeout
    • --container-runtime-endpoint
  • removes iptables output from start of kubelet logs

Issue Fixed:

Requirements:

Notes:

@codecov
Copy link

codecov bot commented Apr 29, 2020

Codecov Report

Merging #3167 into master will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #3167   +/-   ##
=======================================
  Coverage   71.23%   71.24%           
=======================================
  Files         147      147           
  Lines       25745    25751    +6     
=======================================
+ Hits        18340    18346    +6     
  Misses       6268     6268           
  Partials     1137     1137           
Impacted Files Coverage Δ
pkg/engine/templates_generated.go 38.36% <ø> (ø)
pkg/api/defaults-kubelet.go 96.66% <100.00%> (+0.11%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 29dd816...c185b4d. Read the comment docs.

@jackfrancis jackfrancis force-pushed the kubelet-systemd-simplify branch 2 times, most recently from 15f3c71 to f413af8 Compare April 30, 2020 17:09
{{/* This is a partial workaround to this upstream Kubernetes issue: */}}
{{/* https://github.com/kubernetes/kubernetes/issues/41916#issuecomment-312428731 */}}

ExecStartPre=-/sbin/ebtables -t nat --list
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vote we get rid of this extra stdout at the beginning of the kubelet startup. There are better ways to accomplish that, if we want, and journalctl -u kubelet reporting that is misleading as the iptables/ebtables rules do not originate from the kubelet runtime.

ExecStartPre=/bin/bash -c "if [ $(mount | grep \"/var/lib/kubelet\" | wc -l) -le 0 ] ; then /bin/mount --bind /var/lib/kubelet /var/lib/kubelet ; fi"
ExecStartPre=/bin/mount --make-shared /var/lib/kubelet
{{/* This is a partial workaround to this upstream Kubernetes issue: */}}
{{/* https://github.com/kubernetes/kubernetes/issues/41916#issuecomment-312428731 */}}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't able to find what in here relates to the above (closed) issue, so I suspect it's an obsolete comment.

ExecStart=/usr/local/bin/kubelet \
--enable-server \
--node-labels="${KUBELET_NODE_LABELS}" \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell we want to keep the --node-labels configuration like this so we can declare the value of KUBELET_NODE_LABELS at ARM deployment runtime. The KUBELET_CONFIG env var is statically defined in the cloud-init template at generation time, so arbitrary ARM variable/parameters injection isn't possible via that vector.

PRIVATE_IP=$(hostname -i | cut -d" " -f1)
{{- if IsMasterVirtualMachineScaleSets}}
PRIVATE_IP=$(hostname -i | cut -d" " -f1)
sed -i "s|<SERVERIP>|https://$PRIVATE_IP:443|g" "/var/lib/kubelet/kubeconfig"
{{end}}
{{- if gt .MasterProfile.Count 1}}
# Redirect ILB (4443) traffic to port 443 (ELB) in the prerouting chain
{{- /* Redirect ILB (4443) traffic to port 443 (ELB) in the prerouting chain */}}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I'm in here...

@jackfrancis jackfrancis changed the title [WIP] refactor: simplify kubelet systemd service refactor: simplify kubelet systemd service Apr 30, 2020
ExecStart=/usr/local/bin/kubelet \
--enable-server \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--enable-server is the default, we don't need to explicitly set it

ref: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/

@acs-bot acs-bot added size/L and removed size/M labels Apr 30, 2020
mboersma
mboersma previously approved these changes May 1, 2020
Copy link
Member

@mboersma mboersma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Simpler is better, and I think all these changes are justified.

@acs-bot
Copy link

acs-bot commented May 1, 2020

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jackfrancis, mboersma

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [jackfrancis,mboersma]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@acs-bot
Copy link

acs-bot commented May 1, 2020

New changes are detected. LGTM label has been removed.

@acs-bot acs-bot removed the lgtm label May 1, 2020
@jackfrancis jackfrancis added this to In progress in backlog May 1, 2020
@jackfrancis jackfrancis merged commit 7c0a3f0 into Azure:master May 1, 2020
backlog automation moved this from In progress to Done May 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
backlog
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants