Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kubelet config: maxPods, not MaxPods, update eni-max-pods file with new instance types #994

Merged
merged 2 commits into from Jul 17, 2020
Merged

kubelet config: maxPods, not MaxPods, update eni-max-pods file with new instance types #994

merged 2 commits into from Jul 17, 2020

Conversation

etungsten
Copy link
Contributor

@etungsten etungsten commented Jul 16, 2020 •
edited

Issue number:
Fixes #989

Description of changes:

Author: Erikson Tung <etung@amazon.com>
Date:   Thu Jul 16 10:32:02 2020 -0700

    kubelet-config: 'maxPods' not 'MaxPods'
    
    The field for setting max pods should be `maxPods` and not `MaxPods`

Author: Erikson Tung <etung@amazon.com>
Date:   Thu Jul 16 10:32:48 2020 -0700

    kubelet-config: update `eni-max-pods`
    
    Updates `eni-max-pods` with mappings of new instance types to their maximum number of pods supported.

Testing done:

Built 1.17 variant, launched instance
In 1.17 (the version where kubelet complains about MaxPods, other versions didn't log this issue)

kubelet config

bash-5.0# cat /etc/kubernetes/kubelet/config 
---
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: 0.0.0.0
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 2m0s
    enabled: true
  x509:
    clientCAFile: "/etc/kubernetes/pki/ca.crt"
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s
clusterDomain: cluster.local
clusterDNS:
- 10.100.0.10
resolvConf: "/etc/resolv.conf"
hairpinMode: hairpin-veth
cgroupDriver: systemd
cgroupRoot: "/"
runtimeRequestTimeout: 15m
featureGates:
  RotateKubeletServerCertificate: true
  CSIMigration: false
serializeImagePulls: false
serverTLSBootstrap: true
configMapAndSecretChangeDetectionStrategy: Cache
tlsCipherSuites:
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
maxPods: 29

kubelet no longer complains about failing to decode maxPods.

bash-5.0# journalctl -u kubelet
-- Logs begin at Thu 2020-07-16 19:12:59 UTC, end at Thu 2020-07-16 19:14:14 UTC. --
Jul 16 19:13:04 ip-192-168-3-191.us-west-2.compute.internal systemd[1]: Starting Kubelet...
Jul 16 19:13:04 ip-192-168-3-191.us-west-2.compute.internal host-ctr[3132]: time="2020-07-16T19:13:04Z" l
evel=info msg="Pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:602401143452:reposito
ry/eks/pause-amd64:3.1"
Jul 16 19:13:04 ip-192-168-3-191.us-west-2.compute.internal host-ctr[3132]: time="2020-07-16T19:13:04Z" l
evel=info msg="Pulled successfully" img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause-
amd64:3.1"
Jul 16 19:13:04 ip-192-168-3-191.us-west-2.compute.internal host-ctr[3132]: time="2020-07-16T19:13:04Z" l
evel=info msg=Unpacking... img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause-amd64:3.1
"
Jul 16 19:13:04 ip-192-168-3-191.us-west-2.compute.internal host-ctr[3132]: time="2020-07-16T19:13:04Z" l
evel=info msg="Tagging image" imageName="602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause-amd64:3.1
"
Jul 16 19:13:04 ip-192-168-3-191.us-west-2.compute.internal host-ctr[3132]: time="2020-07-16T19:13:04Z" l
evel=info msg="Not starting host container, pull-image-only mode specified"
Jul 16 19:13:05 ip-192-168-3-191.us-west-2.compute.internal kubelet[3182]: Flag --containerd has been dep
recated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns,
 it will follow the standard CLI deprecation timeline before being removed.
Jul 16 19:13:05 ip-192-168-3-191.us-west-2.compute.internal kubelet[3182]: W0716 19:13:05.503509    3182 
server.go:191] Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, 
which should be set in that remote runtime instead
Jul 16 19:13:05 ip-192-168-3-191.us-west-2.compute.internal kubelet[3182]: I0716 19:13:05.539247    3182 
server.go:416] Version: v1.17.8
Jul 16 19:13:05 ip-192-168-3-191.us-west-2.compute.internal kubelet[3182]: W0716 19:13:05.540037    3182 
plugins.go:115] WARNING: aws built-in cloud provider is now deprecated. The AWS provider is deprecated an
d will be removed in a future release
Jul 16 19:13:05 ip-192-168-3-191.us-west-2.compute.internal kubelet[3182]: I0716 19:13:05.540440    3182 
aws.go:1221] Building AWS cloudprovider
....

From 1.16:

kubelet config

bash-5.0# cat /etc/kubernetes/kubelet/config 
---
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: 0.0.0.0
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 2m0s
    enabled: true
  x509:
    clientCAFile: "/etc/kubernetes/pki/ca.crt"
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s
clusterDomain: cluster.local
clusterDNS:
- 10.100.0.10
resolvConf: "/etc/resolv.conf"
hairpinMode: hairpin-veth
cgroupDriver: systemd
cgroupRoot: "/"
runtimeRequestTimeout: 15m
featureGates:
  RotateKubeletServerCertificate: true
serializeImagePulls: false
serverTLSBootstrap: true
configMapAndSecretChangeDetectionStrategy: Cache
tlsCipherSuites:
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
maxPods: 29

kubelet decodes maxPods without issue:

bash-5.0# journalctl -u kubelet
-- Logs begin at Thu 2020-07-16 20:01:58 UTC, end at Thu 2020-07-16 20:06:01 UTC. --
Jul 16 20:02:02 ip-192-168-9-20.us-west-2.compute.internal systemd[1]: Starting Kubelet...
Jul 16 20:02:03 ip-192-168-9-20.us-west-2.compute.internal host-ctr[3135]: time="2020-07-16T20:02:03Z" le
vel=info msg="Pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repositor
y/eks/pause-amd64:3.1"
Jul 16 20:02:03 ip-192-168-9-20.us-west-2.compute.internal host-ctr[3135]: time="2020-07-16T20:02:03Z" le
vel=info msg="Pulled successfully" img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause-a
md64:3.1"
Jul 16 20:02:03 ip-192-168-9-20.us-west-2.compute.internal host-ctr[3135]: time="2020-07-16T20:02:03Z" le
vel=info msg=Unpacking... img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause-amd64:3.1"
Jul 16 20:02:03 ip-192-168-9-20.us-west-2.compute.internal host-ctr[3135]: time="2020-07-16T20:02:03Z" le
vel=info msg="Tagging image" imageName="602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause-amd64:3.1"
Jul 16 20:02:03 ip-192-168-9-20.us-west-2.compute.internal host-ctr[3135]: time="2020-07-16T20:02:03Z" le
vel=info msg="Not starting host container, pull-image-only mode specified"
Jul 16 20:02:04 ip-192-168-9-20.us-west-2.compute.internal kubelet[3183]: Flag --containerd has been depr
ecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, 
it will follow the standard CLI deprecation timeline before being removed.
Jul 16 20:02:04 ip-192-168-9-20.us-west-2.compute.internal kubelet[3183]: W0716 20:02:04.512237    3183 s
erver.go:189] Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, w
hich should be set in that remote runtime instead
Jul 16 20:02:04 ip-192-168-9-20.us-west-2.compute.internal kubelet[3183]: I0716 20:02:04.526984    3183 s
erver.go:410] Version: v1.16.9
....

For 1.15:

kubelet config

bash-5.0# cat /etc/kubernetes/kubelet/config 
---
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: 0.0.0.0
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 2m0s
    enabled: true
  x509:
    clientCAFile: "/etc/kubernetes/pki/ca.crt"
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s
clusterDomain: cluster.local
clusterDNS:
- 10.100.0.10
resolvConf: "/etc/resolv.conf"
hairpinMode: hairpin-veth
cgroupDriver: systemd
cgroupRoot: "/"
runtimeRequestTimeout: 15m
featureGates:
  RotateKubeletServerCertificate: true
serializeImagePulls: false
serverTLSBootstrap: true
configMapAndSecretChangeDetectionStrategy: Cache
tlsCipherSuites:
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
maxPods: 29

kubelet decodes maxPods without problems

bash-5.0# journalctl -u kubelet
-- Logs begin at Thu 2020-07-16 20:29:06 UTC, end at Thu 2020-07-16 20:31:10 UTC. --
Jul 16 20:29:42 ip-192-168-1-208.us-west-2.compute.internal systemd[1]: Starting Kubelet...
Jul 16 20:29:43 ip-192-168-1-208.us-west-2.compute.internal host-ctr[3142]: time="2020-07-16T20:29:43Z" l
evel=info msg="Pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:602401143452:reposito
ry/eks/pause-amd64:3.1"
Jul 16 20:29:43 ip-192-168-1-208.us-west-2.compute.internal host-ctr[3142]: time="2020-07-16T20:29:43Z" l
evel=info msg="Pulled successfully" img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause-
amd64:3.1"
Jul 16 20:29:43 ip-192-168-1-208.us-west-2.compute.internal host-ctr[3142]: time="2020-07-16T20:29:43Z" l
evel=info msg=Unpacking... img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause-amd64:3.1
"
Jul 16 20:29:43 ip-192-168-1-208.us-west-2.compute.internal host-ctr[3142]: time="2020-07-16T20:29:43Z" l
evel=info msg="Tagging image" imageName="602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause-amd64:3.1
"
Jul 16 20:29:43 ip-192-168-1-208.us-west-2.compute.internal host-ctr[3142]: time="2020-07-16T20:29:43Z" l
evel=info msg="Not starting host container, pull-image-only mode specified"
Jul 16 20:29:44 ip-192-168-1-208.us-west-2.compute.internal kubelet[3190]: Flag --containerd has been dep
recated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns,
 it will follow the standard CLI deprecation timeline before being removed.
Jul 16 20:29:44 ip-192-168-1-208.us-west-2.compute.internal kubelet[3190]: W0716 20:29:44.204017    3190 
server.go:191] Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, 
which should be set in that remote runtime instead
Jul 16 20:29:44 ip-192-168-1-208.us-west-2.compute.internal kubelet[3190]: I0716 20:29:44.224923    3190 
server.go:425] Version: v1.15.10

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@etungsten
kubelet-config: 'maxPods' not 'MaxPods'
78b269d 
The field for setting max pods should be `maxPods` and not `MaxPods`
@etungsten
kubelet-config: update eni-max-pods
97b1cf8 
Updates `eni-max-pods` with mappings of new instance types to their maximum number of pods supported.
@etungsten etungsten requested a review from bcressey July 16, 2020 19:18
@etungsten etungsten changed the title Maxpods not maxpods kubectl config: maxPods, not MaxPods Jul 16, 2020
@etungsten etungsten changed the title kubectl config: maxPods, not MaxPods kubectl config: maxPods, not MaxPods, update eni-max-pods file with new instance types Jul 16, 2020
@etungsten etungsten changed the title kubectl config: maxPods, not MaxPods, update eni-max-pods file with new instance types kubelet config: maxPods, not MaxPods, update eni-max-pods file with new instance types Jul 16, 2020
@etungsten etungsten requested a review from tjkirch July 16, 2020 20:33
zmrow
zmrow approved these changes Jul 16, 2020
View reviewed changes
Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☕

@etungsten etungsten merged commit 1ecb62d into bottlerocket-os:develop Jul 17, 2020
@etungsten etungsten deleted the maxpods-not-maxpods branch July 17, 2020 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tjkirch tjkirch tjkirch approved these changes

@bcressey bcressey bcressey approved these changes

@zmrow zmrow zmrow approved these changes

@webern webern webern approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

kubelet 1.17: MaxPods field not recognized in config
5 participants
@etungsten @tjkirch @bcressey @zmrow @webern