New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubelet v1.11.3 ignores parameters from config file (--config) #69665

Closed
mslga opened this Issue Oct 11, 2018 · 3 comments

Comments

Projects
None yet
3 participants
@mslga

mslga commented Oct 11, 2018

/kind bug
/sig node

After upgrading cluster from v1.10.5 to v1.11.3 I encountered information in kubelet log:

Oct 09 15:44:08  kubelet[1636]: Flag --address has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 09 15:44:08  kubelet[1636]: Flag --allow-privileged has been deprecated, will be removed in a future version
...
Oct 09 15:44:08 = kubelet[1636]: Flag --feature-gates has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.

So, I decided to setup kubelet parameter via config file. I created following file:

kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: xxx.xxx.xxx.xxx
port: 10250
staticPodPath: /etc/kubernetes/manifests
nodeStatusUpdateFrequency: 10s
TLSCertFile: /etc/kubernetes/ssl/node.pem
TLSPrivateKeyFile: /etc/kubernetes/ssl/node-key.pem
authentication:
  x509:
    clientCAFile: /etc/kubernetes/ssl/ca.pem
  anonymous:
    enabled: false
cgroupDriver: cgroupfs
cgroupsPerQOS: true
maxPods: 110
failSwapOn: true
EnforceNodeAllocatable: ""
ClusterDNS: 10.233.0.3
ClusterDomain: cluster.local
ResolverConfig: /etc/resolv.conf
KubeReserved:
  cpu=100m,memory=256M
FeatureGates:
  PersistentLocalVolumes=False,VolumeScheduling=False,MountPropagation=False

I used this page as reference: https://github.com/kubernetes/kubernetes/blob/release-1.11/pkg/kubelet/apis/kubeletconfig/types.go

After restarting kubelet, I tried to deploy app into cluster and encountered warning:

Warning  MissingClusterDNS        8m (x6 over 11m)   kubelet,  kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to "Default" policy.

It seems kubelet ignores parameters from config file, because when I come back to previous kubelet configuration no warnings occurred.

  • Kubernetes version (use kubectl version): 1.11.3
  • Cloud provider or hardware configuration: vsphere
  • OS (e.g. from /etc/os-release): ubuntu 16.04
  • Kernel (e.g. uname -a): 4.4.0-134-generic
@Pingan2017

This comment has been minimized.

Show comment
Hide comment
@Pingan2017

Pingan2017 Oct 12, 2018

Contributor

try ClusterDNS: ["10.233.0.3"]

Contributor

Pingan2017 commented Oct 12, 2018

try ClusterDNS: ["10.233.0.3"]

@mslga

This comment has been minimized.

Show comment
Hide comment
@mslga

mslga Oct 15, 2018

@Pingan2017
It seems doesn't work

  Warning  MissingClusterDNS        4s (x10 over 7s)     kubelet,  kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to "Default" policy.

vi kubelet_config.yaml

ClusterDNS: ["10.233.0.3"]

mslga commented Oct 15, 2018

@Pingan2017
It seems doesn't work

  Warning  MissingClusterDNS        4s (x10 over 7s)     kubelet,  kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to "Default" policy.

vi kubelet_config.yaml

ClusterDNS: ["10.233.0.3"]
@mslga

This comment has been minimized.

Show comment
Hide comment
@mslga

mslga Oct 19, 2018

I have found correct configuration:

kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: xxx.xxx.xxx.xxx
port: 10250
staticPodPath: /etc/kubernetes/manifests
nodeStatusUpdateFrequency: 10s
tlsCertFile: /etc/kubernetes/ssl/node.pem
tlsPrivateKeyFile: /etc/kubernetes/ssl/node-key.pem
authentication:
  x509:
    clientCAFile: /etc/kubernetes/ssl/ca.pem
  anonymous:
    enabled: false
cgroupDriver: cgroupfs
cgroupsPerQOS: true
maxPods: 110
failSwapOn: true
EnforceNodeAllocatable: ""
clusterDNS: ["10.233.0.3"]
clusterDomain: cluster.local
resolverConfig: /etc/resolv.conf
kubeReserved:
  {
    "cpu": "100m",
    "memory": "256M"
  }
featureGates:
  {
    "PersistentLocalVolumes": false,
    "VolumeScheduling": false,
    "MountPropagation": false
  }

mslga commented Oct 19, 2018

I have found correct configuration:

kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: xxx.xxx.xxx.xxx
port: 10250
staticPodPath: /etc/kubernetes/manifests
nodeStatusUpdateFrequency: 10s
tlsCertFile: /etc/kubernetes/ssl/node.pem
tlsPrivateKeyFile: /etc/kubernetes/ssl/node-key.pem
authentication:
  x509:
    clientCAFile: /etc/kubernetes/ssl/ca.pem
  anonymous:
    enabled: false
cgroupDriver: cgroupfs
cgroupsPerQOS: true
maxPods: 110
failSwapOn: true
EnforceNodeAllocatable: ""
clusterDNS: ["10.233.0.3"]
clusterDomain: cluster.local
resolverConfig: /etc/resolv.conf
kubeReserved:
  {
    "cpu": "100m",
    "memory": "256M"
  }
featureGates:
  {
    "PersistentLocalVolumes": false,
    "VolumeScheduling": false,
    "MountPropagation": false
  }

@mslga mslga closed this Oct 19, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment