Update master (#8)

* fix(contrib/metallb): adds missing become: true in role (kubernetes-sigs#4356)

On CoreOS, without this, it fails to kubectl apply MetalLB due to lack of privileges.

* Fix kubernetes-sigs#4237: update kube cert path (kubernetes-sigs#4354)

* Use sample inventory file in doc (kubernetes-sigs#4052)

* Revert "Fix kubernetes-sigs#4237: update kube cert path (kubernetes-sigs#4354)" (kubernetes-sigs#4369)

This reverts commit ea7a6f1.

This change modified the certs dir for Kubernetes, but did not move the directories for existing clusters.

* Fix support for ansible 2.7.9 (kubernetes-sigs#4375)

* Use wide for netchecker debug output (kubernetes-sigs#4383)

* Added support of bastion host for reset.yaml (kubernetes-sigs#4359)

* Added support of bastion host for reset.yaml

* Empty commit to triger CI

* Use proxy_env with kubeadm phase commands (kubernetes-sigs#4325)

* clarify that kubespray now supports kubeadm (fixes kubernetes-sigs#4089) (kubernetes-sigs#4366)

* Reduce jinja2 filters in coredns templates (kubernetes-sigs#4390)

* Upgrade to k8s 1.13.5

* Increase CPU flavor for CI (kubernetes-sigs#4389)

* Fix CA cert environment variable for ectd v3 (kubernetes-sigs#4381)

* Added livenessProbe for local nginx apiserver proxy liveness probe (kubernetes-sigs#4222)

* Added configurable local apiserver proxy liveness probe

* Enable API LB healthcheck by default

* Fix template spacing and moved healthz location to nginx http section

* Fix healthcheck listen address to allow kubelet request healthcheck

* Default values for variable dns_servers and dns_domain  are set in two files: (kubernetes-sigs#3999)

values from inventory in roles/kubespray-defaults/defaults/main.yml
hardcoded values in roles/container-engine/defaults/main.yml

dns_servers set empty in roles/container-engine/defaults/main.yml and skydns_server not set in docker_dns_servers variables
also set default value for manual_dns_serve

another variables in roles/container-engine/defaults not need to set

* Fix bootsrap-os role, failing to create remote_tmp (kubernetes-sigs#4384)

* Fix bootsrap-os role, failing to create remote_tmp

* use ansible_remote_tmp hostvar

* Use static files in KubeDNS templating task (kubernetes-sigs#4379)

This commit adapts the "Lay Down KubeDNS Template" task to use the static
files moved by pull request [1]

[1] kubernetes-sigs#4341

* Fix supplementary_addresses rendering error (kubernetes-sigs#4403)

* Corrected cloud name (kubernetes-sigs#4316)

The correct name is Packet, not Packet Host.

* adapt inventory script to python 2.7 version (kubernetes-sigs#4407)

* Calico felix - Fix jinja2 boolean condition (kubernetes-sigs#4348)

* Fix jinja2 boolean condition

* Convert all felix variable to booleans instead.

README.md

@@ -112,7 +112,7 @@ Supported Components
 --------------------
 
 -   Core
-    -   [kubernetes](https://github.com/kubernetes/kubernetes) v1.13.4
+    -   [kubernetes](https://github.com/kubernetes/kubernetes) v1.13.5
     -   [etcd](https://github.com/coreos/etcd) v3.2.24
     -   [docker](https://www.docker.com/) v18.06 (see note)
     -   [rkt](https://github.com/rkt/rkt) v1.21.0 (see Note 2)

contrib/inventory_builder/inventory.py

@@ -197,8 +197,14 @@ def range2ips(self, hosts):
         reworked_hosts = []
 
         def ips(start_address, end_address):
-            start = int(ip_address(start_address).packed.hex(), 16)
-            end = int(ip_address(end_address).packed.hex(), 16)
+            try:
+                # Python 3.x
+                start = int(ip_address(start_address))
+                end   = int(ip_address(end_address))
+            except:
+                # Python 2.7
+                start = int(ip_address(unicode(start_address)))
+                end   = int(ip_address(unicode(end_address)))
             return [ip_address(ip).exploded for ip in range(start, end+1)]
 
         for host in hosts:

contrib/metallb/README.md

@@ -6,5 +6,5 @@ This playbook aims to automate [this](https://metallb.universe.tf/tutorial/layer
 
 ## Install
 ```
-ansible-playbook --ask-become -i inventory/sample/k8s_heketi_inventory.yml contrib/metallb/metallb.yml
+ansible-playbook --ask-become -i inventory/sample/hosts.ini contrib/metallb/metallb.yml
 ```

contrib/metallb/roles/provision/tasks/main.yml

@@ -12,6 +12,7 @@
     kubectl: "{{bin_dir}}/kubectl"
     filename: "{{ kube_config_dir }}/{{ item.item }}"
     state: "{{ item.changed | ternary('latest','present') }}"
+  become: true
   with_items: "{{ rendering.results }}"
   when:
     - "inventory_hostname == groups['kube-master'][0]"

docs/comparisons.md

@@ -19,7 +19,9 @@ on. Had it belonged to the new [operators world](https://coreos.com/blog/introdu
 it may have been named a "Kubernetes cluster operator". Kubespray however,
 does generic configuration management tasks from the "OS operators" ansible
 world, plus some initial K8s clustering (with networking plugins included) and
-control plane bootstrapping. Kubespray [strives](https://github.com/kubernetes-sigs/kubespray/issues/553)
-to adopt kubeadm as a tool in order to consume life cycle management domain
-knowledge from it and offload generic OS configuration things from it, which
-hopefully benefits both sides.
+control plane bootstrapping. 
+
+Kubespray supports `kubeadm` for cluster creation since v2.3 
+(and deprecated non-kubeadm deployment starting from v2.8)
+in order to consume life cycle management domain knowledge from it
+and offload generic OS configuration things from it, which hopefully benefits both sides.

docs/packet.md

@@ -1,7 +1,7 @@
-Packet Host
+Packet
 ===============
 
-Kubespray provides support for bare metal deployments using the [Packet Host bare metal cloud](http://www.packet.com).
+Kubespray provides support for bare metal deployments using the [Packet bare metal cloud](http://www.packet.com).
 Deploying upon bare metal allows Kubernetes to run at locations where an existing public or private cloud might not exist such
 as cell tower, edge collocated installations. The deployment mechanism used by Kubespray for Packet is similar to that used for
 AWS and OpenStack clouds (notably using Terraform to deploy the infrastructure). Terraform uses the Packet provider plugin
@@ -10,7 +10,7 @@ dynamically from the Terraform state file.
 
 ## Local Host Configuration
 
-To perform this installation, you will need a localhost to run Terraform/Ansible (laptop, VM, etc) and an account with Packet Host.
+To perform this installation, you will need a localhost to run Terraform/Ansible (laptop, VM, etc) and an account with Packet.
 In this example, we're using an m1.large CentOS 7 OpenStack VM as the localhost to kickoff the Kubernetes installation.
 You'll need Ansible, Git, and PIP.
 

inventory/sample/group_vars/all/all.yml

@@ -24,6 +24,8 @@ bin_dir: /usr/local/bin
 ## Local loadbalancer should use this port
 ## And must be set port 6443
 nginx_kube_apiserver_port: 6443
+## If nginx_kube_apiserver_healthcheck_port variable defined, enables proxy liveness check.
+nginx_kube_apiserver_healthcheck_port: 8081
 
 ### OTHER OPTIONAL VARIABLES
 ## For some things, kubelet needs to load kernel modules.  For example, dynamic kernel services are needed

inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml

@@ -19,7 +19,7 @@ kube_users_dir: "{{ kube_config_dir }}/users"
 kube_api_anonymous_auth: true
 
 ## Change this to use another Kubernetes version, e.g. a current beta release
-kube_version: v1.13.4
+kube_version: v1.13.5
 
 # kubernetes image repo define
 kube_image_repo: "gcr.io/google-containers"

reset.yml

@@ -13,6 +13,12 @@
   vars:
     ansible_connection: local
 
+- hosts: bastion[0]
+  gather_facts: False
+  roles:
+    - { role: kubespray-defaults}
+    - { role: bastion-ssh-config, tags: ["localhost", "bastion"]}
+
 - hosts: all
   gather_facts: true
 

roles/bootstrap-os/tasks/main.yml

@@ -30,7 +30,7 @@
 
 - name: Create remote_tmp for it is used by another module
   file:
-    path: "{{ lookup('config', 'DEFAULT_REMOTE_TMP', on_missing='skip', wantlist=True) | first | default('~/.ansible/tmp') }}"
+    path: "{{ ansible_remote_tmp | default('~/.ansible/tmp') }}"
     state: directory
     mode: 0700
 

roles/download/defaults/main.yml

@@ -35,7 +35,7 @@ download_delegate: "{% if download_localhost %}localhost{% else %}{{groups['kube
 image_arch: "{{host_architecture | default('amd64')}}"
 
 # Versions
-kube_version: v1.13.4
+kube_version: v1.13.5
 kubeadm_version: "{{ kube_version }}"
 etcd_version: v3.2.24
 
@@ -71,6 +71,7 @@ cni_download_url: "https://github.com/containernetworking/plugins/releases/downl
 # Checksums
 hyperkube_checksums:
   arm64:
+    v1.13.5: 8ffd84ba0cb6382a0ff96000458db8a83c92cac09458defe8496f0f0e155a6a8
     v1.13.4: b9e909e388634d103fe5376aafa313bed5e69293383b0c740de4fe8e18d42d12
     v1.13.3: 588037923b7f4090f5f7a3de23ea49a10345295f0b39bd0c1ebdaa24eaa76731
     v1.13.2: 7f2c2b0c6dcc81102a89fa41957db214416fc8a0cfae664fc0e150a7d3ad337b
@@ -89,6 +90,7 @@ hyperkube_checksums:
     v1.11.1: 43be988ec21bc34c0d1a838098b542016199e4f42466fdf8b0edb26718a1338f
     v1.11.0: 1422f67530888947cc5a7e71a12757ef496efa91cf68888a315d4803cc414294
   amd64:
+    v1.13.5: 1a8a357ebfeab8ec62d0c6f11b59df1a93d6711c3a16e1501da32b55c144c73a
     v1.13.4: 6f2d755a350efec8b3b29e0ddf8362f60475cc10d42dea37f8f2159f7776867b
     v1.13.3: b238c772b5e4b9deed0cdc695fe86324660d037b38c6d6d7eeae7d7a657840c7
     v1.13.2: f159b587ec80ad03bf3b9bb09de5d64b773d01b0e34f2a4f1c816879c56aae6d
@@ -108,6 +110,7 @@ hyperkube_checksums:
     v1.11.0: 7e191c164dc2c942abd37e4b50846e0be31ca959afffeff6b034beacbc2a106a
 kubeadm_checksums:
   arm64:
+    v1.13.5: 59a1995c171e5c1e74f5d02657eb2c155706f2d159ec1847b64dc866228c40d2
     v1.13.4: 4de71d4cfa4dc64127148d48f3a1a1fa7ea24cf0c4fa42957459d0e7f9c03799
     v1.13.3: bef1cbc2d199d32a1a31e70b864dc539b24e3c1cb87b50a1295cf03bec4832b0
     v1.13.2: 08279a3bfeff8c4f6768d6fd92ceff8276a555f9e81bf9d541112fc8eb29963e
@@ -126,6 +129,7 @@ kubeadm_checksums:
     v1.11.1: 6d7c7d5d4b8295ebe18aa2e9fb29917018e41628390909d0de6accbd0f2d56f5
     v1.11.0: f61d9bacdb8306c7e8662010817e0f9d2f380aeb87b2b0fe7801e83843b83d0e
   amd64:
+    v1.13.5: 274bf887039a9993e30f96047a4a474c39e8471c4094acb75aea6beed793f079
     v1.13.4: c4300d1f3ebccad48c8e267e45a736c7d227b0e45ef36582fa8dcfe2ef7b1b10
     v1.13.3: ab767ea53e45aceba628977ef6c8c62eace72d6d232efeaf35ac50cbea5f3739
     v1.13.2: 7cb0ce57c1e6e2d85e05de3780a2f35a191fe93f89cfc5816b424efcf39834b9

roles/etcd/handlers/backup.yml

@@ -52,7 +52,7 @@
     ETCDCTL_API: 3
     ETCDCTL_CERT: "{{ etcd_cert_dir }}/admin-{{ inventory_hostname }}.pem"
     ETCDCTL_KEY: "{{ etcd_cert_dir }}/admin-{{ inventory_hostname }}-key.pem"
-    ETCDCTL_CA_FILE: "{{ etcd_cert_dir }}/ca.pem"
+    ETCDCTL_CACERT: "{{ etcd_cert_dir }}/ca.pem"
   retries: 3
   register: etcd_backup_v3_command
   until: etcd_backup_v3_command.rc == 0

roles/kubernetes-apps/ansible/defaults/main.yml

@@ -7,6 +7,7 @@ dns_min_replicas: 2
 dns_nodes_per_replica: 16
 dns_cores_per_replica: 256
 dns_prevent_single_point_failure: "{{ 'true' if dns_min_replicas|int > 1 else 'false' }}"
+coredns_ordinal_suffix: ""
 
 # nodelocaldns
 nodelocaldns_cpu_requests: 100m

roles/kubernetes-apps/ansible/tasks/kubedns.yml

@@ -1,18 +1,19 @@
 ---
 
 - name: Kubernetes Apps | Lay Down KubeDNS Template
-  template:
-    src: "{{ item.file }}.j2"
+  action: "{{ item.module }}"
+  args:
+    src: "{{ item.file }}{% if item.module == 'template' %}.j2{% endif %}"
     dest: "{{ kube_config_dir }}/{{ item.file }}"
   with_items:
-    - { name: kube-dns, file: kubedns-sa.yml, type: sa }
-    - { name: kube-dns, file: kubedns-config.yml, type: configmap }
-    - { name: kube-dns, file: kubedns-deploy.yml, type: deployment }
-    - { name: kube-dns, file: kubedns-svc.yml, type: svc }
-    - { name: dns-autoscaler, file: dns-autoscaler-sa.yml, type: sa }
-    - { name: dns-autoscaler, file: dns-autoscaler-clusterrole.yml, type: clusterrole }
-    - { name: dns-autoscaler, file: dns-autoscaler-clusterrolebinding.yml, type: clusterrolebinding }
-    - { name: dns-autoscaler, file: dns-autoscaler.yml, type: deployment }
+    - { name: kube-dns, module: template, file: kubedns-sa.yml, type: sa }
+    - { name: kube-dns, module: template, file: kubedns-config.yml, type: configmap }
+    - { name: kube-dns, module: template, file: kubedns-deploy.yml, type: deployment }
+    - { name: kube-dns, module: template, file: kubedns-svc.yml, type: svc }
+    - { name: dns-autoscaler, module: copy, file: dns-autoscaler-sa.yml, type: sa }
+    - { name: dns-autoscaler, module: copy, file: dns-autoscaler-clusterrole.yml, type: clusterrole }
+    - { name: dns-autoscaler, module: copy, file: dns-autoscaler-clusterrolebinding.yml, type: clusterrolebinding }
+    - { name: dns-autoscaler, module: template, file: dns-autoscaler.yml, type: deployment }
   register: kubedns_manifests
   when:
     - dns_mode in ['kubedns','dnsmasq_kubedns']

roles/kubernetes-apps/ansible/templates/coredns-config.yml.j2

@@ -14,15 +14,15 @@ data:
         kubernetes {{ dns_domain }} in-addr.arpa ip6.arpa {
           pods insecure
 {% if resolvconf_mode == 'host_resolvconf' and upstream_dns_servers is defined and upstream_dns_servers|length > 0 %}
-          upstream {{ upstream_dns_servers|join(' ') }}
+          upstream {{ upstream_dns_servers.join(' ') }}
 {% else %}
           upstream /etc/resolv.conf
 {% endif %}
           fallthrough in-addr.arpa ip6.arpa
         }
         prometheus :9153
 {% if resolvconf_mode == 'host_resolvconf' and upstream_dns_servers is defined and upstream_dns_servers|length > 0 %}
-        proxy . {{ upstream_dns_servers|join(' ') }}
+        proxy . {{ upstream_dns_servers.join(' ') }}
 {% else %}
         proxy . /etc/resolv.conf
 {% endif %}

roles/kubernetes-apps/ansible/templates/coredns-deployment.yml.j2

@@ -2,13 +2,13 @@
 apiVersion: extensions/v1beta1
 kind: Deployment
 metadata:
-  name: "coredns{{ coredns_ordinal_suffix | default('') }}"
+  name: "coredns{{ coredns_ordinal_suffix }}"
   namespace: kube-system
   labels:
-    k8s-app: "coredns{{ coredns_ordinal_suffix | default('') }}"
+    k8s-app: "coredns{{ coredns_ordinal_suffix }}"
     kubernetes.io/cluster-service: "true"
     addonmanager.kubernetes.io/mode: Reconcile
-    kubernetes.io/name: "coredns{{ coredns_ordinal_suffix | default('') }}"
+    kubernetes.io/name: "coredns{{ coredns_ordinal_suffix }}"
 spec:
   strategy:
     type: RollingUpdate
@@ -17,17 +17,15 @@ spec:
       maxSurge: 10%
   selector:
     matchLabels:
-      k8s-app: coredns{{ coredns_ordinal_suffix | default('') }}
+      k8s-app: coredns{{ coredns_ordinal_suffix }}
   template:
     metadata:
       labels:
-        k8s-app: coredns{{ coredns_ordinal_suffix | default('') }}
+        k8s-app: coredns{{ coredns_ordinal_suffix }}
       annotations:
         seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
     spec:
-{% if kube_version is version('v1.11.1', '>=') %}
       priorityClassName: system-cluster-critical
-{% endif %}
       nodeSelector:
         beta.kubernetes.io/os: linux
       serviceAccountName: coredns
@@ -42,7 +40,7 @@ spec:
           - topologyKey: "kubernetes.io/hostname"
             labelSelector:
               matchLabels:
-                k8s-app: coredns{{ coredns_ordinal_suffix | default('') }}
+                k8s-app: coredns{{ coredns_ordinal_suffix }}
         nodeAffinity:
           preferredDuringSchedulingIgnoredDuringExecution:
           - weight: 100

roles/kubernetes-apps/ansible/templates/coredns-svc.yml.j2

@@ -2,20 +2,20 @@
 apiVersion: v1
 kind: Service
 metadata:
-  name: coredns{{ coredns_ordinal_suffix | default('') }}
+  name: coredns{{ coredns_ordinal_suffix }}
   namespace: kube-system
   labels:
-    k8s-app: coredns{{ coredns_ordinal_suffix | default('') }}
+    k8s-app: coredns{{ coredns_ordinal_suffix }}
     kubernetes.io/cluster-service: "true"
-    kubernetes.io/name: "coredns{{ coredns_ordinal_suffix | default('') }}"
+    kubernetes.io/name: "coredns{{ coredns_ordinal_suffix }}"
     addonmanager.kubernetes.io/mode: Reconcile
   annotations:
     prometheus.io/path: /metrics
     prometheus.io/port: "9153"
     prometheus.io/scrape: "true"
 spec:
   selector:
-    k8s-app: coredns{{ coredns_ordinal_suffix | default('') }}
+    k8s-app: coredns{{ coredns_ordinal_suffix }}
   clusterIP: {{ clusterIP }}
   ports:
     - name: dns

roles/kubernetes-apps/ansible/templates/dns-autoscaler.yml.j2

@@ -16,20 +16,20 @@
 apiVersion: apps/v1
 kind: Deployment
 metadata:
-  name: dns-autoscaler{{ coredns_ordinal_suffix | default('') }}
+  name: dns-autoscaler{{ coredns_ordinal_suffix }}
   namespace: kube-system
   labels:
-    k8s-app: dns-autoscaler{{ coredns_ordinal_suffix | default('') }}
+    k8s-app: dns-autoscaler{{ coredns_ordinal_suffix }}
     kubernetes.io/cluster-service: "true"
     addonmanager.kubernetes.io/mode: Reconcile
 spec:
   selector:
     matchLabels:
-      k8s-app: dns-autoscaler{{ coredns_ordinal_suffix | default('') }}
+      k8s-app: dns-autoscaler{{ coredns_ordinal_suffix }}
   template:
     metadata:
       labels:
-        k8s-app: dns-autoscaler{{ coredns_ordinal_suffix | default('') }}
+        k8s-app: dns-autoscaler{{ coredns_ordinal_suffix }}
       annotations:
         seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
     spec:
@@ -48,7 +48,7 @@ spec:
           - topologyKey: "kubernetes.io/hostname"
             labelSelector:
               matchLabels:
-                k8s-app: dns-autoscaler{{ coredns_ordinal_suffix | default('') }}
+                k8s-app: dns-autoscaler{{ coredns_ordinal_suffix }}
         nodeAffinity:
           preferredDuringSchedulingIgnoredDuringExecution:
           - weight: 100
@@ -71,9 +71,9 @@ spec:
         - --default-params={"linear":{"preventSinglePointFailure":{{ dns_prevent_single_point_failure }},"coresPerReplica":{{ dns_cores_per_replica }},"nodesPerReplica":{{ dns_nodes_per_replica }},"min":{{ dns_min_replicas }}}}
         - --logtostderr=true
         - --v=2
-        - --configmap=dns-autoscaler{{ coredns_ordinal_suffix | default('') }}
+        - --configmap=dns-autoscaler{{ coredns_ordinal_suffix }}
 {% if dns_mode in ['coredns', 'coredns_dual'] %}
-        - --target=Deployment/coredns{{ coredns_ordinal_suffix | default('') }}
+        - --target=Deployment/coredns{{ coredns_ordinal_suffix }}
 {% endif %}
 {% if dns_mode in ['kubedns', 'dnsmasq_kubedns'] %}
         - --target=Deployment/kube-dns

roles/kubernetes-apps/external_provisioner/local_volume_provisioner/tasks/main.yml

@@ -9,7 +9,7 @@
   delegate_to: "{{ item[0] }}"
   with_nested:
     - "{{ groups['k8s-cluster'] }}"
-    - "{{ local_volume_provisioner_storage_classes.keys() }}"
+    - "{{ local_volume_provisioner_storage_classes.keys() | list}}"
 
 - name: Local Volume Provisioner | Create addon dir
   file:
@@ -59,4 +59,4 @@
   with_items: "{{ local_volume_provisioner_manifests.results }}"
   when: inventory_hostname == groups['kube-master'][0]
   loop_control:
-    label: "{{ item.item.file }}"
+    label: "{{ item.item.file }}"

roles/kubernetes/client/tasks/main.yml

@@ -52,6 +52,7 @@
     --cert-dir {{ kube_config_dir }}/ssl
     --apiserver-advertise-address {{ external_apiserver_address }}
     --apiserver-bind-port {{ external_apiserver_port }}
+  environment: "{{ proxy_env }}"
   run_once: yes
   register: admin_kubeconfig
 

roles/kubernetes/master/tasks/kubeadm-certificate.yml

@@ -27,6 +27,7 @@
 
 - name: Generate new certs and keys
   command: "{{ bin_dir }}/kubeadm init phase certs {{ item }} --config={{ kube_config_dir }}/kubeadm-config.yaml"
+  environment: "{{ proxy_env }}"
   with_items:
     - apiserver
     - apiserver-kubelet-client
@@ -35,6 +36,7 @@
 
 - name: Generate new certs and keys
   command: "{{ bin_dir }}/kubeadm alpha phase certs {{ item }} --config={{ kube_config_dir }}/kubeadm-config.yaml"
+  environment: "{{ proxy_env }}"
   with_items:
     - apiserver
     - apiserver-kubelet-client

roles/kubernetes/master/tasks/kubeadm-kubeconfig.yml

@@ -23,10 +23,12 @@
 
 - name: Generate new configuration files
   command: "{{ bin_dir }}/kubeadm init phase kubeconfig all --config={{ kube_config_dir }}/kubeadm-config.yaml"
+  environment: "{{ proxy_env }}"
   when: kubeadm_version is version('v1.13.0', '>=')
   ignore_errors: yes
 
 - name: Generate new configuration files
   command: "{{ bin_dir }}/kubeadm alpha phase kubeconfig all --config={{ kube_config_dir }}/kubeadm-config.yaml"
+  environment: "{{ proxy_env }}"
   when: kubeadm_version is version('v1.13.0', '<')
   ignore_errors: yes