Support for Kubernetes 1.9

[camilb]

It looks the conformance tests are passing for kubernetes 1.9

https://scanner.heptio.com/1251e49fcd3957ed19b0c2e44549491e/diagnostics/

Necessary changes in kube-proxy configmap:

         - apiVersion: componentconfig/v1alpha1
         + apiVersion: kubeproxy.config.k8s.io/v1alpha1

[whereisaaron]

A significant option for AWS users is the ability to create NLBs for LoadBalancer Service/Ingress resources.

In addition, support for NVMe disks has been added to Kubernetes, and a service of type LoadBalancer can now be backed with an NLB instead of an ELB (alpha).

Plus better handling for the unsolved AWS issue of unreliable EBS attachments.

In v1.9 SIG AWS has improved stability of EBS support across the board. If a Volume is “stuck” in the attaching state to a node for too long a unschedulable taint will be applied to the node, so a Kubernetes admin can take manual steps to correct the error. Users are encouraged to ensure they are monitoring for the taint, and should consider automatically terminating instances in this state.

[whereisaaron]

1.9 and 1.8.6 also fixes the problem you might have noticed with 0.9.9 where kubectl get nodes -o wide where Kubernetes 1.8 can't work out the Docker runtime version (since the switch to Docker 'CE') and shows instead: docker://Unknown

AME                                          STATUS    ROLES     AGE       VERSION           EXTERNAL-IP   OS-IMAGE                                       KERNEL-VERSION      CONTAINER-RUNTIME
ip-172-17-13-157.us-east-2.compute.internal   Ready     <none>    7m        v1.8.4+coreos.0   <none>        Container Linux by CoreOS 1576.4.0 (Ladybug)   4.13.16-coreos-r2   docker://Unknown
ip-172-17-13-215.us-east-2.compute.internal   Ready     master    14m       v1.8.4+coreos.0   <none>        Container Linux by CoreOS 1576.4.0 (Ladybug)   4.13.16-coreos-r2   docker://Unknown
ip-172-17-13-219.us-east-2.compute.internal   Ready     <none>    6m        v1.8.4+coreos.0   <none>        Container Linux by CoreOS 1576.4.0 (Ladybug)   4.13.16-coreos-r2   docker://Unknown
ip-172-17-13-231.us-east-2.compute.internal   Ready     <none>    6m        v1.8.4+coreos.0   <none>        Container Linux by CoreOS 1576.4.0 (Ladybug)   4.13.16-coreos-r2   docker://Unknown
ip-172-17-14-106.us-east-2.compute.internal   Ready     master    14m       v1.8.4+coreos.0   <none>        Container Linux by CoreOS 1576.4.0 (Ladybug)   4.13.16-coreos-r2   docker://Unknown
ip-172-17-14-46.us-east-2.compute.internal    Ready     <none>    7m        v1.8.4+coreos.0   <none>        Container Linux by CoreOS 1576.4.0 (Ladybug)   4.13.16-coreos-r2   docker://Unknown
ip-172-17-14-6.us-east-2.compute.internal     Ready     <none>    7m        v1.8.4+coreos.0   <none>        Container Linux by CoreOS 1576.4.0 (Ladybug)   4.13.16-coreos-r2   docker://Unknown

[ivanilves]

I join to you in this petition! We want 1.9 ❤️

I run v1.9 cluster succesfully and Heptio Sonobuoy "Conformance" E2E tests pass.

My only concern here is that IPVS is broken in 1.9.0 hyperkube image (has no ipset binary), explained here coreos/coreos-kubernetes#915 and image with workaround is available.

[whereisaaron]

The optional IPVS is an efficiency improvement, but not a reason to not to move to 1.9. The NLBs are a big deal because you can assign stable/fixed IP addresses to them. Let's me do a lot of things I couldn't otherwise.

Starting in version 1.9.0, Kubernetes supports Network Load Balancer (NLB). To use a Network Load Balancer on AWS, use the annotation service.beta.kubernetes.io/aws-load-balancer-type with the value set to nlb.

    metadata:
      name: my-service
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: "nlb"

[9len]

1.9 also moves to CNI 0.6.0 which has a portMap plug-in that works properly with flannel, which (I believe) would help close #704

[whereisaaron]

@9len I think it was Calico+flannel that had the hostPort issue? And this was fixed in Kubernetes 1.7. Certainly hostPort works with kube-aws today (with Calico enabled). I updated the kube-aws documentation recently. Is that the same issue you are talking about, or something different?

[9len]

sorry for the X-post, answered in #704

[zmt]

Is there an ETA on the next release that will support kubernetes 1.9 for the NLB support? Will it be 0.9.10 or are you striving for 1.0.0 as the next?

For the moment, I manually edited the generated userdata/cloud-config-controller after `kube-aws render stack to do this for the KubeProxyConfiguration. It did the trick and I am able to spin up an NLB service, which is great.

[whereisaaron]

@camilb @mumoshu is would be great to get an 0.9.10 release or release candidate for 1.9.x support, even if no other significant features are involved.

[mumoshu]

@whereisaaron I can, but honestly i wonder whats the point of doing so? Just wanted to sync with you that we have been making big changes in rc.1 of every release :)

This time the biggest thing would be #1112

[mumoshu]

And you can freely build a kube-aws binary from the current master, which has an official'ish support for 1.9

[cknowles]

v0.9.10 is out with k8s 1.9