Remove hostname requirement in /etc/hosts

[erikwilson]

Fixes #60
Closes #91
Resolves #193
Resolves #299

[ibuildthecloud]

I just want to confirm that kubectl exec/logs/attach/port-forward still works with this change.

[ibuildthecloud]

The reason that node name is set there has to do with how the tunnel server discovers the nodes.

[erikwilson]

Testing shows that exec/logs/attach/port-forward work with this change.

In the tunnel we set the node name here https://github.com/rancher/k3s/blob/master/pkg/agent/tunnel/tunnel.go#L170,
and permit connections to port 10010 here https://github.com/rancher/k3s/blob/master/pkg/agent/tunnel/tunnel.go#L199.

Using the example from https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/, if there is not an entry in hosts for 127.0.0.1 localhost then socat will fail with port-forward:

$ redis-cli -p 7000
Handling connection for 7000
E0822 10:37:10.088983    4997 portforward.go:400] an error occurred forwarding 7000 -> 6379: error forwarding port 6379 to pod 9281e4ac27072ce723da7ad5dd8fa8bcd409200a8eae04bf4cdeec6708f5867c, uid : failed to execute portforward in network namespace "/var/run/netns/cni-afa2dc33-9431-37bd-93fa-fe5c8ad0b89c": socat command returns error: exit status 1, stderr: "2019/08/22 10:37:09 socat[5072] E getaddrinfo(\"localhost\", \"NULL\", {1,2,1,6}, {}): Name or service not known\n"

That failure is on master and with this change, and is probably an upstream issue in the portforward code that should be fixed. This is an issue for the default configuration of Alpine 3.10 in the Vagrantfile I am using.

Also added a commit to prefer "127.0.0.1" over "localhost" for our connection strings, which will help if there is a missing 127.0.0.1 localhost entry in /etc/hosts for Ubuntu. Error shows as level=fatal msg="starting tls server: Get https://localhost:6444/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions: dial ..., doesn't appear to occur in Alpine tho. (related #556)

[ibuildthecloud]

Hmmm.... I'd like to review this one a bit further because something must have change that I don't understand regarding exec. AFAIK exec should be broken.

[ibuildthecloud]

@erikwilson Can you confirm exec works with a multi node cluster where the pod you are execing to is not on the same server as the master.

[erikwilson]

Running with 2 nodes from the Vagrantfile in k3s, on master (hostname k3s-1) I ran k3s with the following command:

K3S_NODE_NAME=k3s-master \
K3S_CLUSTER_SECRET=test \
INSTALL_K3S_SKIP_DOWNLOAD=true \
./install.sh --flannel-iface eth1

and then an agent (hostname k3s-2) with the following command:

K3S_URL=https://10.135.135.101:6443/ \
K3S_NODE_NAME=k3s-worker \
K3S_CLUSTER_SECRET=test \
INSTALL_K3S_SKIP_DOWNLOAD=true \
./install.sh --flannel-iface eth1

Nodes ready:

$ kubectl get nodes
NAME         STATUS   ROLES    AGE     VERSION
k3s-master   Ready    master   4h51m   v1.14.6-k3s.1
k3s-worker   Ready    worker   4h51m   v1.14.6-k3s.1

Launching an example pod:

[ 2019-08-22 16:25:24 ]
root🐮k3s-1:~$ kubectl apply -f https://k8s.io/examples/pods/inject/dapi-envars-pod.yaml
pod/dapi-envars-fieldref created

[ 2019-08-22 16:25:29 ]
root🐮k3s-1:~$ kubectl describe pod/dapi-envars-fieldref
Name:               dapi-envars-fieldref
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               k3s-worker/10.135.135.102
Start Time:         Thu, 22 Aug 2019 16:25:29 -0700
...

Exec and showing info, resolving node ip from inside a pod:

[ 2019-08-22 16:26:49 ]
root🐮k3s-1:~$ kubectl exec -it dapi-envars-fieldref -- sh

/ # printenv
MY_POD_SERVICE_ACCOUNT=default
KUBERNETES_PORT=tcp://10.43.0.1:443
KUBERNETES_SERVICE_PORT=443
REDIS_MASTER_SERVICE_HOST=10.43.208.177
HOSTNAME=dapi-envars-fieldref
SHLVL=1
HOME=/root
REDIS_MASTER_PORT=tcp://10.43.208.177:6379
REDIS_MASTER_SERVICE_PORT=6379
MY_POD_NAMESPACE=default
REDIS_MASTER_PORT_6379_TCP_ADDR=10.43.208.177
REDIS_MASTER_PORT_6379_TCP_PORT=6379
REDIS_MASTER_PORT_6379_TCP_PROTO=tcp
TERM=xterm
MY_POD_IP=10.42.1.5
KUBERNETES_PORT_443_TCP_ADDR=10.43.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
REDIS_MASTER_PORT_6379_TCP=tcp://10.43.208.177:6379
MY_NODE_NAME=k3s-worker
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP=tcp://10.43.0.1:443
PWD=/
KUBERNETES_SERVICE_HOST=10.43.0.1
MY_POD_NAME=dapi-envars-fieldref

/ # nslookup k3s-worker
Server:    10.43.0.10
Address 1: 10.43.0.10 kube-dns.kube-system.svc.cluster.local

Name:      k3s-worker
Address 1: 10.135.135.102 k3s-worker

content of hosts:

$ cat /etc/hosts
127.0.0.1 localhost
::1		localhost localhost.localdomain

and unresolvable hostnames from host vm:

root🐮k3s-1:~$ nslookup k3s-1
nslookup: can't resolve '(null)': Name does not resolve

nslookup: can't resolve 'k3s-1': Name does not resolve

root🐮k3s-2:~$ nslookup k3s-2
nslookup: can't resolve '(null)': Name does not resolve

nslookup: can't resolve 'k3s-2': Name does not resolve

root🐮k3s-1:~$ nslookup k3s-master
nslookup: can't resolve '(null)': Name does not resolve

nslookup: can't resolve 'k3s-master': Name does not resolve

root🐮k3s-1:~$ nslookup k3s-worker
nslookup: can't resolve '(null)': Name does not resolve

nslookup: can't resolve 'k3s-worker': Name does not resolve

Network info:

root🐮k3s-1:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 08:00:27:d7:0c:2c brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fed7:c2c/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 08:00:27:38:1e:f5 brd ff:ff:ff:ff:ff:ff
    inet 10.135.135.101/24 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe38:1ef5/64 scope link
       valid_lft forever preferred_lft forever
4: flannel.1: ...

[ibuildthecloud]

Awesome!