traefik ingress using localhost on linux

[dmlond]

Actual Behavior

When running rancher-desktop on my macbook (intel) I can create a deployment, service, and Ingress, and access the service using 127.0.0.1. I cannot do this on my Ubuntu 20_04 linux with rancher-desktop (both use dockerd)

Steps to Reproduce

create the following resources in rancher-desktop using kubectl

apiVersion: v1
kind: Namespace
metadata:
  name: web
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: web
spec:
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
        env:
          - name: WHOAMI_PORT_NUMBER
            value: "8080"
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: whoami-service
  namespace: web
  labels:
    app: whoami
spec:
  selector:
    app: whoami
  ports:
    - port: 8080
      targetPort: 8080
      protocol: TCP
  type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: whoami
  namespace: web
  labels:
    app: whoami
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  rules:
  - host: whoami.oasis.local.development
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: whoami-service
            port:
              number: 8080

create an entry in /etc/hosts mapping whoami.oasis.local.development to 127.0.0.1

curl http://whoami.oasis.local.development

Result

on the mac, this works, but on linux I get connection refused on port 80

Expected Behavior

I should see the response from the service
Hostname: whoami-658756b586-zjf65
IP: 127.0.0.1
IP: 10.42.0.17
RemoteAddr: 10.42.0.13:48154
GET / HTTP/1.1
Host: whoami.oasis.local.development
User-Agent: curl/7.64.1
Accept: /
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.9
X-Forwarded-Host: whoami.oasis.local.development
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-6bb96f9bd8-rfgtf
X-Real-Ip: 10.42.0.9

Additional Information

No response

Rancher Desktop Version

1.0.1

Rancher Desktop K8s Version

v1.22.6(stable)

Which container runtime are you using?

moby (docker cli)

What operating system are you using?

Ubuntu

Operating System / Build Version

20.04

What CPU architecture are you using?

x64

Linux only: what package format did you use to install Rancher Desktop?

deb

Windows User Only

No response

[evertonlperes]

Thanks for filing the issue @dmlond
I cannot replicate it on macOS (it seems to be working properly).

@adamkpickering
Could you try to replicate it on your env? I tried to replicate it on my virtualbox, but the env is really slow (~20min to startup RD)
Thanks

[dmlond]

I am running Ubuntu 20.04, not macOS.

[evertonlperes]

I am running Ubuntu 20.04, not macOS.

I know, you already described it. I just did a double-check on different macOS version in order to track/find regression issues.

[nunix]

@dmlond @adamkpickering this seems to be related/similar to the rootless mode which doesn't allow to open ports under 1024 (see: https://rootlesscontaine.rs/getting-started/common/sysctl/#optional-allowing-listening-on-tcp--udp-ports-below-1024).

I did the following tests after I deployed the resources @dmlond mentioned:

• Prerequisites

$ kubectl get all -n web

NAME                          READY   STATUS    RESTARTS      AGE
pod/whoami-658756b586-6zhgr   1/1     Running   1 (91s ago)   7m46s

NAME                     TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/whoami-service   ClusterIP   10.43.94.177   <none>        8080/TCP   7m46s

NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/whoami   1/1     1            1           7m46s

NAME                                DESIRED   CURRENT   READY   AGE
replicaset.apps/whoami-658756b586   1         1         1       7m46s


$ kubectl get ingress -A

NAMESPACE   NAME     CLASS    HOSTS                            ADDRESS        PORTS   AGE
web         whoami   <none>   whoami.oasis.local.development   192.168.5.15   80      21m

• Test 1: configuration by default

$ curl http://whoami.oasis.local.development/
curl: (7) Failed to connect to whoami.oasis.local.development port 80 after 0 ms: Connection refused

• Test 2: allowed lower ports to be open and restarted Rancher Desktop

$ sudo sysctl net.ipv4.ip_unprivileged_port_start
net.ipv4.ip_unprivileged_port_start = 1024

$ sudo sysctl net.ipv4.ip_unprivileged_port_start=80
net.ipv4.ip_unprivileged_port_start = 80

$ curl whoami.oasis.local.development
Hostname: whoami-658756b586-6zhgr
IP: 127.0.0.1
IP: 10.42.0.44
RemoteAddr: 10.42.0.47:40448
GET / HTTP/1.1
Host: whoami.oasis.local.development
User-Agent: curl/7.81.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.42
X-Forwarded-Host: whoami.oasis.local.development
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-58b759688b-t5rdx
X-Real-Ip: 10.42.0.42

Can you try it on your end and hopefully this should work.
Last info, I'm using OpenSuse MicroOS, but this is related to generic Linux security, so should work the same on "all" distros (don't quote me on that though 😉 )

[do-m-en]

@nunix I had the same issue (Ubuntu 21.10; only difference from ticket description being that I'm using browser's automatic .localhost url redirect to localhost instead of changing /etc/hosts for that) and yes, your solution solved it.
Big thanks.

Perhaps such NOTE/WARNING messages should be present in General (or at least Port Forwarding or Trhoubleshooting) section of Rancher Desktop app on Linux distros out of the box (or at least titles and links to such known issues in docs).

[dmlond]

I switched to running k3s natively on my machine and I do not have this issue with native k3s. @do-m-en verifies that @nunix solution works, so I agree that some documentation in the FAQ or somewhere could be a solution to this issue.

[jetersen]

Would it not be better to use sudo setcap cap_net_bind_service=+ep /path/to/rancher-desktop
I am currently not running on Linux so I don't know the exact path to rancher-desktop or potentially that would be needed for the k3s binary?
I am guessing here.