Rancher Desktop fails to start with networking tunnel enabled

[jackmtpt]

Actual Behavior

It hangs on the Updating kubeconfig step.

vm-switch.log

time="2024-01-18T10:49:39Z" level=debug msg="using a AF_VSOCK connection file from default namespace: &{0xc000084300}"
time="2024-01-18T10:49:40Z" level=debug msg="created tap device eth0: &{true 0xc0001fa000 eth0}"
time="2024-01-18T10:49:40Z" level=debug msg="successful link setup &{LinkAttrs:{Index:3 MTU:1500 TxQLen:1000 Name:eth0 HardwareAddr:66:1e:80:21:93:9f Flags:broadcast|multicast RawFlags:4098 ParentIndex:0 MasterIndex:0 Namespace:<nil> Alias: Statistics:0xc0002120c0 Promisc:0 Allmulti:0 Multi:1 Xdp:0xc000206078 EncapType:ether Protinfo:<nil> OperState:down PhysSwitchID:0 NetNsID:-1 NumTxQueues:1 NumRxQueues:1 GSOMaxSize:65536 GSOMaxSegs:65535 Vfs:[] Group:0 Slave:<nil>} Mode:tap Flags:0 NonPersist:true Queues:0 Fds:[] Owner:0 Group:0}\n"
time="2024-01-18T10:49:40Z" level=debug msg="setup complete for tap interface eth0(5a:94:ef:e4:0c:ee) + loopback"
time="2024-01-18T10:49:40Z" level=info msg="waiting for packets..."
time="2024-01-18T10:49:40Z" level=debug msg="closed tap device: eth0"
time="2024-01-18T10:49:40Z" level=error msg="reading size from socket failed: EOF"

host-switch.log

time="2024-01-18T10:49:39Z" level=info msg="successfully estabilished a handshake with a peer: d692a146-e272-4703-96a5-b2ec6dfe03a3"
time="2024-01-18T10:49:39Z" level=info msg="successful handshake, waiting for a vsock connection from VMGUID: d692a146-e272-4703-96a5-b2ec6dfe03a3 on Port: 6656"
time="2024-01-18T10:49:39Z" level=info msg="attempt to handshake with [d692a146-e272-4703-96a5-b2ec6dfe03a3], goroutine is terminated"
time="2024-01-18T10:49:39Z" level=error msg="cannot add network services: listen tcp 0.0.0.0:80: bind: An attempt was made to access a socket in a way forbidden by its access permissions."
time="2024-01-18T10:54:41Z" level=fatal msg="handshake with peer process failed: trying to find WSL GUID faild: could not find vsock-peer process on any hyper-v VM(s)"
time="2024-01-18T10:59:42Z" level=fatal msg="handshake with peer process failed: trying to find WSL GUID faild: could not find vsock-peer process on any hyper-v VM(s)"
time="2024-01-18T11:04:43Z" level=fatal msg="handshake with peer process failed: trying to find WSL GUID faild: could not find vsock-peer process on any hyper-v VM(s)"
time="2024-01-18T11:09:44Z" level=fatal msg="handshake with peer process failed: trying to find WSL GUID faild: could not find vsock-peer process on any hyper-v VM(s)"
time="2024-01-18T11:14:45Z" level=info msg="attempt to handshake with [d692a146-e272-4703-96a5-b2ec6dfe03a3], goroutine is terminated"
time="2024-01-18T11:14:45Z" level=fatal msg="handshake with peer process failed: trying to find WSL GUID faild: could not find vsock-peer process on any hyper-v VM(s)"
time="2024-01-18T11:19:47Z" level=fatal msg="handshake with peer process failed: trying to find WSL GUID faild: could not find vsock-peer process on any hyper-v VM(s)"
time="2024-01-18T11:24:48Z" level=fatal msg="handshake with peer process failed: trying to find WSL GUID faild: could not find vsock-peer process on any hyper-v VM(s)"
time="2024-01-18T11:29:49Z" level=fatal msg="handshake with peer process failed: trying to find WSL GUID faild: could not find vsock-peer process on any hyper-v VM(s)"

It seems that k3s isn't able to start because the tunnel never gets set up, and so eth0 doesn't get created in the rancher network namespace. That means there's no default route, which k3s doesn't like.

Steps to Reproduce

New install of 1.12.1. Enable k8s and networking tunnel.

Result

It hangs on the Updating kubeconfig step.

Expected Behavior

It should start up normally and work.

Additional Information

No response

Rancher Desktop Version

1.12.1

Rancher Desktop K8s Version

1.28.5

Which container engine are you using?

moby (docker cli)

What operating system are you using?

Windows

Operating System / Build Version

Windows 11 22H2

What CPU architecture are you using?

x64

Linux only: what package format did you use to install Rancher Desktop?

None

Windows User Only

GlobalProtect VPN client

[jackmtpt]

logs.zip

[MHacker9404]

same problem as #6322 - except I'm not running a VPN

[jackmtpt]

I had this happen on 1.11 as well; it's not specific to 1.12.1

[jackmtpt]

I think I've worked out what the issue is here actually. I had IIS running on my machine which binds to 0.0.0.0:80 and :443. host-switch.exe is launched with the following command line:

"C:\Program Files\Rancher Desktop\resources\resources\win32\internal\host-switch.exe" --port-forward 127.0.0.1:6443=192.168.127.2:6443 --port-forward 0.0.0.0:80=192.168.127.2:80 --port-forward 0.0.0.0:443=192.168.127.2:443

and it seems to fail pretty badly if it's not able to bind 80 or 443.

I stopped IIS and now everything works.

Perhaps the UI application should contain a check to make sure that 80 and 443 (and 6443 presumably) are not in use already and refuse to enable the networking tunnel if they are.

[jandubois]

@Nino-K What I think should happen:

• host-switch should not fail, but simply not forward ports 80 and 443
• a diagnostic should be displayed that explains why the ingress ports are not available on the host

I would also like to know if this is an issue with legacy networking as well; I know we don't have a diagnostic, but I assumed the forwarding would just silently fail, but the app would still start and the rest of the functionality would still be available. Or is that not true?

[Nino-K]

A quick workaround for now is disabling traefik if it's unnecessary. Alternatively, IIS could be configured to listen on different ports. We recognize that this is not a great user experience to disrupt the startup over the occupied ports, as a result, we will be addressing this issue in our later backlog. The new approach will notify the users through diagnostics UI.

[MHacker9404]

@Nino-K that didn't work for me - spun on updating kubeconfig for over 10 minutes before i shut it down

[Nino-K]

@MHacker9404 I looked at the logs that you attached to #6322 and you may be encountring a few issues there, one of which is occupied ports. So I would recommend the following:

1. Make sure no port 80 or 443 is occupied by anyting on the host machine, or altreanatively disable traefik if not needed.
2. If you can, factory reset and restart RD and then enable integration.

Hopefully that works, otherwise we are working on a fix for the integration in this release.

[jandubois]

What I think should happen:

• host-switch should not fail, but simply not forward ports 80 and 443

I've created #6341 to track this.

• a diagnostic should be displayed that explains why the ingress ports are not available on the host

I've created #6342 to track this.

Since @jackmtpt confirmed that the port conflict is the root cause of this failure, I'm going to close this issue. Feel free to subscribe to either (or both) of the new issues if you want to track what is going to happen.