Port conflict of WSL2 with CRC for SSH (usermode networking)

[gbraad]

Issue Description

With the recent introduction of usermode networking, the network stack of podman machine on Windows uses port 2222 to forward ssh. This can conflict with the default of CRC which will ALWAYS use usermode networking and the 2222 port forward for ssh.

Steps to reproduce the issue

n/a

Describe the results you received

Unable to start either depending who comes first

Describe the results you expected

We should both be checking if the port is already in use and suggest an alternative.

podman info output

n/a

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

Windows with Podman machine on WSL2 and CRC

Additional information

Usermode networking

crc-org/crc#3855

[gbraad]

@n1hility
We should both be checking if the port is already in use and suggest an alternative. Though, we might wanna solve this in gvproxy... as multiple instances of a WLS2 machine might run into the same issue.

https://github.com/containers/gvisor-tap-vsock/blob/2a3419da952638147a42db9d49bd74bd10d2340c/pkg/types/gvproxy_command.go#L35

There is re-assignment code, but this might be used more generically: 63a8926

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@gbraad @n1hility any movement on this one?

[n1hility]

I think I should fix this / work around on the wsl side. We can pick an unused port. It’s not actually used in this particular use case.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[Luap99]

@n1hility Was this fixed with the port re-assign logic in machine?

[cfergeau]

The generic vmconfigs code in podman 5 seems to always allocate a random port and to never use 2222 :

podman/pkg/machine/vmconfigs/machine.go

Lines 85 to 95 in bb6b2e9

	sshPort, err := ports.AllocateMachinePort()
	if err != nil {
	return nil, err
	}
	sshConfig := SSHConfig{
	IdentityPath: sshIdentityPath,
	Port: sshPort,
	RemoteUsername: opts.Username,
	}

[cfergeau]

I just installed podman-desktop/podman-machine on a fresh Windows install and podman machine inspect shows a random SSH port being picked up, not 2222

[jeffmaury]

I just installed podman-desktop/podman-machine on a fresh Windows install and podman machine inspect shows a random SSH port being picked up, not 2222

Did you use user-mode-networking ?

[cfergeau]

I just installed podman-desktop/podman-machine on a fresh Windows install and podman machine inspect shows a random SSH port being picked up, not 2222

Did you use user-mode-networking ?

The checkbox in podman-desktop was checked, and gvproxy was listed in the task manager.
One thing I did not do is to check the gvproxy arguments because I don't know how to do this on Windows.

[jeffmaury]

I just installed podman-desktop/podman-machine on a fresh Windows install and podman machine inspect shows a random SSH port being picked up, not 2222

Did you use user-mode-networking ?

The checkbox in podman-desktop was checked, and gvproxy was listed in the task manager. One thing I did not do is to check the gvproxy arguments because I don't know how to do this on Windows.

I just started my machine with user mode networking and tcpview64 gives this for gvproxy:

And crc start gives this:

$ crc start
WARN A new version (2.41.0) has been published on https://developers.redhat.com/content-gateway/file/pub/openshift-v4/clients/crc/2.41.0/crc-windows-installer.zip
INFO Using bundle path C:\Users\Jeff\.crc\cache\crc_microshift_hyperv_4.16.4_amd64.crcbundle
INFO Checking minimum RAM requirements
INFO Check if Podman binary exists in: C:\Users\Jeff\.crc\bin\oc
INFO Checking if running in a shell with administrator rights
INFO Checking Windows release
INFO Checking Windows edition
INFO Checking if Hyper-V is installed and operational
INFO Checking if Hyper-V service is enabled
INFO Checking if crc-users group exists
INFO Checking if current user is in crc-users and Hyper-V admins group
INFO Checking if vsock is correctly configured
INFO Checking if the win32 background launcher is installed
INFO Checking if the daemon task is installed
INFO Checking if the daemon task is running
INFO Checking admin helper service is running
INFO Checking SSH port availability
WARN Preflight checks failed during `crc start`, please try to run `crc setup` first in case you haven't done so yet
port 2222 already in use: listen tcp 127.0.0.1:2222: bind: Une seule utilisation de chaque adresse de socket (protocole/adresse réseau/port) est habituellement autorisée.

[praveenkumar]

I just started my machine with user mode networking and tcpview64 gives this for gvproxy:

For podman machine doesn't it always user mode networking (because depend on gvproxy) ?

[jeffmaury]

I just started my machine with user mode networking and tcpview64 gives this for gvproxy:

For podman machine doesn't it always user mode networking (because depend on gvproxy) ?

On MacOS yes on Windows WSL the default is not to enable user mode networking

[cfergeau]

What happens is that on wsl2, podman-machine starts gvproxy with --listen-stdio=accept and no other arguments. This means the ssh-port argument will default to 2222 https://github.com/containers/gvisor-tap-vsock/blob/d1683b9b6fd0db12e33a44107d61c5d0d1b0726c/cmd/gvproxy/main.go#L65 and gvproxy unconditionally forwards this port to the VM https://github.com/containers/gvisor-tap-vsock/blob/d1683b9b6fd0db12e33a44107d61c5d0d1b0726c/cmd/gvproxy/main.go#L253-L255

This means on wsl2 gvproxy will always bind to port 2222 even though podman-machine does not need this. If wsl2 did something like this

podman/pkg/machine/shim/networking.go

Line 62 in 1ae4b04

cmd.SSHPort = mc.SSH.Port

this bug would not happen.
I could not understand where in the wsl2 code gvproxy is started, so no patch suggestion :(

[cfergeau]

Maybe cfergeau@1a950bc but I have no idea how to make a windows podman build in order to test it.

EDIT: I've managed to build and test it with make GOOS=windows

[cfergeau]

Now that containers/gvisor-tap-vsock#402 is merged in gvisor-tap-vsock, we need a gvisor-tap-vsock release, and then #24070 can be updated to make use of it.

[vyasgun]

/assign