Port forwarding repeated failure on WSL 1.1.0

[rudyzeinoun]

Version

Microsoft Windows [Version 10.0.22623.1095]

WSL Version

• WSL 2
• WSL 1

Kernel Version

5.15.83.1

Distro Version

Ubuntu 20.04

Other Software

Apache/2.4.41
mysqld Ver 10.3.37-MariaDB-0ubuntu0.20.04.1
PHP 8.1 + php8.1fpm
systemd enabled in /etc/wsl.conf

Repro Steps

With WSL 1.1.0 (recently pushed to Store although marked as pre-release), port forwarding fails repeatedly.
Start Apache on the Ubuntu distro, and from Command Prompt on Windows, try to:
telnet localhost 80
It will work.
A few seconds later, repeat the telnet command and it will fail.
Port forwarding no longer works to connect to WSL Ubuntu's running services.

Expected Behavior

telnet command should keep working on the port.

Actual Behavior

telnet command will timeout.

On first try:
netstat -an | findstr /c:"80" | findstr /c:"LISTENING"
Shows port 80 as Listening.

After a few seconds, repeat the netstat command, the port is no longer listed.
This applies to any service running on WSL, and not just Apache.
Port forwarding fails after a few seconds of the service going up.

Restart apache "service apache2 restart". The port will appear on netstat. Wait 10 seconds and check again. It disappears.

Diagnostic Logs

No response

[rudyzeinoun]

Downgrading to 1.0.3 resolves the issue.

$Package = Get-AppxPackage MicrosoftCorporationII.WindowsSubsystemforLinux -AllUsers
Remove-AppxPackage $Package -AllUsers
Add-AppxPackage .\Microsoft.WSL_1.0.3.0_x64_ARM64.msixbundle

[DarkPhoenix2704]

I am having the same issue

Port Forwarding works for a few seconds, then doesn't work.

Distro Version: Ubuntu 20.04
Windows Build: 22621.192
WSL Version: 1.1.0.0

[hensou]

Same issue for me with a nextjs 13 project.
Port Forwarding doesn't work.

Distro Version: Archlinux
Windows Build: Windows 11 Dev Preview Build 25281
WSL Version: 1.1.0.0

Downgrading it to 1.0.3, as suggested by @rudyzeinoun , did make it work again.

[OneBlue]

/logs

[ghost]

Hello! Could you please provide more logs to help us better diagnose your issue?

To collect WSL logs, download and execute collect-wsl-logs.ps1 in an administrative powershell prompt:

Invoke-WebRequest -UseBasicParsing "https://raw.githubusercontent.com/microsoft/WSL/master/diagnostics/collect-wsl-logs.ps1" -OutFile collect-wsl-logs.ps1
Set-ExecutionPolicy Bypass -Scope Process -Force
.\collect-wsl-logs.ps1

The scipt will output the path of the log file once done.

Once completed please upload the output files to this Github issue.

Click here for more info on logging

Thank you!

[ghost]

Thanks for reporting this. I need logs from someone because I can't reproduce. I installed apache2, default config.

[rudyzeinoun]

Here's my log. I removed 1.0.3 and reinstalled 1.1.0 from the Store. The problem is reproducible again.
During the log capture, the port forwarding was already gone. I restarted apache2. The port came back up. Did 1 connection to it, and then it died, as expected.

WslLogs-2023-01-20_09-58-14.zip

[maicol07]

I confirm the same issue. 1 connection is successful and then I can't access it anymore. Here is my log if it can help:
WslLogs-2023-01-20_14-20-05.zip

[john-henry]

I am getting similar problem I believe. My experience is with starting up DDEV with Docker.

Error response from daemon: Ports are not available: exposing port TCP 127.0.0.1:443 -> 0.0.0.0:0: listen tcp 127.0.0.1:443: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.'`

[elsaco]

With WSL 1.1.0 it's connecting over IPv6 only. This is on Windows 10. The web server is running inside a WSL instance:

/usr/local/bin> ss -lt
State         Recv-Q        Send-Q               Local Address:Port               Peer Address:Port       Process
LISTEN        0             128                      127.0.0.1:ipp                     0.0.0.0:*
LISTEN        0             4096                             *:http                          *:*
LISTEN        0             128                          [::1]:ipp                        [::]:*

On Windows side 127.0.0.1 fails to connect:

PS C:\Windows\system32> .\curl.exe -4 -v localhost
*   Trying 127.0.0.1:80...
* connect to 127.0.0.1 port 80 failed: Connection refused
* Failed to connect to localhost port 80 after 2051 ms: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 80 after 2051 ms: Connection refused

but ::1 works, and the connection is steady:

PS C:\Windows\system32> .\curl.exe -6 -v localhost
*   Trying ::1:80...
* Connected to localhost (::1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.83.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< Date: Fri, 20 Jan 2023 17:59:08 GMT
< Server: Apache

More info:

PS C:\Windows\system32> Get-NetTCPConnection -localport 80 | ft -AutoSize

LocalAddress LocalPort RemoteAddress RemotePort State  AppliedSetting OwningProcess
------------ --------- ------------- ---------- -----  -------------- -------------
::1          80        ::            0          Listen                2972

PS C:\Windows\system32> Get-Process -id 2972

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    150      12     1604       7588       0.05   2972   1 wslhost

[ghost]

Ahhh, Thank-you! I was wondering. Is it bound to ::1 on the guest/wsl side?

[ghost]

From the logs I've looked at it appears the binds on the guest are port 80 ipv6

This means that the relay is going to bind the host(windows side) at ::1 port 80.

[ghost]

@rudyzeinoun, I checked your logs and saw the relay bound ::1 80 on the host side, forwarded two connections, and then closed the listening socket 6 seconds later. Did you already verify that the port is still bound on the guest side inside WSL? Using something similar to sudo netstat -lnpt

[ghost]

Non deterministic repro instructions here: #9516 (comment) . If someone is experiencing failure to forward after a period other than exactly 1 minute then please say. Because in that case there may be a second issue. Thanks again for your patience.

[ghost]

Was able to reproduce the bug where the port tracker stops tracking immediately.

[rudyzeinoun]

@rudyzeinoun, I checked your logs and saw the relay bound ::1 80 on the host side, forwarded two connections, and then closed the listening socket 6 seconds later. Did you already verify that the port is still bound on the guest side inside WSL? Using something similar to sudo netstat -lnpt

Yes, the port is still bound on the guest side. If I restart the service (unbind, bind again), it works for another 1 or 2 connections and disappears from host side again.
I see you're able to reproduce it now. 👍

[cheynewallace]

Im having the same issue after updating (automatically via app store) to 1.1.0. I didn't realize It had updated until I went digging around these issues and found this ticket.

Wasted half a day on this yesterday disabling firewalls and resetting networks and ended up giving up and switching to my Macbook so I could actually work before I discovered this.

The issue was as described here, I could usually boot my machine, open WSL, run docker and my dev API, it would work using localhost from the Windows host, in any browser for a about 30 seconds, then it would just stop working. Connection direct to the dynamic IP worked, localhost did not.

Performing wsl --shutdown did nothing and did not restore the localhost routing.

Downgrading to 1.0.3 as suggested by @rudyzeinoun did solve the issue.

[CodeW0lf]

WslLogs-2023-01-24_00-11-56.zip
My experience with this is that having one port forwarded is fine, but as soon as another port attempts to be forwarded, the first port will get disconnected and no further forwarding can occur until wsl --shutdown.

My dev environment starts two servers, one on port 4200 and one on 4201. If I only start one of them and no other services that require a forwarded port, the connection and forwarded port is stable. Seconds after attempting to spawn another service that would require a forwarded port, all port forwarding stops working.

Services are still accessible from within the WSL distribution no matter what is going on with the forwarding.

[dylanirion]

I believe I am experiencing this issue via aws-sam-cli where when running a lambda function locally, docker hits an exception mapping ports Ports are not available: exposing port TCP 127.0.0.1:5400 -> 0.0.0.0:0: listen tcp 127.0.0.1:5400: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.

[xcellsoft]

Similar issues with running docker swarm stack, unable to bind ports. One of our dev machines got an update over the weekend, took us better part of the day to realize it was WSL 1.1.0 update. We downgraded and all is fixed.

Docker stack error snippet:
skutok2kxqwb4sqhjy3cgnwza    \_ mysql_db.1               mysql:5.7.39@sha256:a85b8313feb7298ae240c4beb33a1b4d2e3a3867d3195bab9ed9346d332217c7                                           yoda      Shutdown        Failed less than a second ago     "starting container failed: container b5a3534fc47c51db1edd4a91b4b948b490768b98a25ab533f52121f0809e37b0: endpoint join on GW Network failed: driver failed programming external connectivity on endpoint gateway_e0c5563d9768 (13ddd6d5b49bc7228ed30f198fb44e549fc2e905f077bfbf84ed18b6ef8754f7): Error starting userland proxy: listen tcp4 0.0.0.0:3306: bind: address already in use"

[yejiyang]

Downgrading to 1.0.3 resolves the issue.

$Package = Get-AppxPackage MicrosoftCorporationII.WindowsSubsystemforLinux -AllUsers Remove-AppxPackage $Package -AllUsers Add-AppxPackage .\Microsoft.WSL_1.0.3.0_x64_ARM64.msixbundle

This works for me, great thanks!

[jayg-hive]

I tried to downgrade to 1.0.3 as well but I'm still not able to get the port forwarding to work. 😢

On Windows:

PS C:\Windows\System32> .\curl.exe -4 -v localhost:3000
*   Trying 127.0.0.1:3000...
* connect to 127.0.0.1 port 3000 failed: Connection refused
* Failed to connect to localhost port 3000 after 2050 ms: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 3000 after 2050 ms: Connection refused
PS C:\Windows\System32> .\curl.exe -6 -v localhost:3000
*   Trying ::1:3000...
* connect to ::1 port 3000 failed: Connection refused
* Failed to connect to localhost port 3000 after 2026 ms: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 3000 after 2026 ms: Connection refused
PS C:\Windows\System32> wsl --version
WSL version: 1.0.3.0
Kernel version: 5.15.79.1
WSLg version: 1.0.47
MSRDC version: 1.2.3575
Direct3D version: 1.606.4
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22623.1180
PS C:\Windows\System32>

Pretty much did most of the stuff (i.e., firewall disable on vEthernet, disable Fast Startup) to no effect. This is also a Next.js app as the one of the posters earlier.
WslLogs-2023-01-25_11-55-50.zip

[elsaco]

@jayg-hive open a debug console by adding debugConsole = true to your .wslconfig file. After a wsl --shutdown you'll see a console opened when launching a distro. If you see this kind of messages:

GnsPortTracker: Requested the host for port allocation on port (family 2, port 3000, protocol 6) - returned 0
GnsPortTracker: Tracking bind call: family (2) port (3000) protocol (6)
SecCompDispatcher: Responding to notification with id 6934181779582174820 for pid 928, result 0
GnsPortTracker: No longer tracking bind call: family (2) port (3000) protocol (6)

you're affected by the port proxy issue.

[JshGrn]

This is a massively broken release because of this. Wasted huge amount of time because of this release.

Edit: removed comment asking where auto update was, found it.

[pedrolamas]

@JshGrn disable it in the Microsoft Store app (click photo on top right, Settings, disable app updates)

Just do remember to come here ocasional to manually update the other apps...

[JshGrn]

@pedrolamas Yeah I saw in the end my bad, yeah I will come back in a month and hope its working again, is there any update on the timeline estimated for a fix on this?

[Jont828]

Thanks for linking this thread, I'll go ahead and see if downgrading works.

EDIT: Working on my end!

[OneBlue]

Thanks everyone for the feedback. The fix for this issue is in WSL 1.1.3

[ghost]

More specifically it is now back to the behavior that was part of 1.0.3

[svetoslavenchev]

Thanks everyone for the feedback. The fix for this issue is in WSL 1.1.3

Just tried it - my Docker Compose projects are happy :)

[nickchomey]

Nice work!

How long does it typically take for a new version to arrive in the Windows Store? I just removed 1.0.3 that I installed manually and installed WSL2 from the Windows store, but I have 1.1.2.

[OneBlue]

Nice work!

How long does it typically take for a new version to arrive in the Windows Store? I just removed 1.0.3 that I installed manually and installed WSL2 from the Windows store, but I have 1.1.2.

It's in pre-release for now so only insiders will receive it through the store. It'll be available for everyone once we decide that it's stable enough to go for GA.

In the meantime, non-insider users can download and install the package manually like you did.

[nickchomey]

Thanks. I'm a Windows 11 Beta Insiders user, but it is only giving me 1.1.2. Is there any way to get 1.1.3 automatically through the store so that I can just forget about this, rather than need to monitor releases for manual updates? Also, since 1.1.2 is only a partial fix for this issue, it seems like 1.1.3 should be the one available, with 1.0.3 available for non-insiders

[Cremesis]

I think that 1.1.3 reintroduced the problem in my machine, where the 1.1.2 fixed it :\

[ghost]

@Cremesis, can you open a new bug? 1.1.3 is supposed to be a revert to the behavior present in 1.0.3, feel free to mention me on the issue.

[ghost]

@nickchomey, We don't push builds to everyone at once. This bug is a great example of why we have an insiders ring that we push to before going to GA. The release process isn't our favorite thing either?

[nickchomey]

I understand that. But my point is that you've pushed a partial fix (1.1.2) to me rather than the full fix (1.1.3). It seems to me that I should only be receiving 1.0.3 or 1.1.3...

[ghost]

I get where you're coming from. This is why I wasn't going to mark the bug as closed till I was sure it had rolled out. I can ask someone who knows more about the release process via the store, but I'm pretty sure it's not instantaneous like that.

[nickchomey]

Ok. Anyway I've installed 1.1.3 manually and am not having problems thus far. I'll follow up in a couple weeks to see what version the store has.

[ghost]

@Cremesis, @rajshrimohanks: If you're still experiencing an issue on 1.1.3 I'd greatly appreciate it if you opened a new bug and posted logs.

[JshGrn]

Having to download manually as 1.1.2 still latest in Store on windows insider preview (19/02/2023), checking if fixed now.

EDIT: Fixed for me, great.

[Cremesis]

@Cremesis, @rajshrimohanks: If you're still experiencing an issue on 1.1.3 I'd greatly appreciate it if you opened a new bug and posted logs.

No issue, it was an error on my part: I had a .wslconfig file that disabled ipv6 because I had found it helped with my previous issue on v1.1.0, but in v1.1.3 this was blocking access my server running in WSL... removing that file and restarting WSL solved the problem.

[nickchomey]

Ok. Anyway I've installed 1.1.3 manually and am not having problems thus far. I'll follow up in a couple weeks to see what version the store has.

I just installed from the windows store and received 1.1.3.0. Again, I'm Windows Beta Insider. Hope this helps some people

[androiddisk]

In normal use, execute docker ps or docker restart on the probability line. All forwarding ports fail. Restarting wsl doesn't work. You can only restart windows

[ghost]

Locking this thread because the functionality/feature in question was reverted to the old behavior. Any issues should be filed as new bugs or comments on other bugs.