Skip to content

wslc: Switch port relay AcceptThread from WaitForMultipleObjects to IO completion ports#40042

Open
benhillis wants to merge 37 commits intofeature/wsl-for-appsfrom
user/benhill/fix-port-relay-handle-limit
Open

wslc: Switch port relay AcceptThread from WaitForMultipleObjects to IO completion ports#40042
benhillis wants to merge 37 commits intofeature/wsl-for-appsfrom
user/benhill/fix-port-relay-handle-limit

Conversation

@benhillis
Copy link
Copy Markdown
Member

@benhillis benhillis commented Mar 30, 2026

Replace the WaitForMultipleObjects-based wait loop in relay::MultiHandleWait with an IO completion port (IOCP), removing the MAXIMUM_WAIT_OBJECTS (64) handle limit for all callers (IORelay, container IO, socket accept, etc.).

Switch the port relay AcceptThread from WaitForMultipleObjects to a direct IOCP, removing the 63-port mapping limit.

relay::MultiHandleWait changes (relay.hpp/cpp)

  • Add unique_registered_wait RAII type for RegisterWaitForSingleObject handles
  • Add Register() pure virtual to OverlappedIOHandle, implemented by all subclasses
  • Handles that support IOCP (pipes, sockets) register directly; handles that don't (console handles, events) use a RegisterWaitForSingleObject bridge
  • Convert Run() to use GetQueuedCompletionStatus instead of WaitForMultipleObjects
  • Cancel() posts a key=0 completion to wake Run(); key=0 sets m_cancel on dequeue
  • Fix member order: m_iocp declared before m_handles for correct destruction

Port relay changes (localhost.cpp)

  • Associate listen sockets directly with an IOCP via CreateIoCompletionPort
  • AcceptEx completions go straight to the IOCP — no events or thread pool waits
  • Stop signal via PostQueuedCompletionStatus key=0
  • Remove AcceptEvent from PortRelay (no longer needed)
  • Remove the MAXIMUM_WAIT_OBJECTS port limit check

benhillis and others added 29 commits March 10, 2026 12:34
* test: enable virtiofs tests and enable WSLG during testing

* test fix

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Addresses Dependabot alerts #10 and #11. The Microsoft.NETCore.App.Runtime
packages (win-x64 and win-arm64) at version 10.0.0 are vulnerable to a
denial of service via out-of-bounds read when decoding malformed Base64Url
input (CVSS 7.5 High). Bumped to 10.0.4 which includes the fix.

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
Co-authored-by: WSL notice <noreply@microsoft.com>
…ipt (#14424)

* Ship initrd.img in MSI using build-time generation via tar.exe

Replace the install-time CreateInitrd/RemoveInitrd custom actions with a
build-time step that generates initrd.img using the Windows built-in
tar.exe (libarchive/bsdtar) and ships it directly in the MSI.

The install-time approach had a race condition: wsl.exe could launch
before the CreateInitrd custom action completed, causing
ERROR_FILE_NOT_FOUND for initrd.img.

Changes:
- Add CMake custom command to generate initrd.img via tar.exe --format=newc
- Add initrd.img as a regular file in the MSI tools component
- Remove CreateInitrd/RemoveInitrd custom actions from WiX, DllMain,
  and wslinstall.def
- Remove CreateCpioInitrd helper and its tests (no longer needed)
- Update pipeline build targets to build initramfs instead of init

* pr feedback

* more pr feedback

* switch to using a powershell script instead of tar.exe

* powershell script feedback

* hopefully final pr feedback

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
…uire a call to the service (#14380)

* virtiofs: update logic so querying virtiofs mount source does not require a call to the service

* more pr feedback

* use std::filesystem::read_symlink

* pr feedback and use canonical path in virtiofs symlink

* make sure canonical path is always used

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
* VirtioProxy: Add IPv6 address, gateway, and route support

- Add PreferredIpv6Address field and GetBestGatewayV6* methods to NetworkSettings
- Extend GetHostEndpointSettings() to discover IPv6 unicast address and gateway
- Add UpdateIpv6Address() using ModifyGuestEndpointSettingRequest<IPAddress>
- Push IPv6 default route to guest via UpdateDefaultRoute(AF_INET6)
- Remove AF_INET6 early return in ModifyOpenPorts, use INETADDR_PORT()
- Add EndpointRoute::DefaultRoute() static factory
- Pass client_ip_ipv6 in devicehost options (not yet parsed by devicehost)
- Remove gateway_ip from devicehost options (only needed for DHCP)
- Include IPv6 DNS servers in non-tunneling DNS settings
- Add ConfigurationV6 and DnsResolutionAAAA tests

* cleanup and add more ipv6 tests

* added test coverage and minor updates

* clang format

* pr feedback

* format source

* pr feedback

* test fixes

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
* Initial work

* .

* pr feedback and add unit test

* minor tweaks an fix use after free in logging statement

* implement PR feedback

* hopefully final pr feedback

* pr feedback in test function

* Address PR feedback: add try/catch to TrackPort and PortZeroBind queue push

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
There were instructions already on how to install tcpdump in WSL, but
iptables are also needed for the log collection to be complete, so this
PR adds instructions on how to also install iptables.

Co-authored-by: Andre Muezerie <andremue@linux.microsoft.com>
Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
* Move all supported Ubuntu images to the new format

We backported the build pipeline so all current LTSes come out in the new tar-based format

* Remove the appx based distros

All WSL users can run tar-based distros by now, right?
There is no benefit in maintaining both formats.
- Allow VirtioProxy to keep EnableDnsTunneling=true in config, but clear
  socket-specific options (BestEffortDnsParsing, DnsTunnelingIpAddress)
- Suppress dedicated DNS tunneling hvsocket for VirtioProxy; tunneling
  is handled through the VirtioNetworking device host instead
- Set DnsTunneling flag on VirtioNetworkingFlags so the device host
  knows to tunnel DNS
- Expand SWIOTLB kernel cmdline to cover VirtioFs and VirtioProxy
- Bump DeviceHost package to 1.1.39-0
- Add VirtioProxy DNS test coverage for tunneling on/off
- Skip GuestPortIsReleasedV6 on Windows 10

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
* Refactor: trim unnecessary DLL deps from COMMON_LINK_LIBRARIES

- Split MSI/Wintrust install functions from wslutil.cpp into install.cpp
- Remove MI.lib, wsldeps.lib, msi.lib, Wintrust.lib, computecore.lib,
  computenetwork.lib, Iphlpapi.lib from COMMON_LINK_LIBRARIES
- Add per-target MSI_LINK_LIBRARIES, HCS_LINK_LIBRARIES, SERVICE_LINK_LIBRARIES
- Delay-load msi.dll and WINTRUST.dll for wsl.exe and wslg.exe
- Result: wslhost, wslrelay, wslcsdk, testplugin lose msi/wintrust startup imports;
  wsl.exe and wslg.exe defer msi/wintrust loading until actually needed;
  wslservice is the only target that imports computecore/computenetwork/Iphlpapi

* minor fixes to install.cpp that were caught during PR

* move to wsl::windows::common::install namespace

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
* detach terminal before running mount -a

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* use _exit on error before execv in child process to avoid unintentional resource release

* Add regression test

* Fix clang format issue

* fix all clang format issue

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* resolve ai comments

* move test to unit test

* Fix string literal

* Overwrite fstab to resolve pipeline missing file issue

---------

Co-authored-by: Feng Wang <wangfen@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* test: Add arm64 test distro support

* update unit test baseline

* more test baseline updates

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
* test: remove duplicated DNS test coverage

* format source

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
)

Fail and warn the user when --uninstall is given parameters.
Co-authored-by: WSL localization <noreply@microsoft.com>
…ith DNS over TCP (#14532)

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
…ts (#14531)

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
Co-authored-by: WSL localization <noreply@microsoft.com>
…14387)" (#14538)

* Revert "test: enable virtiofs tests and enable WSLG during testing (#14387)"

* enable wslg for SystemdNoClearTmpUnit test

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
Co-authored-by: WSL localization <noreply@microsoft.com>
* Update cgmanifest to match CMakeLists.txt

* Update CMakeLists.txt

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: WSL notice <noreply@microsoft.com>
* Update Microsoft.WSL.DeviceHost to version 1.1.48-0 (#14575)

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>

* Re-enable WSLG during testing.

This reverts commit bf759a0.

* add back config change (will work with new default, but makes test explicit)

---------

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
@benhillis benhillis requested a review from a team as a code owner March 30, 2026 21:19
Copilot AI review requested due to automatic review settings April 3, 2026 15:27
@benhillis benhillis force-pushed the user/benhill/fix-port-relay-handle-limit branch from 3de1c99 to 3fe1174 Compare April 3, 2026 15:27
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

@benhillis benhillis force-pushed the user/benhill/fix-port-relay-handle-limit branch from 3fe1174 to 8091b71 Compare April 3, 2026 16:35
Copilot AI review requested due to automatic review settings April 5, 2026 18:33
@benhillis benhillis force-pushed the user/benhill/fix-port-relay-handle-limit branch from 8091b71 to a7ab4d5 Compare April 5, 2026 18:33
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Copilot AI review requested due to automatic review settings April 6, 2026 01:52
@benhillis benhillis force-pushed the user/benhill/fix-port-relay-handle-limit branch from b06a0f2 to fdb0b87 Compare April 6, 2026 01:52
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

…ebugging

When /attachdebugger is passed to test.bat, run-tests.ps1 now:
- Starts te.exe with /waitfordebugger in the background
- Polls for the TE.ProcessHost.exe child process via WMI
- Launches WinDbgX attached directly to the test host PID
- With /inproc, attaches to TE.exe itself instead

This replaces the manual workflow of running /waitfordebugger, reading
the PID from the output, and launching WinDbgX separately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@benhillis benhillis force-pushed the user/benhill/fix-port-relay-handle-limit branch from fdb0b87 to a9ecbec Compare April 6, 2026 04:06
Copilot AI review requested due to automatic review settings April 6, 2026 05:12
@benhillis benhillis force-pushed the user/benhill/fix-port-relay-handle-limit branch from a9ecbec to 677ed96 Compare April 6, 2026 05:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

@benhillis benhillis force-pushed the user/benhill/fix-port-relay-handle-limit branch from 677ed96 to c712ebf Compare April 6, 2026 15:27
Copilot AI review requested due to automatic review settings April 6, 2026 17:13
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Ben Hillis and others added 2 commits April 6, 2026 12:10
Per review feedback from @OneBlue:
- Add /inproc when /attachdebugger is set so WinDbgX attaches
  directly to TE.exe instead of polling for TE.ProcessHost.exe
- Simplify exit to pass through TE.exe exit code directly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The previous fix unconditionally called GetOverlappedResult(bWait=TRUE),
which hangs in IOCP mode because Overlapped.hEvent is nullptr and
cancellation completions go to the IOCP queue, not the file handle.

Fix: when RegisteredWithIocp, set a temporary event in the OVERLAPPED
before calling CancelIoEx so GetOverlappedResult can wait on it. This
ensures the kernel is done with the OVERLAPPED before the destructor
frees it, without hanging.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 6, 2026 21:04
@benhillis benhillis force-pushed the user/benhill/fix-port-relay-handle-limit branch from ac96349 to 357a29f Compare April 6, 2026 21:04
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 90 out of 90 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

NOTICE.txt:645

  • NOTICE.txt no longer contains any entries for Microsoft.NETCore.App.Runtime.win-* (and related license text), but packages.config still references these runtimes (now updated to 10.0.4). If NOTICE is the authoritative third-party notices file for shipped dependencies, it likely needs to be regenerated/updated to reflect the current set and versions of redistributed components rather than removing these sections entirely.
MIT License

Copyright (c) <year> <copyright holders>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

---------------------------------------------------------

---------------------------------------------------------

Microsoft.Xaml.Behaviors.WinUI.Managed 3.0.0 - MIT


Comment on lines 964 to 968
void MultiHandleWait::AddHandle(std::unique_ptr<OverlappedIOHandle>&& handle, Flags flags)
{
EnsureIocp();
handle->Register(m_iocp.get(), handle.get());
m_handles.emplace_back(flags, std::move(handle));
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MultiHandleWait::AddHandle calls handle->Register(...) and EnsureIocp() uses m_iocp, but OverlappedIOHandle / MultiHandleWait declarations in relay.hpp currently do not declare a Register() method or any m_iocp member. As-is this TU should fail to compile. Please update relay.hpp to match the new IOCP-based design (add the virtual Register(...) API, required members, and update derived handle class declarations accordingly).

Copilot uses AI. Check for mistakes.
Comment on lines 971 to +977
void MultiHandleWait::Cancel()
{
m_cancel = true;
// Wake up GetQueuedCompletionStatus. The key=0 completion will set m_cancel in Run().
if (m_iocp)
{
PostQueuedCompletionStatus(m_iocp.get(), 0, 0, nullptr);
}
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MultiHandleWait::Cancel() no longer sets m_cancel (it only posts a key=0 completion if m_iocp exists). If Cancel() is invoked before Run() has dequeued the posted packet (or before m_iocp is created), the wait loop may continue scheduling work instead of stopping promptly. Consider setting m_cancel = true in Cancel() in addition to posting to the IOCP.

Copilot uses AI. Check for mistakes.
Comment on lines +55 to +62
set(INITRAMFS ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/${CMAKE_BUILD_TYPE}/initrd.img)
set(INIT ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/${CMAKE_BUILD_TYPE}/init)
add_custom_command(
OUTPUT ${INITRAMFS} "${CMAKE_CURRENT_BINARY_DIR}/CmakeFiles/initramfs"
DEPENDS init ${INIT}
COMMAND powershell.exe -ExecutionPolicy Bypass -NoProfile -NonInteractive -File "${CMAKE_SOURCE_DIR}/tools/create-initrd.ps1" "${INIT}" "${INITRAMFS}"
COMMAND ${CMAKE_COMMAND} -E touch "${CMAKE_CURRENT_BINARY_DIR}/CmakeFiles/initramfs"
VERBATIM)
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The initramfs custom command unconditionally invokes powershell.exe, which will break configuring/building this target on non-Windows hosts. Also, the marker path uses CmakeFiles (nonstandard casing vs CMakeFiles). Consider guarding this block with if (WIN32) (or using pwsh discovery) and fix the output path casing to avoid brittle build behavior.

Copilot uses AI. Check for mistakes.
Comment on lines 8 to +24
@@ -18,10 +18,10 @@
<package id="Microsoft.WSL.bsdtar" version="0.0.2-2" />
<package id="Microsoft.WSL.Dependencies.amd64fre" version="10.0.27820.1000-250318-1700.rs-base2-hyp" targetFramework="native" />
<package id="Microsoft.WSL.Dependencies.arm64fre" version="10.0.27820.1000-250318-1700.rs-base2-hyp" targetFramework="native" />
<package id="Microsoft.WSL.DeviceHost" version="1.1.14-0" />
<package id="Microsoft.WSL.DeviceHost" version="1.1.48-0" />
<package id="Microsoft.WSL.Kernel" version="6.6.114.1-1" targetFramework="native" />
<package id="Microsoft.WSL.LinuxSdk" version="1.20.0" targetFramework="native" />
<package id="Microsoft.WSL.TestDistro" version="2.5.7-47" />
<package id="Microsoft.WSL.TestDistro" version="2.7.1-1" />
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description/title focuses on IOCP changes for relay::MultiHandleWait and port relay accept handling, but this PR also includes substantial unrelated updates (e.g., initrd generation/packaging changes, VirtioProxy IPv6 changes/tests, distro metadata updates, .NET runtime/TestDistro/DeviceHost package bumps, NOTICE/cgmanifest updates). If these are intended, please reflect them in the PR description (or consider splitting) so reviewers and release notes capture the full scope.

Copilot uses AI. Check for mistakes.
@benhillis
Copy link
Copy Markdown
Member Author

Hey @benhillis 👋 — Following up on this PR. The \wsl-github-pr\ check shows action_required status. There are also 2 unresolved review threads remaining (out of 32 total):

  • \AddHandle\/\Register\ ordering concern
  • \Cancel()\ no longer setting the \m_cancel\ flag

This is a large PR (3170+/1359−, 90 files) — great progress resolving 30 of 32 threads! Just those last 2 items and the CI status to sort out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.