Kernel Platform Worker Support

[nibanks]

Description

Extracts/refactors some of the code in #4023 to support platform worker threads, which is a prereq for kernel xdp support.

Testing

CI/CD

Documentation

N/A

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.02%. Comparing base (7b2fbaf) to head (a8a9122).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4605      +/-   ##
==========================================
- Coverage   87.43%   86.02%   -1.42%     
==========================================
  Files          56       56              
  Lines       17634    17634              
==========================================
- Hits        15419    15169     -250     
- Misses       2215     2465     +250     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[nibanks]

Looks like the kernel BVTs are crashing. I will try to investigate later.

[nibanks]

Something seems broken in netperf runs. I can't tell if it's because of this or not. Needs to be investigated before merging.

[ami-GS]

Is the netperf result reliable? wsk related failure is really few. others are more

[nibanks]

Converted this back to draft, because we need to switch to using NtCreateIoCompletion, NtSetIoCompletion, and NtRemoveIoCompletionEx.

[mtfriesen]

Converted this back to draft, because we need to switch to using NtCreateIoCompletion, NtSetIoCompletion, and NtRemoveIoCompletionEx.

I suggest using the Zw variants for KM, unless QUIC is calling on behalf of the user mode thread calling QUIC, and all in the same thread context.

[ami-GS]

Because platform_winkernel.c copies from zwapi.h, quic_platform_winkernel.h does same.

[ami-GS]

msquic/src/core/worker.c

Line 174 in 101f6b9

CxPlatEventWaitForever(Worker->Done);

RegistrationClose doesn't return by here

Edit:
QuicWorkerLoop is not called for some reason

[nibanks]

msquic/src/core/worker.c

Line 174 in 101f6b9

CxPlatEventWaitForever(Worker->Done);

RegistrationClose doesn't return by here
Edit: QuicWorkerLoop is not called for some reason

Likely means the worker thread isn't actually woken to clean up.

[ami-GS]

Randomly hangs by WaitForSingleObject under Registration destructor

[nibanks]

We will need to be sure kernel is fine publishing these, especially since we can't simply use the Zw high-level ones, but needs to use Io* directly to get around the DISPATCH_LEVEL access problem.

[mtfriesen]

ideally we NT_VERIFY(NT_SUCCESS()) the result of closing handles.

[nibanks]

Why?

[mtfriesen]

It doesn't cost anything on release builds, and the assert on debug builds will catch otherwise silent failures.

[mtfriesen]

Historically we haven't done a good job of this, but with the stepped-up focus on security, our code should always at least nominally (e.g., debug-only) inspect return values of functions.