Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wasm-mt] Allow JSImport promises to run on managed threadpool worker threads #83838

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
33b5c4a
[wasm] Bump emscripten to 3.1.34
radekdoulik Mar 27, 2023
dff44d4
Update emsdk deps
radekdoulik Mar 27, 2023
ce1a265
Update icu deps
radekdoulik Mar 27, 2023
ed96a04
Use new images
radekdoulik Mar 27, 2023
8e6a28c
Use emscripten_main_runtime_thread_id
radekdoulik Mar 27, 2023
81672f4
Ignore ExitStatus exceptions
radekdoulik Mar 28, 2023
c2cbb94
Handle UnhandledPromiseRejection for ExitStatus
radekdoulik Mar 28, 2023
f3edd6e
Merge branch 'main' into pr-wasm-emscripten-3-1-34
radekdoulik Mar 28, 2023
4c0cb9e
[wasm][threads] flip YieldFromDispatchLoop; specialize PortableThread…
lambdageek Mar 15, 2023
3d88608
[mono] Implement a LifoJSSemaphore
lambdageek Mar 20, 2023
55b4649
Make managed LowLevelJSSemaphore
lambdageek Mar 20, 2023
8c74dea
copy-paste PortableThreadPool.WorkerThread for threaded WASM
lambdageek Mar 20, 2023
b323f0e
fixup native code for lifo semaphore
lambdageek Mar 21, 2023
51b1fbf
fixup managed code for LowLevelJSSemaphore
lambdageek Mar 21, 2023
3727ff1
Implement PortableThreadPool loop using semaphore callbacks
lambdageek Mar 21, 2023
763edab
manage emscripten event loop from PortableThreadPool.WorkerThread
lambdageek Mar 22, 2023
b45d001
FIXME: thread equality assertion in timeout callback
lambdageek Mar 22, 2023
1f7620c
XXX REVERT ME - minimal async timeout test
lambdageek Mar 22, 2023
06a3cc1
BUGFIX: &wait_entry ===> wait_entry
lambdageek Mar 23, 2023
8574368
nit: log thread id as hex in .ts
lambdageek Mar 23, 2023
c86cb26
XXX minimal sample - fetch on a background thread works
lambdageek Mar 23, 2023
c3cad74
fix non-wasm non-threads builds
lambdageek Mar 24, 2023
e57b262
Add WebWorkerEventLoop internal class to managed event loop keepalive
lambdageek Mar 24, 2023
07eabcd
Start threadpool threads with keepalive checks
lambdageek Mar 24, 2023
474607e
HACK: kind of work around the emscripten_runtime_keepalive_push/pop n…
lambdageek Mar 24, 2023
9f45167
support JS Semaphore with --print-icall-table cross compiler
lambdageek Mar 27, 2023
2e1f31f
make minimal FetchBackground sample more like a unit test
lambdageek Mar 27, 2023
dacc0cb
Share PortableThreadPool.WorkerThread common code
lambdageek Mar 29, 2023
10ca330
make both kinds of lifo semaphore share a base struct
lambdageek Mar 30, 2023
f4a2c02
Unify LowLevelLifoSemaphore for normal and async waiting
lambdageek Mar 30, 2023
42388d8
WebWorkerEventLoop: remove dead code, update comments
lambdageek Mar 31, 2023
853938e
remove unused arg from async wait semaphore
lambdageek Mar 31, 2023
cb8b168
rename native semaphore to LifoSemaphoreAsyncWait
lambdageek Mar 31, 2023
3e8fee4
Rename managed file to LowLevelLifoSemaphore.AsyncWait.Browser.Thread…
lambdageek Mar 31, 2023
552f9a5
Remove unnecessary indirections and allocations from managed AsyncWai…
lambdageek Mar 31, 2023
812524f
fix non-browser+threads builds
lambdageek Mar 31, 2023
50c0f1a
Keep track of unsettled JS interop promises in threadpool workers
lambdageek Apr 3, 2023
c8afaba
change minimal sample's fetch helper to artificially delay
lambdageek Apr 3, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 4 additions & 4 deletions eng/Version.Details.xml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<Dependencies>
<ProductDependencies>
<Dependency Name="Microsoft.NETCore.Runtime.ICU.Transport" Version="8.0.0-preview.3.23163.3">
<Dependency Name="Microsoft.NETCore.Runtime.ICU.Transport" Version="8.0.0-preview.4.23177.3">
<Uri>https://github.com/dotnet/icu</Uri>
<Sha>b3cb54fec4eb845b37038c934d6c9dc17cdfb181</Sha>
<Sha>389d19d09d3cf16ec0143dba065fcd704ab8e48c</Sha>
</Dependency>
<Dependency Name="System.Net.MsQuic.Transport" Version="8.0.0-alpha.1.23166.1">
<Uri>https://github.com/dotnet/msquic</Uri>
Expand Down Expand Up @@ -85,9 +85,9 @@
<Sha>f32e148d67dbf348685c3076a37e8bc68ab3a30f</Sha>
<SourceBuild RepoName="cecil" ManagedOnly="true" />
</Dependency>
<Dependency Name="Microsoft.NET.Workload.Emscripten.Current.Manifest-8.0.100.Transport" Version="8.0.0-preview.4.23170.1">
<Dependency Name="Microsoft.NET.Workload.Emscripten.Current.Manifest-8.0.100.Transport" Version="8.0.0-preview.4.23177.1">
<Uri>https://github.com/dotnet/emsdk</Uri>
<Sha>a464820353b7956538b07c9b53103d793b5e15b6</Sha>
<Sha>e4089ed2abe29bdc25bab2c261940175d0846824</Sha>
<SourceBuild RepoName="emsdk" ManagedOnly="true" />
</Dependency>
</ProductDependencies>
Expand Down
4 changes: 2 additions & 2 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@
<!-- Mono Cecil -->
<MicrosoftDotNetCecilVersion>0.11.4-alpha.23163.1</MicrosoftDotNetCecilVersion>
<!-- ICU -->
<MicrosoftNETCoreRuntimeICUTransportVersion>8.0.0-preview.3.23163.3</MicrosoftNETCoreRuntimeICUTransportVersion>
<MicrosoftNETCoreRuntimeICUTransportVersion>8.0.0-preview.4.23177.3</MicrosoftNETCoreRuntimeICUTransportVersion>
<!-- MsQuic -->
<MicrosoftNativeQuicMsQuicVersion>2.1.7</MicrosoftNativeQuicMsQuicVersion>
<SystemNetMsQuicTransportVersion>8.0.0-alpha.1.23166.1</SystemNetMsQuicTransportVersion>
Expand All @@ -235,7 +235,7 @@
Note: when the name is updated, make sure to update dependency name in eng/pipelines/common/xplat-setup.yml
like - DarcDependenciesChanged.Microsoft_NET_Workload_Emscripten_Current_Manifest-8_0_100_Transport
-->
<MicrosoftNETWorkloadEmscriptenCurrentManifest80100TransportVersion>8.0.0-preview.4.23170.1</MicrosoftNETWorkloadEmscriptenCurrentManifest80100TransportVersion>
<MicrosoftNETWorkloadEmscriptenCurrentManifest80100TransportVersion>8.0.0-preview.4.23177.1</MicrosoftNETWorkloadEmscriptenCurrentManifest80100TransportVersion>
<MicrosoftNETRuntimeEmscriptenVersion>$(MicrosoftNETWorkloadEmscriptenCurrentManifest80100TransportVersion)</MicrosoftNETRuntimeEmscriptenVersion>
<!-- workloads -->
<SwixPackageVersion>1.1.87-gba258badda</SwixPackageVersion>
Expand Down
4 changes: 2 additions & 2 deletions eng/pipelines/common/templates/pipeline-with-resources.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,10 @@ resources:
ROOTFS_DIR: /crossrootfs/ppc64le

- container: browser_wasm
image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-webassembly-net8-20230322221728-80fdceb
image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-webassembly-net8-20230327150025-4404b5c

- container: wasi_wasm
image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-20.04-webassembly-net8-20230322221804-80fdceb
image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-20.04-webassembly-net8-20230327150037-4404b5c

- container: freebsd_x64
image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-freebsd-12
Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/libraries/helix-queues-setup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,6 @@ jobs:

# Browser WebAssembly windows
- ${{ if in(parameters.platform, 'browser_wasm_win', 'wasi_wasm_win') }}:
- (Windows.Amd64.Server2022.Open)windows.amd64.server2022.open@mcr.microsoft.com/dotnet-buildtools/prereqs:windowsservercore-ltsc2022-helix-webassembly-net8-20230319084205-80fdceb
- (Windows.Amd64.Server2022.Open)windows.amd64.server2022.open@mcr.microsoft.com/dotnet-buildtools/prereqs:windowsservercore-ltsc2022-helix-webassembly-net8-20230327150108-4404b5c

${{ insert }}: ${{ parameters.jobParameters }}
Original file line number Diff line number Diff line change
Expand Up @@ -2523,8 +2523,9 @@
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\PortableThreadPool.ThreadCounts.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\PortableThreadPool.WaitThread.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\PortableThreadPool.WorkerThread.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\PortableThreadPool.WorkerThread.NonBrowser.cs" Condition="'$(TargetsBrowser)' != 'true'"/>
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\PortableThreadPool.WorkerTracking.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\PortableThreadPool.Unix.cs" Condition="'$(TargetsUnix)' == 'true' or '$(TargetsBrowser)' == 'true' or '$(TargetsWasi)' == 'true'" />
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\PortableThreadPool.Unix.cs" Condition="'$(TargetsUnix)' == 'true' or ('$(TargetsBrowser)' == 'true' and '$(FeatureWasmThreads)' != 'true') or '$(TargetsWasi)' == 'true'" />
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\PortableThreadPool.Windows.cs" Condition="'$(TargetsWindows)' == 'true'" />
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\LowLevelLifoSemaphore.cs" />
<Compile Include="$(MSBuildThisFileDirectory)System\Threading\LowLevelLifoSemaphore.Windows.cs" Condition="'$(TargetsWindows)' == 'true'" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ public LowLevelLifoSemaphore(int initialSignalCount, int maximumSignalCount, int

public bool Wait(int timeoutMs, bool spinWait)
{
#if TARGET_BROWSER && FEATURE_WASM_THREADS
ThrowIfInvalidSemaphoreKind(LifoSemaphoreKind.Normal);
#endif
Debug.Assert(timeoutMs >= -1);

int spinCount = spinWait ? _spinCount : 0;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System.Diagnostics.Tracing;

namespace System.Threading
{
internal sealed partial class PortableThreadPool
{
/// <summary>
/// The worker thread infastructure for the CLR thread pool.
/// </summary>
private static partial class WorkerThread
{

/// <summary>
/// Semaphore for controlling how many threads are currently working.
/// </summary>
private static readonly LowLevelLifoSemaphore s_semaphore =
new LowLevelLifoSemaphore(
0,
MaxPossibleThreadCount,
AppContextConfigHelper.GetInt32Config(
"System.Threading.ThreadPool.UnfairSemaphoreSpinLimit",
SemaphoreSpinCountDefault,
false),
onWait: () =>
{
if (NativeRuntimeEventSource.Log.IsEnabled())
{
NativeRuntimeEventSource.Log.ThreadPoolWorkerThreadWait(
(uint)ThreadPoolInstance._separated.counts.VolatileRead().NumExistingThreads);
}
});

private static readonly ThreadStart s_workerThreadStart = WorkerThreadStart;

private static void WorkerThreadStart()
{
Thread.CurrentThread.SetThreadPoolWorkerThreadName();

PortableThreadPool threadPoolInstance = ThreadPoolInstance;

if (NativeRuntimeEventSource.Log.IsEnabled())
{
NativeRuntimeEventSource.Log.ThreadPoolWorkerThreadStart(
(uint)threadPoolInstance._separated.counts.VolatileRead().NumExistingThreads);
}

LowLevelLock threadAdjustmentLock = threadPoolInstance._threadAdjustmentLock;
LowLevelLifoSemaphore semaphore = s_semaphore;

while (true)
{
bool spinWait = true;
while (semaphore.Wait(ThreadPoolThreadTimeoutMs, spinWait))
{
WorkerDoWork(threadPoolInstance, ref spinWait);
}

if (WorkerTimedOutMaybeStop(threadPoolInstance, threadAdjustmentLock))
{
break;
}
}
}


private static void CreateWorkerThread()
{
// Thread pool threads must start in the default execution context without transferring the context, so
// using UnsafeStart() instead of Start()
Thread workerThread = new Thread(s_workerThreadStart);
workerThread.IsThreadPoolThread = true;
workerThread.IsBackground = true;
// thread name will be set in thread proc
workerThread.UnsafeStart();
}
}
}
}