Support for WASI threads #98120

Milek7 · 2024-02-07T17:58:29Z

wasi-libc provides pthread implementation, so use that. This way it behaves almost like POSIX-like platform (it does not support signals). Unfortunately macrology does get pretty complicated, because it is WASM, but does not need to use most existing WASM support code but generic POSIX instead.

wasi-sdk is updated to version 21 for WebAssembly/wasi-sdk@c891cd2 and WebAssembly/wasi-libc@ec4566b

pthread_exit does not exist, but returning from start_wrapper seems to work fine, not sure when other ways of exiting a thread are invoked.

Conditionals in csproj are patched to work correctly with WASI threads, but I'm not really sure whether FeatureWasmManagedThreads should be defined at all for this configuration. It's not a browser, has POSIX threads, so this whole complication with ifdefing unsupported browser attributes and adding special assemblies for WebAssembly.Threading seems completely unnecessary. I don't know how to fix this cleanly, though.

When testing on wasmtime it needs bytecodealliance/wasmtime#7884

Milek7 · 2024-02-07T18:04:06Z

@dotnet-policy-service agree company="SimRail S.A."

kg · 2024-02-07T21:03:50Z

src/mono/CMakeLists.txt

 message ("CMAKE_SYSTEM_NAME=${CMAKE_SYSTEM_NAME}")
 message ("CMAKE_SYSTEM_VARIANT=${CMAKE_SYSTEM_VARIANT}")

 set(CLR_CMAKE_HOST_OS ${CMAKE_SYSTEM_NAME})
-string(TOLOWER ${CLR_CMAKE_HOST_OS} CLR_CMAKE_HOST_OS)
+set(CLR_CMAKE_HOST_VARIANT ${CMAKE_SYSTEM_VARIANT})


i'm not super familiar with cmake, but it seems weird that the host variant is either threads or not threads depending on the target you're building for. don't all cmake build hosts have threads?

I think this is referring not to build host but mono runtime host. (as opposed to target system for which runtime will do codegen)
Still, I'm not sure how that interacts with DISABLE_THREADS (shouldn't there be TARGET/HOST_DISABLE_THREADS?), but in any case it was used this way for wasm before.

ghost · 2024-02-07T22:31:53Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

wasi-libc provides pthread implementation, so use that. This way it behaves almost like POSIX-like platform (it does not support signals). Unfortunately macrology does get pretty complicated, because it is WASM, but does not need to use most existing WASM support code but generic POSIX instead.

wasi-sdk is updated to version 21 for WebAssembly/wasi-sdk@c891cd2 and WebAssembly/wasi-libc@ec4566b

pthread_exit does not exist, but returning from start_wrapper seems to work fine, not sure when other ways of exiting a thread are invoked.

Conditionals in csproj are patched to work correctly with WASI threads, but I'm not really sure whether FeatureWasmManagedThreads should be defined at all for this configuration. It's not a browser, has POSIX threads, so this whole complication with ifdefing unsupported browser attributes and adding special assemblies for WebAssembly.Threading seems completely unnecessary. I don't know how to fix this cleanly, though.

When testing on wasmtime it needs bytecodealliance/wasmtime#7884

Author:	Milek7
Assignees:	-
Labels:	`arch-wasm`, `area-Build-mono`, `community-contribution`, `os-wasi`
Milestone:	-

pavelsavara · 2024-02-08T08:28:49Z

This is WASIX extensions, right ? What would happen when we upgrade to WASI preview 2 ?

vargaz · 2024-02-08T08:39:41Z

src/mono/mono/utils/mono-threads-wasm.c

+#elif defined(USE_PTHREAD_WASM_BACKEND)
+
+struct __pthread {
+	struct pthread *self;


This looks pretty brittle, what if this structure changes ?

I don't know how to get stack bounds otherwise, because wasi-libc doesn't have pthread_getattr_np.

WebAssembly/wasi-libc#470

Milek7 · 2024-02-08T13:57:07Z

This is WASIX extensions, right ? What would happen when we upgrade to WASI preview 2 ?

It uses pthreads from wasi-libc, which is currently implemented using wasi-threads proposal (https://github.com/WebAssembly/wasi-threads)
When wasi-libc switches underlying implementation it shouldn't require changes here, but as far I know component-model based preview2 isn't expected to support threads soon, so pthread-supporting toolchain will stay on preview1 for quite some time.

pavelsavara · 2024-02-12T16:54:57Z

@Milek7 we appreciate your efforts!

I would like to share some context to set your and our expectations. As you know, the wasm threading is still developing and the final implementation may be very different from what you are trying to achieve in this PR.

Our view is that we want to have dotnet based on the standard component based WASI, rather than POSIX/WASIX extensions. On the other hand, as you pointed out, threads are not ready.

We are willing to explore this, given that we all know this API would eventually go away.
(The WASI flavor of dotnet is still experimental, probably until WASI itself reaches v1.0, we will not guarantee any compatibility or behavior)

As you probably noticed, our big focus at the moment is to enable threads in the browser (with emscripten). Browser limitations as well as interop with JS requires that we work-around many concepts. Historically we only had Mono on emscripten and that combination was referred to as wasm in the codebase. About year ago we added WASI as another OS and recently made refactoring of the source code directories to reflect browser OS. But part of the #ifdef probably still reflects wasm meaning wasm-browser. Story of FeatureWasmManagedThreads in csproj is similar. It surely makes sense sort it out. This is all Mono, but now there is also NativeAOT experimental branch of dotnet runtime on WASM.

For context, in the browser major features which influence threads are:

you need to keep yielding to browser event loop
- otherwise lot of things don't work: networking, rendering, debugging, new thread creation ...
you can't synchronously block on UI thread (roughly for same reasons)
you can schedule yourself callback via setTimeout.
- This is the way how we overcome limitations above.
- It allows us to implement single-threaded "thread pool" and timer and await JS promises.
Some of those limitations are lifted for multi-threaded (MT) browser, but there is JS object thread affinity.

We have following combinations ahead

ST browser - stable
MT browser - under heavy development
ST WASI - possibly after we finish MT browser
MT WASIX - we did not prioritize it in any way yet
MT WASI - we are waiting for WASI component model

What need to be done for WASI

We need to conceptually figure out which of dotnet features we solved for (single-threaded) browser, could be solved for single-threaded WASI.
- ST "thread pool" to schedule async Task to single thread.
- Timers - there is no setTimeout in WASI. Maybe subscribe to clock and poll ?
- PLINQ
- GC, finalizer
- knowing this would inform us on the #ifdef
- enable unit test coverage for all BCL. Right now we only have "hello world" smoke test running on CI.
make WASI MT with WASIX shared memory work
- this may be side-step for us.
- implement thread-pool, timers, PLINQ, GC - this may be very close to normal unix. We don't need to yield to (browser) event loop etc.
- enable unit tests for all BCL on MT, fix problems
adopt WASIp2 component model in dotnet PAL.
- Possibly call that API directly rather than via libc. Would wasm-threads libc dependency hold us back ?
- This may bring networking. Browser HTTP and WS adapters are non-trivial. WASIp3 with futures and streams will change it again. Shall we wait for it to arrive ?
make WASI MT with component model thread
- Is component based MT going to bring WASM linear memory shared across threads ?
- Would there be host "event loop" ? Because there will be WASIp3 futures.
- it may mean that we will need to partially align it with browser MT and that WASIX MT will be complexity to get rid of, at that point.

Right now, we are not actively working on any of the above.

Are you still interested ?
Which parts you would like to work on ?
What's your motivation or use-case for working on this ?

Milek7 · 2024-02-12T18:47:15Z

What's your motivation or use-case for working on this ?

My motivation is running existing C# codebase in wasm runtime. As you noted, single-threaded requires yielding to caller, which requires changes to both library code and applications (which may assume that threads exist, use locks, and blocking functions). Multi-threading though makes it fit into posix-like model, which enables library and application code to work without changes.

make WASI MT with WASIX shared memory work

This PR uses wasi-threads proposal (as implemented in eg. wasmtime) and does not use wasmer WASIX extensions.

implement thread-pool, timers, PLINQ, GC - this may be very close to normal unix. We don't need to yield to (browser) event loop etc.

Indeed that's why I'm pursuing this approach. Admittedly I didn't do extensive testing yet, but this (Thread, ThreadPool, Task, etc.) should already work with this PR.

Given that it seems component model multithreading won't be available for quite some time (I have seen statements that "The Component Model does not have support for threads at this time and that's not expected to be supported until the time scale of years in the future"), I think it is reasonable to merge this approach for time being. When it comes I think it is expected that wasi-libc will still provide pthread implementation requiring no changes here. Targeting component model directly bypassing libc would likely require some changes, though I'm not exactly sure why you would do this, considering that in practice it would probably mean copy-pasting musl code. (unless it would also bring Wasm GC, but that would be complete architecture change anyway).

I can look into threading bugs if some will come up with more testing, and potentially untangle that FeatureWasmManagedThreads flag if there would be guidelines what to do with it. (like I said before, it works but feels superfluous to ifdef UnsupportedOS for browser when this isn't a browser anyway)

pavelsavara · 2024-02-12T19:57:07Z

What's your motivation or use-case for working on this ?
My motivation is running existing C# codebase in wasm runtime

I'm interested in more details, if you are willing to share them.
What makes wasm attractive compared with say "chiselled" docker ? (this is Q from wasm fan boy)

I didn't do extensive testing yet, but this (Thread, ThreadPool, Task, etc.) should already work with this PR.

We can help enable WASI full library test suite for you on this PR. Is it ok for us to push into your branch ?

There are also some failing CI legs on this PR, because we will need to bump SDK in the Helix VM images too.

potentially untangle that FeatureWasmManagedThreads flag

We are discussing that with the team. TBH we don't know exactly what we want yet. Will keep you posted.

vargaz · 2024-02-12T20:02:16Z

Maybe the bump of the wasi sdk should be done in a separate PR ?

Milek7 · 2024-02-12T20:26:15Z

We can help enable WASI full library test suite for you on this PR. Is it ok for us to push into your branch ?

Thanks, feel free to do it.

I'm interested in more details, if you are willing to share them.

It's probably rather unconventional use-case :) Actually we are embedding it in desktop application, for various reasons it is convenient to have it compiled as WASM module (no extra processes, easy state cleanup for restart).

PS: For testing with wasmtime it will need fresh build (for bytecodealliance/wasmtime#7884)

pavelsavara · 2024-02-13T13:08:22Z

PS: For testing with wasmtime it will need fresh build (for bytecodealliance/wasmtime#7884)

Is there feed with nighty build ?

runtime/eng/testing/wasi-provisioning.targets

Lines 21 to 23 in 954ead3

    
           <WasmtimeURL>https://github.com/bytecodealliance/wasmtime/releases/download/v$(WasmtimeVersion)/wasmtime-v$(WasmtimeVersion)-x86_64-linux.tar.xz</WasmtimeURL> 
        
           <WasmtimeURL Condition="'$(HostOS)' == 'osx'" >https://github.com/bytecodealliance/wasmtime/releases/download/v$(WasmtimeVersion)/wasmtime-v$(WasmtimeVersion)-x86_64-macos.tar.xz</WasmtimeURL> 
        
           <WasmtimeURL Condition="'$(HostOS)' == 'windows'" >https://github.com/bytecodealliance/wasmtime/releases/download/v$(WasmtimeVersion)/wasmtime-v$(WasmtimeVersion)-x86_64-windows.zip</WasmtimeURL>

Milek7 · 2024-02-13T14:15:36Z

https://github.com/bytecodealliance/wasmtime/releases/download/dev/wasmtime-dev-x86_64-linux.tar.xz

vargaz · 2024-02-13T14:39:53Z

The wasi sdk bump was merged.

vargaz · 2024-02-13T19:27:57Z

/azp run runtime-wasm

azure-pipelines · 2024-02-13T19:28:15Z

Azure Pipelines successfully started running 1 pipeline(s).

pavelsavara · 2024-03-19T12:42:29Z

I didn't forget about it and it's still on my TODO list. I plan to cleanup the ifdef situation first, thanks for your patience.

Milek7 · 2024-03-19T14:40:38Z

In the meantime I encountered problem with WasmBuildNative, added fix for that.

Milek7 requested review from lewing, pavelsavara, lambdageek, vargaz, SamMonoRT, lateralusX, BrzVlad, thaystg and marek-safar as code owners February 7, 2024 17:58

ghost added the community-contribution Indicates that the PR has been added by a community member label Feb 7, 2024

dotnet-issue-labeler bot added the area-Build-mono label Feb 7, 2024

Milek7 force-pushed the wasi-threads branch from 0c64a18 to 25889c0 Compare February 7, 2024 18:20

marek-safar added the os-wasi Related to WASI variant of arch-wasm label Feb 7, 2024

Milek7 force-pushed the wasi-threads branch from 25889c0 to e5c2019 Compare February 7, 2024 20:35

kg reviewed Feb 7, 2024

View reviewed changes

lambdageek added the arch-wasm WebAssembly architecture label Feb 7, 2024

vargaz reviewed Feb 8, 2024

View reviewed changes

This was referenced Feb 8, 2024

PortableSourceBuild failures in runtime-dev-innerloop #98160

Closed

[browser][MT] HttpClientCancelTest.SendAsync_Cancel_Success #98201

Closed

pavelsavara mentioned this pull request Feb 12, 2024

[wasi] bump WASI SDK #98315

Closed

vargaz mentioned this pull request Feb 12, 2024

[wasi] Update wasi-sdk to version 21. dotnet/dotnet-buildtools-prereqs-docker#967

Merged

pavelsavara mentioned this pull request Feb 13, 2024

[wasi] Bump wasi sdk to 21. #98340

Merged

vargaz force-pushed the wasi-threads branch from e5c2019 to 5cee9d3 Compare February 13, 2024 15:25

This was referenced Feb 21, 2024

System.Text.Json failing some large file tests #59678

Closed

[wasm] System.Text.Json.Tests running out of memory #98578

Closed

Milek7 added 3 commits March 19, 2024 14:43

Add support for WASI threads

6a88a5c

Patch csproj conditionals to work with WASI threads

06d5061

Fix pthread linking for WasmBuildNative

c48c6ab

Milek7 force-pushed the wasi-threads branch from c3d2988 to c48c6ab Compare March 19, 2024 13:44

Milek7 requested review from maraf, akoeplinger and steveisok as code owners March 19, 2024 13:44

build-analysis bot mentioned this pull request Mar 19, 2024

RestApiException`1: The response contained an invalid status code 500 Internal Server Error dotnet/dnceng#2298

Open

3 tasks

pavelsavara self-assigned this Apr 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for WASI threads #98120

Support for WASI threads #98120

Milek7 commented Feb 7, 2024

Milek7 commented Feb 7, 2024

kg Feb 7, 2024

Milek7 Feb 7, 2024 •

edited

ghost commented Feb 7, 2024

pavelsavara commented Feb 8, 2024

vargaz Feb 8, 2024

Milek7 Feb 8, 2024

Milek7 Feb 8, 2024

Milek7 commented Feb 8, 2024

pavelsavara commented Feb 12, 2024 •

edited

Milek7 commented Feb 12, 2024

pavelsavara commented Feb 12, 2024

vargaz commented Feb 12, 2024

Milek7 commented Feb 12, 2024 •

edited

pavelsavara commented Feb 13, 2024

Milek7 commented Feb 13, 2024

vargaz commented Feb 13, 2024

vargaz commented Feb 13, 2024

azure-pipelines bot commented Feb 13, 2024

pavelsavara commented Mar 19, 2024

Milek7 commented Mar 19, 2024

Support for WASI threads #98120

Are you sure you want to change the base?

Support for WASI threads #98120

Conversation

Milek7 commented Feb 7, 2024

Milek7 commented Feb 7, 2024

kg Feb 7, 2024

Choose a reason for hiding this comment

Milek7 Feb 7, 2024 • edited

Choose a reason for hiding this comment

ghost commented Feb 7, 2024

pavelsavara commented Feb 8, 2024

vargaz Feb 8, 2024

Choose a reason for hiding this comment

Milek7 Feb 8, 2024

Choose a reason for hiding this comment

Milek7 Feb 8, 2024

Choose a reason for hiding this comment

Milek7 commented Feb 8, 2024

pavelsavara commented Feb 12, 2024 • edited

Milek7 commented Feb 12, 2024

pavelsavara commented Feb 12, 2024

vargaz commented Feb 12, 2024

Milek7 commented Feb 12, 2024 • edited

pavelsavara commented Feb 13, 2024

Milek7 commented Feb 13, 2024

vargaz commented Feb 13, 2024

vargaz commented Feb 13, 2024

azure-pipelines bot commented Feb 13, 2024

pavelsavara commented Mar 19, 2024

Milek7 commented Mar 19, 2024

Milek7 Feb 7, 2024 •

edited

pavelsavara commented Feb 12, 2024 •

edited

Milek7 commented Feb 12, 2024 •

edited