--shared-memory is disallowed when using -sSHARED_MEMORY=1 compiling C++ code

[toyobayashi]

Version of emscripten/emsdk:

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.13 (531257621816c200bc7c3be53129494afd029aec)
clang version 15.0.0 (https://github.com/llvm/llvm-project 5c6ed60c517c47b25b6b25d8ac3666d0e746b0c3)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: C:\Users\toyobayashi\Projects\emsdk\upstream\bin

Failing command line in full:

C:\Users\toyobayashi\Projects\emxx-shared-memory-repro>em++ -sSHARED_MEMORY=1 -sALLOW_MEMORY_GROWTH=1 --js-library=library_napi.js -o out/myobject.js emnapi.c myobject.cc main.cc
clang++: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
wasm-ld: error: --shared-memory is disallowed by cxa_default_handlers.o because it was not compiled with 'atomics' or 'bulk-memory' features.
em++: error: 'C:/Users/toyobayashi/Projects/emsdk/upstream/bin\wasm-ld.exe -o out/myobject.wasm C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_8n173m2y\emnapi_0.o C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_8n173m2y\myobject_1.o C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_8n173m2y\main_2.o -LC:\Users\toyobayashi\Projects\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten C:\Users\toyobayashi\Projects\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten\crtbegin.o -lGL -lal -lhtml5 -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lcompiler_rt -lc++-noexcept -lc++abi-noexcept -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__main_argc_argv --export-if-defined=fflush --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__wasm_call_ctors --export=__errno_location --export-table -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=2147483648 --global-base=1024' failed (returned 1)

Full link command and output with -v appended:

 "C:/Users/toyobayashi/Projects/emsdk/upstream/bin\wasm-ld.exe" -o out/myobject.wasm C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_d1j8pdc_\emnapi_0.o C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_d1j8pdc_\myobject_1.o C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_d1j8pdc_\main_2.o -LC:\Users\toyobayashi\Projects\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten C:\Users\toyobayashi\Projects\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten\crtbegin.o -lGL -lal -lhtml5 -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lcompiler_rt -lc++-noexcept -lc++abi-noexcept -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__main_argc_argv --export-if-defined=fflush --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__wasm_call_ctors --export=__errno_location --export-table -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=2147483648 --global-base=1024
wasm-ld: error: --shared-memory is disallowed by cxa_default_handlers.o because it was not compiled with 'atomics' or 'bulk-memory' features.
em++: error: 'C:/Users/toyobayashi/Projects/emsdk/upstream/bin\wasm-ld.exe -o out/myobject.wasm C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_d1j8pdc_\emnapi_0.o C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_d1j8pdc_\myobject_1.o C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_d1j8pdc_\main_2.o -LC:\Users\toyobayashi\Projects\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten C:\Users\toyobayashi\Projects\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten\crtbegin.o -lGL -lal -lhtml5 -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lcompiler_rt -lc++-noexcept -lc++abi-noexcept -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__main_argc_argv --export-if-defined=fflush --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__wasm_call_ctors --export=__errno_location --export-table -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=2147483648 --global-base=1024' failed (returned 1)

Repro:
https://github.com/toyobayashi/emxx-shared-memory-repro

---

C++ hello world is also failed to compile with -sSHARED_MEMORY=1

main.cc

#include <iostream>

int main() {
  std::cout << "em++" << std::endl;
  return 0;
}

C:\Users\toyobayashi\Projects\emxx-shared-memory-repro>em++ -sSHARED_MEMORY=1 -o main.js main.cc
wasm-ld: error: --shared-memory is disallowed by ios.o because it was not compiled with 'atomics' or 'bulk-memory' features.
em++: error: 'C:/Users/toyobayashi/Projects/emsdk/upstream/bin\wasm-ld.exe -o main.wasm C:\Users\toyobayashi\AppData\Local\Temp\emscripten_temp_z4sx8wwx\main_0.o -LC:\Users\toyobayashi\Projects\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten C:\Users\toyobayashi\Projects\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten\crtbegin.o -lGL -lal -lhtml5 -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lcompiler_rt -lc++-noexcept -lc++abi-noexcept -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__main_argc_argv --export-if-defined=fflush --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__wasm_call_ctors --export=__errno_location --export-table -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024' failed (returned 1)

But C is all OK.

main.c

#include <stdio.h>

int main() {
  printf("emcc\n");
  return 0;
}

C:\Users\toyobayashi\Projects\emxx-shared-memory-repro>emcc -sSHARED_MEMORY=1 -o main.js main.c
# OK! output main.js and main.wasm

Forgive my stupidity if this is not a bug.

[kripken]

@juj it looks like SHARED_MEMORY does not link in -mt system libraries, which I think is the issue here (

emscripten/tools/system_libs.py

Line 497 in 17ea9f0

return super().get_default_variation(is_mt=settings.USE_PTHREADS, is_ww=settings.WASM_WORKERS and not settings.USE_PTHREADS, **kwargs)

). So building hello world with SHARED_MEMORY builds that object with atomics + bulk memory, but we link with non-atomic-supporting libc++ for example, which fails to link.

What should SHARED_MEMORY be doing here? The docs in settings.js say

// [compile+link] - affects user code at compile and system libraries at link.

Should it affect system libraries? If so, however, wouldn't it end up linking in pthreads dependencies? (libc++ might use pthreads when atomics are enabled) I'm not sure how this fits together...

[juj]

This does certainly look like a bug.

The way I understood the requested use case of -sSHARED_MEMORY from comment #12833 to work is that it would enable targeting shared codegen, but not use the pthreads or Wasm Workers API. That is, the users would be essentially building a singlethreaded application, but still want to get a Shared WebAssembly.Memory backing, and atomics aware codegen. (so that they can do some manual magic with custom JS-based Workers)

However implementing that is tricky, since to enable shared codegen for LLVM means to pass -pthread, which I believe has been developed from the assumption that a pthreads runtime would be available.

Building libc with multithreading enabled also makes it assume that pthreads is available, so I think libc-mt will have spots that depend on pthreads.

I think we are probably missing an implementation of a "libc-mt-without-pthreads" and possibly a LLVM "-shared-mem-without-pthread" build flags here which would enable targeting such cases.

In the original PR, since we don't have such an adapted "libc-mt-without-pthreads", I configured system_libs.py to build libc singlethreaded, thinking that it would be linked to the final app (and that apps won't care/observe that anyway since all they want is a shared SAB) - but now I see with that error message that this won't work because WebAssembly linker is strict and disallows linking singlethreaded object files against multithreading enabled object files.

[toyobayashi]

The way I understood the requested use case of -sSHARED_MEMORY from comment #12833 to work is that it would enable targeting shared codegen, but not use the pthreads or Wasm Workers API. That is, the users would be essentially building a singlethreaded application, but still want to get a Shared WebAssembly.Memory backing, and atomics aware codegen. (so that they can do some manual magic with custom JS-based Workers)

It seems that I understand correctly. I'm not familiar with how the underlying toolchain works. I just wanted to try to write my custom Worker in JavaScript and do something in worker with SharedArrayBuffer but not use the pthreads, then I experienced the link error. Hope this could be fixed soon, or any workaround available?

[sbc100]

1. Do you expect to be running the compiled wasm code at the same time that another worker is accessing the shared memory?
2. If so, is that other worker also running the same wasm code, or some other code, such as custom JS?
3. What kind of synchronization primitives to you plan on using?
4. Do you want/need to core C/C++ libraries to be thread safe?

[toyobayashi]

@sbc100

Do you expect to be running the compiled wasm code at the same time that another worker is accessing the shared memory?

Yes.

If so, is that other worker also running the same wasm code, or some other code, such as custom JS?

Yes. Running the same wasm code and custom JS.

What kind of synchronization primitives to you plan on using?

Maybe Atomics?

Do you want/need to core C/C++ libraries to be thread safe?

I'm not sure.

I am the author of emnapi which is a library implemented a subset of Node-API in JavaScript for Emscripten platform. All APIs declared in js_native_api.h has been implemented. Recently I'm thinking how to implement the simple asynchronous operation APIs declared in node_api.h:

• napi_create_async_work
• napi_delete_async_work
• napi_queue_async_work
• napi_cancel_async_work

I plan to invoke the C side napi_async_execute_callback in another Worker, so I also need to running my JS code in that worker. Now I'm not sure if it is the correct and best way to do this. In this case should C/C++ libraries be thread safe?

[sbc100]

Is there some reason why you can't use the existing pthread support? (-sUSE_PTHREADS)

[toyobayashi]

I need to do these things in JavaScript, using dynCalls, and listen worker message myself to interact with my emnapi JS runtime code. Maybe using existing pthread support can also achieve this, but if user don't need to use pthread, they still need to use -sUSE_PTHREADS to use napi_*_async_work API

[kripken]

@juj Thanks, makes sense. There isn't an obvious solution, then, and we'd need to do more work to support use cases here. Maybe we need to figure those out more first.

[sbc100]

I need to do these things in JavaScript, using dynCalls, and listen worker message myself to interact with my emnapi JS runtime code. Maybe using existing pthread support can also achieve this, but if user don't need to use pthread, they still need to use -sUSE_PTHREADS to use napi_*_async_work API

IIRC, pthreads can call JS code in the all the same ways that the main thread can.

If your main goal is to proxy async work then we have specific API that can be used for this purpose: https://github.com/emscripten-core/emscripten/blob/main/system/include/emscripten/proxying.h

In terms of listen for worker messages I believe you can do that to. See the discussion here: #16239

[lucasjinreal]

Same error here:

wasm-ld: error: --shared-memory is disallowed by CMakeFiles/nanodet-simd-threads.dir/nanodet.cpp.o because it was not compiled with 'atomics' or 'bulk-memory' features.

Just to know how to resolve here?

[sbc100]

This is because nanodet.cpp was not compiled with -pthread.

[sbc100]

If you want to build a multithreaded program you need to apply -pthread (or whatever link flag you are using, e.g. -sSHARED_MEMORY) to every object file in your project.

[vadimkantorov]

Hitting a similar issue while compiling a recent fontconfig (was not the case before one year ago)

[attilaolah]

Looks like this issue becomes pretty much inevitable with a sufficiently large dependency tree. Building ffmpeg, which now requires -pthread since version 6.0 (the libraries not, but the binary itself does). Once I start to link in codecs and their dependencies, and I start recompiling all those dependencies with -sSHARED_MEMORY, I inevitably hit some dependency that happens to use part of the standard library that is not compiled with bulk memory or atomics.

It would at least be useful to have a standard library that is either thread-safe, or maybe unsafe but at least it should be possible to link it with thearding-enabled code. Maybe then it is UB to concurrently call those standard library methods, but well-behaved apps would hopefully not do that.

[sbc100]

When linking fails in this way its because thread-safety has been stripped/removed from a given object file. This means the resulting code is guarantees to unsafe to use from multiple threads (i.e. the specific object in question had usages of atomics or TLS that were lowered away by the compiler).

It true that you could take that guaranteed thread-unsafe code and link it into a multithreaded program if you are sure that all the usage of that library will come from just one thread. I can imagine that could be useful in a few application but one would need to be extremely careful and aware of the risks involved. Is that really what you are asking for? How common is this case? How hard would it be to instead just rebuild everything with the -pthread flag?

Being able to rebuild your entire project, including all object files from scratch (for example when updating from one version of emscripten to the next) is already something that folks kind of need to be able to do.

Certainly if you were going to try to distribute a version libffmpeg.a to a wide audience you would want to distrubute two flavors of it (libffmpeg.a and libffmpeg-mt.a) having folks link the non-threaded version into a threaded application without at least a warning sounds like a bad idea.

[attilaolah]

To be clear here, I would much rather recompile everything down to the standard library, rather than link against thread-unsafe code. I am however, currently unsure how to do that. (Namely, how to recompile the standard library — or, are there already -mt versions of it ready for consumption?)

Indeed, building a libffmpeg.a and a libffmpeg-mt.a with and without thread support, along with all the dependencies, is not a big deal. The problem seems to be that, as of now, even when I build a multi-threaded static library, I am unable to link it when building a binary (a .wasm file, that is) since the linker complains that some dependencies are not thread safe. I then go ahead and update all those dependencies, however, I eventually end up with some that need standard library headers that are not thread safe.

Maybe those dependencies are not correctly passing -pthread when building the static lib, I'll have to double-check each. It is a bit painful since they all use different build systems… But I'll get there.

And to add to all that, I'm only interested in a multithreaded version at the moment, since we will need to use the ffmpeg binary in our product, and the binary, as of 6.0, can only be build with threads enabled.

Thanks for the quick response btw!

[sbc100]

To be clear here, I would much rather recompile everything down to the standard library, rather than link against thread-unsafe code. I am however, currently unsure how to do that. (Namely, how to recompile the standard library — or, are there already -mt versions of it ready for consumption?)

Emscripten already takes care of linking the correct version of the standard libraries yes.

[sbc100]

however, I eventually end up with some that need standard library headers that are not thread safe.

I'm not sure what you mean by this, perhaps you could give and example of elaborate?

[attilaolah]

While I got it working now, the issue I gat was while trying to build, of all things, bz2. I think the build script might have dropped -pthread for me, not entirely sure but in the end I got it working. Sorry to make noise on this thread, I think my main mistake was not realising that Emscripten would already link with the correct version of the standard libraries — it didn't, for me, but likely because the build system removed some flags, and I haven't yet figured out why. I'll update this issue when I figure that out.