build with pthreads always includes library_webgl.js

[niklasf]

Example invocation:

em++ -s USE_PTHREADS=1 tests/pthread/test_pthread_clock_drift.cpp

Does not use any graphics, but output a.out.js contains

var GL={counter:1,buffers:[],programs:[],framebuffers:[],renderbuffers:[],textures:[],uniforms:[],shaders:[],

and so on.

Not eliminated when using -O3 or even --closure 1.

[juj]

This is a known current limitation, unfortunately the pthreads proxying mechanism adds dependencies to other libraries (GL is one of them), that we don't currently build variations of to get them stripped away.

[juj]

If you'd like to strip this away manually, edit src/library_pthread.c and remove anything related to webgl there (and probably also canvas if you don't need that), and then do emcc --clear-cache and rebuild the project. Not sure if that is enough, but probably a good start.

[niklasf]

Thanks. In my application the following hack reduces generated js from 36K to 32K (-O3 --closure 1):

diff --git a/system/lib/pthread/library_pthread.c b/system/lib/pthread/library_pthread.c
index 7811167ed..717246828 100644
--- a/system/lib/pthread/library_pthread.c
+++ b/system/lib/pthread/library_pthread.c
@@ -218,15 +218,17 @@ static void _do_call(em_queued_call* q) {
       q->returnValue.i =
         pthread_create(q->args[0].vp, q->args[1].vp, q->args[2].vp, q->args[3].vp);
 #endif
       break;
+#if 0
     case EM_PROXIED_CREATE_CONTEXT:
       q->returnValue.i = emscripten_webgl_create_context(q->args[0].cp, q->args[1].vp);
       break;
     case EM_PROXIED_RESIZE_OFFSCREENCANVAS:
       q->returnValue.i =
         emscripten_set_canvas_element_size(q->args[0].cp, q->args[1].i, q->args[2].i);
       break;
+#endif
     case EM_PROXIED_JS_FUNCTION:
       q->returnValue.d =
         emscripten_receive_on_main_thread_js((int)q->functionPtr, q->args[0].i, &q->args[1].d);
       break;

[niklasf]

The only other remaining large unused function is strftime. Is that another part of this issue or something else?

[juj]

Hmm, that sounds like something else. Off the top of my head I think pthreads should not be pulling in strftime (although maybe via some convoluted dependency chain it does end up happening)

[kripken]

The strftime issue does seem unrelated, I don't see any references to it from pthreads code, and it doesn't show up with building hello world with pthreads. Do you have a testcase @niklasf that shows it is included even though it's not used?

[niklasf]

It's happening in https://github.com/niklasf/stockfish.wasm, but so far I haven't been able to reduce it to a smaller test case. I can confirm it's not included in tests/pthread/test_pthread_clock_drift.cpp, for example.

Meanwhile, since I am now patching emscripten anyway, as above, I just replaced the implementation with a call to abort().

[sbc100]

If you want to see why a given symbol is included you can try the -Wl,--trace-symbol=strftime flag. It should say which object file is pulling in the symbol and where its coming from.

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.

[niklasf]

Still relevant.

[sbc100]

There seem to be two different issues here: GL and strftime.

For the GL issue it sounds like you have a workaround and we have no plans to fix.

For the strftime issue it should like are waiting for a report on adding -Wl,--trace-symbol=strftime which should tell us why your project depends on that symbol.

[niklasf]

Ok, closing the main part (GL) as won'tfix, then. Will reopen the strftime issue if I manage to create a small reproducible example. -Wl,--trace-symbol=strftime unfortunately did not work (no output), even though the symbol is clearly there.

[sbc100]

Can you maybe try with you try -Wl,--trace-symbol=strftime_l (with underscore and lowercase L at the end)?

[niklasf]

Oh, that yields the following output while linking:

/home/niklas/.emscripten_cache/sysroot/lib/wasm32-emscripten/lto/libc++-mt-noexcept.a(locale.o): reference to strftime_l

Followed by:

PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /opt/emscripten-llvm/bin/wasm-ld -o stockfish.wasm benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o --trace-symbol=strftime_l -L/home/niklas/.emscripten_cache/sysroot/lib/wasm32-emscripten/lto /home/niklas/.emscripten_cache/sysroot/lib/wasm32-emscripten/lto/crtbegin.o -lc-mt -lcompiler_rt-mt -lc++-mt-noexcept -lc++abi-mt-noexcept -ldlmalloc-mt -lc_rt_wasm -lsockets-mt --fatal-warnings -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=stackSave --export-if-defined=stackRestore --export-if-defined=stackAlloc --export-if-defined=__wasm_call_ctors --export-if-defined=__errno_location --export-if-defined=__emscripten_pthread_data_constructor --export-if-defined=__pthread_tsd_run_dtors --export-if-defined=_emscripten_call_on_thread --export-if-defined=_emscripten_main_thread_futex --export-if-defined=_emscripten_thread_init --export-if-defined=_emscripten_thread_exit --export-if-defined=emscripten_current_thread_process_queued_calls --export-if-defined=_emscripten_allow_main_runtime_queued_calls --export-if-defined=emscripten_futex_wake --export-if-defined=emscripten_get_global_libc --export-if-defined=emscripten_main_browser_thread_id --export-if-defined=emscripten_main_thread_process_queued_calls --export-if-defined=emscripten_run_in_main_runtime_thread_js --export-if-defined=emscripten_stack_set_limits --export-if-defined=emscripten_sync_run_in_main_thread_2 --export-if-defined=emscripten_sync_run_in_main_thread_4 --export-if-defined=emscripten_tls_init --export-if-defined=pthread_self --export-if-defined=pthread_testcancel --export-if-defined=memalign --export-if-defined=malloc --export-if-defined=free --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-table -z stack-size=5242880 --initial-memory=71303168 --entry=main --max-memory=2147483648 --global-base=1024
 #0 0x000055f891ca9001 (/opt/emscripten-llvm/bin/wasm-ld+0x60d001)
 #1 0x000055f891ca686e (/opt/emscripten-llvm/bin/wasm-ld+0x60a86e)
 #2 0x00007ffac4f0c870 __restore_rt sigaction.c:0:0
 #3 0x000055f892675840 llvm::DIE::getUnitDie() const (/opt/emscripten-llvm/bin/wasm-ld+0xfd9840)
 #4 0x000055f892683d4c llvm::DwarfDebug::finishEntityDefinitions() (/opt/emscripten-llvm/bin/wasm-ld+0xfe7d4c)
 #5 0x000055f892698165 llvm::DwarfDebug::finalizeModuleInfo() (/opt/emscripten-llvm/bin/wasm-ld+0xffc165)
 #6 0x000055f89269d420 llvm::DwarfDebug::endModule() (/opt/emscripten-llvm/bin/wasm-ld+0x1001420)
 #7 0x000055f89266756a llvm::AsmPrinter::doFinalization(llvm::Module&) (/opt/emscripten-llvm/bin/wasm-ld+0xfcb56a)
 #8 0x000055f8940d58cd llvm::FPPassManager::doFinalization(llvm::Module&) (/opt/emscripten-llvm/bin/wasm-ld+0x2a398cd)
 #9 0x000055f8940e2130 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/opt/emscripten-llvm/bin/wasm-ld+0x2a46130)
#10 0x000055f892a7b5fe (/opt/emscripten-llvm/bin/wasm-ld+0x13df5fe)
#11 0x000055f892a7bbb3 llvm::lto::backend(llvm::lto::Config const&, std::function<std::unique_ptr<llvm::lto::NativeObjectStream, std::default_delete<llvm::lto::NativeObjectStream> > (unsigned int)>, unsigned int, llvm::Module&, llvm::ModuleSummaryIndex&) (/opt/emscripten-llvm/bin/wasm-ld+0x13dfbb3)
#12 0x000055f892a6ee34 llvm::lto::LTO::runRegularLTO(std::function<std::unique_ptr<llvm::lto::NativeObjectStream, std::default_delete<llvm::lto::NativeObjectStream> > (unsigned int)>) (/opt/emscripten-llvm/bin/wasm-ld+0x13d2e34)
#13 0x000055f892a6f368 llvm::lto::LTO::run(std::function<std::unique_ptr<llvm::lto::NativeObjectStream, std::default_delete<llvm::lto::NativeObjectStream> > (unsigned int)>, std::function<std::function<std::unique_ptr<llvm::lto::NativeObjectStream, std::default_delete<llvm::lto::NativeObjectStream> > (unsigned int)> (unsigned int, llvm::StringRef)>) (/opt/emscripten-llvm/bin/wasm-ld+0x13d3368)
#14 0x000055f89208f0b3 lld::wasm::BitcodeCompiler::compile() (/opt/emscripten-llvm/bin/wasm-ld+0x9f30b3)
#15 0x000055f8920979d8 lld::wasm::SymbolTable::addCombinedLTOObject() (/opt/emscripten-llvm/bin/wasm-ld+0x9fb9d8)
#16 0x000055f89207d396 (/opt/emscripten-llvm/bin/wasm-ld+0x9e1396)
#17 0x000055f89207e609 lld::wasm::link(llvm::ArrayRef<char const*>, bool, llvm::raw_ostream&, llvm::raw_ostream&) (/opt/emscripten-llvm/bin/wasm-ld+0x9e2609)
#18 0x000055f891c17827 (/opt/emscripten-llvm/bin/wasm-ld+0x57b827)
#19 0x000055f891ba4933 main (/opt/emscripten-llvm/bin/wasm-ld+0x508933)
#20 0x00007ffac494cb25 __libc_start_main (/usr/lib/libc.so.6+0x27b25)
#21 0x000055f891c1678e _start (/opt/emscripten-llvm/bin/wasm-ld+0x57a78e)
em++: error: '/opt/emscripten-llvm/bin/wasm-ld -o stockfish.wasm benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o --trace-symbol=strftime_l -L/home/niklas/.emscripten_cache/sysroot/lib/wasm32-emscripten/lto /home/niklas/.emscripten_cache/sysroot/lib/wasm32-emscripten/lto/crtbegin.o -lc-mt -lcompiler_rt-mt -lc++-mt-noexcept -lc++abi-mt-noexcept -ldlmalloc-mt -lc_rt_wasm -lsockets-mt --fatal-warnings -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=stackSave --export-if-defined=stackRestore --export-if-defined=stackAlloc --export-if-defined=__wasm_call_ctors --export-if-defined=__errno_location --export-if-defined=__emscripten_pthread_data_constructor --export-if-defined=__pthread_tsd_run_dtors --export-if-defined=_emscripten_call_on_thread --export-if-defined=_emscripten_main_thread_futex --export-if-defined=_emscripten_thread_init --export-if-defined=_emscripten_thread_exit --export-if-defined=emscripten_current_thread_process_queued_calls --export-if-defined=_emscripten_allow_main_runtime_queued_calls --export-if-defined=emscripten_futex_wake --export-if-defined=emscripten_get_global_libc --export-if-defined=emscripten_main_browser_thread_id --export-if-defined=emscripten_main_thread_process_queued_calls --export-if-defined=emscripten_run_in_main_runtime_thread_js --export-if-defined=emscripten_stack_set_limits --export-if-defined=emscripten_sync_run_in_main_thread_2 --export-if-defined=emscripten_sync_run_in_main_thread_4 --export-if-defined=emscripten_tls_init --export-if-defined=pthread_self --export-if-defined=pthread_testcancel --export-if-defined=memalign --export-if-defined=malloc --export-if-defined=free --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-table -z stack-size=5242880 --initial-memory=71303168 --entry=main --max-memory=2147483648 --global-base=1024' failed (received SIGSEGV (-11))
make[1]: *** [Makefile:535: stockfish.js] Error 1
make[1]: Leaving directory '/home/niklas/Projekte/stockfish.wasm/src'
make: *** [Makefile:451: build] Error 2

[sbc100]

So it seems that strftime is actually used via strftime_l via some symbol in locale.o. What do you mean when you say its unused? Isn't strftime_l imported by the resulting wasm module?

[kleisauke]

It looks like PR #14912 makes the pthread library no longer depend on WebGL.

[kleisauke]

PR #16405, which has just landed, makes the pthread library no longer dependent on WebGL.

[niklasf]

Awesome. Thanks for the heads-up!