emmake fails and outputs thousands of libcxx errors

[jeffRTC]

I'm on Windows.

I activate env vars by running C:/Users/Administrator/emsdk/emsdk_env.bat then I run emcmake .. && emmake make and it started to print gazillions of errors related to libcxx and libc

Example,

mingw32-make[2]: Entering directory 'C:/b/browserClient/build'
[ 33%] Building CXX object CMakeFiles/browserClient.dir/CoreModules/browser/Client/browserClient.cpp.o
C:\Users\Administrator\emsdk\upstream\emscripten\em++.bat   @CMakeFiles/browserClient.dir/includes_CXX.rsp -Oz --bind -s WASM=1 -s ASYNCIFY=1 -s NO_EXIT_RUNTIME=1 -s ALLOW_MEMORY_GROWTH=1 -s EXTRA_EXPORTED_RUNTIME_METHODS=[ccall,cwrap] -s EXPORT_ALL=1 -s MODULARIZE=1 -s EXPORT_NAME=browserClient-std=gnu++17 -o CMakeFiles\browserClient.dir\CoreModules\browser\Client\browserClient.cpp.o -c C:\b\CoreModules\browser\Client\browserClient.cpp

C:\Users\Administrator\emsdk\upstream\emscripten\system\include\libcxx\streambuf:141:5: error: unknown type name 'locale'; did you
      mean 'locale_t'?
    locale pubimbue(const locale& __loc) {
    ^
C:\Users\Administrator\emsdk\upstream\emscripten\system\lib\libc\musl\arch\emscripten\bits/alltypes.h:391:34: note: 'locale_t'
      declared here
typedef struct __locale_struct * locale_t;

[jeffRTC]

cc @sbc100 @kripken

[jeffRTC]

Lol I ended up getting foobar challenge from Google while looking ways to fix this. Quite iconic moment.

[kripken]

Something is probably wrong in your include paths. Try to build a hello world. And run with -v to debug the include paths.

[jeffRTC]

@kripken

-IC:/deps/libdatachannel/deps/json/single_include/nlohmann -IC:/deps/libdatachannel/deps/json/single_include -IC:/deps/libdatachannel/deps/json/test/reports/2016-08-29-fuzz -IC:/deps/libdatachannel/deps/json/test/reports

[jeffRTC]

@kripken

diagnostics:DEBUG: disabled warning: -I or -L of an absolute path "-IC:/deps/single-header" encountered. If this is to a local system header/library, it may cause problems (local system files make sense for compiling natively on your system, but not necessarily to JavaScript). [-Wabsolute-paths]

[sbc100]

The locale class is defined in
system/include/libcxx/__locale
which is included from
system/include/libcxx/ios
which is included from
system/include/libcxx/streambuf

This works fine under normal circumstances.. perhaps you have file called __locale or ios somewhere in your include path that is getting picked up instead?

When I have problems like this I will sometimes at a #error foo to the file in question. In this case ``system/include/libcxx/__locale`. If you don't see that error then it means that file is not being included and you can do that same with next level up. If you do see the error then its a different problem and we know its not the wrong include being picked up.

[jeffRTC]

There is a folder called ios in one of includes path. It has no relation to libcxx

How come clang is that stupid to pick that up?

[sbc100]

I didn't think clang would pick up directories.. but I guess its possible.

I imagine it should be looking for files only... perhaps this should be fixed in upstream clang?

[jeffRTC]

@sbc100 Seriously, it has to do with include paths somewhere. I was able to fix the error by manually including the paths by hands.

I was using this macro before to automate the process,

MACRO(HEADER_DIRECTORIES return_list dir)
    FILE(GLOB_RECURSE new_list "${dir}/*.h" "${dir}/*.hpp")
    SET(dir_list "")
    FOREACH(file_path ${new_list})
        GET_FILENAME_COMPONENT(current_dir_path ${file_path} PATH)
        GET_FILENAME_COMPONENT(prev_dir_path ${current_dir_path} PATH)
        SET(dir_list ${dir_list} ${current_dir_path})
        SET(dir_list ${dir_list} ${prev_dir_path})
    ENDFOREACH()
    LIST(REMOVE_DUPLICATES dir_list)
    SET(${return_list} ${dir_list})
ENDMACRO()

In big projects, it would be painful to include them by hand. Nevertheless, you may investigate this case further if you want to.

[jeffRTC]

@sbc100

Re-opening this. I found an another error,

emscripten:DEBUG: emscript: js compiler glue
error: undefined symbol: _emval_as (referenced by top-level compiled C/C++ code)
warning: Link with `-s LLD_REPORT_UNDEFINED` to get more information on undefined symbols
warning: To disable errors for undefined symbols use `-s ERROR_ON_UNDEFINED_SYMBOLS=0`
warning: __emval_as may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_await (referenced by top-level compiled C/C++ code)
warning: __emval_await may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_call_method (referenced by top-level compiled C/C++ code)
warning: __emval_call_method may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_call_void_method (referenced by top-level compiled C/C++ code)
warning: __emval_call_void_method may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_decref (referenced by top-level compiled C/C++ code)
warning: __emval_decref may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_get_global (referenced by top-level compiled C/C++ code)
warning: __emval_get_global may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_get_method_caller (referenced by top-level compiled C/C++ code)
warning: __emval_get_method_caller may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_get_property (referenced by top-level compiled C/C++ code)
warning: __emval_get_property may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_incref (referenced by top-level compiled C/C++ code)
warning: __emval_incref may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_new_cstring (referenced by top-level compiled C/C++ code)
warning: __emval_new_cstring may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_run_destructors (referenced by top-level compiled C/C++ code)
warning: __emval_run_destructors may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: _emval_take_value (referenced by top-level compiled C/C++ code)
warning: __emval_take_value may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcAddRemoteCandidate (referenced by top-level compiled C/C++ code)
warning: _rtcAddRemoteCandidate may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcCreateDataChannel (referenced by top-level compiled C/C++ code)
warning: _rtcCreateDataChannel may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcCreatePeerConnection (referenced by top-level compiled C/C++ code)
warning: _rtcCreatePeerConnection may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcDeleteDataChannel (referenced by top-level compiled C/C++ code)
warning: _rtcDeleteDataChannel may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcDeletePeerConnection (referenced by top-level compiled C/C++ code)
warning: _rtcDeletePeerConnection may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcGetBufferedAmount (referenced by top-level compiled C/C++ code)
warning: _rtcGetBufferedAmount may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcGetDataChannelLabel (referenced by top-level compiled C/C++ code)
warning: _rtcGetDataChannelLabel may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSendMessage (referenced by top-level compiled C/C++ code)
warning: _rtcSendMessage may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetBufferedAmountLowCallback (referenced by top-level compiled C/C++ code)
warning: _rtcSetBufferedAmountLowCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetBufferedAmountLowThreshold (referenced by top-level compiled C/C++ code)
warning: _rtcSetBufferedAmountLowThreshold may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetDataChannelCallback (referenced by top-level compiled C/C++ code)
warning: _rtcSetDataChannelCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetErrorCallback (referenced by top-level compiled C/C++ code)
warning: _rtcSetErrorCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetLocalCandidateCallback (referenced by top-level compiled C/C++ code)
warning: _rtcSetLocalCandidateCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetLocalDescriptionCallback (referenced by top-level compiled C/C++ code)
warning: _rtcSetLocalDescriptionCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetMessageCallback (referenced by top-level compiled C/C++ code)
warning: _rtcSetMessageCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetOpenCallback (referenced by top-level compiled C/C++ code)
warning: _rtcSetOpenCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetRemoteDescription (referenced by top-level compiled C/C++ code)
warning: _rtcSetRemoteDescription may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: rtcSetUserPointer (referenced by top-level compiled C/C++ code)
warning: _rtcSetUserPointer may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: wsCreateWebSocket (referenced by top-level compiled C/C++ code)
warning: _wsCreateWebSocket may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: wsDeleteWebSocket (referenced by top-level compiled C/C++ code)
warning: _wsDeleteWebSocket may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: wsSendMessage (referenced by top-level compiled C/C++ code)
warning: _wsSendMessage may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: wsSetErrorCallback (referenced by top-level compiled C/C++ code)
warning: _wsSetErrorCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: wsSetMessageCallback (referenced by top-level compiled C/C++ code)
warning: _wsSetMessageCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: wsSetOpenCallback (referenced by top-level compiled C/C++ code)
warning: _wsSetOpenCallback may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: wsSetUserPointer (referenced by top-level compiled C/C++ code)
warning: _wsSetUserPointer may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
Error: Aborting compilation due to previous errors
em++: error: 'C:/Users/Administrator/emsdk/node/12.18.1_64bit/bin/node.exe C:\Users\Administrator\emsdk\upstream\emscripten\src\compiler.js C:\Users\ADMINI~1\AppData\Local\Temp\2\emscripten_temp\tmpz69ew0dg.txt' failed (1)

I export all, see my compiler flags,

add_compile_options("SHELL: --bind")
add_compile_options("SHELL: -s WASM=1")
add_compile_options("SHELL: -s ASYNCIFY=1")
add_compile_options("SHELL: -s NO_EXIT_RUNTIME=1")
add_compile_options("SHELL: -s ALLOW_MEMORY_GROWTH=1")
add_compile_options("SHELL: -s EXTRA_EXPORTED_RUNTIME_METHODS=['ccall','cwrap']")
add_compile_options("SHELL: -s EXPORT_ALL=1")
add_compile_options("SHELL: -s MODULARIZE=1")

[jeffRTC]

@sbc100

Made a change and that fixed out the error

add_link_options("SHELL: --bind")
add_link_options("SHELL: -s ASYNCIFY=1")
add_link_options("SHELL: -s NO_EXIT_RUNTIME=1")
add_link_options("SHELL: -s ALLOW_MEMORY_GROWTH=1")
add_link_options("SHELL: -s EXTRA_EXPORTED_RUNTIME_METHODS=['ccall','cwrap']")
add_link_options("SHELL: -s EXPORT_ALL=1")
add_link_options("SHELL: -s MODULARIZE=1")

Any way to know which flags are link ones versus compile ones?

[jeffRTC]

Found it,

https://github.com/emscripten-core/emscripten/blob/master/src/settings.js

// In general it is best to pass the same arguments at both compile and link
// time, as whether wasm object files are used or not affects when codegen
// happens (without wasm object files, codegen is done entirely during
// link; otherwise, it is during compile). Flags affecting codegen must
// be passed when codegen happens, so to let a build easily switch when codegen
// happens (LTO vs normal), pass the flags at both times. The flags are also
// annotated in this file:
//
// [link] - Should be passed at link time. This is the case for all JS flags,
// as we emit JS at link (and that is most of the flags here, and
// hence the default).
// [compile+link] - A flag that has an effect at both compile and link time,
// basically any time emcc is invoked. The same flag should be
// passed at both times in most cases.
//
// If not otherwise specified, a flag is [link]. Note that no flag is only
// relevant during compile time, as during link we may do codegen for system
// libraries and other support code, so all flags are either link or
// compile+link.
//

Better to add this to your official website docs

[sbc100]

Yes, those are useful. I think it would be even better if we could issue warning when they are used in the wrong situation. Like clang does "warning: argument unused during compilation".

Also this only covers -s settings, not other options :(