Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ReferenceError: asyncifyStubs is not defined #21104

Closed
Charles-Schleich opened this issue Jan 18, 2024 · 16 comments · Fixed by #21674
Closed

ReferenceError: asyncifyStubs is not defined #21104

Charles-Schleich opened this issue Jan 18, 2024 · 16 comments · Fixed by #21674

Comments

@Charles-Schleich
Copy link

Version of emscripten/emsdk: using Docker image FROM emscripten/emsdk:3.1.51

Failing command line in full:
This is a runtime failure for me, in the browser.
Context, compiling CPP -> WASM to run in the browser.
Packaging with Webpack.
Fails in both Chrome and Firefox.

Full link command and output with -v appended:
Building emscripten with CMakeLists.txt

set(WASM_LINK_FLAGS "--js-library=${CMAKE_SOURCE_DIR}/func-wrapper.js \
--extern-pre-js=${CMAKE_SOURCE_DIR}/COPYWRITE.js \
-s TEXTDECODER=0 \
-s ENVIRONMENT=web,worker \
-s EXPORTED_RUNTIME_METHODS=[\"writeArrayToMemory\",\"getValue\",\"setValue\",\"registerJSCallback\",\"onRuntimeInitialized\",\"cwrap\",\"UTF8ToString\",\"stringToUTF8OnStack\"] \
-s EXPORTED_FUNCTIONS=['_test_call'] \
-s PTHREAD_POOL_SIZE=16 \
-s ERROR_ON_UNDEFINED_SYMBOLS=0 \
-s ALLOW_TABLE_GROWTH \
-s RESERVED_FUNCTION_POINTERS=100 \
-s ASYNCIFY=2 \
-s ASYNCIFY_STACK_SIZE=65535 \
-s EXPORT_ES6=1 \
-s MODULARIZE=1 \
-s USE_ES6_IMPORT_META=1 \
-s LINKABLE=1 \
-s EXPORT_ALL=1 \
-s EXPORT_NAME=zenohWasm \
-s ALLOW_TABLE_GROWTH \
-pthread \
-fPIC ")
  EM_JS(void, testReadFile, (), {
      Asyncify.handleAsync(async () => {
          // console.log(content);
      });
  });

Exact error that i am getting:
Uncaught (in promise) ReferenceError: asyncifyStubs is not defined

Any pointers as to the correct incantation of Linker flags,
or if you require any other information, please let me know.

@sbc100
Copy link
Collaborator

sbc100 commented Jan 18, 2024

What is in COPYWRITE.js and func-wrapper.js? Can you reproduce the error without those flags?

Do you have a backtrace for where the ReferenceError is coming from?

Can you try removing -s ERROR_ON_UNDEFINED_SYMBOLS=0?

BTW, you can use simple list for EXPORTED_RUNTIME_METHODS and EXPORTED_FUNCTIONS. e.g. -s EXPORTED_RUNTIME_METHODS=writeArrayToMemory,getValue,setValue,registerJSCallback,onRuntimeInitialized,cwrap,UTF8ToString,stringToUTF8OnStack.

@Charles-Schleich
Copy link
Author

Hi @sbc100, thanks for the lightning fast response,

  1. COPYWRITE.js is just some comments that are prepended to each file that gets generated.
    func-wrapper.js is a file that contains some attempts to wrap callbacks

func-wrapper.js - Callback

mergeInto(LibraryManager.library, {
    $func_map: "new Map()",
    $func_idx: "0",
    $registerJSCallback__deps: ['$func_map', '$func_idx'],
    $registerJSCallback: function (fn) {
        index = func_idx
        func_map.set(index, fn)
        func_idx++;
        return index;
    },
    
    call_js_callback__proxy: 'sync',
    call_js_callback__sig: 'ipi',
    call_js_callback__deps: ['$func_map', '$func_idx'],
    call_js_callback: function (id, value, ctx) {
        var fn = func_map.get(id)
        fn(value, ctx)
        // if call_js_callback__proxy is `async` then after calling the fn we
        // must check if it return a promise, in that case we should use the `.then(..)` 
        // and free the memory there.
    },
    remove_js_callback__proxy: 'sync',
    remove_js_callback__sig: 'i',
    remove_js_callback__deps: ['$func_map', '$func_idx'],
    remove_js_callback: function (id) {
        func_map.delete(id)
    }
});


mergeInto(LibraryManager.library, {

    test_call_js_callback__proxy: 'sync',
    test_call_js_callback__sig: 'ipi',
    test_call_js_callback__deps: ['$func_map', '$func_idx'],
    test_call_js_callback: function () {
        console.log("Hello world from Callback");
        // if call_js_callback__proxy is `async` then after calling the fn we
        // must check if it return a promise, in that case we should use the `.then(..)` 
        // and free the memory there.
    },
});

Question 1: Will I still need this function wrapper with the ASYNCIFY linker flags ?
Also are there any other flags that I should be adding ?

  1. Yes asyncifyStubs appears first on line 7398 on my generated foo_wasm.js file, this is what appears at those lines.
  /** @type {function(...*):?} */
  function __emval_await(
  ) {
  abort('missing function: _emval_await');
  }
  __emval_await.sig = 'pp';
  __emval_await.stub = true;
  asyncifyStubs['_emval_await'] = undefined;

  /** @type {function(...*):?} */
  function __emval_call(
  ) {
  abort('missing function: _emval_call');
  }
  __emval_call.sig = 'dpppp';
  __emval_call.stub = true;
  asyncifyStubs['_emval_call'] = undefined;

  /** @type {function(...*):?} */
  function __emval_decref(
  ) {
  abort('missing function: _emval_decref');
  }
  __emval_decref.sig = 'vp';
  __emval_decref.stub = true;
  asyncifyStubs['_emval_decref'] = undefined;

  /** @type {function(...*):?} */
  function __emval_get_method_caller(
  ) {
  abort('missing function: _emval_get_method_caller');
  }
  __emval_get_method_caller.sig = 'pipi';
  __emval_get_method_caller.stub = true;
  asyncifyStubs['_emval_get_method_caller'] = undefined;

  /** @type {function(...*):?} */
  function __emval_run_destructors(
  ) {
  abort('missing function: _emval_run_destructors');
  }
  __emval_run_destructors.sig = 'vp';
  __emval_run_destructors.stub = true;
  asyncifyStubs['_emval_run_destructors'] = undefined;

asyncifyStubs only appears those 5 times on those lines of generated code.

Removing -s ERROR_ON_UNDEFINED_SYMBOLS=0
Then adding --bind as per the discussion #2372 (comment)
Has moved me forward to now to having the error

WebAssembly.Function is not a constructor
in the Function instrumentWasmImports

  instrumentWasmImports(imports) {
        var importPattern = /^(invoke_.*|__asyncjs__.*)$/;
  
        for (let [x, original] of Object.entries(imports)) {
          let sig = original.sig;
          if (typeof original == 'function') {
            let isAsyncifyImport = original.isAsync || importPattern.test(x);
            // Wrap async imports with a suspending WebAssembly function.
            if (isAsyncifyImport) {
              assert(sig, `Missing __sig for ${x}`);make 
              let type = sigToWasmTypes(sig);
              // Add space for the suspender promise that will be used in the
              // Wasm wrapper function.
              type.parameters.unshift('externref');
              imports[x] = original = new WebAssembly.Function(
                type,
                original,
                { suspending: 'first' }
              );
            }
          }
        }
      },

Is there a flag that i need to be adding to let the codegen know that i want WebAssembly.Function to exist ?

Further Context:
I have a C library, which i am compiling to WASM with Emscripten, I am writing a C++ layer in between C and TS
With the C++ layer hopefully making it a little bit easier to achieve the async function calls that i want to get from TS.

I basically am looking for a way for the WASM to asynchronously send values to the TS, on some event.
Enabling something like Rust channels.

Any guidance is greatly appreciated.

Thank you for your help, and your patience !

@sbc100
Copy link
Collaborator

sbc100 commented Jan 19, 2024

-sASYNCIFY=2 used the experimental JSPI proposal under the hood so most likely your browser doesn't have that enabled. @brendandahl do we not have a nice warning in debug mode for this case?

@sbc100
Copy link
Collaborator

sbc100 commented Jan 19, 2024

I'd still like to understand how you got the original issue with asyncifyStubs being undefined so we can fix that too.

@brendandahl
Copy link
Collaborator

-sASYNCIFY=2 used the experimental JSPI proposal under the hood so most likely your browser doesn't have that enabled. @brendandahl do we not have a nice warning in debug mode for this case?

We don't currently have a nice warning. It got sidetracked on how to properly check if JSPI is supported with the possibility of suspender being removed.

@brendandahl
Copy link
Collaborator

That discussion is taking awhile, so maybe we should just add the warning for how it currently works.

@sbc100
Copy link
Collaborator

sbc100 commented Jan 19, 2024

Sounds like simply checking for WebAssembly.Function would be a reasonable first approximation?

@brendandahl
Copy link
Collaborator

That will check if type reflection is enabled (needed by JSPI), but won't tell us about JSPI. For now, we can check "Suspender" in WebAssembly.

@sbc100
Copy link
Collaborator

sbc100 commented Feb 8, 2024

Could you add this check? Then I think we can close this issue

@woodser
Copy link

woodser commented Feb 8, 2024

I'm also getting this error: asyncifyStubs is not defined (currently trying 3.1.53, but saw this when I tried 3.1.48 too).

Unfortunately I don't think I can remove the -s ERROR_ON_UNDEFINED_SYMBOLS=0 flag because it's expected that only a subset of the codebase is imported, so there have always been undefined symbols.

My linker flags:

-Wall -Wl,--allow-undefined -std=c++11 -Oz \
--bind \
-s MODULARIZE=1 \
-s USE_PTHREADS=0 \
-s 'EXPORT_NAME=\"monero_ts\"' \
-s ERROR_ON_UNDEFINED_SYMBOLS=0 \
-s ASSERTIONS=0 \
-s EXIT_RUNTIME=0 \
-s PRECISE_F32=1 \
-s EXCEPTION_DEBUG=0 \
-s DEMANGLE_SUPPORT=1 \
-s NO_DYNAMIC_EXECUTION=1 \
-s NODEJS_CATCH_EXIT=0 \
-s RESERVED_FUNCTION_POINTERS=5 \
-s EXPORTED_FUNCTIONS='[\"_malloc\",\"_free\"]' \
-s EXPORTED_RUNTIME_METHODS='[\"UTF8ToString\",\"stringToUTF8\",\"lengthBytesUTF8\",\"intArrayToString\",\"getTempRet0\",\"addFunction\"]' \
-s WASM=1 \
-s WASM_BIGINT=1 \
-s ALLOW_MEMORY_GROWTH=1 \

@caiiiycuk
Copy link
Contributor

Have same error on 3.1.56 if -s ERROR_ON_UNDEFINED_SYMBOLS=0

@sbc100
Copy link
Collaborator

sbc100 commented Apr 1, 2024

@caiiiycuk is this a regression? If so it would be great if you could help by bisecting to find out what revision broke this behaviour: https://emscripten.org/docs/contributing/developers_guide.html#bisecting

@caiiiycuk
Copy link
Contributor

@sbc100, sure, here it is:

5fe277a9ce9de4fc6bdc5d55780cafa695b10f5b is the first bad commit
commit 5fe277a9ce9de4fc6bdc5d55780cafa695b10f5b
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date:   Tue Aug 8 00:14:25 2023 +0000

    Roll emscripten from 0538c9dba690 to bc1516253af6 (3 revisions)
    
    https://chromium.googlesource.com/external/github.com/emscripten-core/emscripten.git/+log/0538c9dba690..bc1516253af6
    
    2023-08-07 sbc@chromium.org Mark "missing symbol" stub functions as such (#19990)
    2023-08-07 sbc@chromium.org Remove unnecessary `heapAndOffset` helper function. NFC (#19984)
    2023-08-07 sbc@chromium.org Simplify proxiedFunctionTable. NFC (#19987)
    
    If this roll has caused a breakage, revert this CL and stop the roller
    using the controls here:
    https://autoroll.skia.org/r/emscripten-emscripten-releases
    Please CC dschuff@google.com,wasm-waterfall@grotations.appspotmail.com on the revert to ensure that a human
    is aware of the problem.
    
    To report a problem with the AutoRoller itself, please file a bug:
    https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
    
    Documentation for the AutoRoller is here:
    https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
    
    Tbr: wasm-waterfall@grotations.appspotmail.com
    Change-Id: I481334cb0539840eff7a1e4afbe35c40dc93159c
    Reviewed-on: https://chromium-review.googlesource.com/c/emscripten-releases/+/4757347
    Bot-Commit: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
    Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>

 DEPS | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

@sbc100
Copy link
Collaborator

sbc100 commented Apr 1, 2024

@caiiiycuk can you share the linker flags you are using to reproduce this?

@caiiiycuk
Copy link
Contributor

caiiiycuk commented Apr 2, 2024

@sbc100

    set(LIBS "-sUSE_SDL")
    set(CMAKE_C_FLAGS "-O3 -w ${LIBS}")
    set(CMAKE_CXX_FLAGS "-O3 -w -std=c++11 ${LIBS}")
    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} \
        -O3 ${LIBS} \
        --emit-symbols-map \
        -s ASSERTIONS=0 \
        -s SAFE_HEAP=0 \
        -s TOTAL_MEMORY=134217728 \
        -s INVOKE_RUN=0 \
        -s NO_EXIT_RUNTIME=1 \
        -s ASYNCIFY=1 \
        -s 'ASYNCIFY_IMPORTS=[\"syncSleep\"]' -s ASYNCIFY_WHITELIST=@${CMAKE_CURRENT_LIST_DIR}/gpx/asyncify.txt \
        -s EXPORTED_FUNCTIONS='[\"_main\"]' \
        -s EXTRA_EXPORTED_RUNTIME_METHODS=\"['lengthBytesUTF8', 'stringToUTF8', 'UTF16ToString', 'callMain']\" \
        -s FORCE_FILESYSTEM=1 \
        -s ERROR_ON_UNDEFINED_SYMBOLS=0"
    )

If you want, I can share project with you

@sbc100
Copy link
Collaborator

sbc100 commented Apr 2, 2024

Thanks, I think I have enough info to make a repro case now.

sbc100 added a commit to sbc100/emscripten that referenced this issue Apr 2, 2024
sbc100 added a commit that referenced this issue Apr 2, 2024
impact-maker pushed a commit to impact-maker/emscripten that referenced this issue Apr 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants