New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stack overflow error when large string passed #6860

Closed
h-matsuo opened this Issue Jul 15, 2018 · 1 comment

Comments

Projects
None yet
1 participant
@h-matsuo

h-matsuo commented Jul 15, 2018

I uploaded some basic code on my gist to replicate my situation.
Please clone the gist and test my codes.


Symptom

I wrote a C sample code (sample.c) to count the length of input string.
I used Emscripten to compile sample.c and WebAssembly bytecode was successfully generated.

However, when I pass a string of 1.3 MB or more to the C function, "Stack overflow" error will be risen.
I tried building with -s ALLOW_MEMORY_GROWTH=1 but nothing was changed.
You can see the full console messages here.

How can I pass a large data to the C function?
I want to pass dozens MB of input data ultimately.

How to replicate

  1. Install docker and Google Chrome.

  2. Clone my gist:

    $ git clone git@gist.github.com:ac360733ae4a69afb4fc9f3a24bc0206.git wasm-issue-sample
    $ cd wasm-issue-sample
  3. Run nginx-up.sh:

    $ ./nginx-up.sh
  4. Open localhost:8000 in Chrome and you will see "Stack overflow" error message in console.

You can easily change the length of input string by editing index.html.

Environment

  • Google Chrome: 67.0.3396.99
  • OS: macOS 10.13.6 High Sierra
  • Emscripten: 1.38.8
    emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 1.38.8
    clang version 6.0.1  (emscripten 1.38.8 : 1.38.8)
    Target: x86_64-unknown-linux-gnu
    Thread model: posix
    InstalledDir: /emsdk_portable/llvm/clang/bin
    Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8
    Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8.4
    Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9
    Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.2
    INFO:root:(Emscripten: Running sanity checks)
    WARNING:root:java does not seem to exist, required for closure compiler, which is optional (define JAVA in /emsdk_portable/data/.config if you want it)
    WARNING:root:closure compiler will not be available
    

@h-matsuo h-matsuo changed the title from Stack overflow error when > 1.2 MB string passed to Stack overflow error when > 1.3 MB string passed Jul 15, 2018

@h-matsuo

This comment has been minimized.

h-matsuo commented Jul 18, 2018

I self-solved this issue. I'll write down what I did.
Thank you for great project, @kripken !


Cause

Essentially what I want to do is passing a very large string from JavaScript to the compiled C function.
Here is the C function declaration:

// sample.c
void sample(const char*);

There are many ways to pass an argument to this sample().
At first, I directly passed the string to Module.cwrap() but this will invoke stack overflow:

// NG
const sample = Module.cwrap('sample', null, ['string']);
sample(veryLargeString); // stack overflow error!

In this case, passed string will be on Emscripten's stack.
However, the stack will not automatically grow even if built with -s ALLOW_MEMORY_GROWTH=1 (#4344).
It is not suitable for large input data to interact through the stack.

Solution

I used Module._malloc() and Module.stringToUTF8() to put the large data on Emscripten's heap:

// OK
const bufferSize = Module.lengthBytesUTF8(veryLargeString);
const bufferPtr = Module._malloc(bufferSize + 1);
Module.stringToUTF8(veryLargeString, bufferPtr, bufferSize + 1);

The heap can automatically grow when built with -s ALLOW_MEMORY_GROWTH=1, so no errors will be risen.
After allocating input data on the heap, I can pass the data to the function using pointer:

// OK
const sample = Module.cwrap('sample', null, ['number']); // not 'string', pointer is a number
sample(bufferPtr);
Module._free(bufferPtr);

@h-matsuo h-matsuo closed this Jul 18, 2018

@h-matsuo h-matsuo changed the title from Stack overflow error when > 1.3 MB string passed to Stack overflow error when large string passed Jul 18, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment