Memory allocation problem

[MidnightJava]

I get the following error on my console after several minutes of processing vorbis data streaming to the client.

Cannot enlarge memory arrays. Either (1) compile with -s TOTAL_MEMORY=X with X higher than the current value 16777216, (2) compile with ALLOW_MEMORY_GROWTH which adjusts the size at runtime but prevents some optimizations, or (3) set Module.TOTAL_MEMORY before the program runs.

I don't know if this means that I have a memory leak, or that the setting used when compiling vorbis for the js port needs to be higher, but it will not grow indefinitely. I grabbed the prebuilt vorbis.js from here, and I'm having problems getting the build to work as documented on the web site.

What is the best way to get a version of vorbis.js compiled per the options given in the error message?

[devongovett]

Interesting. Does it happen with a specific file or every file you test?

Emscripten defaults to 16 MB for its heap size by default. We use 16 KB for our output buffer, plus whatever libvorbis uses internally.

[MidnightJava]

I'm not playing a file, but rather streaming data indefinitely via WebSocket from a server running a software-defined radio. I believe that aurora-websocket treats the stream as a single file, so maybe I'm running into a problem streaming a file longer than the typical media file? I didn't time it, but I beleive it ran for 10 or 15 min before failing.

As I mentioned, I'm using the pre-built version of vorbis.js. I'd like to build my own version so I can change the compile parameters. Here's what happens when I try to build.

1. I installed emscripten, apparently successful, and sourced the emsk_env.sh file to set the PATH.
2. npm install produces the following error output (std out was re-directed)

sed: can't read s/-O20/-O2/g: No such file or directory
sed: can't read s/-O4/-O2/g: No such file or directory
configure: error: No 16 bit type found on this platform!
ERROR    root: Configure step failed with non-zero return code 1! Command line: ['./configure', '--prefix=/opt/git/vorbis.js/libogg', '--disable-static'] at /opt/git/vorbis.js/libogg
sed: can't read s/$srcdir\/configure/#/: No such file or directory
sed: can't read s/-O20/-O2/g: No such file or directory
sed: can't read s/-O4/-O2/g: No such file or directory
sed: can't read s/$ac_cv_func_oggpack_writealign/yes/: No such file or directory
configure: error: Ogg >= 1.0 required !
ERROR    root: Configure step failed with non-zero return code 1! Command line: ['./configure', '--prefix=/opt/git/vorbis.js/libvorbis/build', '--disable-oggtest', '--disable-static', '--with-ogg=/opt/git/vorbis.js/libogg', '--with-ogg-libraries=/opt/git/vorbis.js/libogg/lib'] at /opt/git/vorbis.js/libvorbis
libtool: install: warning: relinking `libvorbisfile.la'
libtool: install: warning: relinking `libvorbisenc.la'
WARNING  root: emcc: cannot find library "ogg"
WARNING  root: emcc: cannot find library "vorbis"
Traceback (most recent call last):
  File "/opt/emsdk_portable/emscripten/master/emscripten.py", line 1655, in <module>
    _main(environ=os.environ)
  File "/opt/emsdk_portable/emscripten/master/emscripten.py", line 1643, in _main
    temp_files.run_and_clean(lambda: main(
  File "/opt/emsdk_portable/emscripten/master/tools/tempfiles.py", line 39, in run_and_clean
    return func()
  File "/opt/emsdk_portable/emscripten/master/emscripten.py", line 1651, in <lambda>
    DEBUG_CACHE=DEBUG_CACHE,
  File "/opt/emsdk_portable/emscripten/master/emscripten.py", line 1535, in main
    shared.Building.ensure_struct_info(struct_info)
  File "/opt/emsdk_portable/emscripten/master/tools/shared.py", line 1758, in ensure_struct_info
    gen_struct_info.main(['-qo', info_path, path_from_root('src/struct_info.json')])
  File "/opt/emsdk_portable/emscripten/master/tools/gen_struct_info.py", line 505, in main
    struct_info = inspect_code(header_files, cpp_opts, structs, defines)
  File "/opt/emsdk_portable/emscripten/master/tools/gen_struct_info.py", line 380, in inspect_code
    info = subprocess.check_output([shared.LLVM_INTERPRETER, bin_file[1]]).splitlines()
AttributeError: 'module' object has no attribute 'check_output'
Traceback (most recent call last):
  File "/opt/emsdk_portable/emscripten/master/emcc", line 1524, in <module>
    flush_js_optimizer_queue()
  File "/opt/emsdk_portable/emscripten/master/emcc", line 1430, in flush_js_optimizer_queue
    run_passes(chunks[0], title, just_split=False, just_concat=False)
  File "/opt/emsdk_portable/emscripten/master/emcc", line 1396, in run_passes
    final = shared.Building.js_optimizer(final, passes, jcache, debug_level >= 4, js_optimizer_extra_info, just_split=just_split, just_concat=just_concat)
  File "/opt/emsdk_portable/emscripten/master/tools/shared.py", line 1629, in js_optimizer
    ret = js_optimizer.run(filename, passes, NODE_JS, jcache, debug, extra_info, just_split, just_concat)
  File "/opt/emsdk_portable/emscripten/master/tools/js_optimizer.py", line 431, in run
    return temp_files.run_and_clean(lambda: run_on_js(filename, passes, js_engine, jcache, source_map, extra_info, just_split, just_concat))
  File "/opt/emsdk_portable/emscripten/master/tools/tempfiles.py", line 39, in run_and_clean
    return func()
  File "/opt/emsdk_portable/emscripten/master/tools/js_optimizer.py", line 431, in <lambda>
    return temp_files.run_and_clean(lambda: run_on_js(filename, passes, js_engine, jcache, source_map, extra_info, just_split, just_concat))
  File "/opt/emsdk_portable/emscripten/master/tools/js_optimizer.py", line 351, in run_on_js
    pre_1, pre_2 = pre.split(start_asm)
ValueError: need more than 1 value to unpack

The root problem seems to be the sed errors, but I have no idea what's happening there. I run npm as root user (since it installs things in /usr/local/bin), with Emscripten env setup for the root user and the PATH set.

[MidnightJava]

The above was happening on my CentOS system at work. I was able to build libvorbis.js successfully on my OSX system at home. I'd like to compile it with ALLOW_MEMORY_GROWTH, and I'll try it on my work computer on Monday. I just noticed that I can set Module.TOTAL_MEMORY in the javascript file, without having to set a compiler option. But is that in vorbis.js or llibvorbis.js?

Also can you tell me how I set the compiler option ALLOW_MEMORY_GROWTH?

[MidnightJava]

Running the files built on my OSX system at home on the CentOs system at work fails. It was worth a shot, but it looks like cross-compiling is not readily workable. Compiling on CentOs is a lost cause because I have no idea what the sed errors mean.

So I tried setting Module.TOTAL_MEMORY=[some value]. That doesn't work either. The instructions say to set it after the program started. So I tried setting it in the aurora-websocket script and in the script for my app. The setting is not recognized, because exactly 13 min and 25 seconds into playback, it throws the memory allocation error, and reports the same value for the current setting of TOTAL_MEMROY. I want to see if increasing the value gets me to a steady state, though it seems there is a memory leak.

I'm getting Vorbis decode errors -135, and my colleague who wrote the encoder on the server is trying to figure out why. His test decoder on the server reports no problems, but in the client app I get a decode error on almost every frame, though the audio sounds good (until it dies). Could the decode errors be causing the memory errors?