Bound WASM memory on long outputs + ship a scalar (non-SIMD) wasm fallback - #1
Merged
Conversation
The npm/README.md already covers the full JS/WASM API. Replace the parallel section in the root README with a pointer to it.
renderMono/renderStereo materialise the entire output as one contiguous vector in WASM linear memory and then copy it again into the returned Float32Array, so peak memory is ~2x the output (and both channels at once for stereo). A large stretch (e.g. a few seconds stretched several hundred times into an hour-plus of audio) exceeds the wasm32 heap and aborts. The module was also linked with ALLOW_MEMORY_GROWTH but no MAXIMUM_MEMORY, so the cap was Emscripten's 2 GiB default. - Add OfflineRenderer::render_mono_chunked / render_stereo_chunked, which run the same DSP loop (refactored into shared stream_channel / stream_stereo cores) but deliver the result one bufsize() chunk at a time. render_mono/render_stereo output is unchanged. - Expose renderMonoChunked / renderStereoChunked in the WASM bindings; each chunk is copied out to a fresh JS-heap Float32Array, so peak WASM memory stays ~ input + one chunk regardless of output length. - Raise MAXIMUM_MEMORY to the 4 GiB wasm32 maximum for immediate headroom. - Add TypeScript declarations and an npm README section. - Add tests/chunked_test.cpp: structural parity between the chunked and whole-buffer paths, plus an allocation-free check that a representative extreme render overflows the known wasm32 caps while the chunked working set stays tiny. Extend the browser smoke test to exercise renderMonoChunked.
olilarkin
force-pushed
the
chunked-offline-render
branch
from
June 7, 2026 16:11
e0c520d to
06f61c2
Compare
Ship a second WebAssembly artifact, paulstretch.nosimd.wasm, built without -msimd128 alongside the SIMD paulstretch.wasm. The SIMD build fails to compile in WebViews without WASM SIMD support (notably macOS WKWebView before Safari 16.4 / macOS 13), where the module can't be parsed at all. Consumers feature-detect SIMD at runtime and load the matching binary; both are built with the same Emscripten version so a single glue drives either one. - scripts/build-wasm.sh: build both variants and assemble npm/dist. Deletes the linked output before each variant build so an up-to-date tree still refreshes it (CMake links straight into npm/dist and won't relink an up-to-date target). - npm/package.json: 0.3.0; export + pack dist/paulstretch.nosimd.wasm. - CI/release: build both via the script; smoke-test each wasm in the browser. - README: document the scalar fallback and the feature-detect probe.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Stretching a short sound by a large factor (e.g. ~7 s × 482× → an hour-plus of audio) aborts the WebAssembly module with the generic "build with -sASSERTIONS for more info" message. This is an out-of-memory
abort(), not a logic bug — reported downstream in paulstretch-for-live#2. It happens on the OfflineRenderer path (clicking Apply).Root cause.
renderMono/renderStereobuild the entire output as one contiguousstd::vector<float>in WASM linear memory, thento_js_float32_arrayallocates a second full-size copy — so peak memory is ≈ 2× the output, doubled again for stereo. The module links with-sALLOW_MEMORY_GROWTH=1but no-sMAXIMUM_MEMORY, so the cap is Emscripten's 2 GiB default. A 7 s × 482× stereo render peaks well past that.Fix
OfflineRenderer::render_mono_chunked/render_stereo_chunked(C++) andrenderMonoChunked/renderStereoChunked(WASM). Same DSP — the existing render loops are refactored into sharedstream_channel/stream_stereocores, sorender_mono/render_stereooutput is unchanged — but the result is delivered onebufsize()chunk at a time. Each chunk is copied out to a fresh JS-heapFloat32Array, so peak WASM memory stays ≈ input + one chunk regardless of output length. The consumer accumulates chunks on the JS heap (or streams them to disk / an encoder).-sMAXIMUM_MEMORY=4294967296(4 GiB wasm32 max) for immediate headroom. The chunked API is what removes the dependency on output length.SIMD fallback for older WebViews (fixes paulstretch-for-live#6)
Separately, the production
paulstretch.wasmis a-msimd128build (PFFFTWASM_SIMD128,v128locals). Some WebViews can't parse a SIMD module at all and fail to compile it — notably macOS WKWebView before Safari 16.4 / macOS 13, which aborts withCompileError … can't get function local's type. On those systems the extension is completely dead (preview and render), which is paulstretch-for-live#6.scripts/build-wasm.shbuilds both variants and assemblesnpm/dist:paulstretch.wasm(SIMD) pluspaulstretch.nosimd.wasm(scalar, zero SIMD opcodes). Both are built with the same Emscripten, so the generated glue is byte-identical — onepaulstretch.jsdrives either binary, and consumers feature-detect at runtime (WebAssembly.validateprobe) and pointlocateFileat the matching.wasm.npm/distand won't relink an up-to-date target — otherwise the second copy is a stale wasm from the other variant), and asserts the two binaries actually differ.npm/package.json→ 0.3.0; exports + packsdist/paulstretch.nosimd.wasm. README documents the fallback and the detection probe.Tests
tests/chunked_test.cpp(new):streaming_testconvention), every chunk is exactlybufsize, empty input is a no-op.estimate_output_framesagainst named wasm32 caps) that a representative extreme render's buffered footprint overflows the 2 GiB default cap, and a bigger one overflows even the 4 GiB max, while the chunked working set stays ~16 KB.scripts/build-wasm.shand runs the headless-Chrome browser smoke test once per binary (SIMD and scalar) through the shared glue; the release workflow publishes both.Verified locally
ctestpasses on PFFFT and ACCELERATE backends..wasmreports max 65536 pages (4 GiB).v128/SIMD opcodes; SIMD wasm reportsfftSimdArch()="WASM_SIMD128", scalar reports"4xScalar", both produce identical output through the same glue (Node smoke test).data-status="ok") for both binaries.Opening for CI before merge.