Add WebAssembly (wasm) target for scsynth #5309

Sciss · 2021-01-01T20:38:22Z

Purpose and Motivation

Types of changes

Documentation
Bug fix
New feature

To-do list

Code is tested
All tests are passing
Updated documentation
This PR is ready for review

- introduce a hypothetical new audio api 'openal' - some EMSCRIPTEN conditionals - ignore missing boost pthreads for now The cmake command for now is: $ emcmake cmake -DSC_WII=no -DSC_EL=no -DSUPERNOVA=no -DSC_HIDAPI=no -DNO_LIBSNDFILE=yes -DSC_QT=no -DNO_AVAHI=yes -DSC_ABLETON_LINK=no -DCMAKE_BUILD_TYPE="Release" -Wno-dev -s USE_PTHREADS=1 -DPTHREADS_LIBRARY=ignore --target scsynth ..

- until the missing audio API is detected

- add some more preprocessor conditionals - now we get an initial list of undefined symbols (mostly: initialization of audio API, some pthread, some file system)

- we have unstaged commits in `external_libraries/nova-tt`; thus next step will be to commit them in a fork of nova-tt

- use nova-tt fork (branch wasm) - I ran `update --remote`; was this a good idea?

- make succeeds now

- seems we had flags on emcmake that were never passed to emcc - we have a strange linker error regarding shared memory - now with shared memory actually enabled, we cannot run in Firefox without user adjustments - in Chromium, there is an error enumerating audio/video devices - some problem now in boost at boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>::priv_add_segment(void*, unsigned long) (http://0.0.0.0:8000/scsynth.wasm:wasm-function[4452]:0xceebe) This may be a problem due to assuming _no_ shared memory in previous preprocessor tuning; perhaps we need to remove those diversions again

``` abort @ scsynth.js:1475 ___assert_fail @ scsynth.js:2082 boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>::priv_add_segment(void*, unsigned long) @ scsynth.wasm:1 boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>::rbtree_best_fit(unsigned long, unsigned long) @ scsynth.wasm:1 boost::interprocess::segment_manager_base<boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul> >::segment_manager_base(unsigned long, unsigned long) @ scsynth.wasm:1 boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index>::segment_manager(unsigned long) @ scsynth.wasm:1 boost::interprocess::ipcdetail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index, 16ul>::create_impl(void*, unsigned long) @ scsynth.wasm:1 boost::interprocess::ipcdetail::create_open_func<boost::interprocess::ipcdetail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index, 16ul> >::operator()(void*, unsigned long, bool) const @ scsynth.wasm:1 void boost::interprocess::ipcdetail::managed_open_or_create_impl<boost::interprocess::shared_memory_object, 16ul, true, false>::priv_open_or_create<boost::interprocess::ipcdetail::create_open_func<boost::interprocess::ipcdetail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index, 16ul> > >(boost::interprocess::ipcdetail::create_enum_t, char const* const&, unsigned long, boost::interprocess::mode_t, void const*, boost::interprocess::permissions const&, boost::interprocess::ipcdetail::create_open_func<boost::interprocess::ipcdetail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index, 16ul> >) @ scsynth.wasm:1 boost::interprocess::ipcdetail::managed_open_or_create_impl<boost::interprocess::shared_memory_object, 16ul, true, false>::managed_open_or_create_impl<boost::interprocess::ipcdetail::create_open_func<boost::interprocess::ipcdetail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index, 16ul> > >(boost::interprocess::open_or_create_t, char const* const&, unsigned long, boost::interprocess::mode_t, void const*, boost::interprocess::ipcdetail::create_open_func<boost::interprocess::ipcdetail::basic_managed_memory_impl<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index, 16ul> > const&, boost::interprocess::permissions const&) @ scsynth.wasm:1 boost::interprocess::basic_managed_shared_memory<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index>::basic_managed_shared_memory(boost::interprocess::open_or_create_t, char const*, unsigned long, void const*, boost::interprocess::permissions const&) @ scsynth.wasm:1 detail_server_shm::server_shared_memory_creator::server_shared_memory_creator(unsigned int, unsigned int) @ scsynth.wasm:1 World_New @ scsynth.wasm:1 ```

- somehow it hangs if we call `EventLoop::run([world]() { World_WaitForQuit(world, true); });` so skip that for now

- add a 'boot' button so we don't run into the problem of having an AudioContext that doesn't start

- so all pieces are in place, OSC messages seem to be correctly encoded - audio driver start/stop now in place - but we get these errors: Uncaught TypeError: Failed to execute 'copyToChannel' on 'AudioBuffer': The provided ArrayBufferView value must not be shared. at ScriptProcessorNode.ad.proc.onaudioprocess (scsynth.js:1688) ad.proc.onaudioprocess @ scsynth.js:1688 It seems that because of `-pthread`, the HEAP is now backed by `SharedArrayBuffer`, and that in turn cannot be used in `copyToChannel`. Hopefully, we can just make one additional copy. If not, we're f*cked :-O

- for example, trying to d_recv analog-bubbles, the first UGen encountered, 'Control' is apparently not installed - probably we need to include some binary file for the emscripten virtual file system

- DiskIO is disabled because we currently don't build with libsndfile (perhaps it's possible -- I don't remember why I had to disable it)

- we need to understand how to allocate (and free) the memory for OSC replies. is web_reply_func called from different threads? Can we just use malloc, or should we use rt memory?

- noticed a bug in `NumOutputBuses`

- extraneous constructor argument `int inNumSamples` was causing crashes

- wasm/configure.sh : use release build type (we're still catching exceptions, though)

- probably the originally missing symbols were due to wrong pthread linker settings - go back to timblechmann's repo

- seems the assertion in rbtree_best_fit.hpp does hold now

- build file seems ok for now

- Module.oscDriver is now a dictionary from virtual port number to an object that at least defines the property `receive`, a function taking two arguments: sender's port, and Uint8Array raw OSC packet.

dyfer · 2021-01-04T00:23:49Z

Thank you for this, @Sciss !
I don't have the expertise to do a full review, but I've noticed that submodule updates are part of this PR. Is this intentional? If not, then they should probably be reverted. Feel free to submit a separate PR for submodule updates, if desired.

Sciss · 2021-01-04T00:42:24Z

Not an expert with git submodules: I ran init and update a few times. How would I revert them? And to what commit?

dyfer · 2021-01-04T17:35:40Z

I'm not an expert either...
I think that in order to check which commit they are at on the develop branch, you could switch to that branch, then go to each affected submodule directory, and use git status to check and take note of commit SHA. Then switch back to your PR branch, go to submodule directory again and checkout that commit...
Alternatively you can try deleting submodules and then run update --init --recursive but that sometimes messes things up... see #5306 (comment) (and the rest of that issue, especially if you switched remotes on your submodules)

- needed to do anything more involving like allocating larger buffers

dyfer · 2021-09-20T16:33:59Z

Should this be closed in favor of #5571 ?

dylans · 2021-09-21T07:07:41Z

Should this be closed in favor of #5571 ?

Probably, I retained the full commit history from @Sciss , but didn’t have an easy path to merge the updates back into this PR.

dyfer · 2022-01-19T18:36:17Z

Hello @Sciss
Are you interested in continuing work on this PR?

Sciss · 2022-01-19T18:47:37Z

Definitely interested; I mean I'm using scsynth.wasm. I don't have time resources at the moment, though, so it'll have to wait a bit. I'm happy to comment on questions.

dyfer · 2022-01-19T19:12:31Z

Thanks, got it. Should we close this PR and move the discussion to #5571 (which has all your commits rebased and with resolved conflicts, AFAIU) or should we close that one and keep the discussion here? Just trying to tidy things up a bit.

Sciss · 2022-01-19T19:16:23Z

Yes, makes sense, thank you

dyfer · 2022-01-19T21:31:06Z

(continue conversation in #5571)

Sciss added 30 commits December 21, 2020 19:19

step: some ugens compile (?)

05a9834

- until the missing audio API is detected

step

4a2a73f

- add some more preprocessor conditionals - now we get an initial list of undefined symbols (mostly: initialization of audio API, some pthread, some file system)

disable boost shared memory

dcf0664

pretend the file system is unix

a801d5d

update readme

fc7e8d8

thoughts on how to add the new Audio Driver

fa4882b

step. add blank SC_WebAudio.cpp ; change 'openal' to 'webaudio'

5ee1823

- we have unstaged commits in `external_libraries/nova-tt`; thus next step will be to commit them in a fork of nova-tt

update submodules

613ff99

- use nova-tt fork (branch wasm) - I ran `update --remote`; was this a good idea?

step: dummy impl of SC_WebAudio.cpp

92ef16d

- make succeeds now

add HTML target

14f3815

fix linker/boost problem

f3fe089

giving up on cmake

8e35ca5

step; disable assertion in boost

d305c2d

try with Embind. we get sr and buffer size

df2f452

store audio context

d4d82b4

step. allow omission of -u, -t; set em main loop

470ab94

- somehow it hangs if we call `EventLoop::run([world]() { World_WaitForQuit(world, true); });` so skip that for now

explicit UI boot

50c1db9

- add a 'boot' button so we don't run into the problem of having an AudioContext that doesn't start

add print-error-to-console hook

d04827e

driver setup ok from web audio point of view

cca695c

begin main-loop analogous to Jack and PortAudio

745eac4

we may have a running driver -- but can't test yet without osc

b0b670f

introduce WebInPort (dummy for now)

552bfa8

example OSC messages; UGens are not 'installed'!

3c189a6

- for example, trying to d_recv analog-bubbles, the first UGen encountered, 'Control' is apparently not installed - probably we need to include some binary file for the emscripten virtual file system

plugins are statically linking!

6002639

- DiskIO is disabled because we currently don't build with libsndfile (perhaps it's possible -- I don't remember why I had to disable it)

undo debug printing; update read-me

0adb2b7

improve README_WASM.md; continue with SC_WebPortIn reply

761c29a

- we need to understand how to allocate (and free) the memory for OSC replies. is web_reply_func called from different threads? Can we just use malloc, or should we use rt memory?

Sciss added 13 commits December 31, 2020 13:04

thought on web_reply_func

7ac6e18

implement OSC replies

39abd35

audio input works

153e963

- noticed a bug in `NumOutputBuses`

typo

0c88d87

fix bug in several DelayUGens Ctors

8107ca5

- extraneous constructor argument `int inNumSamples` was causing crashes

prepare for clang-format; web-port-in destructor

676ccbb

Format with clang-format

d5e5154

- wasm/configure.sh : use release build type (we're still catching exceptions, though)

clang-format-8 instead of -9

109dbee

fork of nova-tt no longer needed

f781268

- probably the originally missing symbols were due to wrong pthread linker settings - go back to timblechmann's repo

web-port-in: unused ip is not a 'todo'

dab9ab6

reenable boost assertion (no longer failing)

1e1e6fe

- seems the assertion in rbtree_best_fit.hpp does hold now

remove todo comments from CMakeLists.txt

15e1f5c

- build file seems ok for now

more symmetric OSC interface

9f3d9e5

- Module.oscDriver is now a dictionary from virtual port number to an object that at least defines the property `receive`, a function taking two arguments: sender's port, and Uint8Array raw OSC packet.

Sciss mentioned this pull request Jan 4, 2021

scsynth.wasm crucialfelix/supercolliderjs#92

Open

increase static heap size

89c431a

- needed to do anything more involving like allocating larger buffers

dylans mentioned this pull request Sep 19, 2021

Add WebAssembly (wasm) target for scsynth (rebased) #5571

Open

4 tasks

Sciss closed this Jan 19, 2022

Sciss mentioned this pull request Jan 20, 2022

[servers] porting audio servers for Emscripten/WebAssembly #5224

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add WebAssembly (wasm) target for scsynth #5309

Add WebAssembly (wasm) target for scsynth #5309

Sciss commented Jan 1, 2021

dyfer commented Jan 4, 2021

Sciss commented Jan 4, 2021

dyfer commented Jan 4, 2021

dyfer commented Sep 20, 2021

dylans commented Sep 21, 2021

dyfer commented Jan 19, 2022

Sciss commented Jan 19, 2022

dyfer commented Jan 19, 2022

Sciss commented Jan 19, 2022

dyfer commented Jan 19, 2022

Add WebAssembly (wasm) target for scsynth #5309

Add WebAssembly (wasm) target for scsynth #5309

Conversation

Sciss commented Jan 1, 2021

Purpose and Motivation

Types of changes

To-do list

dyfer commented Jan 4, 2021

Sciss commented Jan 4, 2021

dyfer commented Jan 4, 2021

dyfer commented Sep 20, 2021

dylans commented Sep 21, 2021

dyfer commented Jan 19, 2022

Sciss commented Jan 19, 2022

dyfer commented Jan 19, 2022

Sciss commented Jan 19, 2022

dyfer commented Jan 19, 2022