Multithread Support with Emscripten #110
Replies: 9 comments 28 replies
-
|
To enable a build with pThreads, currently you need to make sure that all 3x sub-projects are compiled with flags to enable shared memory and multi-threading. The project builds a jolt-static library, so that needs it The project also builds a glue.o, and then finally the wasm module again And set the environment to run in a worker: You will need to copy the /dist/*.worker.js file to your normal JS folder so that the main thread can spawn the child-threads Note: Chrome is fine with loading this on localhost as long as you have COOP/COEP enabled The biggest blocker on enabling threads so far is Safari has problems with the SharedMemory access threads. This causes problems where suddenly collision stops working all together, so usually after running with pThread enabled, you will just have bodies stop responding to collision detection and fall through each other. I run the test every so often, and have been looking at new Safari Preview release notes Non-Preview (existing Safari, iPhone) - Safari 17.2.1, iPhone iOS 17.3.1
Safari Preview - Release 188 (Safari 17.4) - requires HTTPS even on localhost + COEP/COOP:
The usage of custom threads and pools is because of the way that Javascript interfaces with the Jolts internal event system. To some degree it was also me seeing if I could hack my way around the Safari bug, but also was to give myself extra hooks into the Thread Spawn logic. Even if it does become stable, you'd want to ensure that you actually get a benefit from running the physics in more than 1x thread. Even with single-threading the physics, you can still setup to use an external SharedMem blob to syncs Position/Rotations and run [UI Main Thread, 3D Render Thread, PhysicsThread] without multi-threading the Physics Library |
Beta Was this translation helpful? Give feedback.
-
|
To see some of the initial problems in the bottom issue described, run a multi-threaded build with the following added to the bottom of your examples (such as 2d_funnel) const contactListener = new Jolt.ContactListenerJS();
contactListener.OnContactValidate = (body1, body2, baseOffset, collideShapeResult) => {
return Jolt.ValidateResult_AcceptAllContactsForThisBodyPair;
};
contactListener.OnContactAdded = (body1, body2, manifold, settings) => {};
contactListener.OnContactPersisted = (body1, body2, manifold, settings) => {};
contactListener.OnContactRemoved = (subShapePair) => {};
physicsSystem.SetContactListener(contactListener);Upon running in debug, you will see errors thrown of We can emulate that behavior by modifying the self.startWorker = (instance) => {
setTimeout(() => {
Module = instance;
// Notify the main thread that this thread has loaded.
postMessage({ 'cmd': 'loaded' });
// Process any messages that were queued before the thread was ready.
for (let msg of messageQueue) {
handleMessage(msg);
}
// Restore the real message handler.
self.onmessage = handleMessage;
})
};After this, the code works kind-of as expected, but all workers will throw the expected Errors due to Currently this will fail almost all JS-callback dependent demos: PathConstraint, ConveyorBelt, Buoyancy, ContactListener |
Beta Was this translation helpful? Give feedback.
-
|
I thought I'd make it a bit easier to try this out and add an ENABLE_MULTI_THREADING option to CMakeLists.txt following your instructions above (see branch) but when I try to build it it fails in the very last step with: I'm running: Any ideas? |
Beta Was this translation helpful? Give feedback.
-
|
@jrouwe Technically it should be possible in regular JS, but the |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the super in-depth response! I appreciate your insights and all the work you've clearly done on this path. Regarding your main blocker, running in safari:
Is this only an issue on Mobile safari or is the desktop version showing similar issues? Have other platforms/orgs discussed similar issues with Obviously, I'd like to see everything working perfectly on every platform, but if Mobile Safari is the only platform with such issues then I think it not a bad idea to move forward with a working threaded version. We could disclaimer the limitation for Safari continue to test/work towards safari working properly. If we had a multi-thread version we could build examples for the different usecases/setups and deal with other issues like the ones you mentioned in your second reply. Once safari multi-threading became stable they could use the same examples/docs for their setup...
Totally agree. For me, it's adding/removing bodies to the system as well as syncing position data that would be most advantageous. Also having a single point to listen to contact/body events without having to pipe events through |
Beta Was this translation helpful? Give feedback.
-
|
Safari on desktop also appears to have the same issues, though I need to OS update to get the very latest revision of Safari. Those builds may want to be their own distro target, like wasm/wasm-compat, although I think they haven't been too unstable when launched with a max thread of 1x Regarding the events, many collision listeners methods have a response, like contact-validate, or use ioParameters allowing modifications in the middle of a worker thread's Job, so you if those are used, you need to design in a way that the worker threads can correctly respond to those instead of just firing off an event. Likely you would not want to use postMessage and await a reply from a single logic processing thread or you've erased all the parallel benefits. It is possible to never use any of those fields, in which case it's not an issue. Also, anything living on the C++ layer is automatically handed across threads. Ex: Collision Filter and Sensor flags can substitute for some (but not all) contact validation use cases. Compiling your own needed logic against the source is a viable solution in some special cases as well. If custom JS classes are used, likely the ideal solution is to design such that you can replicate the exact custom collision logic to the workers, ex: serialize your constructors and pass them to the threads or use a shared buffer, so that all workers know how to correctly respond the exact same way given what limited logic they know. |
Beta Was this translation helpful? Give feedback.
-
|
I'd need to double check, but from my experience, without some custom Jolt C++ side job code, a pthread worker will never respond to a postMessage once it's started, since pThreads are a single routine that starts and never stops or pauses unless instructed from the C++ side. This means for practical tasks you'd either need to build and pass everything over prior to initializing and running the first Step, or we'd need to design some custom Job task worker. |
Beta Was this translation helpful? Give feedback.
-
|
I didn’t mean to distract with my to keep this thread on track, what is your test setup? I can run it against safari latest on desktop and mobile and see if I can replicate the results. also I like the idea of multi-thread being a build target, what would the steps be to make that happen? |
Beta Was this translation helpful? Give feedback.
-
|
To build your own version, the reference branch from jrouwe [https://github.com/jrouwe/JoltPhysics.js/tree/feature/multithreading] works as-is, if you change line 26 to only build wasm-compat using the --target param to avoid the binareyn wasm2js atomics issue we mentioned: cmake --build Build/$BUILD_TYPE -j`nproc` --target jolt-wasm-compatThen run the command provided above.: sh build.sh Distribution -DENABLE_MULTI_THREADING=YESYou need to specify 'Distribution' as the arg2 and not just rely on it defaulting to Distribution, because it blindly takes arg2 as the script $BUILD_TYPE, and will build a CMake build literally named You will need to copy the resulting WIthout modifying any of the Examples, you can produce issues using the Regarding developers not needing access to workers, internally the WASM threads can access each others WASM/C++ memory, but nothing on each others JS layer, so Each thread will have it's own JS Listener callbacks trigger and would be expected by Jolt to independently handle and respond to them as required. Jolt expects from the C++ side that shared-mem means shared functions, so this logic is sound from a C++ viewpoint. Regarding a kind of flexible solution to the worker-sync issue, I think we can subclass JobSystemThreadPool and create a EM_JS( void, __synchronize_jolt_worker, (), { globalThis.synchronizeWorker && globalThis.synchronizeWorker() });I'm not sure on the best logic or location of calling it, but possibly it can be called in the ThreadMain before the My PostMessage comment itself was just as a callout to anyone who uses Worker Threads in JS, that once the pThread starts and the thread-pool is running, the Thread is essentially locked in a while(true) loop and will never respond to you from the JS side, even with a postMessage, which JS developers may expect would work. Someone who uses VanillaJS may expect that as a trivial solve, they would just add a logic step every cycle to postMessage pass over any pointers from the main thread and ensure they sync everything to the workers before they run the next Physics System Step. But, if just using emscripten's default behavior, that strategy won't work. |
Beta Was this translation helpful? Give feedback.
-
|
I was able to update Safari and confirm the existing issue on Desktop - Desktop Safari 17.3.1, Released January 22, 2024 This is even with setting a 0-thread-count on it and spawning 0-workers, though Safari works fine if using the compiled single-threaded Jolt. |
Beta Was this translation helpful? Give feedback.
-
I think I have an idea why this happens, I had to increment the thread pool size and match it with the max threads that the job scheduler will take or else I got hangs in Chrome as well. Maybe you can try again with the new code and see if it is fixed now? Anyway the change is now merged and when enabled it gives a very good speed boost (the stress test runs 15fps without it and 60fps with it on my machine). B.t.w. I tried making a web server that enables COOP/COEP but this is googled together, let me know if I did it totally wrong (especially how you're supposed to work with packages is a mystery to me). |
Beta Was this translation helpful? Give feedback.
-
|
The new CMake/Build script appears to work end-to-end for me. The only change I would recommend as a Node standard is to use either "description": "Serves the JoltPhysics.js examples",
"main": "serveexamples.js",
"scripts": {
"serve": "node serveexamples.js"
},After that, this worked fine |
Beta Was this translation helpful? Give feedback.
-
|
All the headers we need for COOP/COEP are showing for me as well: The fact that CharacterVirtual is not multi-threaded itself since it runs outside the physics Step call at least gives some flexibility in character-driven games, and if you only ever care about only the character hitting things in callbacks, that route can work for an entire game, but the Vehicles dependency on JS physics callbacks fails any cars and motorcycles driving games. (without a fix on the workers) |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the tip! Did the Safari Preview version work without hanging now as well? |
Beta Was this translation helpful? Give feedback.
-
|
I hadn't had too much of a problem with that prior to your modification on Safari Build 189, as long as it was a Distribution build with no bugs. Safari really doesn't do well with trying to load a debug instance or any other large JS files, even for regular front-end web development. The Safari Preview status seems to change every-other-week, given it is the rolling every-2-week webkit build (https://webkit.org/blog/). If there is ever a crash in a thread, likely you need to terminate the entire tab and restart another one from scratch because the entire process can lock up. That also sometimes seems to be the case in single-threaded Chrome/Firefox sometimes as well, especially if a debug-build with how massive debug-binaries are, or if you frequently recompile a Distro build. I think as long as the wasm binary doesn't change, the browser will just use the cached bytecode compile in the process, but if you rebuilt the wasm module, the browser will recompile wasm-to-bytecode but not always free the old tab-process memory, until finally Jolt instantiates but is only give a 0-byte heap and mallocs will fail on soft reload of the page. Crashes in any WASM code seem to also leak memory in a tab-process over time, so frequent WASM crashes will also destabilize a browser-tab-process until it needs to be killed and restarted (duplicate-tab seems fine) Parallelism: |
Beta Was this translation helpful? Give feedback.
-
|
A perk of the PTHREAD_POOL_SIZE logic is that it will pre-init all the threads, then pause until you actually create the physicsSystem. This gives both the ability to fully start up the thread and Jolt, and you almost have a solid barrier where you can configure the worker threads. Taking that into account, when you do a multi-threaded JS run of Jolt, you can pass a custom script that will only run on the workers in the joltInit This means that we have a solid block where between when threads are created ( initJolt) and when they are run (new JoltInterface). The kind-of-zero communication logic that you can use (and I have in the example below) is that you can overwrite the WebIDL cache with a JS Proxy, meaning that any arbitrary request for a C++ Pointer will return whatever JS Interface code you want every time. This technique uses mangled names if the library is distro-mode due to minification, but there are ways around that to prevent them from being minified away, using the -post-js emscripten compiled parameter. If this proxy concept wasn't so hacky, that could be handled with a custom setClassProxy method to the Wasm Module Interface. I'm still not solid on a good way to pass data worker thread, mainly pointers, with emscripten's internal scoping of everything Worker related. I was tempted to use BroadcastChannel, but there's not a good way to barrier until all 16 workers are running then send the data and then barrier until they are done. I'd tried working through LocalStorage, BroadcastChannel, IndexedDB ... none are exactly clean, and the solid cleanest way is recoding the Worker JS to add a new Note: The code on this branch is * not * the way you'd want to actually do this approach, I just wanted a temp branch to show a way that gets you 80% of the way there. Even if you did plan to just proxy all JS Listeners of a specific class, you'd probably still want to an actual new JS file instead of dynamic JS-blob of text. |
Beta Was this translation helpful? Give feedback.
-
|
Worst case, we tack on a static |
Beta Was this translation helpful? Give feedback.
-
|
I am not sure if it is of help, but want to share my setup, where I run Jolt in a webworker. I do allow JS callbacks, but without going to main thread. The main concept is that on main thread I stringify the JS function that is supposed to become a callback, send to worker, and evaluate it to turn it back into an executable JS function. // main thread
function onContactAdded(b1Pointer, b2Pointer, manifoldPointer, settingsPointer) {
// custom code
}
const overrides = {
onContactAdded: new String(onContactAdded),
onContactValidate: ...
};
// send overrides to worker// worker
listener.OnContactAdded = eval('(' + funcStr + ')').bind(this);This way the worker can have a custom JS callback, but without a need to talk to main thread. This is where the User Data fields become useful on the objects. Since these callback have no access to main thread or the app scope, they can use User Data fields to toggle a branch, e.g.: function onContactSolve(character, ...) {
...
const allowSliding = character.GetShape().GetUserData() === 1 ? true : false;
if (!allowSliding && ...) {
...
}
} |
Beta Was this translation helpful? Give feedback.
-
|
What is your worker setup? My current limit is this the passing of anything given the worker is launched deep in the emscripten code and the worker pointers, Is this Jolt running in actual multithreading compiled mode, or running single threaded but executing in its own worker, or possibly a combo (shared memory, single thread compile, be careful) |
Beta Was this translation helpful? Give feedback.
-
|
I see. Sounds like a pain to access them then :( Yeah, my setup is simple - single thread, executed in own worker (non-emscripten worker). |
Beta Was this translation helpful? Give feedback.
-
|
iOS 17.4 now appears to be public. This has the Safari fix that allows for multi-threading, so multithreading is theoretically now available on iPhones. I've updated and confirmed the 16-thread-pool build (enable_multi_threading) works on my phone. The challenge would be how to filter so that iOS 17.3 devices do not attempt to load the multithread builds and fail. ++ Edit: (note, these don't call out anything WASM-Multhreading-Related) |
Beta Was this translation helpful? Give feedback.
-
|
I have initial work on recoding all JS dependent examples to work in multi-threading. Most of them do a no-op or just duplicate blindly the logic of a callback in the worker thread. The one that actually needs to signal back is the buoyancy demo that needs to inform the main thread if the ball is in the moving or still water, so that uses the SharedMemory. I currently do not have the SharedMemory doing more than storing a fixed value, but the intention was for it to store a pointer to a memory block that all threads could collaborate in. https://github.com/PhoenixIllusion/JoltPhysics.js/tree/feature/multi-thread-examples |
Beta Was this translation helpful? Give feedback.
-
|
Do you have a resource for the updates to this safari version? |
Beta Was this translation helpful? Give feedback.
-
|
Not sure there is a reliable way to detect it. Apple is doing all sorts of nonsense to avoid webpages tracking a user via specific hardware fingerprints. Usually what worked were detecting some major features, e.g. if https://developer.apple.com/documentation/safari-release-notes/safari-17_4-release-notes |
Beta Was this translation helpful? Give feedback.
-
|
According to the spec they added support for "
So this would work: const arrayBuffer = new ArrayBuffer(8);
const detachedExists = 'detached' in arrayBuffer;
console.log('detached exists', detachedExists); |
Beta Was this translation helpful? Give feedback.
-
|
Prototype checks are usually enough vs allocating and checking: !!ArrayBuffer.prototype.detachedThat specific case will fail on Firefox, which hasn't added There are edge cases (likely not on ArrayBuffer) where checking prototype vs instantiated objects this may cause bugs, like caused by prototype pollution, or special cases where instantiating a class may have special checks in the constructor that forcefully reject/delete/freeze attempts to modify, but those are usually only in the sensitive security sections of the browser or other libraries. |
Beta Was this translation helpful? Give feedback.
-
|
If example updates are desired, I'm not sure how we would want to update the examples to support this new build variant I believe the required update is very minor for most cases, mainly just making sure anything important tags the Bodies with a UserData, but I do not know if it would be confusing to have it mixed in the middle of single-threaded versions. In the best cases, it is just having an optional worker-loader JS file, and you can blindly pass that URL to the initJolt() call and it will never be used unless the code kicks off workers. The three most complicated ones are:
|
Beta Was this translation helpful? Give feedback.
-
This just meant to do it somehow on the JavaScript side in the examples instead of it being part of the library. |
Beta Was this translation helpful? Give feedback.
-
|
An ugly method, but one that is mentioned in emscripten-core/emscripten#20235 Technically, if you don't have the Thread Pool, you would use [allocate unused workers] to create the pool on-demand yourself with that method. To access all the threads in the pool, I added another post-js method: if(Module['PThread']) {
Module['PThread']['getAllWorkers'] = () => ([... PThread.unusedWorkers, ... PThread.runningWorkers]);
}Then in the worker, used the pattern in the Pull Request that several of their pThread issues said was a 'standard' way to daisy-chain worker startups: // extra *.worker.js file, passed to { mainScriptUrlOrBlob: string }
import initJolt from '../jolt-physics.wasm-compat.js';
export default async (module) => {
const Jolt = await initJolt(module);
const handleMessage = globalThis.onmessage;
const messageQueue = [];
await new Promise(resolve => {
function continueLoading() {
for (let msg of messageQueue) {
handleMessage(msg);
}
globalThis.onmessage = handleMessage;
resolve();
}
globalThis.onmessage = (msg) => {
if(msg.data.cmd == 'js-shared') {
const { sharedBuffer, index} = msg.data;
// TODO: use or store SharedArrayBuffer as needed
continueLoading();
} else {
messageQueue.push(msg);
}
if(msg.data.cmd == 'run') {
continueLoading();
}
}
})
return Jolt
}Then prior to Jolt Interface constructor, you can pass the SharedArrayBuffer to the children threads. They may not all load before the first call to Disabling the ThreadPool in the compile, but exposing the PThread.allocateUnusedWorker() on one hand would mean that the same build will run in Single-Threaded mode on non-COOP, but on the other it also requires another work around. |
Beta Was this translation helpful? Give feedback.
-
|
After staring at this code for 10 mins, I think I get what it does (not sure what Github pages doesn't seem to support COOP/COEP so if we enable multi threading on the demos at https://jrouwe.github.io/JoltPhysics.js/ it won't work (although there seem to be hacky workarounds for this). This means we may need a solution that allows a single threaded fallback (or we just create a separate build for multithreading vs non-multithreading). |
Beta Was this translation helpful? Give feedback.
-
|
I had not posted an example of using the new method yet, no. That would be something like: import initJolt from './js/jolt-physics.wasm-compat.js';
// point to a script with the awaiting Workers, that will sit and wait for a Shared Buffer
initJolt({ mainScriptUrlOrBlob: './worker/load_shared_array.worker.js' }).then(function (Jolt) {
// allocate a shared memory buffer
const sharedBuffer = new SharedArrayBuffer(24);
// use the new method, with newly exposed direct access to Workers, to post a reference
// of the SharedArrayBuffer to all of them
Jolt.PThread.getAllWorkers().forEach((worker, i) =>
worker.postMessage({cmd: 'js-shared', sharedBuffer, index: i }));
// This command here will create the PhysyicsSystem, which is an active 'execute thread' and
// will lock the Workers going forward in the while(true) loop. All threads must at least have a
// pending postMessage of the SharedArrayBuffer before this or forever not have access to
// it until you destroy and recreate a new PhysicsSystem
initExample(Jolt);
|
Beta Was this translation helpful? Give feedback.
-
|
Ok, I understand. Let me know what you think is the best way to proceed (if at all). Perhaps add a new version to the package called |
Beta Was this translation helpful? Give feedback.
-
|
I'm having a little bit of a problem following at the end here, but it seems like we're trying to solve many problems at once. In the future the library could be responsible for deciding which version of the platform to use but for now leave it to the end user. I think that means we need a basic build like @jrouwe mentions Right now, I have to build my own communication api/channels and run Jolt on a separate worker to not have it block the rendering thread. How do we build a simple (understandably limited) version of the Falling Shapes demo that does that? How can we demonstrate the benefits of being multi-threaded? crank the shapes up to 1,000? Is that possible? |
Beta Was this translation helpful? Give feedback.
-
|
For anything that does not actually have callbacks (setup so that during JoltInterface Step, Jolt is a black box), such as Friction, SoftBody, Rigged, Constraints (non-path), Falling Shapes, Add/Remove, 2D Funnel, Stress Test (possibly best example for this), [non-exhaustive list] there is no JS code change or special instruction needed other than building with the ReadMe flags and ensuring the worker-js is in the JS folder. Regarding browser notes, although this is experimental, it should work equally on newest Safari 17.4, Chrome, Firefox, and Chromium-powered-Edge, so we would not need to state chrome-only. The only spike in difficulty is if you require even one callback during the physics step, given it is occurring potentially on any spawned external worker process. |
Beta Was this translation helpful? Give feedback.
-
I saw the other discussion HERE but that person is trying to implement their own pooling/situation without the use of Pthreads.
As Emscripten supports pThreads and C++ Threads are fairly common in WebAssebly I'm wondering what it would take to get this functionality in Jolt.
From my limited understanding,
std::threadsis built on-top ofpThreadsso from other examples I've seen should compile with Emscripten without too much hassle. There are limitations and things that might have to be done but I don't think it as big of a task as that other user.From the feature table Threads are well supported in all major browsers, so this looks like a doable thing.
One thing to note, is regardless of thread support with Jolt, Web users would likely still be required to use workers to access things as the point with Jolt is we can access them INSIDE other threads. But this way at least, we would push a lot of the body/object tracking back into jolt vs doing it all in duplicate.
I haven't seen anything in the jolt docs that would let you get like 4 threads going for collision detection vs 1, but that's probably due to my lack of C++ skills.
Beta Was this translation helpful? Give feedback.
All reactions