do you have a sense how hard it would be to adapt this for liquidfun?

[danbri]

Liquidfun (an interesting fork of box2d that adds particle/fluid capabilities) has some pre-WASM emscripten tooling, but it seems out of date. I was wondering if your approach here would be more maintainable, and if you'd have any interest in trying it...

[Birch-san]

Hm, I agree that the approach used in this repository (consume C++ via git submodule, bind via WebIDL) would be more maintainable.

Might not be too difficult. Some observations:

• Liquidfun is forked from Box2D v2.3.0, which is as old as November 2013
  • notably, it predates the official CMake build system introduced in Box2D v2.4.0 (July 2020), and provides its own CMake build.
• Liquidfun goes to some effort in its binding code to make use of offsets to avoid JS->WASM call overhead:

  • Offsets was an idea I had for optimizing
    the overhead of calling into emscripten code. Emscripten code is half
    native speed, but the call overhead is unbelievable(like 50x the cost of
    a JS only call).

  • this is probably unnecessary as of October 2018
• Liquidfun goes to a lot of manual effort to author binding code
  • WebIDL would be more maintainable. box2d.js's v2.3.1 bindings could be a good starting point.
  • using WebIDL as the basis for bindings also means that the webidl-to-ts typings generator in this repository could be used to generate TypeScript typings

Based on file dates, it kinda looks like Liquidfun leaves most of the Box2D source untouched; the fork contribution is primarily the Particle directory?

The approach used for binding in this repository would not be interface-compatible with the manual bindings used in Liquidfun currently, so any testbed examples would need tweaking in order to work.

The path of least resistance may be to fork Box2D at its latest commit, and add the Particle folder back in, (and confront any breaking API changes). then consume that in this repository, update the WebIDL bindings to include Particle classes.

[danbri]

Totally agree with your observations (it is old and under-maintained), and the clever trickery to speed up calls looked both like a maintenance burden, and unnecessary in 2021. I believe the original intent was gamedev but these days box2d is getting a lot of usage as a simple physics engine for machine learning / AI. Much of that is in a Python-led world e.g. openai environments like BipedalWalker; but also increasingly in JS (example, see also [tensorflow.js]).(https://www.tensorflow.org/js/demos).
I thought a WASM build could be a path towards drumming up attention/interest for it (or something similar) in an ML context. But yeah it should also be fun for web games, with WASM maturing and everything being faster than 2013...

Based on file dates, it kinda looks like Liquidfun leaves most of the Box2D source untouched; the fork contribution is primarily the Particle directory?

AFAIK that sounds plausible

The approach used for binding in this repository would not be interface-compatible with the manual bindings used in Liquidfun currently, so any testbed examples would need tweaking in order to work.

Noted. Personally I don't see that as a problem (it's possible others have other plans, but things seem quiet...).

The path of least resistance may be to fork Box2D at its latest commit, and add the Particle folder back in, (and confront any breaking API changes). then consume that in this repository, update the WebIDL bindings to include Particle classes.

I had the impression from comments in the liquidfun repo that this would be a substantial undertaking, definitely beyond me, though I'd love to see it!

[Birch-san]

some things that really made it challenging like some git settings that affect the recognition of newline differences, \r\n, vs \n

I've made a commit of "all non-whitespace changes to C++ source", based on Box2D v2.3.0:
https://github.com/erincatto/box2d/compare/master...Birch-san:liquidfun?expand=1

Looks like it's more than just the Particle folder that changed. Quite a few files are touched, but it's mostly additions rather than contradictions.

Not sure how much the world moved in the years since then (i.e. Box2D v2.4.1). All files were renamed (i.e. to eliminate capital letters), which will add some challenge to a rebase. I think there's a lot of changes, but most are so light-touch that it'd be easy to figure out where to insert them in v2.4.1. May need manual attention though.

[Birch-san]

I've had a stab at manually copying all those changes onto Box2D v2.4.1:

https://github.com/erincatto/box2d/compare/master...Birch-san:liquidfun-rebase?expand=1

Haven't tried compiling or running it. There weren't many errors in my IDE, except that I'll need to change all float32s into floats.

Bost of the changes don't look like they'd cause problems, but I saw substantial rewrites to timers and to memory allocation.

[Birch-san]

Updated WebIDL bindings:
https://github.com/Birch-san/box2d-wasm/compare/liquidfun?expand=1

Fixed other compile errors (e.g. missing symbols that Emscripten noticed at link-time):
https://github.com/erincatto/box2d/compare/master...Birch-san:liquidfun-rebase?expand=1

It's successfully compiled to WASM locally on my computer, and my original examples still work (haven't regressed).

Various new symbols (i.e. the ones that I exposed via WebIDL) are accessible via JS:

b2ParticleBodyContact
b2ParticleColor
b2ParticleContact
b2ParticleDef
b2ParticleGroup
b2ParticleGroupDef
b2ParticleHandle
b2ParticlePair
b2ParticleSystem
b2ParticleSystemDef
b2ParticleTriad

Next step would be to build a demo to see if they work.

[Birch-san]

Works. Released:
https://github.com/Birch-san/box2d-wasm/releases/tag/v4.0.0-liquidfun.0

[danbri]

I've only had a little time to poke at this, but it looks great - can confirm it builds fine, the integration-test app works and shows particles, the demos still function etc.

[danbri]

I've (as a temporary step for testing/exploration) put a copy of the generated .js and .wasm on my site, so it can be loaded from elsewhere (please nobody make anything that will break if this goes away!).

I did this to see how the WASM approach might look in other environments. I only got this far in the online P5 editor: https://editor.p5js.org/danbri/sketches/hWpLvdEB5 - for some reason it only works for me when using an incognito window, but fails while logged in. It could also be interesting to adapt the backend examples and try a python-wasm runtime e.g. via Colab. The P5 example doesn't try using particles yet.

curl --head https://danbri.org/2021/lf-wasm/demo/Box2D.wasm
HTTP/1.1 200 OK
Date: Fri, 26 Feb 2021 13:10:23 GMT
Server: Apache/2.4.25 (Debian)
Last-Modified: Fri, 26 Feb 2021 11:38:56 GMT
ETag: "164aeb-5bc3bb6a495c4"
Accept-Ranges: bytes
Content-Length: 1460971
Cache-Control: max-age=31536000, public
Access-Control-Allow-Origin:*
Content-Type: application/wasm

See also Twitter thread