Description
The problem
PHP.wasm crashes if the ASYNCIFY_ONLY functions list in packages/php-wasm/compile/Dockerfile is not exhaustive enough:
Whenever it crashes, the developer must analyze the stacktrace, update the list, and rebuild PHP. That DX is rough and many developers won't be willing to approach Asyncify issues.
Tl;dr solution idea
Autogenerate the list of ASYNCIFY_ONLY functions with an automated test suite that would exercise all possible code paths that may trigger a network call in PHP.
The full story:
Technical details
Asyncify lets synchronous C or C++ code interact with asynchronous JavaScript. Technically, it saves the entire C call stack before yielding control back to JavaScript, and then restores it when the asynchronous call is finished. This is called stack switching.
Stack switching requires wrapping all C functions that may be found at a call stack at a time of making an asynchronous call. Blanket-wrapping of every single C function adds a significant overhead, which is why we maintain a list of specific function names:
wordpress-playground/packages/php-wasm/compile/Dockerfile
Lines 624 to 632 in 15a6609
Unfortunately, missing even a single item from that list results in a WebAssembly crash whenever that function is a part of the call stack when an asynchronous call is made.
@danielbachhuber asked:
It seems like we need to find a more permanent solution to these function crashes. I don't think we'll be able to tag a "stable" release with this behavior.
Is it possible to reliably list all the relevant functions by hand?
I think so.
The current list is a result of a long trial-and-error process, but there is a away to systematize it, or perhaps even automate it.
Source of the crash
The only async operations are network requests – we can work backwards from here to find all the relevant Zend engine functions.
The only way to trigger an async request is via Emscripten's createPeer function which is triggered by connect(2), listen(2) and sendmsg(2).
The internal network functions are called by the following top-of-the-stack code paths:
- PHP stream handlers like
getimagesize("https://...") - PHP network-related functions like
fsockopenormysql_connect.
Top-of-the-stack functions can be called by the following bottom-of-the-stack code paths:
- A function call (
myfunc(),$obj->method(),$reflection->callMethod(),new MyClas(), etc.) - Resource cleanup (destructors, request shutdown handlers, etc.)
The big idea
- Find all top-of-the-stack functions that trigger network calls
- Find all bottom-of-the-stack functions that trigger the above
- Write a test suite for a cartesian product of the two
- Auto-append any missing items to
ASYNCIFY_ONLY - Rebuild, rinse and repeat
This way, we'd get a more reliable and systematic list of ASYNCIFY_ONLY functions.
There's still a risk of missing some code paths, but:
- It should happen much less frequently
- The fix would only involve creating a unit test that reproduces the crash
- We could involve the output of ASYNCIFY_ADVISE in the process to minimize the risks
Furthermore, perhaps the error message could include a link to a new issue form with the stack trace and other relevant details already prepopulated.
Why other ideas won't work so well and what we can do about it
Auto-listing all the functions
Asyncify can auto-list all the required C functions when built without ASYNCIFY_ONLY.
Auto-detection is overeager and ends up listing about 70,000 C functions which increases the startup time to 4.5s. This can be seen by adding -sASYNCIFY_ADVISE to the final emcc in the Dockerfile. Edit: I just tried this again and seems to be significantly less now. There's still a lot of unrelated functions there, though. Dynamic calls issued by internal functions like zend_call_method don't play well with static analysis.
Eventually, V8 will likely handle stack switching for us and remove this problem entirely. Unfortunately, the timeline for landing this in Node.js is unclear. @ThomasTheDane – do you have any insights about that, by any chance?
Recovering from a crash
We cannot recover. A crash leaves PHP in an undefined state. Were we in a middle of a request? In a request shutdown handler? Do we need to clean up any dangling resources? There's no telling! Once a crash happened, we must shutdown the PHP runtime.
We can, however, automatically open a new runtime. It will crash again eventually, but at least the developer won't have to manually restart the app.
Automating recompilation
@danielbachhuber asked:
Could we abstract the ASYNCIFY functions to some configuration file, and then have this error message code do a smarter job identifying the missing function?
The answer is yes, but only for developers working with node.js packages inside WordPress Playground repo. No bueno for npm package consumers and on the web. This could still be useful.