[RFC] Add support for building to the wasm32-wasi target #10457

ereslibre · 2023-01-26T20:39:00Z

Add support for building to the wasm32-wasi target

This change performs initial adaptations allowing the PHP interpreter
to be compiled for the wasm32-wasi target.

Co-Authored-By: Asen Alexandrov alexandrov@vmware.com

ereslibre · 2023-01-26T20:39:15Z

This adds support for building php-cgi to the wasm32-wasi platform.

In order to build in your own environment, you can use wasi-sdk directly, or use a build container image to make the build process easier. wasi-sdk is a project that makes it easier to build C projects to WebAssembly + WASI. It bundles clang and the wasi-libc project.

As an improvement, we can update the configure.ac to accept a --wasi-sdk argument, akin to what the sqlite project is going to do in order to support WebAssembly + WASI.

Building

You can build either by using wasi-sdk directly, or by using a container image we have prepared that contains everything required.

Build with `wasi-sdk`

Install wasi-sdk first:

$ wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-19/wasi-sdk-19.0-linux.tar.gz
$ tar xvf wasi-sdk-19.0-linux.tar.gz
$ export WASI_SDK_PATH=`pwd`/wasi-sdk-19.0
$ export CC="${WASI_SDK_PATH}/bin/clang --sysroot=${WASI_SDK_PATH}/share/wasi-sysroot"
$ export LD="${WASI_SDK_PATH}/bin/wasm-ld"

Set common flags:

$ export CFLAGS="-D_POSIX_SOURCE=1 -D_GNU_SOURCE=1 -DHAVE_FORK=0 -D_WASI_EMULATED_GETPID -D_WASI_EMULATED_SIGNAL -D_WASI_EMULATED_PROCESS_CLOCKS"
$ export LDFLAGS="-lwasi-emulated-getpid -lwasi-emulated-signal -lwasi-emulated-process-clocks"

Configure:

$ ./buildconf --force
$ ./configure --host=wasm32-wasi host_alias=wasm32-musl-wasi --target=wasm32-wasi target_alias=wasm32-musl-wasi --without-libxml --disable-dom --without-iconv --without-openssl --disable-simplexml --disable-xml --disable-xmlreader --disable-xmlwriter --without-pear --disable-phar --disable-opcache --disable-zend-signals --without-pcre-jit --with-sqlite3 --enable-pdo --with-pdo-sqlite --disable-fiber-asm --without-sqlite3

And build!, make -j cgi.

You can find the binary under sapi/cgi/php-cgi as usual.

Notes and automated build artifacts

In this example we have only built the CGI binary, but the CLI binary can also be adapted to build for wasm32-wasi.

At the webassembly-language-runtimes project we are continuously releasing PHP binaries. You can find released artifacts here, and on every commit of the repo.

These artifacts are also linked against a sqlite3 build ported to WASI -- the sqlite3 project will officially release the WASI version of sqlite3. --

Running the binaries

You can download any WebAssembly runtime compatible with WASI (most of them). For example, wasmtime:

$ cat hello-world.php
<?php
  echo "Hello, world!";
?>
$ wasmtime --mapdir .::. php-cgi.wasm -- hello-world.php
X-Powered-By: PHP/8.3.0-dev
Content-type: text/html; charset=UTF-8

Hello, world!

It's required to map the current directory, so that the hello-world.php is found by the WebAssembly module within the sandbox.

Tests

Running the test suite in our automation is still work in progress. We will improve this changes so that all test that are relevant to the wasm32-wasi platform are green.

setjmp/longjmp emulation

We have a contribution that enables the setjmp/longjmp emulation in the same way as the Ruby interpreter has done. This contribution is still to be merged in our own repository, but we are also excited to integrate it on the changes we aim to upstream.

Similarly, we also have work in progress that enables PHP fibers using the same Asyncify strategy.

Feedback

We are looking forward for feedback on these changes. We want to contribute this changes upstream, and are willing to update this contribution so that it fits your needs and requirements.

Please, let us know your thoughts, or anything you might need in order for this change to land.

Thank you!

ereslibre · 2023-01-26T20:46:58Z

Further context: we have successfully run wordpress in the browser, and also in the server side with Apache and mod_wasm.

ext/posix/posix.c

mvorisek · 2023-01-27T15:52:30Z

Please add a CI workflow for WASM, see https://github.com/php/php-src/blob/master/.github/workflows/push.yml

sapi/fpm/fpm/fpm_unix.c

ereslibre · 2023-02-06T11:49:18Z

Please add a CI workflow for WASM, see https://github.com/php/php-src/blob/master/.github/workflows/push.yml

Thanks @mvorisek. Updated PR. Current result as of now:

Number of tests :   379               334
Tests skipped   :    45 ( 11.9%) --------
Tests warned    :     0 (  0.0%) (  0.0%)
Tests failed    :    17 (  4.5%) (  5.1%)
Tests passed    :   317 ( 83.6%) ( 94.9%)
---------------------------------------------------------------------
Time taken      :   125 seconds

Can somebody allow GH actions to execute on this PR?

ereslibre · 2023-03-13T11:01:09Z

Thanks a lot for the reviews. I have updated the PR.

Let me know if by the compat layer you meant something specifically different than what this patch provides. I looked at the win32 compat layer, but I also see many ifdefs here and there on the source code, so I tried to update the patch from scratch in a similar form, while trying not to be very intrusive with the current codebase. Please, let me know if you have a specific example that would look better on the current codebase, I would be happy to adapt.

The CI result with the wasi execution can be found at https://github.com/ereslibre/php-src/actions/runs/4403303495

cc/ @bukka @mvorisek @Girgias @derickr

mvorisek · 2023-03-13T11:14:06Z

There are a lot of segfaults in the CI: https://github.com/ereslibre/php-src/actions/runs/4403303495/jobs/7711483600#step:7:9775

bukka

I think it tries to disable way too many function. As I understand wasm is missing some of libc functions, right? If so, then it would be probably better to wrap the missing pieces with functions that have the same signature like libc functions and always fail. Then you can link those for wasm and it won't be necessary to add that many ifdefs.

ext/standard/basic_functions.c

bukka · 2023-03-13T12:28:34Z

Zend/zend.c

@@ -1175,9 +1177,9 @@ ZEND_COLD void zenderror(const char *error) /* {{{ */
 }
 /* }}} */

+#ifndef __wasi__


This might need some more checking of places where the bailout is used and veryfying if it might have any impact on the apllication under wasm. We basically don't want to introduce something that will result to future bug and I would imagine that disabling bailout might be one of those things without a proper checking.

Yes, absolutely. Also, we will implement the bailout logic once we include setjmp/longjmp support.

ext/standard/basic_functions.c

ext/standard/filestat.c

ext/standard/pageinfo.c

ext/standard/wasi/dns.c

bukka · 2023-03-13T13:54:18Z

main/fastcgi.c

@@ -1209,6 +1223,12 @@ static int fcgi_read_request(fcgi_request *req)

 	return 1;
 }
+#else // __wasi__
+static int fcgi_read_request(fcgi_request *req)


what's the actual point of using FCGI if it's not re-using connection (if it's 0, then it closes the connection)? Why don't you just use plain CGI?

Updated the patch; fastcgi.c requires some changes because cgi_main uses certain symbols that come from fastcgi.c; and we are interested in php-cgi.

bukka · 2023-03-13T13:56:44Z

wasi/main/php_open_temporary_file.c

+
+PHPAPI const char* php_get_temporary_directory(void)
+{
+  return NULL;


use tabs and not spaces (this is not the place where you indent with spaces so please all other cases as well)

Sorry, I am going to check everything, my editor is pretty stubborn at times.

mvorisek · 2023-06-09T21:30:32Z

.github/workflows/push.yml

+  WASM32_WASI:
+    name: "WASM32_WASI_${{ matrix.debug && 'DEBUG' || 'RELEASE' }}_${{ matrix.zts && 'ZTS' || 'NTS' }}"


Suggested change

WASM32_WASI:

name: "WASM32_WASI_${{ matrix.debug && 'DEBUG' || 'RELEASE' }}_${{ matrix.zts && 'ZTS' || 'NTS' }}"

WASI:

name: "WASI_${{ matrix.debug && 'DEBUG' || 'RELEASE' }}_${{ matrix.zts && 'ZTS' || 'NTS' }}"

just to make the CI job name shorter/nicer...

I would keep WASM32_WASI if that's fine, given that WASM64 be added eventually

mvorisek · 2023-06-09T21:32:13Z

.github/workflows/push.yml

+        run: make -j$(/usr/bin/nproc) cgi cli
+      - name: Optimize
+        uses: ./.github/actions/optimize-wasi
+      # To be defined how to approach test coverage on this platform.


All core tests (tests that does not require any unsupported extension) must pass or they must be exluded if they are expected to fail ATM.

setjmp/longjmp emulation

Fibers are core part of the PHP language since PHP 8.0 release. Can the support be added from the day zero?

All core tests (tests that does not require any unsupported extension) must pass or they must be exluded if they are expected to fail ATM.

Will exclude tests that don't apply on this platform in a next pass.

Fibers are core part of the PHP language since PHP 8.0 release. Can the support be added from the day zero?

We should look into that. If it's a prerequisite for this patch to land, then yes. If it's fine to iterate I'd prefer so, to not block the whole patch on smaller specifics, given the patch is already somewhat big.

iluuu1994

I haven't actually finished the review. But the trend seemed to continue. I think we need a better approach for missing functions/features than stubbing them out.

Zend/zend_wasi.c

iluuu1994 · 2023-06-19T13:47:15Z

Zend/zend_wasi.c

+ZEND_API ZEND_COLD ZEND_NORETURN void _zend_bailout(const char *filename,
+                                                    uint32_t lineno) /* {{{ */
+{
+  // Not implemented yet, as WebAssembly has no stack switching


Is there a reason why this is in its own file?

The idea was to be as less intrusive as possible with the existing tree, avoiding unnecessary ifdefs that might clutter existing files.

If you think there's a better way I'm fine with any proposal!

Zend/zend_alloc.c

sapi/cgi/cgi_main.c

iluuu1994 · 2023-06-19T14:20:59Z

main/unistd_wasi.c

+
+#include <errno.h>
+
+int dup(int fildes)


Does this need to be in a separate file? Separate files are a bit of a pain with autoconf because it require calling ./configure again.

Yes, same rationale as #10457 (comment), to be as less intrusive as possible.

I understand the ./configure re-run concern, but at this stage I think is better to keep them well separated, and as WASI evolves with future snapshots start merging with the main tree as things establish on upcoming wasi-libc versions.

Zend/zend_alloc.c

Zend/zend_fibers_wasi.c

This change performs initial adaptations allowing the PHP interpreter to be compiled for the wasm32-wasi target. Co-Authored-By: Asen Alexandrov <alexandrov@vmware.com>

WebAssembly + WASI GH actions have specific tasks for the following steps: - apt: an existing PHP installation will be used as the test runner, as opposed to the binary being built. It also installs other dependencies used by later stages, such as Wasmtime (WebAssembly runtime), and Binaryen (contains `wasm-opt`, to be used on the optimization phase.) - configure-wasi: specific configure values. - optimize-wasi (new action): performs binaryen optimization passes on the resulting binary. - test-wasi: requires a wrapper that will use `wasmtime` to run the `php` CLI compiled to WebAssembly.

ereslibre · 2023-07-12T11:05:23Z

Thanks for your review @iluuu1994. I have updated the patch fixing all your comments, and I also commented about the separate files here: #10457 (comment).

This change will evolve as the wasi-libc evolves. However, the principle is always to be as less intrusive as possible with the current codebase. When there are concepts that are not directly mappable to WebAssembly/WASI, we stubbed them out; in this patch this happened in a case by case basis, so please, let us know if you'd like to see things handled in a different way.

The goal is to propose this patch for landing on the PHP tree, which I think might need an RFC process, but before that happens I think is worth getting done as much as possible. Because of this, feedback from PHP contributors is highly valuable.

derickr

There are still plenty of places where functions just "appear" to exist, but without doing anything.

derickr · 2024-05-22T08:11:16Z

ext/standard/basic_functions.c

 			php_log_err_with_severity(message, LOG_NOTICE);
+#else
+			php_log_err_with_severity(message, 5);


Magic numbers are not a good idea. If WASM doesn't define LOG_NOTICE, then have a block somewhere else that defines this (and I suspect other constants) in one go.

derickr · 2024-05-22T08:14:48Z

ext/standard/exec.c

+
+#else
+
+void php_shell_exec_ex(INTERNAL_FUNCTION_PARAMETERS);


This reads really odd - if WASI is active, you only use a forward declaration. I would rather do it like:

#ifndef PHP_WASI static void php_shell_exec_ex(INTERNAL_FUNCTION_PARAMETERS) /* {{{ { … } #else static void php_shell_exec_ex(INTERNAL_FUNCTION_PARAMETERS) /* {{{ { php_wasi_shell_exec_ex(INTERNAL_FUNCTION_PARAMETERS_PASSTHRU); } #endif // PHP_WASI

And then have your new files implement the php_wasi_* variant(s).

But considering that doesn't do anything on WASI, this function should also just not even exist on that platform.

derickr · 2024-05-22T08:16:06Z

ext/standard/file_wasi.c

+
+PHPAPI void php_umask(INTERNAL_FUNCTION_PARAMETERS)  /* {{{ */
+{
+	RETURN_LONG(0);


Again, these functions do nothing, so they shouldn't exist.

ereslibre · 2024-05-22T16:00:59Z

I thank everyone who has provided feedback on this PR. I'm closing it as is. If there is interest in reviving this work I encourage folks to take the lead on that.

github-actions bot added Category: Engine Extension: fileinfo Extension: pdo_sqlite Extension: posix Extension: session Extension: standard SAPI: cgi SAPI: fpm SAPI: litespeed labels Jan 26, 2023

ereslibre marked this pull request as draft January 26, 2023 20:39

ereslibre force-pushed the wasm32-wasi branch from b726faa to a94519d Compare January 26, 2023 20:54

derickr reviewed Jan 27, 2023

View reviewed changes

ext/posix/posix.c Outdated Show resolved Hide resolved

Girgias reviewed Jan 27, 2023

View reviewed changes

sapi/fpm/fpm/fpm_unix.c Outdated Show resolved Hide resolved

ereslibre mentioned this pull request Jan 30, 2023

Streamline PHP patches vmware-labs/webassembly-language-runtimes#5

Open

3 tasks

ereslibre force-pushed the wasm32-wasi branch from a94519d to de67bf7 Compare February 6, 2023 11:40

github-actions bot added SAPI: cli and removed SAPI: fpm SAPI: litespeed Extension: posix labels Feb 6, 2023

ereslibre force-pushed the wasm32-wasi branch from de67bf7 to 000954b Compare February 6, 2023 11:46

ereslibre requested review from derickr, Girgias and mvorisek and removed request for derickr and Girgias February 6, 2023 11:46

ereslibre force-pushed the wasm32-wasi branch from 000954b to d2fada3 Compare February 6, 2023 11:52

ereslibre force-pushed the wasm32-wasi branch 2 times, most recently from 8ce4db8 to e033a5a Compare March 12, 2023 12:08

github-actions bot removed Extension: fileinfo Extension: session labels Mar 12, 2023

ereslibre force-pushed the wasm32-wasi branch 2 times, most recently from 38b96cf to 4584dc2 Compare March 13, 2023 09:22

bukka requested changes Mar 13, 2023

View reviewed changes

ereslibre force-pushed the wasm32-wasi branch from 4584dc2 to 5d77869 Compare March 16, 2023 10:12

eliot-akira mentioned this pull request May 10, 2023

PHP WASM: Compile PHP to WASI WordPress/wordpress-playground#290

Open

ereslibre force-pushed the wasm32-wasi branch from 5d77869 to 32a7f81 Compare May 30, 2023 08:43

ereslibre force-pushed the wasm32-wasi branch 2 times, most recently from f25535d to 0188a82 Compare June 9, 2023 21:05

mvorisek reviewed Jun 9, 2023

View reviewed changes

ereslibre requested review from bukka and Girgias June 13, 2023 11:13

iluuu1994 reviewed Jul 10, 2023

View reviewed changes

ereslibre force-pushed the wasm32-wasi branch 3 times, most recently from 060ea6f to b7883f5 Compare July 12, 2023 10:45

ereslibre and others added 2 commits July 12, 2023 13:02

Add support for building to the wasm32-wasi target

0e4ef52

This change performs initial adaptations allowing the PHP interpreter to be compiled for the wasm32-wasi target. Co-Authored-By: Asen Alexandrov <alexandrov@vmware.com>

ereslibre force-pushed the wasm32-wasi branch from b7883f5 to 908a7a7 Compare July 12, 2023 11:03

bukka mentioned this pull request Feb 17, 2024

consider official support for libphp.wasm build #13418

Open

derickr requested changes May 22, 2024

View reviewed changes

ereslibre closed this May 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] Add support for building to the wasm32-wasi target #10457

[RFC] Add support for building to the wasm32-wasi target #10457

ereslibre commented Jan 26, 2023 •

edited

Loading

ereslibre commented Jan 26, 2023 •

edited

Loading

ereslibre commented Jan 26, 2023

mvorisek commented Jan 27, 2023

ereslibre commented Feb 6, 2023 •

edited

Loading

ereslibre commented Mar 13, 2023

mvorisek commented Mar 13, 2023

bukka left a comment

bukka Mar 13, 2023

ereslibre Jun 9, 2023

bukka Mar 13, 2023

ereslibre Jun 9, 2023

bukka Mar 13, 2023

ereslibre Jun 9, 2023

mvorisek Jun 9, 2023

ereslibre Jul 12, 2023 •

edited

Loading

mvorisek Jun 9, 2023

ereslibre Jun 12, 2023

iluuu1994 left a comment

iluuu1994 Jun 19, 2023

ereslibre Jul 11, 2023

iluuu1994 Jun 19, 2023

ereslibre Jul 11, 2023 •

edited

Loading

ereslibre commented Jul 12, 2023 •

edited

Loading

derickr left a comment

derickr May 22, 2024

derickr May 22, 2024

derickr May 22, 2024

ereslibre commented May 22, 2024

		WASM32_WASI:
		name: "WASM32_WASI_${{ matrix.debug && 'DEBUG' \|\| 'RELEASE' }}_${{ matrix.zts && 'ZTS' \|\| 'NTS' }}"

[RFC] Add support for building to the wasm32-wasi target #10457

[RFC] Add support for building to the wasm32-wasi target #10457

Conversation

ereslibre commented Jan 26, 2023 • edited Loading

ereslibre commented Jan 26, 2023 • edited Loading

Building

Build with wasi-sdk

Notes and automated build artifacts

Running the binaries

Tests

setjmp/longjmp emulation

Feedback

ereslibre commented Jan 26, 2023

mvorisek commented Jan 27, 2023

ereslibre commented Feb 6, 2023 • edited Loading

ereslibre commented Mar 13, 2023

mvorisek commented Mar 13, 2023

bukka left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ereslibre Jul 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iluuu1994 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ereslibre Jul 11, 2023 • edited Loading

Choose a reason for hiding this comment

ereslibre commented Jul 12, 2023 • edited Loading

derickr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ereslibre commented May 22, 2024

ereslibre commented Jan 26, 2023 •

edited

Loading

ereslibre commented Jan 26, 2023 •

edited

Loading

Build with `wasi-sdk`

ereslibre commented Feb 6, 2023 •

edited

Loading

ereslibre Jul 12, 2023 •

edited

Loading

ereslibre Jul 11, 2023 •

edited

Loading

ereslibre commented Jul 12, 2023 •

edited

Loading