Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add docker build #4

Merged
merged 4 commits into from
Sep 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 0 additions & 27 deletions .github/workflows/build.yml

This file was deleted.

19 changes: 19 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Build Docker image
on:
pull_request:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: false
tags: soyuka/php-wasm:latest
58 changes: 58 additions & 0 deletions .github/workflows/demo.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Simple workflow for deploying static content to GitHub Pages
name: Deploy static content to Pages

on:
# Runs on pushes targeting the default branch
push:
branches: ["main"]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: true

jobs:
# Single deploy job since we're just deploying
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
-
name: Build PHP WASM
uses: docker/build-push-action@v5
with:
context: .
push: false
tags: soyuka/php-wasm:latest
- name: Setup Pages
uses: actions/configure-pages@v3
- name: Build website
working-directory: demo
run: |
docker create --name=php-wasm soyuka/php-wasm:latest
mkdir -p public/ dist/
docker cp php-wasm:/build/php-web.mjs ./dist
docker cp php-wasm:/build/php-web.wasm ./public
docker run -v $(pwd)/src:/src -v $(pwd)/public:/public -v $(pwd)/dist:/dist php-wasm python3 /emsdk/upstream/emscripten/tools/file_packager.py /public/php-web.data --use-preload-cache --lz4 --preload "/src" --js-output=/dist/php-web.data.js --no-node --exclude '*/.*' --export-name=createPhpModule
sed '/--pre-js/r dist/php-web.data.js' dist/php-web.mjs > public/php-web.mjs
- name: Upload artifact
uses: actions/upload-pages-artifact@v2
with:
path: 'demo/public'
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v2
58 changes: 0 additions & 58 deletions .github/workflows/release.yml

This file was deleted.

17 changes: 9 additions & 8 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ WORKDIR /src/sqlite
RUN emcc -Oz -DSQLITE_OMIT_LOAD_EXTENSION -DSQLITE_DISABLE_LFS -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_THREADSAFE=0 -DSQLITE_ENABLE_NORMALIZE -c sqlite3.c -o sqlite3.o

FROM build_tool as php_src
ARG PHP_BRANCH=PHP-8.2.9
ARG PHP_BRANCH=PHP-8.2.10
RUN git clone https://github.com/php/php-src.git php-src \
--branch $PHP_BRANCH \
--single-branch \
Expand All @@ -49,7 +49,8 @@ RUN git clone https://github.com/php/php-src.git php-src \
FROM php_src AS php-wasm
ARG WASM_ENVIRONMENT=web
ARG ASSERTIONS=0
ARG OPTIMIZE=-O2
ARG OPTIMIZE=-O1
# TODO: find a way to keep this, it can't be empty if defined...
# ARG PRE_JS=
ARG INITIAL_MEMORY=256mb
COPY --from=libxml /src/libxml2/build/ /src/usr
Expand All @@ -59,8 +60,7 @@ ENV LIBXML_LIBS "-L/src/usr/lib"
ENV LIBXML_CFLAGS "-I/src/usr/include/libxml2"
ENV SQLITE_CFLAGS "-I/src/usr/include/sqlite3"
ENV SQLITE_LIBS "-L/src/usr/lib"
WORKDIR /src/php-src
RUN ./buildconf --force \
RUN cd /src/php-src && ./buildconf --force \
&& emconfigure ./configure \
--enable-embed=static \
--with-layout=GNU \
Expand Down Expand Up @@ -89,11 +89,11 @@ RUN ./buildconf --force \
--enable-pdo \
--with-pdo-sqlite \
--with-sqlite3
RUN emmake make -j8
RUN cd /src/php-src && emmake make -j8
# PHP7 outputs a libphp7 whereas php8 a libphp
RUN bash -c '[[ -f .libs/libphp7.la ]] && mv .libs/libphp7.la .libs/libphp.la && mv .libs/libphp7.a .libs/libphp.a && mv .libs/libphp7.lai .libs/libphp.lai || exit 0'
RUN cd /src/php-src && bash -c '[[ -f .libs/libphp7.la ]] && mv .libs/libphp7.la .libs/libphp.la && mv .libs/libphp7.a .libs/libphp.a && mv .libs/libphp7.lai .libs/libphp.lai || exit 0'
COPY ./source /src/source
RUN emcc $OPTIMIZE \
RUN cd /src/php-src && emcc $OPTIMIZE \
-I . \
-I Zend \
-I main \
Expand All @@ -102,7 +102,7 @@ RUN emcc $OPTIMIZE \
/src/source/phpw.c \
-o /src/phpw.o \
-s ERROR_ON_UNDEFINED_SYMBOLS=0
RUN mkdir /build && emcc $OPTIMIZE \
RUN mkdir /build && cd /src/php-src && emcc $OPTIMIZE \
-o /build/php-$WASM_ENVIRONMENT.mjs \
--llvm-lto 2 \
-s EXPORTED_FUNCTIONS='["_phpw", "_phpw_flush", "_phpw_exec", "_phpw_run", "_chdir", "_setenv", "_php_embed_init", "_php_embed_shutdown", "_zend_eval_string"]' \
Expand All @@ -122,3 +122,4 @@ RUN mkdir /build && emcc $OPTIMIZE \
# -s DECLARE_ASM_MODULE_EXPORTS=0 \
-lidbfs.js \
/src/phpw.o /src/usr/lib/sqlite3.o .libs/libphp.a /src/usr/lib/libxml2.a
RUN rm -r /src/*
118 changes: 60 additions & 58 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,32 +3,70 @@
Project based on https://github.com/seanmorris/php-wasm which was forked from https://github.com/oraoto/pib

I fixed some inconsistencies in the Makefile and removed non-essential things. This fork:
- does build sqlite3
- does build libxml
- has no javascript abstraction
- builds sqlite3
- builds libxml
- no javascript abstraction
- exposes FS and builds with [IDBFS](https://emscripten.org/docs/api_reference/Filesystem-API.html#FS.syncfs)
- does not build https://github.com/seanmorris/vrzno which allows javascript access from PHP (TODO add this back opt-in cause it's really cool)
- does not add preloaded data, having this separatly from php-wasm builds allows for more flexibility (see [preload data section](#preload-data))
- exposes FS and builds with [IDBFS](https://emscripten.org/docs/api_reference/Filesystem-API.html#FS.syncfs)
- reduces memory to 256mb

## Usage

Some build

## Build

```
docker build . -t php-wasm
# create a temporary container
docker create --name=php-wasm php-wasm
# to copy builded files
docker cp php-wasm:/build/php-web.wasm ./build
docker cp php-wasm:/build/php-web.mjs ./build/php-web.mjs
```

Builded files will be located in `build/php-web.js` and `build/php-web.wasm`.
The module we export in this image is called `createPhpModule`.

### Build arguments

Use this as template to build PHP with emscripten. At build these arguments are available:

```
LIBXML2_TAG=v2.9.10
PHP_BRANCH=PHP-8.2.9
```

The next args are used for [emcc options](https://github.com/soyuka/php-wasm/blob/513f284e1ba8f26d66e08a97291f484b3dd7de1b/Dockerfile#L108) `-sOPTION`
see [settings.js](https://github.com/emscripten-core/emscripten/blob/9bdb310b89472a0f4d64f36e4a79273d8dc7fa98/src/settings.js#L633).
In fact it's even easier for you to set them directly in [the Dockerfile](https://github.com/soyuka/php-wasm/blob/513f284e1ba8f26d66e08a97291f484b3dd7de1b/Dockerfile#L108).

```
WASM_ENVIRONMENT=web
ASSERTIONS=0
OPTIMIZE=-O2
INITIAL_MEMORY=256mb
```

### Preload data

My prefered option is to use the [`file_packager`](https://github.com/emscripten-core/emscripten/blob/9bdb310b89472a0f4d64f36e4a79273d8dc7fa98/tools/file_packager) tool to build the preloaded data in a `php-web.data.js` (and `php-web.data` file). These are preloaded into IDBFS. That can be changed changing the `-lidbfs.js` argument to `emcc`.

This will preload `SOME_DIR` into the `/src` directory inside the WASM filesystem:

```
mkdir -p php-wasm
docker run -v SOME_DIR:/src -v $(pwd)/php-wasm:/dist -w /dist soyuka/php-wasm:8.2.9 python3 /emsdk/upstream/emscripten/tools/file_packager.py php-web.data --use-preload-cache --lz4 --preload "/src" --js-output=php-web.data.js --no-node --exclude '*/.*' --export-name=createPhpModule
ls php-wasm/
```

Note that the `php-web.data.js` must be either used as `PRE_JS` argument to emcc or it needs to be included inside the `php-web.js`:

```
sed '/--pre-js/r php-wasm/php-web.data.js' php-wasm/php-web.mjs > this-has-preloaded-data-php-web.mjs
```

We match the `export-name` with the emcc `EXPORT_NAME` option. Use excludes to downsize the preloaded data weight.

## Usage

To execute some php, call `pib_exec` using [`ccall`](https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html#interacting-with-code-ccall-cwrap), for example:
To execute some php, call `phpw_exec` using [`ccall`](https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html#interacting-with-code-ccall-cwrap), for example:

```javascript
const phpBinary = require('build/php-web');
Expand All @@ -46,69 +84,33 @@ return phpBinary({
})
.then(({ccall, FS}) => {
const phpVersion = ccall(
'pib_exec'
'phpw_exec'
, 'string'
, ['string']
, [`phpversion();`]
);
})
```

The builded PHP WASM version exposes these functions that will help you execute PHP: `_pib_init`, `_pib_destroy`, `_pib_run`, `_pib_exec`, `_pib_refresh`. The source code behind them is located under `source/pib_eval.c` and the exported function are declared in the final build command (see `Makefile`).

Thanks to [@seanmorris](https://github.com/seanmorris/php-wasm) you can also use persistent calls to keep things in memory by using:
### API

```javascript
let retVal = ccall(
'pib_init'
, NUM
, [STR]
, []
);

console.log('PHP initialized', retVal)

function runCode(phpCode) {
ccall(
'pib_run'
, NUM
, [STR]
, [`?>${phpCode}`]
)
}

// Remove things kept in-memory
function refresh() {
ccall(
'pib_refresh'
, NUM
, []
, []
);
}
phpw_exec(string code): string
phpw_run(string code): void
phpw(string filePath): void
```

More examples on the code usage are available on [@seanmorris's repository](https://github.com/seanmorris/php-wasm/tree/master/docs-source) or on [APIPlatform By Examples]().

## Preload data
### Example calls:

You can build preloaded assets (`PRELOAD_ASSETS=/data`) using `make preload-data`. Then you'll need to add the `php-web.data.js` to the actual `php-web.js` file.

This step on API Platform By Examples is done with the following Makefile using the original file as `php-web.ori.js`:

```make
UID=1000
API_PLATFORM_DIR=examples
DOCKER_RUN=docker run --rm -e ENVIRONMENT=web -v $(CURDIR):/src -v $(CURDIR)/../${API_PLATFORM_DIR}:/src/${API_PLATFORM_DIR} -w /src soyuka/php-emscripten-builder:latest

preload:
${DOCKER_RUN} python3 /emsdk/upstream/emscripten/tools/file_packager.py ./public/php-web.data --preload "/src/${API_PLATFORM_DIR}" --js-output=./php-wasm/php-web.data.js
${DOCKER_RUN} chown ${UID} ./php-wasm/php-web.data.js
sed -e '/function(Module) {/r./php-wasm/php-web.data.js' php-wasm/php-web.ori.js > php-wasm/php-web.js
rmdir ${API_PLATFORM_DIR}
```javascript
const STR = 'string';
ccall("phpw", null, [STR], ["public/index.php"]);
console.log(ccall("phpw_exec", STR, [STR], ["phpversion();"]));
```

[More about how to call exposed functions](https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html?highlight=call#interacting-with-code-ccall-cwrap)

## TODO

- cache libxml and sqlite to speed up builds
- add opt-in / opt-out sqlite libxml vrzno and mb more
4 changes: 4 additions & 0 deletions demo/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
dist/
public/php-web.mjs
public/php-web.data
public/php-web.wasm
Loading