From 912cade0bea74733dafc88a91567247e4032aace Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 6 Feb 2024 15:28:10 -0800 Subject: [PATCH 1/5] Document how to do a WASI build --- getting-started/setup-building.rst | 97 ++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index b2dc2f1c2..01ae95808 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -345,6 +345,103 @@ and how to build. .. _Git for Windows download from the official Git website: https://git-scm.com/download/win +.. _wasi-compiling: + +WASI +---- + +WASI_ is a system interface standard for WebAssembly_. Through a combination of +C compilers that can target WebAssembly and `wasi-libc`_ providing +POSIX-compatible shims for WASI, it's possible for CPython to run on a WASI +host/runtime as a *guest*. + +.. note:: + + The instructions below assume a Unix-based OS due to cross-compilation for + CPython being designed for ``./configure`` / ``make``. + +To build for WASI, you will need to cross-compile CPython. This requires a C +compiler just like building for :ref:`Unix ` as well as: + +1. A C compiler that can target WebAssembly (e.g. `WASI-SDK`_) +2. A WASI host/runtime (e.g. wasmtime_) + +All of this is provided in the :ref:`devcontainer `. You can +also use what's installed in the container as a reference of what versions of +these tools are known to work. + +.. note:: + + CPython has only been verified with the above tools for WASI. Using + other compilers, hosts, or WASI versions *should* work, but the above tools + and their versions specified in the container are tested via a + :ref:`buildbot `. + +Building for WASI requires doing a cross-build where you have a *build* Python +to help produce a WASI build of CPython (technically it's a "host x host" +cross-build because the build Python is also the target Python while the host +build is the WASI build). This means you effectively build CPython twice. + +The easiest way to get a debug build of CPython for WASI is to use the +``Tools/wasm/wasi.py build`` command: + +.. code-block:: shell + + $ python Tools/wasm/wasi.py build --quiet -- --config-cache --with-pydebug + +That single command will configure and build both the build Python and the +WASI build in ``cross-build/build`` and ``cross-build/wasm32-wasi``, +respectively. + +You can also do each configuration and build step separately; the command above +is just a convenience wrapper around the following commands: + +.. code-block:: shell + + $ python Tools/wasm/wasi.py configure-build-python --quiet -- --config-cache --with-pydebug + $ python Tools/wasm/wasi.py make-build-python --quiet + $ python Tools/wasm/wasi.py configure-host --quiet -- --config-cache + $ python Tools/wasm/wasi.py make-host --quiet + +.. note:: + + The ``configure-host`` command infers the use of ``--with-pydebug`` from the + build Python. + +Running the separate commands after ``wasi.py build`` is useful if you want to +run just the e.g. ``make-host`` step after making code change. + +Once everything is complete, there will be a +``cross-build/wasm32-wasi/python.sh`` helper file which you can use to run the +``python.wasm`` file (see the output from the `configure-host` subcommand): + +.. code-block:: shell + + $ cross-build/wasm32-wasi/python.sh --version + +You can also use ``Makefile`` targets and they will work as expected thanks to +the ``HOSTRUNNER`` environment variable having been set to a similar value as +used in ``python.sh``: + +.. code-block:: shell + + $ make -C cross-build/wasm32-wasi test + +.. note:: + + WASI uses a *capability-based* security model. This means that the WASI host + does not give full access to your machine unless you tell it to. What this + also means is things like files can end up being mapped to a different path + inside the WASI host. So, if you try passing a file path to + ``python.wasm``/ ``python.sh``, it needs to match the path **inside** the + WASI host, not the path on your machine (much like using a container). + +.. _WASI: https://wasi.dev +.. _wasi-libc: https://github.com/WebAssembly/wasi-libc +.. _WASI-SDK: https://github.com/WebAssembly/wasi-sdk +.. _wasmtime: https://wasmtime.dev +.. _WebAssembly: https://webassembly.org + .. _build-dependencies: .. _deps-on-linux: .. _macOS and OS X: From 2dfe09aeb88f7be757ad86932b8d5b5c89a43336 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 6 Feb 2024 15:29:57 -0800 Subject: [PATCH 2/5] Fix a Markdown mistake --- getting-started/setup-building.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index 01ae95808..d2d79929f 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -413,7 +413,7 @@ run just the e.g. ``make-host`` step after making code change. Once everything is complete, there will be a ``cross-build/wasm32-wasi/python.sh`` helper file which you can use to run the -``python.wasm`` file (see the output from the `configure-host` subcommand): +``python.wasm`` file (see the output from the ``configure-host`` subcommand): .. code-block:: shell From 204d4599c258874e8df1ee8dbeadace2d073d88f Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 7 Feb 2024 14:51:03 -0800 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- getting-started/setup-building.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index d2d79929f..eb0395921 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -364,7 +364,7 @@ To build for WASI, you will need to cross-compile CPython. This requires a C compiler just like building for :ref:`Unix ` as well as: 1. A C compiler that can target WebAssembly (e.g. `WASI-SDK`_) -2. A WASI host/runtime (e.g. wasmtime_) +2. A WASI host/runtime (e.g. Wasmtime_) All of this is provided in the :ref:`devcontainer `. You can also use what's installed in the container as a reference of what versions of @@ -394,7 +394,7 @@ WASI build in ``cross-build/build`` and ``cross-build/wasm32-wasi``, respectively. You can also do each configuration and build step separately; the command above -is just a convenience wrapper around the following commands: +is a convenience wrapper around the following commands: .. code-block:: shell @@ -408,8 +408,8 @@ is just a convenience wrapper around the following commands: The ``configure-host`` command infers the use of ``--with-pydebug`` from the build Python. -Running the separate commands after ``wasi.py build`` is useful if you want to -run just the e.g. ``make-host`` step after making code change. +Running the separate commands after ``wasi.py build`` is useful if you, for example, only want to +run the ``make-host`` step after making code changes. Once everything is complete, there will be a ``cross-build/wasm32-wasi/python.sh`` helper file which you can use to run the @@ -430,8 +430,8 @@ used in ``python.sh``: .. note:: WASI uses a *capability-based* security model. This means that the WASI host - does not give full access to your machine unless you tell it to. What this - also means is things like files can end up being mapped to a different path + does not give full access to your machine unless you tell it to. This + also means things like files can end up being mapped to a different path inside the WASI host. So, if you try passing a file path to ``python.wasm``/ ``python.sh``, it needs to match the path **inside** the WASI host, not the path on your machine (much like using a container). From 592966ad1c504b90fbcf890626da8c9ac48c4f4c Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 7 Feb 2024 14:59:31 -0800 Subject: [PATCH 4/5] Clarify that `wasi.py` should be run by an installed copy of Python --- getting-started/setup-building.rst | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index eb0395921..2f579924b 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -380,14 +380,18 @@ these tools are known to work. Building for WASI requires doing a cross-build where you have a *build* Python to help produce a WASI build of CPython (technically it's a "host x host" cross-build because the build Python is also the target Python while the host -build is the WASI build). This means you effectively build CPython twice. +build is the WASI build). This means you effectively build CPython twice: once +to have a version of Python for the build system to use and another that's the +build you ultimately care about (i.e. the build Python is not meant for use by +you directly, only the build system). The easiest way to get a debug build of CPython for WASI is to use the -``Tools/wasm/wasi.py build`` command: +``Tools/wasm/wasi.py build`` command (which should be run w/ a recent version of +Python you have installed on your machine): .. code-block:: shell - $ python Tools/wasm/wasi.py build --quiet -- --config-cache --with-pydebug + $ python3 Tools/wasm/wasi.py build --quiet -- --config-cache --with-pydebug That single command will configure and build both the build Python and the WASI build in ``cross-build/build`` and ``cross-build/wasm32-wasi``, From b79aa71b0e42e2a882b192edf36ef6c350820862 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Thu, 8 Feb 2024 11:13:46 -0800 Subject: [PATCH 5/5] Update the naming of "WASI SDK" --- getting-started/setup-building.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index 2f579924b..0da5ba557 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -363,7 +363,7 @@ host/runtime as a *guest*. To build for WASI, you will need to cross-compile CPython. This requires a C compiler just like building for :ref:`Unix ` as well as: -1. A C compiler that can target WebAssembly (e.g. `WASI-SDK`_) +1. A C compiler that can target WebAssembly (e.g. `WASI SDK`_) 2. A WASI host/runtime (e.g. Wasmtime_) All of this is provided in the :ref:`devcontainer `. You can @@ -442,7 +442,7 @@ used in ``python.sh``: .. _WASI: https://wasi.dev .. _wasi-libc: https://github.com/WebAssembly/wasi-libc -.. _WASI-SDK: https://github.com/WebAssembly/wasi-sdk +.. _WASI SDK: https://github.com/WebAssembly/wasi-sdk .. _wasmtime: https://wasmtime.dev .. _WebAssembly: https://webassembly.org