cross compilation

[pmp-p]

Following discord discussion on cross compiling, trying to canvas things to do.

reminder

    build: the machine you are building on
    host: the machine you are building for
    target: the machine that compiler will produce binary for

What is really defining a target ?

it's an ecosystem defined by a combination of a libc+libstdc++ ( API ) , a linker (ABI) and an hardware architecture(CPU/ARCH) to run on.

What currently potential and usefull targets for mys ?

• emscripten ( works , happens to be the most used platform on this earth (web) !)
• wasi ( does not work yet but may be soon very interesting ).
• android ( works too , happens to be the most widespread OS on this earth )

emscripten ( runs on javascript engine or wasm , all to be found in modern browsers eg chromium 81+ )

CPU/ARCH: asm.js or wasm
API : good supports for C/C++ standards and exceptions and with limited threading, no jit no native call ( browser sandboxing )
ABI: better use static linking of PIC objects.
executables are multifile : one .html or .js , a .wasm for bitcode and possibly a .data for a posix filesystem archive ( used to provide files in browsers )

wasi ( wasm / use a runtime interpreter on bytecode )

CPU/ARCH: wasm
API : no support for C setjmp, and not C++ exceptions and limited threading, can do ffi and native calls
ABI: better use static linking of PIC objects.
executables : .wasm that use a specific ABI revision from a key/value mapping ( older key called "wasi_unstable" most recent "wasi_snapshot_preview1" ) usually evolved from "cloudabi" https://github.com/NuxiNL/cloudlibc.

very close to emscripten may merge with it sooner or later.

android ( a linux mobile kernel flavour easily embedded )

CPU/ARCH: mips arm5/arm7 arm64 x86 x86_64
API: numbered from 1 to 30 with C broken locales until api 24 but overall good C++ support, https://en.wikipedia.org/wiki/Android_version_history
ABI: ELF executables using a linux-ish custom linker located at /system/bin/linker that does not respect RPATH and RTLD_ params
must be packaged in a signed ZIP file called APK.

more info https://pythondev.readthedocs.io/android.html
signing zip/apk may require external tools or specific java versions and esoteric knowledge.

how to compile for a target ?

These three arch have usually the common problem that the build cannot run the host binaries ( except maybe (quite) complex linux build + some tweak with binfmt / proot / chroot ) or that native compilation via emulation is very slow and may require admin rights.

so short answer : use cross compilation.

toolchains are provided by vendors for all those targets or usually already supported by recent versions of clang 10+

all toolchains listed here are clang versions customized for target to run on linux x86_64 systems. It may be possible to deploy those on darwin, aosp, win32 systems but it's yet out of scope.

emscripten :
clang based easy to install with

git clone https://github.com/emscripten-core/emsdk.git
cd emsdk && ./emsdk update-tags && ./emsdk install --embedded latest && ./emsdk activate --embedded latest

wasi:
get https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-12/wasi-sdk-12.0-linux.tar.gz and unpack.
standard clang cross compilation rules apply and work with all classic posix build tools.

android:
clang toolchains ( a lot of them ) grouped under the acronym NDK ( native developpement kit )
goto https://developer.android.com/ndk/downloads and pick a know-to-work version because ndk new versions break things, have silly layout and exotic build tools and TLDR licensing. cmake may work.

good to know: github actions provide ready to use NDK CI. python-for-android/beeware projects may demonstrate how to setup a ndk headless without too much user stress.

Mys ?

Mys need to know :
which toolchain is to be used
where it is located ( and how to set it up )
which api/abi/cpu
which custom restrictions/warning to apply when transpiling : maybe use a C preprocessor ? common macro are __EMSCRIPTEN__ __wasi__ __ANDROID__ __ANDROID_API__

That's too much at once and should be decomposed to avoid "black box" effect and too huge/complicated scripts unless you plan to provide some docker/vm image or unsigned prebuilts !

proposal part 1:

three root packages:
mys-cross-emsdk mys-cross-wasi mys-cross-android that can download toolchains, present and accept licenses and set compiler up

i'm strongly in favor of using a custom python sysconfig template because it would help narrow down and maybe solve lots of pip extension building problems on c-python ecosystem ( upstream step 2 )

proposal part 2:

a parent package mys-cross-compile
a generic robust parent package files which by default would target the OLDER and MOST COMPATIBLE target of each ecosystem to ensure large audience.

run real TESTSUITES on real or emulated host and offer support only on those "old stables", and maybe integrate with pip testrunner ( upstream step 3).

proposal part 3:

use small patch files to apply versionning on the parent package from a folder with a specific name scheme.
( and maybe use https://pypi.org/project/patch in case not gnu/linux for applying conversion )

matching each target with a MULTIARCH tag inspired from python/pip and some earlied proposals on bugs.python.org (bpo) for android. https://bugs.python.org/issue?%40search_text=android+multiarch&submit=search

which CPU/ARCH to pass to the underlying C++ compiler ( clang )
which API to select
which ABI to interface

that method also allow extra tag like "-testmynewapi" on simple and small patches folders to test contributors proposals.

draft for naming : pmp-p/pydk#22

eg:

./mys/cli/templates/build/Makefile
./mys/lib/mys.cpp
./mys/transpiler/source_visitor.py

When addressed by mys build/install with a --target parameter.
for eg --target=wasm_1_emscripten you would have those to apply from the matching target folder:

wasm_1_emscripten/cli/templates/build/Makefile.diff
wasm_1_emscripten/lib/mys.cpp.diff
wasm_1_emscripten/transpiler/source_visitor.py.diff

try to do it as much as "pip/MULTIARCH" as possible and eventually ( upstream step 1 )

---

Sidenotes :

• All this process as been used for real ( except automated testing) with cpython/panda3d that includes ** a lot ** of third parties that mys can gain access to at least for emscripten and android via https://github.com/pmp-p/pydk

• As one can see upstreaming to pip steps order is not the same as Mys logical progress that may explain why cross compilation is stalled on Python ecosystem.