Migrating NaCl toolchain From PNaCl to Saigo

[illwieckz]

Saigo allows us to get rid of PNaCl limitations (like old C++14 from Clang 3.6) before doing a bigger jump to Wasm, here are some discussions on the topic:

• Consider updating NaCl as an alternative to WebAssembly DaemonEngine/Daemon#795
• Compiling Freetype and Lua as submodules for nexe/wasm cgame/sgame #3189

Roadmap:

• Add support for Saigo into the CMake build configuration
  • Early support of the Saigo toolchain DaemonEngine/Daemon#981
  • Early support of the Saigo toolchain #2862
• Make possible to build Lua and Freetype with Saigo
  • Lua and FreeType submodule #3195
  • FreeType submodule DaemonEngine/Daemon#1446
• Provide c/sgame dependencies as submodules
  • Clone Lua repository
  • Add Lua as submodule
  • Clone Freetype repository
  • Add Freetype as submodule
• Provide Saigo in external_deps
  • Find a way to build Saigo compiler
  • Find a way to package Saigo compiler
  • Find a way to build Saigo binutils
  • Find a way to package Saigo binutils
  • Find a way to fetch Saigo libc/libc++
  • Find a way to package Saigo libc/libc++
  • Package all of that in external_deps
• Fix bugs uncovered by Saigo or Clang 19+
  • Model bug with Clang 19 DaemonEngine/Daemon#1312
• Fix bugs only happening with Saigo
  • Don't use a C++ exception when parsing numbers #3196
  • System: use nacl_minidump header from libs/, Saigo doesn't ship it unlike PNaCl DaemonEngine/Daemon#1501
  • The Saigo-built arm nexe crashes #3297
    cmake: build Saigo arm nexe with -O0 to avoid a bug DaemonEngine/Daemon#1516
  • Workarounds for Saigo Clang 21 DaemonEngine/Daemon#1878

The non-blocking issues are listed in the next comment.

[illwieckz]

Non-blocking issues:

• NaCl amd64 crash dumps no longer work on Linux DaemonEngine/Daemon#1504
  Same with PNaCl-built nexe, maybe a Linux kernel issue.
• The 32-bit arm Native Client loader requires a 4k PageSize kernel, newer RaspiOS boots a 16k PageSize kernel by default on arm64 #3286
  Same with PNaCl-built nexe, that's an NaCl loader issue.
  Make it work on 16k page size Arm kernel DaemonEngine/native_client#8
• The Linux i686 debugger provided with PNaCl doesn't work with amd64 nexe
  Same with PNaCl-built nexe.

They are listed in a separate comment so they don't affect the completeness of the issue.

---

So, it's now possible with current master to build the game with Saigo. The Saigo toolchain isn't shipped yet with the external deps so one has to do fetch and install Saigo by hand.

It currently only works on Linux because Saigo isn't built for other platforms yet.

Here follow some instructions on how to get Saigo and build with it.

Abstract:

Download and extract https://dl.unvanquished.net/deps/linux-amd64-default_10.tar.xz and put the saigo_newlib folder in daemon/external_deps/linux-amd64-default_10 to install the Saigo toolchain, then use the -DBUILD_GAME_NACL=ON CMake option to switch from the PNaCl toolchain to the Saigo toolchain before building.

Full instructions:

# Get the source
git clone https://github.com/Unvanquished/Unvanquished.git
cd Unvanquished

# Fetch all the submodules
git submodule update --init --recursive

# Install external deps with Saigo
cd daemon/external_deps
rm -rf linux-amd64-default_10
wget https://dl.unvanquished.net/deps/linux-amd64-default_10.tar.xz
tar xvf linux-amd64-default_10.tar.xz
cd linux-amd64-default_10
wget https://dl.illwieckz.net/b/saigo/preview/saigo_newlib_20241119.txz
tar xvf saigo_newlib_20241119.txz
cd ../../..

# Build with Saigo
mkdir -p build; cd build
cmake .. -DBUILD_GAME_NACL=ON -DUSE_NACL_SAIGO=ON
cmake --build . --parallel "$(nproc)"

[illwieckz]

Saigo not only builds faster, but also allows to build code requiring an higher C++ standard.

Here are some CMake command line example with options to request an higher C++ standard (using C++ 17 with GNU extensions):

cmake .. -DUSE_RECOMMENDED_CXX_STANDARD=OFF -DCMAKE_CXX_FLAGS='-std=gnu++17'

[illwieckz]

So, here is the repository where I implement the compilation build script of Saigo:

• https://github.com/DaemonEngine/DaemonSaigoNativeClientToolkit

The toolchain binaries have been successfully rebuilt for:

	Linux	Windows	macOS	FreeBSD
amd64	✅️	✅️	✅️	✅️
i686	✅️	✅️
arm64	✅️		✅️
armhf	✅️

Some informations on the binaries (here the C++ compiler and the linker for the amd64 nexe, per platform):

freebsd-amd64/prefix/bin/x86_64-nacl-clang++:     ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.3, FreeBSD-style, stripped
freebsd-amd64/prefix/bin/x86_64-nacl-ld.bfd:      ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.3, FreeBSD-style, stripped
linux-amd64/prefix/bin/x86_64-nacl-clang++:       ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=bcd4f4aff30f10c64e6515d41476459666385f2e, stripped
linux-amd64/prefix/bin/x86_64-nacl-ld.bfd:        ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=4d709354e7fef75946bfce8b24fa1727a352cd5f, stripped
linux-arm64/prefix/bin/x86_64-nacl-clang++:       ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=4d47428bd568c0bf8405b7be52e9141d646215a4, for GNU/Linux 3.7.0, stripped
linux-arm64/prefix/bin/x86_64-nacl-ld.bfd:        ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=543c7314b72e186f6da728f20d307f92bf11dcc4, for GNU/Linux 3.7.0, stripped
linux-armhf/prefix/bin/x86_64-nacl-clang++:       ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=6ca2e2971d760c15025c0bf0ac33ca4be4960f49, for GNU/Linux 3.2.0, stripped
linux-armhf/prefix/bin/x86_64-nacl-ld.bfd:        ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=52ad3c6b880745c5f16058c2734d4bae4b15dd79, for GNU/Linux 3.2.0, stripped
linux-i686/prefix/bin/x86_64-nacl-clang++:        ELF 32-bit LSB pie executable, Intel 80386, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=98e2f239f5d7123a788eaf759e66e873d5bd3e67, for GNU/Linux 3.2.0, stripped
linux-i686/prefix/bin/x86_64-nacl-ld.bfd:         ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=0936276e32670e43facdafbeed7a1c3d6d56d7fe, for GNU/Linux 3.2.0, stripped
macos-amd64/prefix/bin/x86_64-nacl-clang++:       Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE|HAS_TLV_DESCRIPTORS>
macos-amd64/prefix/bin/x86_64-nacl-ld.bfd:        Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>
macos-arm64/prefix/bin/x86_64-nacl-clang++:       Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE|HAS_TLV_DESCRIPTORS>
macos-arm64/prefix/bin/x86_64-nacl-ld.bfd:        Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>
windows-amd64/prefix/bin/x86_64-nacl-clang++.exe: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows, 11 sections
windows-amd64/prefix/bin/x86_64-nacl-ld.bfd.exe:  PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows, 10 sections
windows-i686/prefix/bin/x86_64-nacl-clang++.exe:  PE32 executable (console) Intel 80386 (stripped to external PDB), for MS Windows, 9 sections
windows-i686/prefix/bin/x86_64-nacl-ld.bfd.exe:   PE32 executable (console) Intel 80386 (stripped to external PDB), for MS Windows, 9 sections

The NaCl libc and libc++ aren't fetched yet so the built toolchains are still unusable.

Once this is done, it means the Saigo toolchain will be usable on any platform that can run the game.

With PNaCl only some Windows and Linux platforms can run the toolchain (no toolchain for Arm), it is broken with macOS and FreeBSD can't run the pexe translator on Linuxulator. So this will be a big progress, as any platform that can run the engine (and run the nacl game, either natively or through some compatibility layer) would be able to be a working development platform running the toolchain natively.

[slipher]

Debugging update:

• Interactive debugging remains somewhat usable. Tested with an old nacl-gdb.
• Postmortem crash analysis (which we actually use) does not work. Breakpad's dump_syms crashes when fed an nexe build by Saigo. Using the upstream Breakpad makes no difference.

[illwieckz]

We got the game working without exceptions using setjmp/longjmp, but I wonder if a newer nacl loader supports exceptions with Saigo (the compiler doesn't error on -fexceptions and -fcxx-exceptions if I'm right). There is no priority on that as the game builds and runs. But we may investigate this.

[illwieckz]

Here is the most recent Google build of Saigo for Linux amd64, repackaged:

• https://dl.illwieckz.net/b/saigo/preview/saigo_newlib_20250219.txz

To use it, the saigo_newlib folder should be stored in the external deps folder, so basically external_deps/linux-amd64-default_10/saigo_newlib.

It is based on Clang 20:

$ saigo_newlib/bin/x86_64-nacl-clang -v 2>&1 | head -n3
clang version 20.0.0git (https://chromium.googlesource.com/a/native_client/nacl-llvm-project-v10.git 0b939f58669775bad14c93d6d89ce046600e8eb4)
Target: x86_64-unknown-nacl
Thread model: posix

According to some file date, the build may be from January 30, 2025.

[slipher]

In DaemonEngine/Daemon#1638 we gain the ability to rebuild nacl_loader for 64-bit Linux. Note that this is not really a necessary part of a switch to Saigo: The nacl_loader, irt_core, and compiler toolchain components are seem to be freely compatible between old and recent versions.

I push some further commits to https://github.com/DaemonEngine/native_client and now it is possible to rebuild irt_core outside the Chromium ecosystem as well, given a Saigo toolchain. I added instructions atop the README. I used the https://dl.illwieckz.net/b/saigo/preview/saigo_newlib_20250219.txz posted above.

Then I tried to build irt_core with a Saigo toolchain produced from https://github.com/DaemonEngine/DaemonSaigoNativeClientToolkit, but hit a snag that most headers are missing. I found a short document about the headers stating that some are taken directly from Newlib. Oddly, I couldn't find any copies of the Newlib source in Chromium code search. I guess we'll have to hunt for a build script to see where the headers come from.

[illwieckz]

Excellent!

[illwieckz]

Then I tried to build irt_core with a Saigo toolchain produced from https://github.com/DaemonEngine/DaemonSaigoNativeClientToolkit, but hit a snag that most headers are missing.

Yes the whole nexe libc/libc++ isn't installed yet by this repository. I'll start with copying them from google builds (rebuilding them is much more complicated as it requires some very old nacl gcc to be built as well). I assume the missing headers are part of the libc/libc++.

[slipher]

Where is the build system for the libc/libc++?