Embedded LLD Phase 7 Prerequisite: Native sanitizer linking (Linux)

[SeanTAllen]

Implementation plan for Embedded LLD,
the prerequisite work for Phase 7 (Cleanup). Phase 7 removes the legacy
system() linker and the --linker/--link-ldcmd options. That removal
can't happen while live code still routes through the legacy path on its own,
without anyone asking for --linker. The
Phase 7 scoping comment
identified two things that still do: native sanitizer builds, and FreeBSD
use=dtrace.

This document covers the native sanitizer half, and Linux only. macOS
and the BSDs run the same legacy fallback for native sanitizer builds, but the
plan is to land Linux first, where the one sanitizer CI job actually runs and
the work can be verified end to end, then take what's learned there to macOS
and the BSDs as separate follow-ups. The FreeBSD dtrace question and the
legacy-code deletion itself are also separate follow-ups, not part of this
work.

Divergences

One behavior change from what exists today.

Aspect	Today	Proposed	Why
Native Linux link, sanitizer build	Legacy compiler-driver path (system()), so the driver can pass -fsanitize=… and pull in the sanitizer runtime	Embedded LLD (link_exe_lld_elf), with the sanitizer runtime linked explicitly	Removes the last self-routing use of the legacy path on native Linux, unblocking its deletion in Phase 7

Non-sanitizer native Linux builds already link through embedded LLD and don't
change. Sanitizer cross-compilation, macOS, and the BSDs don't change either:
their guards stay in place.

The problem

link_exe (src/libponyc/codegen/genexe.cc:1945) dispatches to embedded LLD.
The Linux branch reads:

if(c->opt->linker == NULL
  && target_is_linux(c->opt->triple)
#if defined(PONY_SANITIZER)
  && is_cross_compiling(c)
#endif
  )
  return link_exe_lld_elf(c, program, file_o);

In a non-sanitizer build the #if block is absent, so native Linux already
uses embedded LLD. The guard only bites in a sanitizer build: there, a native
link falls through to the legacy compiler-driver path so the driver can pass
-fsanitize=….

That flag isn't decoration. When ponyc is built with sanitizers, libponyrt's C
code is instrumented, so every executable ponyc links references __asan_*
(or the ubsan equivalents) and has to link the sanitizer runtime. The embedded
ELF path doesn't add that runtime today. Drop the guard without adding it and
every native sanitizer link fails with undefined __asan_* symbols.

PONY_SANITIZER is a build-time string ("address,undefined", "thread", and so
on), set in CMakeLists.txt from use=address_sanitizer /
undefined_behavior_sanitizer / thread_sanitizer. The only sanitizer build
CI exercises is the weekly with_sanitizers job: x86-64 Linux glibc,
address_sanitizer,undefined_behavior_sanitizer, native, debug and release.

What the compiler driver emits

Verified with clang -fsanitize=address,undefined -### on x86-64 Linux. The
driver inserts, immediately before the object file:

--whole-archive  libclang_rt.asan_static-x86_64.a  --no-whole-archive
--whole-archive  libclang_rt.asan-x86_64.a          --no-whole-archive
--dynamic-list=  libclang_rt.asan-x86_64.a.syms

and, after the object, the system libraries -lrt -lresolv on top of the
-lpthread -lm -ldl ponyc already emits. undefined alone resolves to
ubsan_standalone; thread to tsan; combining undefined with
address/thread folds ubsan into the larger runtime, so there's no separate
ubsan_standalone. Every one of these flags is accepted by LLD's ELF driver.

Approach: capture the sanitizer link fragment at build time

This follows the recommendation in section 9 of #4941: query the compiler at
build time for the runtime paths and embed them as compile-time constants.
PONY_SANITIZER is already a build-time decision, and a sanitizer-enabled
ponyc is a developer and CI artifact that runs on the machine that built it, so
build-machine-absolute paths in the binary are fine.

Asking the driver for the fragment, rather than resolving each library by name
with --print-file-name, is the point. The driver owns the mapping from
-fsanitize=X to a set of libraries, the ubsan-folding rule, and the
whole-archive/dynamic-list wrapping. Resolving libraries by name means
re-implementing all of that in ponyc's build, and that hidden knowledge is
exactly what rots when a compiler version changes its sanitizer runtime layout.
Capture sidesteps it: whatever combination PONY_SANITIZER names, the driver
reports the exact fragment, address and undefined and thread and any future
combination alike.

CMake

Done in pure CMake. Python is not a ponyc build dependency, and this work
doesn't make it one; CMake's list operations are enough.

1. Compile a tiny dummy object at configure time. -### emits a link line only
   for a real input.
2. Run two queries, capturing stderr, with the same -march the native build
   uses: one with -fsanitize=<value>, one without, as a baseline. Query the C
   compiler (the one that compiles and instruments libponyrt), so the resolved
   runtime matches the instrumentation.
3. Tokenize both link lines into CMake lists. The sanitized line is a superset
   of the baseline (a set-difference confirms nothing is removed). A two-pointer
   ordered scan emits each sanitized token that doesn't match the next
   unconsumed baseline token: exactly what -fsanitize injected, in driver
   order, the whole-archive/dynamic-list block and the extra system libraries
   both. No library names or wrapping flags are hardcoded. The implementation
   confirms the driver doesn't reorder the non-sanitizer tokens (the scan
   consumes every baseline token in order); if that ever failed, the fallback is
   to extract the contiguous clang_rt block and append the glibc-standard
   sanitizer system libraries explicitly.
4. Generate a header declaring the captured fragment as a
   static const char* const[] plus its count, only inside the sanitizer
   branch, and add its directory to libponyc's include path.

genexe.cc

Remove the is_cross_compiling guard from the Linux dispatch branch. Splice the
captured arguments into link_exe_lld_elf immediately before the object file,
gated on !is_cross_compiling(c). The gate matters because link_exe_lld_elf
serves both the Linux and BSD branches, and the captured paths are
host-arch-absolute, so they must not reach a cross link. The four cases:

• native Linux + sanitizer: enters here, not cross, fragment spliced. This is
  the fix.
• cross Linux + sanitizer: enters here, cross, fragment suppressed. No
  host-arch paths in a cross link. Unexercised by CI and without a sanitizer
  runtime today either, so behavior is preserved, not regressed.
• native BSD + sanitizer: never enters here. The BSD guard stays intact, so the
  native BSD sanitizer build still falls to legacy.
• cross BSD + sanitizer: enters here, cross, fragment suppressed. Same as cross
  Linux.

The splice carries a comment recording that the function serves both branches,
why the gate is there, and the ordering. The whole-archived clang_rt static
archives precede the object so their members are pulled in. The injected -lrt
and -lresolv aren't re-emitted later by ponyc, so they appear only here,
before the object, which is safe because they're shared libraries: DT_NEEDED
placement isn't position-sensitive the way static-archive symbol resolution is.
The header include is guarded to match the generated header's sanitizer-only
existence, which also keeps the array out of non-sanitizer translation units and
clear of the unused-variable warning under -Werror.

The legacy path keeps its own -fsanitize= handling. Legacy stays reachable
through --linker, and a sanitizer build forced down that path still needs it.
The flag goes dead only for the default native case, which is the whole point.

Verification

All on an x86-64 Linux host, mirroring the weekly with_sanitizers job.

1. Counterfactual: with the guard removed but the fragment not yet spliced, a
   native sanitizer link must fail with undefined __asan_* symbols. Confirms
   the splice is load-bearing.
2. Build a sanitizer ponyc
   (use=pool_memalign,address_sanitizer,undefined_behavior_sanitizer), link
   and run a small Pony program through it, confirm it links via embedded LLD
   (verbose output shows ld.lld with the clang_rt arguments and no
   system() driver call) and that the sanitizer runtime is present. Pony code
   isn't instrumented, only libponyrt's C is, so the live signal is the
   runtime's presence and a fault triggered in instrumented runtime code, not a
   Pony-level fault.
3. Run test-ci-core debug and release under the same ASAN_OPTIONS and
   UBSAN_OPTIONS the CI job uses; its full-program tests are the ones that
   link executables through link_exe.

No CI workflow change is needed; with_sanitizers already builds and tests, and
will now link through embedded LLD instead of the driver. The change is internal
build and linker plumbing with no user-facing difference, so there are no
release notes.

Out of scope, tracked as follow-ups

• Native sanitizer linking on macOS and the BSDs.
• FreeBSD use=dtrace on embedded LLD.
• Deleting the legacy system() linker and the --linker/--link-ldcmd
  options (Phase 7 proper).

[SeanTAllen]

Update on the capture mechanism, now that it's implemented and verified.

The original plan above captured the sanitizer link fragment for Clang only and left macOS, the BSDs, and GCC on the legacy path. That last one is a problem: the whole point of this work is to let Phase 7 delete the legacy system() linker, and ponyc supports GCC as a build compiler. Keeping GCC on the legacy path would mean the legacy path can never actually die. So the implementation covers both Clang and GCC native sanitizer builds on Linux.

Both compilers go through the same mechanism: at configure time, run the driver twice with -### (with and without -fsanitize=), and take the ordered difference of the two link lines. Whatever the driver adds is the fragment, in driver order. The difference between compilers lives entirely in what the driver emits — Clang whole-archives its libclang_rt static archives and adds a --dynamic-list; GCC adds libasan_preinit.o and links shared -lasan/-lubsan. Neither shape is hardcoded, so this also rides over Clang's runtime-directory layout differences across versions (the flat lib/linux/libclang_rt.asan-<arch>.a vs the per-target lib/<triple>/libclang_rt.asan.a) without caring which one a given Clang uses.

Two wrinkles surfaced while making the diff work for both. The link line has to be tokenized respecting shell quoting, because Clang double-quotes every argument and GCC leaves most bare; separate_arguments(UNIX_COMMAND) handles both. And GCC's LTO plugin injects a -fresolution=<tmp>.res argument with a fresh temp path on every invocation, which differs between the two probe runs and desyncs the diff — filtering that one volatile token restores a clean supersequence. With both in place the diff consumes the baseline exactly and yields just the sanitizer additions: for address,undefined, Clang gives the --whole-archive/clang_rt/--dynamic-list block plus -lrt -lresolv; GCC gives libasan_preinit.o -lasan -lubsan.

If a future driver ever violates the supersequence assumption — a new non-deterministic token, or reordered arguments — the capture stops at configure time with a clear error rather than emitting a fragment it can't trust.

Verified on this host with both compilers: full builds, a program linking through embedded LLD with the sanitizer runtime live (for GCC, libasan.so.8/libubsan.so.1 resolve as DT_NEEDED), and the core test suite passing under ASan plus UBSan, 2018 tests, on each of Clang and GCC. CI exercises the Clang sanitizer build only, matching the project's clang-testing policy; GCC sanitizer support is best-effort, made to work here and fixed reactively if a report comes in, not gated in CI.

Still scoped to native Linux. macOS, the BSDs, FreeBSD dtrace, and the legacy-code deletion itself remain separate follow-ups.

[SeanTAllen]

Implemented and verified (clang + gcc): PR #5409.