Skip to content

Fix ZJIT test linking#16479

Closed
nirvdrum wants to merge 2 commits intoruby:masterfrom
nirvdrum:fix-zjit-test-linking
Closed

Fix ZJIT test linking#16479
nirvdrum wants to merge 2 commits intoruby:masterfrom
nirvdrum:fix-zjit-test-linking

Conversation

@nirvdrum
Copy link
Copy Markdown
Contributor

I ran into an issue running the ZJIT test suite on a macOS system using a build with both YJIT and ZJIT enabled. It appears to be the result of duplicate ZJIT symbols and the undefined way they're resolved. As far as I can tell, we have ZJIT symbols in both the ZJIT lib built for testing as well as libruby.o. Since the test build does not include YJIT code, resolving any YJIT symbols loads libruby.o, which also contains ZJIT. If a ZJIT symbol is resolved from libruby.o instead of the test binary, we can end up with uninitialized global state, manifesting in issues such as:

    running 1 test
    test hir_type::tests::integer_has_ruby_class ... FAILED

    failures:

    failures:
        hir_type::tests::integer_has_ruby_class

    test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 1521 filtered out; finished in 0.01s

  stderr ───

    thread 'hir_type::tests::integer_has_ruby_class' (19194206) panicked at zjit/src/options.rs:485:28:
    called `Option::unwrap()` on a `None` value
    stack backtrace:
       0: __rustc::rust_begin_unwind
       1: core::panicking::panic_fmt
       2: core::panicking::panic
       3: core::option::unwrap_failed
       4: core::option::Option<T>::unwrap
                 at /nix/var/nix/builds/nix-98420-2996139808/rustc-1.93.0-src/library/core/src/option.rs:1016:21
       5: zjit::options::update_profile_threshold
                 at ./src/options.rs:229:53
       6: zjit::options::set_call_threshold
                 at ./src/options.rs:495:5
       7: zjit::cruby::test_utils::boot_rubyvm
                 at ./src/cruby.rs:1173:17
       8: core::ops::function::FnOnce::call_once
                 at /nix/var/nix/builds/nix-98420-2996139808/rustc-1.93.0-src/library/core/src/ops/function.rs:250:5
       9: std::sync::once::Once::call_once::{{closure}}
                 at /nix/var/nix/builds/nix-98420-2996139808/rustc-1.93.0-src/library/std/src/sync/once.rs:159:41
      10: std::sys::sync::once::queue::Once::call
      11: std::sync::once::Once::call_once
                 at /nix/var/nix/builds/nix-98420-2996139808/rustc-1.93.0-src/library/std/src/sync/once.rs:159:20
      12: zjit::cruby::test_utils::with_rubyvm
                 at ./src/cruby.rs:1203:22
      13: zjit::hir_type::tests::integer_has_ruby_class
                 at ./src/hir_type/mod.rs:819:9
      14: zjit::hir_type::tests::integer_has_ruby_class::{{closure}}
                 at ./src/hir_type/mod.rs:818:32
      15: core::ops::function::FnOnce::call_once
                 at /nix/var/nix/builds/nix-98420-2996139808/rustc-1.93.0-src/library/core/src/ops/function.rs:250:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

        FAIL [   0.031s] zjit hir_type::tests::set_has_ruby_class
  stdout ───

    running 1 test
    test hir_type::tests::set_has_ruby_class ... FAILED

    failures:

    failures:
        hir_type::tests::set_has_ruby_class

    test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 1521 filtered out; finished in 0.01s

There are two commits in this PR, but only one should be retained. The two commits attempt to resolve the issue in different ways. The first provides stubs for all of the YJIT symbols in the test build so libruby.o doesn't need to be loaded. This, however, requires keeping that list synchronized and given the non-deterministic nature of the issue, missing one stub may not reliably induce a test failure.

The second approach uses a build script to construct the list of stubs instead of hard-coding the list. It's more reliable, and my preferred option, but does complicate the build script. It currently builds on top of the first commit, but I'll squash them if we go this route.

nirvdrum and others added 2 commits March 20, 2026 10:40
`make zjit-test` can fail on macOS because the test binary ends up with
two copies of ZJIT code: one from the Rust test crate (compiled by Cargo)
and one from `libruby.o` inside `libminiruby.a`. Each copy has its own
independent global state (`OPTIONS`, `ZJIT_STATE`, etc.), so initialization
through one copy leaves the other uninitialized, causing panics at runtime.

The linker is forced to pull `libruby.o` from the archive because C code
references YJIT symbols (e.g., `rb_yjit_init`, `rb_yjit_bop_redefined`)
that only `libruby.o` defines. When `libruby.o` is pulled in for YJIT, it
brings along duplicate ZJIT globals. On Linux, `--allow-multiple-definition`
masked the issue. On macOS, the linker silently resolves duplicates but can
pick the wrong definition for cross-codegen-unit GOT entries, splitting
state.

This eliminates the root cause by preventing `libruby.o` from being linked
into the test binary:

- `zjit/build.rs` now copies `libminiruby.a` to Cargo's `OUT_DIR` and runs
  `ar d` to remove `libruby.o` before linking. The Linux
  `--allow-multiple-definition` workaround is no longer needed.

- A new `zjit/src/yjit_stubs.rs` module (compiled only for `#[cfg(test)]`)
  provides no-op definitions for all 48 YJIT symbols that the Ruby VM's C
  code references. These satisfy the linker without pulling in `libruby.o`
  or `yjit.o` from the archive. The stubs are safe because YJIT is never
  enabled during ZJIT tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… hand

Replace the manually-maintained yjit_stubs.rs with build-time generation
in build.rs. The build script now:

1. Runs `nm -u` on the cleaned archive to discover all undefined YJIT
   symbols that the remaining C objects reference.
2. Parses `yjit.h` to identify which symbols are variables (extern
   declarations) vs functions.
3. Generates a minimal C file with zero-initialized variables and void()
   function stubs.
4. Compiles it with the system C compiler and inserts the resulting
   object into the archive.

This stays synchronized automatically whenever YJIT's symbol set changes,
with no manual maintenance required.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
XrXr added a commit to Shopify/ruby that referenced this pull request Mar 23, 2026
Closes <ruby#16479>.
It was reported that the duplicate ZJIT symbols among libminiruby and
the ones built into the test binary in zjit-test cause test failures.
This patch removes the duplication, and as such also removes
`--allow-multiple-definition` when on Binutils.
@nirvdrum
Copy link
Copy Markdown
Contributor Author

I'm closing this out in favor of #16503. Adjusting the build system is the cleaner fix.

@nirvdrum nirvdrum closed this Mar 23, 2026
XrXr added a commit that referenced this pull request Mar 23, 2026
Closes <#16479>.
It was reported that the duplicate ZJIT symbols among libminiruby and
the ones built into the test binary in zjit-test cause test failures.
This patch removes the duplication, and as such also removes
`--allow-multiple-definition` when on Binutils.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant