Skip to content

[DRAFT] add allocator libraries compatible with the new rustc mangling #3403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from

Conversation

krasimirgg
Copy link
Collaborator

@krasimirgg krasimirgg commented Apr 10, 2025

This adds a version of the allocator support libraries implemented in rust, to account for the new internal allocator symbol mangling applied by rustc (rust-lang/rust#127173).

This mechanism relies on unstable language features and requires a nightly rustc from 2025-04-05 or later. It is designed to be compatible with:

  • all allocator flavors (the default and global)
  • all flavors of no_std
  • both situations for when a rust_library is used as a dependency of a cc_binary and when the experimental_use_cc_common_link setting is used.

Rustc generates references to internal allocator symbols when building rust libraries. At link time, rustc generates
the definitions of these symbols. When rustc is not used as the final linker, we need to generate the definitions
ourselves. This happens for example when a rust_library is used as a dependency of a rust_binary, or when the
experimental_use_cc_common_link setting is used.

For older versions of rustc, the allocator symbol definitions can be provided via the rust_toolchain's
allocator_library or global_allocator_library attributes, with sample targets like
//ffi/cc/allocator_library and //ffi/cc/global_allocator_library.

Recent versions of rustc started mangling these allocator symbols (rust-lang/rust#127173). The
mangling uses a scheme that is specific to the exact version of the compiler. This makes the cc allocator library
definitions ineffective. To work around this, we provide rust versions of the symbol definitions annotated with an
unstable language attribute that instructs rustc to mangle them consistently. Because of the usage of unstable language features, this is only compatible with nightly versions of the compiler.

Since the new symbol definitions are written in rust, we cannot just attach them as attributes on the rust_toolchain
as the old cc versions, as that would create a build graph cycle:

  1. any rust_library depends on a rust_toolchain,
  2. the rust_toolchain depends on the allocator library,
  3. so the allocator library cannot just be a rust_library directly.

The bootstrapping cycle can be avoided by defining a separate internal "initial" rust toolchain specifically for
building the rust allocator libraries, and use a transition to attach the generated allocator libraries to the "main" rust
toolchain. But that duplicates the whole subgraph of the build around the rust toolchains, repository and supporting
tools used for them.

Instead, we define a new custom rust_allocator_libraries rule, which builds the new rust allocator libraries and exposes them via a new AllocatorLibrariesInfo provider. Since we cannot attach such a target as an attribute to the rust_toolchain, we attach it as a new allocator_libraries common attribute to the rust rules. We ported the cc versions into the new rust implementations of the (default) allocator and global allocator flavors in //ffi/rs/allocator_library and //ffi/rs/global_allocator_library.

The rust standard libraries themselves have references to the allocator libraries. For correct linking order of the standard libraries, we need to establish the new rust allocator libraries as link-time dependencies of the standard libraries. The specific set of linker inputs depends on the no_std flavor and the choice of (default) allocator vs. global allocator. We establish the linking dependencies by extending the rust_toolchain _make_libstd_and_allocator_ccinfo feature to let us inject a custom allocator library as a standard library dependency and using that in the rust_allocator_libraries implementation to generate the final standard libraries linker graph.

The new functionality is opt-in and gated via:

  1. the new setting //rust/settings:experimental_use_allocator_libraries_with_mangled_symbols, which supplies the default value for the repository, and
  2. the new rust_toolchain attribute experimental_use_allocator_libraries_with_mangled_symbols, which lets the user override the choice of using the new-style rust-based allocator symbols, or the old-style cc-based ones for a specific toolchain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant