Move all LLVM externs into the rustc_llvm crate

[Diggsey]

#17795 expresses a desire to test the LLVM bindings with ctest. A pre-requisite for that is to have all the LLVM externs in once place, rather than spread across multiple crates.

Actually testing with ctest is blocked because the current version of ctest uses a long out-of-date parser which cannot cope with eg. extern blocks among other things. However there appears to be active work to build a new version of ctest (see https://github.com/rust-lang/libc/tree/main/ctest-next)

This PR does not intend to change any behaviour, only to move all FFI code into the rustc_llvm crate so that it may be tested at some future time.

[rustbot]

r? @cuviper

rustbot has assigned @cuviper.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

triagebot.toml has been modified, there may have been changes to the review queue.

cc @davidtwco, @wesleywiser

Some changes occurred in compiler/rustc_codegen_llvm/src/builder/autodiff.rs

cc @ZuseZ4

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

Some changes occurred in compiler/rustc_codegen_llvm/src/llvm/enzyme_ffi.rs

cc @ZuseZ4

Some changes occurred in coverage instrumentation.

cc @Zalathar

[rust-log-analyzer]

The job x86_64-gnu-llvm-19 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

#19 exporting to docker image format
#19 sending tarball 26.9s done
#19 DONE 34.8s
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-19]
[CI_JOB_NAME=x86_64-gnu-llvm-19]
debug: `DISABLE_CI_RUSTC_IF_INCOMPATIBLE` configured.
---
sccache: Listening on address 127.0.0.1:4226
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-19', '--enable-llvm-link-shared', '--set', 'rust.randomize-layout=true', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'build.print-step-timings', '--enable-verbose-tests', '--set', 'build.metrics', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--set', 'gcc.download-ci-gcc=true', '--enable-new-symbol-mangling']
configure: build.build          := x86_64-unknown-linux-gnu
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-19/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.randomize-layout := True
configure: rust.thin-lto-import-instr-limit := 10
---
[RUSTC-TIMING] rustc_driver_impl test:false 8.320
   Compiling rustc_driver v0.0.0 (/checkout/compiler/rustc_driver)
error: linking with `cc` failed: exit status: 1
  |
  = note:  "cc" "-Wl,--version-script=/tmp/rustcRjp3Sl/list" "-Wl,--no-undefined-version" "-m64" "/tmp/rustcRjp3Sl/symbols.o" "<1 object files omitted>" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/rustc_driver-34400ccebd970507.3oaphi11shgvqpb0ixxffsd77.rcgu.rmeta" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/{librustc_driver_impl-52ffae76512c0b8a.rlib,libctrlc-013681b5285cdaeb.rlib,libnix-19b3b98101b1f1f8.rlib,librustc_log-eb74dc97849e6186.rlib,libtracing_tree-fcfa9b1e06c74b55.rlib,libtracing_log-6873c449fd0d0779.rlib,libnu_ansi_term-5417242e32ad9045.rlib,libtracing_subscriber-b237629531702914.rlib,libnu_ansi_term-45a568220e5b71b6.rlib,liboverload-028dc2ad05f88120.rlib,libsharded_slab-0dbd06c5cef1ce63.rlib,liblazy_static-e4c143cd1490ea98.rlib,libmatchers-ab478bd58c20cfea.rlib,libregex_automata-4c548855388c22f3.rlib,libregex_syntax-ac6c6ce25fda5984.rlib,libthread_local-5adc05e9be496341.rlib,libjiff-5ca1282c467d0bf6.rlib,libstable_mir-7c8caa85bea1981b.rlib,librustc_smir-0af3be5d841053b4.rlib,librustc_interface-bf3a259b98685d93.rlib,librustc_codegen_llvm-7ecf747e7b5253ed.rlib,librustc_llvm-0e8185cff39ae3c3.rlib,librustc_sanitizers-9f9dd76969405fb4.rlib,librustc_hir_typeck-2c98eafcabf1c831.rlib,librustc_hir_analysis-f685b486f7cdaf0a.rlib,librustc_monomorphize-8be63df59214c878.rlib,librustc_mir_transform-04b870e86b1ea6f1.rlib,librustc_mir_build-8ff196aea70cc243.rlib,librustc_pattern_analysis-c1f55887a8fad970.rlib,librustc_borrowck-0418db4395cb5758.rlib,librustc_traits-396c617cb94e32c3.rlib,librustc_const_eval-bbcd4354ffccef2f.rlib,librustc_mir_dataflow-21cf9fb6b461760d.rlib,librustc_builtin_macros-497aa2a07f2eacbc.rlib,librustc_resolve-76c20548a52b47b8.rlib,libpulldown_cmark-9a2f027550b860aa.rlib,libunicase-7b146e1d394817cf.rlib,libpulldown_cmark_escape-b9f7564d289aae6c.rlib,librustc_passes-d694f17eb3222b7e.rlib,librustc_ast_lowering-e01ccd8d815ce723.rlib,librustc_privacy-b3538cf4d166e786.rlib,librustc_ty_utils-b229fd8c017decc0.rlib,librustc_lint-5a9cc7492ba6d091.rlib,libunicode_security-0bbecb5329b3551a.rlib,libunicode_script-ef7f1a587f84f904.rlib,librustc_query_impl-51d372e45f42ee42.rlib,librustc_codegen_ssa-271895422dabcde0.rlib,libwasm_encoder-f7ea881594178a63.rlib,libleb128-58d3d5507e2819d3.rlib,libthorin-323c01c5b57fb2f8.rlib,libgimli-d9fead7ea8600c7c.rlib,libfallible_iterator-eed05a88b93f6a17.rlib,libbstr-dd56532dfb199fe0.rlib,librustc_trait_selection-736a39064c385c9a.rlib,librustc_next_trait_solver-7c5aebf00245359c.rlib,librustc_parse_format-e7e672113965142f.rlib,librustc_transmute-e4150848e281ac05.rlib,librustc_infer-94c523db2075ddcd.rlib,librustc_incremental-ade8590f494773ab.rlib,librustc_symbol_mangling-961a975538b840e7.rlib,librustc_demangle-dd5a84401da95ae1.rlib,libpunycode-d652322daff78ac3.rlib,libpathdiff-d88f77b06587ec53.rlib,libregex-050454cb8770d763.rlib,libregex_automata-11523339afff3634.rlib,libaho_corasick-0b468428e64453c9.rlib,libregex_syntax-6decbe19c0b2aaaa.rlib,libcc-13691ab27340e2ee.rlib,libshlex-5deb4bad964e93da.rlib,libar_archive_writer-824d23136dc19749.rlib,libobject-a6febaa0b91ca511.rlib,libruzstd-7df0486a5058b532.rlib,libtwox_hash-54acf34b990d5090.rlib,librand-d955395bc57cb6a3.rlib,librand_chacha-70b0ba5b73af6607.rlib,librand_core-a30e006543c4fc28.rlib,libgetrandom-c7dd2e9dbb95ef13.rlib,libstatic_assertions-04dd6bd89460663c.rlib,libflate2-4abb987ec168b61f.rlib,libminiz_oxide-23ffddebcc8fda33.rlib,libadler2-e7cc772b0334c48c.rlib,librustc_metadata-e834352f56128475.rlib,liblibloading-af20115b9f067569.rlib,librustc_expand-a8233fc82fa0e70e.rlib,librustc_ast_passes-947bd7f1f82693b6.rlib,librustc_proc_macro-970f6284f25c31bd.rlib,librustc_parse-3ff548a4c78ca44f.rlib,libunicode_normalization-7e7d2b712ee483cc.rlib,libtinyvec-2dcf63617ff4c0b9.rlib,libtinyvec_macros-50b80f3d008a4189.rlib,librustc_attr_parsing-85c2e5231b76a484.rlib,librustc_middle-3a12640d73bc1cac.rlib,librustc_apfloat-1a5099fb8ff271d1.rlib,libgsgdt-1a5f2a2217d74218.rlib,libpolonius_engine-9b0cd5c3d0daa96d.rlib,libdatafrog-0345bcea204e6544.rlib,librustc_query_system-ced1994ca738f089.rlib,librustc_session-6af80cd27e7d1569.rlib,libgetopts-afe6940913304150.rlib,librustc_feature-305a085d932bb076.rlib,librustc_hir_pretty-ba42e425862f66e9.rlib,librustc_errors-0400a61b4e40380b.rlib,libtermize-af231eb4f8a15111.rlib,librustc_error_codes-5fc2580158d12b4b.rlib,librustc_type_ir-f2fc8f3ffbd39329.rlib,libannotate_snippets-463d1128677c16f1.rlib,libanstyle-85660c8c87095b29.rlib,libtermcolor-f95b4c759008a181.rlib,librustc_lint_defs-3654b1a122c85c46.rlib,librustc_error_messages-09c44d831ee3182f.rlib,librustc_baked_icu_data-48bb3eee3f5663ff.rlib,libicu_list-d886d0d295d622a7.rlib,libicu_list_data-88572c18c41316b2.rlib,libregex_automata-5ddc29ef364f4dfa.rlib,libicu_provider_adapters-00374e7b18389ad7.rlib,libicu_locid_transform-654c2a51ffeac097.rlib,libicu_locid_transform_data-1f6402223b499fbb.rlib,libicu_provider-6885ee058b1d42fb.rlib,libicu_locid-203f999f22739ef4.rlib,liblitemap-0ea0f7f05e59c9fe.rlib,libtinystr-4555add1884dc014.rlib,libzerovec-27924c1992a40814.rlib,libwriteable-b3867fc7342e684f.rlib,libyoke-463d6b6f048f12cb.rlib,libzerofrom-b72bf09e0271accf.rlib,libfluent_bundle-f99cf2a2692a79a1.rlib,libfluent_langneg-fb9296fdc25e342b.rlib,libintl_pluralrules-7a92f8b959934df0.rlib,libself_cell-9a94c0a530c88bf6.rlib,libintl_memoizer-3dcc8c7b541d45cd.rlib,libtype_map-15eaf672474c5f5c.rlib,libunic_langid-aa7f2b22f6fe5340.rlib,libunic_langid_macros-e6b7ffdca10adff2.rlib,libunic_langid_impl-b41eed1e7bfaee33.rlib,libtinystr-be557cfc5ffe365e.rlib,libfluent_syntax-fc9e6a3bd3601128.rlib,libthiserror-07add9a62fc5f77c.rlib,librustc_hir-fb0c4cea6ae61d68.rlib,libodht-167003f37387060b.rlib,librustc_target-a93181816f1b1209.rlib,libobject-279718b59850bedb.rlib,libcrc32fast-33522421df7d16d9.rlib,libwasmparser-39bc14a31c7c8bac.rlib,librustc_fs_util-0ec66fce4ee9cf48.rlib,libserde_json-3afdbe5aa6730e4e.rlib,libryu-a1f1633c91843935.rlib,libserde-4878a9049b9e0c2a.rlib,librustc_attr_data_structures-ec7a849ca9355863.rlib,librustc_ast_pretty-de839e3b02ee64ce.rlib,librustc_lexer-7511fdf552f13ed5.rlib,libunicode_xid-5c815a4d18474a8b.rlib,libunicode_properties-b5c900d196b2c76a.rlib,libitertools-28427f6ecd16ea65.rlib,librustc_abi-4e387c7856e896d0.rlib,librand_xoshiro-76aa1fc395675bd2.rlib,librand-88b57c197f8af9bf.rlib,librand_chacha-da6689f153e1513c.rlib,libppv_lite86-2a726be6688b1fd7.rlib,libzerocopy-5647e32066beb24c.rlib,librand_core-82bab5ce68d325db.rlib,librustc_ast-7d8134255a1dbfe9.rlib,libmemchr-1bbbd25962d969ed.rlib,librustc_ast_ir-cfe469a564df569d.rlib,librustc_literal_escaper-bcc66caa6b1800a4.rlib,librustc_span-1107932744cc495c.rlib,libunicode_width-f847fcd963526f7e.rlib,libblake3-1bb51fb138af1087.rlib,libconstant_time_eq-00d603b80a01fd68.rlib,libarrayref-9d427196e124bab6.rlib,libitoa-f4f20165a4e2556d.rlib,libscoped_tls-c990a5c466658562.rlib,libsha2-fba72b326adc9445.rlib,libsha1-b1f441bc57535024.rlib,libcpufeatures-b2bdacd19b060d69.rlib,libmd5-534490611c61c902.rlib,libdigest-bd9e82fc605d8fc8.rlib,libblock_buffer-059f045d049c0756.rlib,libcrypto_common-d129f33f97567a75.rlib,libgeneric_array-1d860bdd56a3772c.rlib,libtypenum-9dbe078f0db44bd2.rlib,librustc_data_structures-89e980d57b738e38.rlib,libelsa-0877aa3a4aacd06b.rlib,libstable_deref_trait-bbe56d02fd895a64.rlib,librustc_thread_pool-b6d4bc4a4adac2cd.rlib,libcrossbeam_deque-b36287de5afb7e33.rlib,libcrossbeam_epoch-bb0e86b5362fc60d.rlib,libcrossbeam_utils-5c1978bba55ab52a.rlib,libstacker-7a930ff4c3eded65.rlib,libpsm-d66348f3cddb8058.rlib,librustc_arena-a0f8473031421b14.rlib,libtempfile-9b3b7d88a7f7f1e7.rlib,libgetrandom-2c0db444209ee03f.rlib,libfastrand-ad3a768c9eb043d9.rlib,librustix-07bae014dc6c8d80.rlib,libbitflags-3b517991b12954b9.rlib,liblinux_raw_sys-a51d68dff6bd5b7a.rlib,libarrayvec-94d65f5611aba573.rlib,libeither-5d8b437a692dacb1.rlib,libmeasureme-299e6048d5697509.rlib,librustc_hash-1338aa8fb37c8a58.rlib,libperf_event_open_sys-b0b9213b61a19b80.rlib,libmemmap2-186bdb8d63e99cf2.rlib,librustc_graphviz-2847c0ab5340c15f.rlib,libparking_lot-66293082e4314a39.rlib,libparking_lot_core-44c5d7d08ef06963.rlib,liblock_api-a1c4310830e1612c.rlib,libscopeguard-dbec1508fb35b2f9.rlib,libjobserver-6601a31fa13af791.rlib,liblibc-06024cf60798dd6b.rlib,libtracing-c12261008cc0da43.rlib,libcfg_if-ad991742b793b4f2.rlib,libpin_project_lite-62e41bf0c283a538.rlib,libtracing_core-5964735060598bc1.rlib,libonce_cell-099ed8eca7128110.rlib,librustc_hash-15c104e147413582.rlib,librustc_index-9499e5ec30752e46.rlib,librustc_serialize-c9e900a8eff6ee5f.rlib,libindexmap-c6601dccfd206e26.rlib,libhashbrown-e706ceae84d73654.rlib,libfoldhash-e320545c9de2d8bb.rlib,libequivalent-26218c8da007351f.rlib,libthin_vec-5b457f9d78ae7bb1.rlib,librustc_hashes-d1c029d697fdae22.rlib,librustc_stable_hash-47f11f4c80f06b69.rlib,libsmallvec-f82fe871448e84cf.rlib,libena-7ad5940f4558cb88.rlib,liblog-1f0c626ed107bf4e.rlib}.rlib" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*,libcfg_if-*,liblibc-*,liballoc-*,librustc_std_workspace_core-*,libcore-*,libcompiler_builtins-*}.rlib" "-Wl,-Bdynamic" "-lLLVM-19" "-ldl" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-L" "/tmp/rustcRjp3Sl/raw-dylibs" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/psm-c55ad074624aa3d9/out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/blake3-63df129abfd2bfaf/out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/blake3-63df129abfd2bfaf/out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/rustc_llvm-382aed539aa29e78/out" "-L" "/usr/lib/llvm-19/lib" "-L" "/usr/lib/gcc/x86_64-linux-gnu/14" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_driver-34400ccebd970507.so" "-shared" "-Wl,-soname=librustc_driver-34400ccebd970507.so" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-z,origin" "-Wl,-rpath,$ORIGIN/../lib"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: /usr/bin/ld: LLVMGetNextFunction: undefined version: 
          /usr/bin/ld: LLVMGetFirstFunction: undefined version: 
          /usr/bin/ld: failed to set dynamic section sizes: bad value
          collect2: error: ld returned 1 exit status

[bjorn3]

Can you please revert all formatting changes?