libload / dlopen Enzyme/autodiff

[ZuseZ4]

A draft @davidlattimore and I developed at RustChina Unconf.

Closes #143295

r? @bjorn3 Does that generally look like what you had in mind?

I'll update the compiler/rustc_codegen_llvm/src/llvm/enzyme_ffi.rs to also go through libloading, and load the right path rather than hardcoding it. So far it passes some tests, I assume the other ones only fail due to enzyme_ffi not being updated yet.

[bjorn3]

You could do something like

rust/compiler/rustc_interface/src/util.rs

Lines 371 to 381 in 8a1b399

	let target = host_tuple();
	let sysroot = sysroot
	.all_paths()
	.map(|sysroot| {
	filesearch::make_target_lib_path(sysroot, target).with_file_name("codegen-backends")
	})
	.find(|f| {
	info!("codegen backend candidate: {}", f.display());
	f.exists()
	})

[rust-log-analyzer]

A job failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[Kobzol]

Sounds exciting! With this, will we still have to compile LLVM in some "special" way, so that it has support for Enzyme? Or is having the libEnzyme.so file on disk enough?

[ZuseZ4]

That's not an issue anymore, I had removed all extra handling multiple months ago, in my previous push to get it on nightly. The only thing left would now be to add Enzyme as a downloadable rustup component. Would you be able to help with that and take over once I cleaned up and landed this PR @Kobzol? I don't know much about distributions and would like to avoid to have this laying around too many extra months.

[Kobzol]

Yeah, if having libEnzyme.so on disk is enough, then I can help getting that into rustup. There are still some questions to resolve, but we can leave that for a separate issue.