Fix autodiff builds on Apple CI

[ZuseZ4]

Summary

We intend to enable autodiff next weekend.
Currently the dist-x86_64-linux and dist-aarch-linux runner succeeded, so we can ship autodiff in nightly for linux.
It would be nice, if std::autodiff would simultaneously become available for macos users.
For that, we'd want to fix the three apple dist builders, which all fail with the same error, of not finding the llvm dir, after building enzyme and llvm. This fails pretty late in the compilation pipeline, so I assume it's an easy fix, by just updating or adding the correct path to llvm.

The autodiff backend (which builds Enzyme) was originally introduced here: https://github.com/rust-lang/rust/pull/129176/files
The current implementations life at

rust/src/bootstrap/src/core/build_steps/compile.rs

Line 1190 in d6c1e45

// We want to link against registerEnzyme and in the future we want to use additional

and

rust/src/bootstrap/src/core/build_steps/llvm.rs

Line 984 in d6c1e45

.define("LLVM_DIR", builder.llvm_out(target));

.
Especially this commit fixed local MacOS builds, but as we can see that's not enough for CI yet.

You can test if your changes are successful through try builds (you'll need to ask a rustc member to run one for you).

To avoid having to guess the name of the right LLVM dir, you can probably download the logs of either of the 3 failed apple dist builds. I am not an expert here though, so I'd suggest to ping onur-ozkan who helped with previous autodiff bootstrap issues, or maybe jieyouxu (just add an @ before their name). Since it would be nice to have this as part of the first release and the fix should be quite small, I think it's fine if more than one person looks into it, so I think there's no need to claim it.

Command used

See https://github.com/rust-lang/rust/pull/140064/files

Expected behaviour

Build succeeds, similar to dist-x86_64-linux
#140064 (comment)

Actual behaviour

You can see the failing issue

failed try with plugins & llvm-enzyme enabled: dist-x86_64-apple (#140064 (comment))
failed try with plugins & llvm-enzyme enabled: dist-apple-various (#140064 (comment))
failed try with plugins & llvm-enzyme enabled: dist-aarch64-apple (#140064 (comment))

Bootstrap configuration (bootstrap.toml)

Varies a bit based on the runner, see #140064

Operating system

MacOS, different ones.

HEAD

HEAD

Additional context

Build Log

<log>

[ZuseZ4]

I just found https://github.com/rust-lang/rustc-dev-guide/pull/1964/files#diff-2698d0add3d852fa6248afe666409fa1fd874e2226bfa58be18a9b815ed80b5a, and I think this vaguely matches the examples for when to reach out to them (sorry if it doesn't, please let me know). I hope someone might either know how paths or builds differ, or if there's a good way to reproduce this without having to re-run the slow try builds. Locally "it works for me".
@rustbot ping apple

[rustbot]

Hey Apple notification group! This issue or PR could use some Apple-specific
guidance. Could one of you weigh in? Thanks <3

(In case it's useful, here are some instructions for tackling these sorts of
issues).

cc @BlackHoleFox @hkratz @inflation @madsmtm @nvzqz @shepmaster @thomcc

[nidnogg]

I can try to hop on this for a few days. @ZuseZ4 could you maybe assign me to this in the meantime?

[ZuseZ4]

Sure, any help is appreciated. The last time I tried I got at least beyond building LLVM and Enzyme here: #140799
The issue seems to be that it tries to link Enzyme against cranelift, but Enzyme is part of the LLVM toolchain (sysroot maybe?), and thus isn't available there. We shouldn't try to link against cranelift, but I wasn't able to reproduce it locally and testing in CI is quite slow (>30 min per try).

[nidnogg]

quick question:

You can test if your changes are successful through try builds (you'll need to ask a rustc member to run one for you).

Is this still the only reliable way of testing this? Are you a rustc member that can run this?

[ZuseZ4]

You can try to reproduce the build config from the dist-aarch64-apple (or any other apple dist runner) in https://github.com/rust-lang/rust/pull/140799/files locally. If you manage to reproduce the failure locally you can experiment much faster afterwards, since it won't rebuild LLVM or Enzyme. Then you can probably get feedback in around a minute, instead of 30 minutes.
If you don't manage to reproduce it locally I can also approve try runs, I'm a member of the compiler team.

[nidnogg]

Sweet, this should be enough to go on. Thanks!