Sort the fat LTO modules to produce deterministic output.

[jgalenson]

Some projects that use LTO for their release builds are not reproducible. We can fix this by sorting the fat LTO modules before using them.

It might also be useful to do this for thin LTO, but I couldn't get that to work to test it so I didn't do it.

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @estebank (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[estebank]

r? @alexcrichton

[michaelwoerister]

Thanks for the PR, @jgalenson! This seems like a reasonable thing to do. Would you mind moving the sorting into generate_lto_work?

[alexcrichton]

I'd actually also prefer if this matches ThinLTO as much as possible as well. It looks like we don't do any sorting there, and I think that's because ThinLTO is inherently more deterministic in its passes.

Could the sort be pushed even further into the fat_lto method in the backend somewhere around here? I believe the only piece which should affect determinisim is the order in which we link the modules together through LLVM, and the serialized_modules list is what's going to get linked in there.

(there's also already a helpful name to sort everything by at that point)

[alexcrichton]

Oh also, mind adding a test which exercises this?

[jgalenson]

That's actually exactly where I initially put the sorting code, but I moved it earlier in case that helped any other uses. Since it seems there aren't, I'll be happy to move it back. :)

I don't know the rustc test framework well. Where would you recommend I add the test?

[alexcrichton]

I think we have a few determinism tests in src/test/run-make-fulldeps/* already, and they can probably be extended with "fat lto" versions as well

[jgalenson]

I made the requested change and added a test in a second commit.

Note, however, that the test is not very good, as it passes even without the fix. I'm not sure how to trigger the nondeterminism in a small testcase like this.

[jgalenson]

And of course I got the test working right after I pushed it. :) It seems to work now.

[alexcrichton]

Nice, thanks! Could you also throw in a comment about why the sorting is done? Other than that looks great to me!

[jgalenson]

Done.

[alexcrichton]

@bors: r+

[bors]

📌 Commit 3e6a927 has been approved by alexcrichton

[Centril]

Failed in #63389 (comment), @bors r-

[jgalenson]

It looks like the new test is no deterministic on Windows (it is on Linux, as the other bots passed). I tried reverting the opt-level=1 to see if that fixes it. If it does not, should we just remove the test altogether?

[alexcrichton]

Hm so the nondeterminism may be coming from the linker rather than rustc, could this perhaps produce an LTO'd staticlib to rule that out?

[jgalenson]

Unfortunately, doing that seems to make the test reproducible even without this patch, whatever opt-level I use. I could still add it, of course, but I'm not sure how useful it would be like that.

[alexcrichton]

Hm ok in that case it's probably best to execute this test just on Linux for now where we have a high level of confidence that the linker used is deterministic.

[jgalenson]

Done. I couldn't tell how to stop only a single part of the test from running on Windows, so I instead copied the code into a new non-Windows test. Is there a way to start a Windows buildbot manually to ensure it works there?

As an aside, should we keep all these commits separate or squash them to avoid intermediate states with the broken test?

[alexcrichton]

@bors: r+

[bors]

📌 Commit b6767b3 has been approved by alexcrichton

[bors]

⌛ Testing commit b6767b3 with merge 6f70adc...

[bors]

☀️ Test successful - checks-azure
Approved by: alexcrichton
Pushing 6f70adc to master...