LLVM upgrade #34743

Merged
merged 33 commits into from Aug 1, 2016

Projects

None yet
@badboy
Contributor
badboy commented Jul 9, 2016 edited

As discussed in https://internals.rust-lang.org/t/need-help-with-emscripten-port/3154/46 I'm trying to update the used LLVM checkout in Rust.

I basically took @shepmaster's code and applied it on top (though I did the commits manually, the original commits have better descriptions.

With these changes I was able to build rustc. make check throws one last error on run-pass/issue-28950.rs. Output: https://gist.github.com/badboy/bcdd3bbde260860b6159aa49070a9052

I took the metadata changes as is and they seem to work, though it now uses the module in another step. I'm not sure if this is the best and correct way.

Things to do:

  • Make run-pass/issue-28950.rs pass unrelated
  • Find out how the PositionIndependentExecutable setting is now used
  • Is the llvm::legacy still the right way to do these things?

cc @brson @alexcrichton

@rust-highfive
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @pnkfelix (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@badboy
Contributor
badboy commented Jul 9, 2016

Ah, forgot to mention that I am using a LLVM checkout of http://llvm.org/git/llvm.git

@frewsxcv frewsxcv commented on the diff Jul 9, 2016
src/rustllvm/PassWrapper.cpp
@@ -267,7 +267,7 @@ LLVMRustAddLibraryInfo(LLVMPassManagerRef PMB,
// similar code in clang's BackendUtil.cpp file.
extern "C" void
LLVMRustRunFunctionPassManager(LLVMPassManagerRef PM, LLVMModuleRef M) {
- FunctionPassManager *P = unwrap<FunctionPassManager>(PM);
+ llvm::legacy::FunctionPassManager *P = unwrap<llvm::legacy::FunctionPassManager>(PM);
@frewsxcv
frewsxcv Jul 9, 2016 Member

llvm::legacy::FunctionPassManager

Tangential comment to this pull request: Does anyone know what this got replaced with since it's presumably legacy now?

@eefriedman
eefriedman Jul 9, 2016 Contributor

LLVM pass manager infrastructure is currently getting rewritten to be more flexible. See https://github.com/llvm-mirror/llvm/blob/d1aa4ea020d932ffc6a3c3e4d1bbab659391c725/tools/opt/NewPMDriver.cpp#L50 for a usage example for the new API. That said, the rewrite isn't complete yet.

@alexcrichton
alexcrichton Jul 10, 2016 Member

Looks like clang is also still using this, so we probably don't need to update just yet until they do at least.

@shepmaster shepmaster and 2 others commented on an outdated diff Jul 9, 2016
src/rustllvm/PassWrapper.cpp
@@ -188,7 +188,7 @@ LLVMRustCreateTargetMachine(const char *triple,
}
TargetOptions Options;
- Options.PositionIndependentExecutable = PositionIndependentExecutable;
+ //Options.PositionIndependentExecutable = PositionIndependentExecutable;
@shepmaster
shepmaster Jul 9, 2016 Contributor

I'm pretty sure this is a hack that only makes sense in my fork (there's no PIE for microcontrollers :-)).

Even if we wanted to remove this (doubtful), it should be removed and not commented-out. I believe that LLVM applies this flag at a more granular level now, so looking downward for another flag to apply it to seems reasonable.

@badboy
badboy Jul 9, 2016 Contributor

Right, this is one of the things mentioned above. I left it in, so I remember to fix that.

@alexcrichton
alexcrichton Jul 10, 2016 Member

This was deleted here which appears to be replaced by this which is a new setPIELevel function on the LLVM module itself.

The LLVM module isn't available at this time so we'll just have to arrange for it to be set later. Perhaps something like this though:

  • Leave this block, but gated by #if LLVM_VERSION_MINOR <= 8 for compatibility with older versions.
  • Add LLVMRustSetModulePIELevel as a function which is a noop on pre-LLVM-3.9 and afterwards calls the setPIELevel function on the LLVM module.
@badboy
badboy Jul 11, 2016 Contributor

Did you mean we have a LLVMRustSetModulePIELevel with the #ifdef for calling the setPIELevel method?
If so, at what point do we call it?

@shepmaster
shepmaster Jul 11, 2016 Contributor

If so, at what point do we call it?

I'd believe you want to call it every time an LLVM module is created.

@shepmaster shepmaster and 2 others commented on an outdated diff Jul 9, 2016
src/librustc_trans/debuginfo/mod.rs
@@ -168,7 +168,7 @@ pub fn finalize(cx: &CrateContext) {
}
debug!("finalize");
- let _ = compile_unit_metadata(cx);
+ //let _ = compile_unit_metadata(cx);
@shepmaster
shepmaster Jul 9, 2016 Contributor

Shouldn't commit commented-out code. This appears to be moved to the constructor, so I believe it's safe and correct to just remove this.

@badboy
badboy Jul 9, 2016 Contributor

Right. This will be removed.

@shepmaster
shepmaster Jul 11, 2016 Contributor

Although thinking of what @alexcrichton was saying, it might have to just be placed in an #if guard to maintain compatibility with earlier LLVM...

@alexcrichton
alexcrichton Jul 11, 2016 Member

Yeah we may want this for compatibility with older llvm versions, but I'm not sure if the fix here is backwards compatible (it may be?)

@badboy
badboy Jul 11, 2016 Contributor

It will definitely need to change, as we changed the function.
However, the new version might even work with old LLVM, doesn't it?

@alexcrichton
alexcrichton Jul 11, 2016 Member

I think maybe yeah? I'm not actually sure what changed here, it looks like just a bit of shuffling around to me...

@shepmaster
shepmaster Jul 11, 2016 Contributor

It's indeed possible that the updated code (that creates the metadata early) will work with older LLVMs. Specifically, LLVM inverted the way the debug information graph is built. All the builders inside LLVM magically handle this, as long as there is something to add the debug information to. Creating the debug information as early as possible enables this.

@shepmaster shepmaster and 1 other commented on an outdated diff Jul 9, 2016
src/rustllvm/PassWrapper.cpp
@@ -359,7 +359,17 @@ LLVMRustAddAlwaysInlinePass(LLVMPassManagerBuilderRef PMB, bool AddLifetimes) {
extern "C" void
LLVMRustRunRestrictionPass(LLVMModuleRef M, char **symbols, size_t len) {
llvm::legacy::PassManager passes;
- passes.add(llvm::createInternalizePass());
+
+ auto PreserveFunctions = [=](const GlobalValue &GV) {
@shepmaster
shepmaster Jul 9, 2016 Contributor

How did this work before? symbols wasn't even being used...

@badboy
badboy Jul 9, 2016 Contributor

It was used in the old version, see 0763041#diff-2030f049f8df2454e3720ea9403bc14aL363.

I should have squashed that.

@shepmaster
Contributor

I'm no core contributor, but I don't see the value in commit X introducing less-than-great variable names and then commit X+1 improving them. I'd just squash those two together.

@badboy
Contributor
badboy commented Jul 9, 2016

@shepmaster yes, I should have squashed them beforehand. I can still do this.

@alexcrichton alexcrichton commented on an outdated diff Jul 10, 2016
src/librustc_trans/debuginfo/metadata.rs
@@ -981,14 +981,14 @@ fn pointer_type_metadata<'a, 'tcx>(cx: &CrateContext<'a, 'tcx>,
return ptr_metadata;
}
-pub fn compile_unit_metadata(cx: &CrateContext) -> DIDescriptor {
- let work_dir = &cx.sess().working_dir;
- let compile_unit_name = match cx.sess().local_crate_source_file {
- None => fallback_path(cx),
+pub fn compile_unit_metadata(scc: &::context::SharedCrateContext, debug_context: &CrateDebugContext, sess: &::session::Session) -> DIDescriptor {
@alexcrichton
alexcrichton Jul 10, 2016 Member

Right now our line limit is 100 characters so I think this will trigger make tidy, could you also add imports instead of using absolute paths here?

@alexcrichton alexcrichton commented on the diff Jul 10, 2016
src/rustllvm/PassWrapper.cpp
@@ -358,9 +358,18 @@ LLVMRustAddAlwaysInlinePass(LLVMPassManagerBuilderRef PMB, bool AddLifetimes) {
extern "C" void
LLVMRustRunRestrictionPass(LLVMModuleRef M, char **symbols, size_t len) {
- PassManager passes;
- ArrayRef<const char*> ref(symbols, len);
- passes.add(llvm::createInternalizePass(ref));
+ llvm::legacy::PassManager passes;
+
+ auto PreserveFunctions = [=](const GlobalValue &GV) {
+ for (size_t i=0; i<len; i++) {
+ if (GV.getName() == symbols[i]) {
+ return true;
+ }
+ }
+ return false;
+ };
+
+ passes.add(llvm::createInternalizePass(PreserveFunctions));
@alexcrichton
alexcrichton Jul 10, 2016 Member

Can you wrap this in a #if to only execute this new part on LLVM version >= 9 and keep the old code around for 8 and below? That should help us keep compiling against older versions of LLVM.

Come to think of that, can you also ensure that the llvm::legacy:: above compiles on LLVM 3.7 and 3.8?

@alexcrichton alexcrichton commented on an outdated diff Jul 10, 2016
@@ -1 +1 @@
-Subproject commit 80ad955b60b3ac02d0462a4a65fcea597d0ebfb1
+Subproject commit c90294dc24393d973ed18bbc8bc87b73cb67e484
@alexcrichton
alexcrichton Jul 10, 2016 Member

This is currently the rust-llvm-2016-03-13 branch in the rust-lang/llvm repo, we'll want to make a new branch which points to this LLVM version and also has some of our custom patches, which includes.

I've done this at the rust-lang-2016-07-09 branch as well as including some of our own patches (mostly just related to compiling LLVM itself). Can you update this commit to that patch?

@alexcrichton
Member

A failure on issue-28950 looks like it may be specific to your system or something like that, I doubt that the LLVM changes would cause it to fail. Are you sure that it succeeds before this PR on your local computer?

@badboy
Contributor
badboy commented Jul 10, 2016

@alexcrichton Indeed, with a fresh checkout of rust-lang/rust I see the same error on issue-28950

@alexbool
Contributor

This is supposed to fix #34119 ?

@badboy
Contributor
badboy commented Jul 11, 2016

@alexbool If it is fixed by something in LLVM upstream, then yes

@brson
Contributor
brson commented Jul 11, 2016

cc @kripken Here's Rust's LLVM upgrade.

@kripken
kripken commented Jul 11, 2016

@brson: cool, emscripten's merges of that same LLVM commit are in https://github.com/kripken/emscripten-fastcomp/tree/next-merge and https://github.com/kripken/emscripten-fastcomp-clang/tree/next-merge .

The tracking issue is kripken/emscripten#4430. Looks like only some SIMD issues block it, which probably wouldn't be a problem for rust, so it could be usable for testing already.

@shepmaster
Contributor

I'd strongly encourage having links to the original LLVM issues / commits that are forcing these changes. These links should be in the commit message that modifies the relevant aspect of the code. That should make it much easier for people in the future to track down the why.

@shepmaster
Contributor

That should make it much easier for people in the future to track down the why.

I say this having been one of those people to track down such information in the past.

@alexcrichton
Member

@shepmaster to clarify, you mean the changes to LLVM bindings and why they're changing? (e.g. this)

@shepmaster
Contributor

you mean the changes to LLVM bindings and why they're changing

Yeah; sorry to be unclear. Having the commit message be something like

[LLVM 3.9] Configure PIE at the module level instead of compilation unit level

This was deleted here[linky] which appears to be replaced by this[linky] which is a new setPIELevel function on the LLVM module itself.

would make that commit πŸ’― in my book. Otherwise the commit is like "moving some functions around!"

Again, I stress that I hold no power to enforce these suggestions; I just would have found commits like this to be useful when I was digging into the existing code to make these kind of changes. 🌴

@alexcrichton
Member

Sounds like a good plan to me!

@badboy
Contributor
badboy commented Jul 12, 2016 edited

I will rebase the commits and change accordingly (force-push incoming!)

@shepmaster Thanks for mentioning this, it's indeed a good idea, especially in the long run

@solson
Member
solson commented Jul 13, 2016

@alexcrichton @eddyb Can this PR also be placed under the "Launch MIR into Orbit" milestone, since it's blocking #34119?

@alexcrichton alexcrichton added this to the Launch MIR into Orbit milestone Jul 13, 2016
@alexcrichton
Member

@solson sure!

@badboy
Contributor
badboy commented Jul 14, 2016

I just force-pushed with the new commits

  • Reorganized all commits into logical blocks, including proper descriptions and links
  • I had to upgrade compiler-rt as well to get it to compile, this is not yet reflected here
  • I had to re-enable the std::thread thing in LLVM to get it to compile at least once. We probably need to find a proper workaround first.
  • When choosing anything other than Default in ffc5a8608f1418f6a52f754232ba5edb90427370 I get the following error on compile:
error: linking with `cc` failed: exit code: 1
note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps/std-b8d6c224bdd7e5ff.0.o" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps/std-b8d6c224bdd7e5ff.1.o" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps/std-b8d6c224bdd7e5ff.2.o" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps/std-b8d6c224bdd7e5ff.3.o" "-o" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps/libstd-b8d6c224bdd7e5ff.so" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps/std-b8d6c224bdd7e5ff.metadata.o" "-nodefaultlibs" "-L" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps" "-L" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps" "-L" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/build/std-26167b16044fc6c3/out/.libs" "-L" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/build/alloc_jemalloc-0986af1f8480d6ae/out/lib" "-L" "/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-Wl,--whole-archive" "-l" "backtrace" "-Wl,--no-whole-archive" "-Wl,-Bdynamic" "-l" "dl" "-l" "rt" "-l" "pthread" "-Wl,--whole-archive" "/tmp/rustc.FqfLoqjnVH7z/libcollections-af9a9d510058b25a.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FqfLoqjnVH7z/libpanic_unwind-17c382f05cff5951.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FqfLoqjnVH7z/liballoc-3e8adf33e113f98f.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FqfLoqjnVH7z/liballoc_jemalloc-565df9f061d73123.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FqfLoqjnVH7z/librustc_unicode-f4dd78d2f1b9c500.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FqfLoqjnVH7z/libunwind-f70560ece060458e.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FqfLoqjnVH7z/liblibc-b8c0e2a7b980b655.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FqfLoqjnVH7z/librand-f0667cf9d06dbb0e.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FqfLoqjnVH7z/libcore-d7d9994298ae504c.rlib" "-Wl,--no-whole-archive" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "util" "-shared" "-Wl,-rpath,$ORIGIN/../lib" "-l" "compiler-rt"
note: /usr/bin/ld: /home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps/std-b8d6c224bdd7e5ff.0.o: relocation R_X86_64_TPOFF32 against `_ZN3std11collections4hash3map11RandomState3new4KEYS7__getit5__KEY17hc8a582c36b9a9c3eE' can not be used when making a shared object; recompile with -fPIC
/home/jer/src/rust/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/debug/deps/std-b8d6c224bdd7e5ff.0.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

error: aborting due to previous error
error: Could not compile `std`.

It seems something is not compiled correctly with -fPIC, but I'm not sure what to do here.

@brson
Contributor
brson commented Jul 14, 2016

Thanks @badboy. This is a great looking branch.

@badboy
Contributor
badboy commented Jul 14, 2016

Btw, I tested with commit 14f04db1b53e21922728767b440a44741c9e292c from upstream compiler-rt.

@alexcrichton alexcrichton and 2 others commented on an outdated diff Jul 14, 2016
src/librustc_trans/context.rs
@@ -322,6 +322,7 @@ unsafe fn create_context_and_module(sess: &Session, mod_name: &str) -> (ContextR
let llvm_target = sess.target.target.llvm_target.as_bytes();
let llvm_target = CString::new(llvm_target).unwrap();
llvm::LLVMRustSetNormalizedTarget(llmod, llvm_target.as_ptr());
+ llvm::LLVMRustSetModulePIELevel(llmod);
@alexcrichton
alexcrichton Jul 14, 2016 Member

This is actually currently conditional when the argument is passed to LLVMRustCreateTargetMachine:

!any_library && reloc_model == llvm::RelocPIC

Perhaps that logic could be extracted to a method and that could be passed down here as well?

@badboy
badboy Jul 14, 2016 edited Contributor

Adding the exact same code to calculate any_library and reloc_model and then gating that call seems to work now, even with PIELevel::Level::Large (which, to my understanding, should be equivalent to the old option).

@badboy
badboy Jul 14, 2016 Contributor

I will refactor the code into a method, so it can be called where necessary

@alexcrichton
alexcrichton Jul 14, 2016 Member

Ok, awesome! So long as we're doing the same as before (whatever that PIELevel is), sounds good to me

@shepmaster
shepmaster Jul 14, 2016 Contributor

whatever that PIELevel is

All I know is that my PIELevel is always too low. 🍰

@alexcrichton alexcrichton and 1 other commented on an outdated diff Jul 14, 2016
src/librustc_trans/debuginfo/metadata.rs
flags.as_ptr() as *const _,
0,
split_name.as_ptr() as *const _)
};
- fn fallback_path(cx: &CrateContext) -> CString {
- CString::new(cx.link_meta().crate_name.clone()).unwrap()
+ fn fallback_path(scc: &::context::SharedCrateContext) -> CString {
@alexcrichton
alexcrichton Jul 14, 2016 Member

one more absolute import here

@badboy
badboy Jul 14, 2016 Contributor

Oh right, will fix it.

@alexcrichton alexcrichton and 1 other commented on an outdated diff Jul 14, 2016
src/rustllvm/ArchiveWrapper.cpp
for (size_t i = 0; i < NumMembers; i++) {
auto Member = NewMembers[i];
assert(Member->name);
if (Member->filename) {
-#if LLVM_VERSION_MINOR >= 8
- Members.push_back(NewArchiveIterator(Member->filename));
+#if LLVM_VERSION_MINOR >= 9
+ Expected<NewArchiveMember> MOrErr = NewArchiveMember::getFile(Member->filename, true);
@alexcrichton
alexcrichton Jul 14, 2016 Member

I swear this changes literally every single LLVM release

@badboy
badboy Jul 14, 2016 Contributor

I'm sure they find the best API eventually

@alexcrichton alexcrichton and 1 other commented on an outdated diff Jul 14, 2016
src/rustllvm/ArchiveWrapper.cpp
for (size_t i = 0; i < NumMembers; i++) {
auto Member = NewMembers[i];
assert(Member->name);
if (Member->filename) {
-#if LLVM_VERSION_MINOR >= 8
- Members.push_back(NewArchiveIterator(Member->filename));
@alexcrichton
alexcrichton Jul 14, 2016 Member

I think this needs to stick around for LLVM 3.8, right?

@badboy
badboy Jul 14, 2016 Contributor

Hm, I'll check that again, testing it with 3.8 as well

@alexcrichton
alexcrichton Jul 14, 2016 Member

FWIW in the past I've used the precompiled 3.7/3.8 distributions that LLVM ships and I just have a few build directories when I test out these changes, I just double check the C++ compiles on each and shouldn't take too long.

@alexcrichton alexcrichton and 1 other commented on an outdated diff Jul 14, 2016
src/rustllvm/ArchiveWrapper.cpp
@@ -150,19 +158,33 @@ LLVMRustWriteArchive(char *Dst,
const LLVMRustArchiveMember **NewMembers,
bool WriteSymbtab,
Archive::Kind Kind) {
- std::vector<NewArchiveIterator> Members;
+ std::vector<NewArchiveMember> Members;
@alexcrichton
alexcrichton Jul 14, 2016 Member

This'll probably need a #ifdef for 3.7-8 compat

@badboy
badboy Jul 14, 2016 Contributor

Oops, right.

@alexcrichton
Member

When choosing anything other than Default in ffc5a86 I get the following error on compile:

That seems fine to me, we probably want Default for now anyway.

I had to upgrade compiler-rt as well to get it to compile, this is not yet reflected here

Do you have a branch which contains our patches as well? If so, I'll push it up to rust-lang! We unfortunately have a lot of patches here :(

I had to re-enable the std::thread thing in LLVM to get it to compile at least once.

Drat, could you send a PR to the LLVM branch in rust-lang/llvm though?

@badboy
Contributor
badboy commented Jul 14, 2016

Do you have a branch which contains our patches as well? If so, I'll push it up to rust-lang! We unfortunately have a lot of patches here :(

I will try to combine those.

Drat, could you send a PR to the LLVM branch in rust-lang/llvm though?

I can, but this will break mingw-w64 then again.

@alexcrichton
Member

Ah yeah and if you have difficulty merging the compiler-rt patches, I can also help out rebasing those. Also yeah we need to get this working on mingw-w64, so were the patches to enable usage of std::thread? If so we just need to figure out how to get it working there.

Not sure how to deal with upstream adding more of what we need to remove...

@badboy
Contributor
badboy commented Jul 14, 2016

New commits tackle the latest comments. I tried with LLVM 3.8 and stage1 still compiles. For some reason the LLVM 3.7 fails due to changed arguments of a function called in RustWrapper.cpp.

Next on: compiler-rt merging

@alexcrichton
Member

Well this is gonna get interested, wanted to test some things out locally and got:

CMake 3.4.3 or higher is required.  You are running version 3.2.2

I'm on Ubuntu 14.04 (which a bunch of our builders are as well), so we'll have to resolve this one way or another. We'll also need to update the README I believe to reflect that 3.4.3 is the minimum version, not 2.8.8 any more

@badboy
Contributor
badboy commented Jul 14, 2016

Oh right, that cmake thing. D'oh!

@alexcrichton
Member

Oh dear, do we also need to require a super-new clang now? I'm getting:

//home//alex//code//clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-14.04//include/llvm/Support/Format.h:79:7: error:
      'llvm::format_object<unsigned int>' has virtual functions but non-virtual destructor

My clang version is Ubuntu clang version 3.4-1ubuntu3

@alexcrichton
Member

Oh wait no nevermind that's just because -Wnon-virtual-dtor is in llvm-config --cxxflags from those pre built binaries, disregard!

@alexcrichton
Member

Ok, confirmed 3.8 at least compiles, and 3.7.1 compiles for me as well. Note that 3.7.0 doesn't compile for known reasons, so maybe you're testing against that locally? If travis passes though we should be fine.

@badboy
Contributor
badboy commented Jul 14, 2016 edited

D'oh! Ok, it is 3.7.0 breaking, thanks Ubuntu…

@alexcrichton
Member

Ok, CMake versions are:

$ docker run --entrypoint bash -it alexcrichton/rust-slave-linux-cross:2016-06-21 -c 'cmake --version'
cmake version 3.2.2
$ docker run --entrypoint bash -it alexcrichton/rust-slave-dist:2016-05-31 -c 'cmake --version'
cmake version 3.3.2
$ docker run --entrypoint bash -it alexcrichton/rust-slave-android:2016-07-01 -c 'cmake --version'
cmake version 3.5.1
$ docker run --entrypoint bash -it alexcrichton/rust-slave-linux:2016-03-30 -c 'cmake --version'
cmake version 3.2.2
$ ssh macstadium-prod1 /usr/local/bin/cmake --version
cmake version 3.5.0

And the Windows bots are running cmake 3.2.1

Hurray! Almost everything needs an upgrade! The linux bots should all be easy enough as they're just tweaking some docker images. Most of them are on ubuntu 14.04 for no particular reason, and Android which is on 16.04 shows that it should be far enough forward. The Windows bots will be... interesting, but I'll figure it out.

As soon as this is ready to go we can r+ and confirm it actually needs an upgrade, but I'll start upgrading them anyway

@badboy
Contributor
badboy commented Jul 14, 2016

Btw, this is the error I get with the current compiler-rt:

CMake Error at cmake/Modules/AddCompilerRT.cmake:1 (include):
  include could not find load file:

    AddLLVM
Call Stack (most recent call first):
  lib/CMakeLists.txt:4 (include)

It's missing some files, upstream checkouts has these files.
I'm not sure why it works in master right now.
Maybe it is enough if we place the right files there instead of doing a full compiler-rt upgrade?

@alexcrichton
Member

Hm that means that build system changes probably happened, in theory a compiler-rt upgrade should be pretty easy, so I think it'd be worth trying that out first

@badboy
Contributor
badboy commented Jul 14, 2016

If you say so, I will try that first in the morning.

@badboy
Contributor
badboy commented Jul 15, 2016

I have a branch of compiler-rt here where I merged master from http://llvm.org/git/compiler-rt and fixed the conflicts. Right now it compiles on my test server, but I might have broken other stuff.

@alexcrichton
Member

Ok cool, thanks! I'll try to go through that soon. Unfortunately it was a bit rocky updating the CMake version. So in updating the linux-cross image for Ubuntu 16.04 it means we couldn't use our old "vendored" MIPS compilers, we could now switch to the "official" MIPS compilers. Unfortunately they don't support compiling with -msoft-float which we pass. It looks like not passing them, however, is gonna make it past all the bots (test are still running).

cc @japaric, do you have an idea of how crucial it is to pass -msoft-float for the MIPS target? I can't imagine we have all that many users of the mips-unknown-linux-gnu target right now as we haven't even had to worry about glibc compatibility just yet, but you never know!

Note that the mipsel target doesn't pass -msoft-float, so I'm tempted to just not worry about it and stop passing that flag.

@badboy I'll get back to you about the compiler-rt upgrade.

@shepmaster
Contributor

alexcrichton and badboy as they make progress

@shepmaster shepmaster commented on an outdated diff Jul 15, 2016
src/rustllvm/ArchiveWrapper.cpp
@@ -150,19 +158,40 @@ LLVMRustWriteArchive(char *Dst,
const LLVMRustArchiveMember **NewMembers,
bool WriteSymbtab,
Archive::Kind Kind) {
+
+#if LLVM_VERSION_MINOR >= 9
@shepmaster
shepmaster Jul 15, 2016 Contributor

Perhaps these should reference the old version, like the previous set? LLVM_VERSION_MINOR <= 8 and flip the clauses?

@shepmaster shepmaster commented on an outdated diff Jul 15, 2016
src/rustllvm/ArchiveWrapper.cpp
Members.push_back(NewArchiveIterator(Member->filename));
#else
Members.push_back(NewArchiveIterator(Member->filename, Member->name));
#endif
} else {
+#if LLVM_VERSION_MINOR >= 9
@shepmaster
shepmaster Jul 15, 2016 Contributor

Ditto the flip.

@alexcrichton
Member

@badboy ok it looks like the LLVM commit I gave you didn't actually build, the std::thread patch was indeed bad. I tweaked it again to hopefully be more future proof this time, and I think fewer hacks are needed now that we've upgraded our OSX compilers. Can you update the llvm submodule to rust-lang/llvm@6e665b7?

@alexcrichton
Member

Ok, and I have an initial stack of patches for compiler-rt.

I experimented dropping:

  • Old MIPS patches, our compilers are much newer and hopefully won't need them
  • Old ARM patches, merged upstream
  • Old Android patches, hopefully our compiler is new enough to not need them
  • Old SJ/LJ patches, current changes may have subsumed this? Not sure (cc @vhbit)

Other than that couldn't drop too much unfortunately

Still testing locally, but if you want to update this PR we can start sending it to bors.

@alexcrichton alexcrichton added a commit to alexcrichton/rust that referenced this pull request Jul 15, 2016
@alexcrichton alexcrichton mk: Don't pass -msoft-float on mips-gnu
Soon the LLVM upgrade (#34743) will require an updated CMake installation, and
the easiest way to do this was to upgrade the Ubuntu version of the bots to
16.04. This in turn brings in a new MIPS compiler on the linux-cross builder,
which is now from the "official" ubuntu repositories. Unfortunately these
new compilers don't support compiling with the `-msoft-float` flag like we're
currently passing, causing compiles to fail.

This commit removes these flags as it's not clear *why* they're being passed, as
the mipsel targets also don't have it. At least if it's not supported by a
debian default compiler, perhaps it's not too relevant to support?
5f43817
@alexcrichton
Member

Er actually, to send to the bots we have to pull in all the new images, and to do that we have to land #34841 first, but you get my point :)

@alexcrichton
Member

Fixed some minor issues with compiler-rt, should pass the linux-cross bot now and hopefully Android as well. That's where most changes were targeted, so it should be good to go now I think. New commit to update to is rust-lang/compiler-rt@3c51a62

@bors bors added a commit that referenced this pull request Jul 16, 2016
@bors bors Auto merge of #34841 - alexcrichton:no-mips-soft-float, r=brson
mk: Don't pass -msoft-float on mips-gnu

Soon the LLVM upgrade (#34743) will require an updated CMake installation, and
the easiest way to do this was to upgrade the Ubuntu version of the bots to
16.04. This in turn brings in a new MIPS compiler on the linux-cross builder,
which is now from the "official" ubuntu repositories. Unfortunately these
new compilers don't support compiling with the `-msoft-float` flag like we're
currently passing, causing compiles to fail.

This commit removes these flags as it's not clear *why* they're being passed, as
the mipsel targets also don't have it. At least if it's not supported by a
debian default compiler, perhaps it's not too relevant to support?
145f0ec
@japaric
Member
japaric commented Jul 16, 2016

@alexcrichton

cc @japaric, do you have an idea of how crucial it is to pass -msoft-float for the MIPS target?

I looked into this mips-gcc float ABI thing. Here are my notes:

  • The current rust-buildbot MIPS toolchain (15.10 ppa package) uses the soft float ABI
  • with that toolchain, we don't need to pass -msoft-float because it's already the default ABI

However:

  • the new rust-buildbot toolchain (16.04 official package) uses the hard float ABI(!)
  • I think that's why @alexcrichton is getting an error building C dependencies with -msoft-float -- these two float ABIs are incompatible and can't be linked together
  • more clearly: using the -msoft-float flag you are building new C binaries (*.o) with the soft float ABI and trying to link them with the system libraries that have a hard float ABI

Next:

  • On the Rust ABI side, the mips(el)-gnu backend is generating binaries with the soft float ABI

Therefore:

  • We should continue to use a soft float ABI toolchain to avoid mixing two different ABIs i.e. we shouldn't use the 16.04 official package, which is hard float ABI, but instead use a soft-float ABI one e.g. built with crosstool-ng

Now, I'm actually surprised that #34841 passes on on the upgraded docker images, which have the 16.04 mips toolchain, ... I think this is because mixing these two float ABIs only generates a linker warning and not a hard error:

# ubuntu 16.04
$ mips-linux-gnu-gcc-5 -msoft-float foo.c
/usr/lib/gcc-cross/mips-linux-gnu/5/../../../../mips-linux-gnu/bin/ld: Warning: a.out uses -mhard-float (set by /usr/lib/gcc-cross/mips-linux-gnu/5/../../../../mips-linux-gnu/lib/../lib/crt1.o), /tmp/ccJcvBHC.o uses -msoft-float

Perhaps std appears to cross compile OK (albeit with linker warnings, which rustc doesn't bubble up (?)) but using that std to cross compile rust programs generates bad binaries (because of the mixed ABIs)? -- we don't really know because we don't have tests that execute cross compiled binaries. Anyway, I would advise against mixing two different float ABIs.

@alexcrichton
Member

Looks like our mips targets were turned into soft float way back when, so I'm not sure they necessarily need to be soft float. @japaric do you know if there's a reason to do one or the other? It seems odd to me that the mipsel target would be hard float when the mips target isn't...

It's true though that we definitely shouldn't mix, but I think I'd personally prefer to just move everything over to hard float to match upstream toolchains if possible.

@alexcrichton
Member

Oh dear, so... this will probably break external builds. Look like compiler-rt can't be built against older LLVM sources!

@brson thoughts on doing #34400 now?

@alexcrichton alexcrichton referenced this pull request Jul 18, 2016
Open

rustc hangs #34858

@alexcrichton
Member

Ok, thanks again for all the patience @badboy! I think we have a plan of action to solve the remaining problems:

  1. To update cmake (to build LLVM 3.9) we will update the bots to Ubuntu 16.04, which pulls in a new MIPS toolchain (all other toolchains remain the same), which as @japaric points out has some differences in float support. I'll send a PR to change us over to hard-float for the MIPS targets (more consistent with debian), and it shouldn't affect this PR I believe.
  2. To fix compiling compiler-rt against older LLVMs (or any other LLVM) we're going to pursue #34873. This will cut our ties with compiler-rt's build system, so all these questions go away!

So the remaining steps here are:

  • I will send a PR to use hard-float on MIPS, that can land whenever
  • We'll land #34873
  • I'll tweak the compiler-rt submodule to jettison all our build system patches and only keep the source patches.
  • You'll rebase this PR, including updates to the compiler-rt submodule
  • We'll wait for a green light on Travis
  • I'll restart buildbot, updating all the images
  • We'll send to @bors

Sound ok to you? Any last things you want to throw in on this branch?

@badboy
Contributor
badboy commented Jul 18, 2016

πŸ‘ That sounds like a good plan (don't worry about my patience, I'm happy to get so much support in bringing this forward). I'll take a look at #34873 to understand what it does.

@brson
Contributor
brson commented Jul 18, 2016

@brson thoughts on doing #34400 now?

Yes.

@alexcrichton alexcrichton added a commit to alexcrichton/rust that referenced this pull request Jul 18, 2016
@alexcrichton alexcrichton rustc: Remove soft-float from MIPS targets
Right now two MIPS targets in the compiler, `mips-unknown-linux-{gnu,musl}` both
generate object files using the soft-float ABI through LLVM by default. This is
also expressed as the `-C soft-float` codegen option and otherwise isn't used
for any other target in the compiler. This option was added quite some time ago
(back in #9617), and nowadays it's more appropriate to be done through a codegen
option.

This is motivated by #34743 which necessitated an upgrade in the CMake
installation on our bots which necessitated an upgrade in the Ubuntu version
which invalidated the MIPS compilers we were using. The new MIPS compilers
(coming from Debian I believe) all have hard float enabled by default and soft
float support not built in. This meant that we couldn't upgrade the bots
until #34841 landed because otherwise we would fail to compile C code as the
`-msoft-float` option wouldn't work.

Unfortunately, though, this means that once we upgrade the bots the C code we're
compiling will be compiled for hard float and the Rust code will be compiled
for soft float, a bad mismatch! This PR remedies the situation such that Rust
will compile with hard float as well.

If this lands it will likely produce broken nightlies for a day or two while we
get around to upgrading the bots because the current C toolchain only produces
soft-float binaries, and now rust will be hard-float. Hopefully, though, the
upgrade can go smoothly!
617c65e
@alexcrichton
Member

Changing MIPS to hard float is #34910

@alexcrichton alexcrichton added a commit to alexcrichton/rust that referenced this pull request Jul 19, 2016
@alexcrichton alexcrichton rustc: Remove soft-float from MIPS targets
Right now two MIPS targets in the compiler, `mips-unknown-linux-{gnu,musl}` both
generate object files using the soft-float ABI through LLVM by default. This is
also expressed as the `-C soft-float` codegen option and otherwise isn't used
for any other target in the compiler. This option was added quite some time ago
(back in #9617), and nowadays it's more appropriate to be done through a codegen
option.

This is motivated by #34743 which necessitated an upgrade in the CMake
installation on our bots which necessitated an upgrade in the Ubuntu version
which invalidated the MIPS compilers we were using. The new MIPS compilers
(coming from Debian I believe) all have hard float enabled by default and soft
float support not built in. This meant that we couldn't upgrade the bots
until #34841 landed because otherwise we would fail to compile C code as the
`-msoft-float` option wouldn't work.

Unfortunately, though, this means that once we upgrade the bots the C code we're
compiling will be compiled for hard float and the Rust code will be compiled
for soft float, a bad mismatch! This PR remedies the situation such that Rust
will compile with hard float as well.

If this lands it will likely produce broken nightlies for a day or two while we
get around to upgrading the bots because the current C toolchain only produces
soft-float binaries, and now rust will be hard-float. Hopefully, though, the
upgrade can go smoothly!
f77bcc8
@bors bors added a commit that referenced this pull request Jul 20, 2016
@bors bors Auto merge of #34910 - alexcrichton:hard-float-mips, r=brson
rustc: Remove soft-float from MIPS targets

Right now two MIPS targets in the compiler, `mips-unknown-linux-{gnu,musl}` both
generate object files using the soft-float ABI through LLVM by default. This is
also expressed as the `-C soft-float` codegen option and otherwise isn't used
for any other target in the compiler. This option was added quite some time ago
(back in #9617), and nowadays it's more appropriate to be done through a codegen
option.

This is motivated by #34743 which necessitated an upgrade in the CMake
installation on our bots which necessitated an upgrade in the Ubuntu version
which invalidated the MIPS compilers we were using. The new MIPS compilers
(coming from Debian I believe) all have hard float enabled by default and soft
float support not built in. This meant that we couldn't upgrade the bots
until #34841 landed because otherwise we would fail to compile C code as the
`-msoft-float` option wouldn't work.

Unfortunately, though, this means that once we upgrade the bots the C code we're
compiling will be compiled for hard float and the Rust code will be compiled
for soft float, a bad mismatch! This PR remedies the situation such that Rust
will compile with hard float as well.

If this lands it will likely produce broken nightlies for a day or two while we
get around to upgrading the bots because the current C toolchain only produces
soft-float binaries, and now rust will be hard-float. Hopefully, though, the
upgrade can go smoothly!
b3707ff
@bors bors added a commit that referenced this pull request Jul 21, 2016
@bors bors Auto merge of #34910 - alexcrichton:hard-float-mips, r=brson
rustc: Remove soft-float from MIPS targets

Right now two MIPS targets in the compiler, `mips-unknown-linux-{gnu,musl}` both
generate object files using the soft-float ABI through LLVM by default. This is
also expressed as the `-C soft-float` codegen option and otherwise isn't used
for any other target in the compiler. This option was added quite some time ago
(back in #9617), and nowadays it's more appropriate to be done through a codegen
option.

This is motivated by #34743 which necessitated an upgrade in the CMake
installation on our bots which necessitated an upgrade in the Ubuntu version
which invalidated the MIPS compilers we were using. The new MIPS compilers
(coming from Debian I believe) all have hard float enabled by default and soft
float support not built in. This meant that we couldn't upgrade the bots
until #34841 landed because otherwise we would fail to compile C code as the
`-msoft-float` option wouldn't work.

Unfortunately, though, this means that once we upgrade the bots the C code we're
compiling will be compiled for hard float and the Rust code will be compiled
for soft float, a bad mismatch! This PR remedies the situation such that Rust
will compile with hard float as well.

If this lands it will likely produce broken nightlies for a day or two while we
get around to upgrading the bots because the current C toolchain only produces
soft-float binaries, and now rust will be hard-float. Hopefully, though, the
upgrade can go smoothly!
77ba66d
@alexcrichton
Member

Ok #34873 is almost ready to land (fingers crossed for this round). Unfortunately this branch doesn't build for me but with brson@7f5f54d it seems to work. I had to update the llvm-mc syntax b/c it changed in 3.9 and there was just one modification to a C++ file we had to keep from before. Other than that things are looking good so far.

If everything goes super well I'll just make a new PR with all these commits, that one, and the updated compiler-rt. Then I'll r+ that and see it merged when I wake up. I can dream can't I?

@alexcrichton
Member

Alas perhaps not tonight, the g++ on the MinGW buildbots refuses to build LLVM no matter how I'm coaxing it right now, so gonna just have to sort that out tomorrow.

@alexcrichton
Member

@badboy can you cherry pick 31daccb and f7c8e80 in here? No need to preserve commit messages or authorship or anything, they'll just need to be in this PR. The LLVM submodule will likely change again as I figure out how to compile it on MinGW again.

@badboy
Contributor
badboy commented Jul 21, 2016

Cherry-picks incoming πŸ’

@badboy
Contributor
badboy commented Jul 21, 2016

There we go, I picked them apart a bit, but the changes are the same.

@GuillaumeGomez GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request Jul 21, 2016
@GuillaumeGomez GuillaumeGomez Rollup merge of #34910 - alexcrichton:hard-float-mips, r=brson
rustc: Remove soft-float from MIPS targets

Right now two MIPS targets in the compiler, `mips-unknown-linux-{gnu,musl}` both
generate object files using the soft-float ABI through LLVM by default. This is
also expressed as the `-C soft-float` codegen option and otherwise isn't used
for any other target in the compiler. This option was added quite some time ago
(back in #9617), and nowadays it's more appropriate to be done through a codegen
option.

This is motivated by #34743 which necessitated an upgrade in the CMake
installation on our bots which necessitated an upgrade in the Ubuntu version
which invalidated the MIPS compilers we were using. The new MIPS compilers
(coming from Debian I believe) all have hard float enabled by default and soft
float support not built in. This meant that we couldn't upgrade the bots
until #34841 landed because otherwise we would fail to compile C code as the
`-msoft-float` option wouldn't work.

Unfortunately, though, this means that once we upgrade the bots the C code we're
compiling will be compiled for hard float and the Rust code will be compiled
for soft float, a bad mismatch! This PR remedies the situation such that Rust
will compile with hard float as well.

If this lands it will likely produce broken nightlies for a day or two while we
get around to upgrading the bots because the current C toolchain only produces
soft-float binaries, and now rust will be hard-float. Hopefully, though, the
upgrade can go smoothly!
91fd838
@alexcrichton
Member

Ok, I think I got LLVM working through some serious surgery. Looks like I messed up a bit with the grep condition in configure.

Also looks like tests are failing on the master branch because LLVM changed one of their custom target specifications, which we assert is the same as our own. If we change it, however, then we break with other LLVMs because they have a different data-layout as well. I'm just gonna disable that check for custom LLVMs for now and open an issue. Still working through failing tests though.

@brson
Contributor
brson commented Jul 21, 2016

Wow that is a horrible patch :(

@badboy
Contributor
badboy commented Jul 22, 2016

@alexcrichton Should I already pull in the updates to the LLVM branch and the WIP changes?

@badboy
Contributor
badboy commented Jul 22, 2016

Ok, so I'm unable to find the actual commits in any cloned version of these repositories…

@alexcrichton
Member
alexcrichton commented Jul 22, 2016 edited

@badboy not quite yet, I'm still working on testing. Unfortunately there are tons of failures on the bots:

And... they're basically all failing for different reasons.

I had to apply this patch to get to that state, which:

  • Should pass tests on linux64 (locally for me)
    • The data-layout for aarch64 was changed, the data-layout assertion was relaxed for custom LLVMs
    • A test was removed which I don't think ever actually needed to work.
  • Uses an LLVM that actually compiles on MinGW
  • Fixes a condition in the configure script.
@alexcrichton
Member

I'm currently playing whack-a-mole with the bots in dev, trying to get a full suite of the failures so we can work through them.

@alexcrichton
Member

I unfortunately won't get a chance to look into these test failures yet, so any help investigating them would be much appreciated!

@eddyb eddyb and 1 other commented on an outdated diff Jul 22, 2016
src/rustllvm/ArchiveWrapper.cpp
rai->cur = ar->child_begin();
+#else
+ Error err;
+ rai->cur = ar->child_begin(err);
@eddyb
eddyb Jul 22, 2016 Member

The err Error value needs to be checked, otherwise this happens with debug LLVM:

Program aborted due to an unhandled Error:
Error value was Success. (Note: Success values must still be checked prior to being destroyed).
@badboy
badboy Jul 24, 2016 Contributor

will take care of that

@eddyb eddyb commented on the diff Jul 22, 2016
src/librustc_trans/context.rs
@@ -322,6 +355,11 @@ unsafe fn create_context_and_module(sess: &Session, mod_name: &str) -> (ContextR
let llvm_target = sess.target.target.llvm_target.as_bytes();
let llvm_target = CString::new(llvm_target).unwrap();
llvm::LLVMRustSetNormalizedTarget(llmod, llvm_target.as_ptr());
+
+ if is_pie_binary(sess) {
+ llvm::LLVMRustSetModulePIELevel(llmod);
@eddyb
eddyb Jul 22, 2016 edited Member

You might need to set the PICLevel too, even if the relocation model is PIC.
One potentially related failure is:

note: ld: illegal text-relocation to '_panic_loc21555' in i686-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/std-f53fb285.0.o from '__ZN103_$LT$core..slice..RSplitN$LT$$u27$a$C$$u20$T$C$$u20$P$GT$$u20$as$u20$core..iter..iterator..Iterator$GT$4next17he6277dfc64030327E' in i686-apple-darwin/stage1/lib/rustlib/i686-apple-darwin/lib/std-f53fb285.0.o for architecture i386

EDIT: Not that it would help, given how it's only used in PowerPC.

@eddyb
Member
eddyb commented Jul 23, 2016

All the relocation nonsense was because, well, this mismatch:

pub enum RelocMode {
    RelocDefault = 0,
    RelocStatic = 1,
    RelocPIC = 2,
    RelocDynamicNoPic = 3,
}
typedef enum {
    LLVMRelocDefault,
    LLVMRelocStatic,
    LLVMRelocPIC,
    LLVMRelocDynamicNoPic
} LLVMRelocMode;
...
enum Model { Static, PIC_, DynamicNoPIC };

So the Rust side had LLVMRelocMode, but LLVMRustCreateTargetMachine was taking Reloc::Model which meant that RelocMode::RelocPIC became Reloc::DynamicNoPIC.

The LLVMRust* functions MUST ONLY TAKE C TYPES.
In this case, LLVMRelocMode, and do the conversion LLVMCreateTargetMachine does.

@eddyb
Member
eddyb commented Jul 23, 2016

Aarch64 Android also needs a data-layout update:

data-layout for builtin `aarch64-linux-android` target,
`e-m:e-i64:64-i128:128-n32:64-S128`, differs from LLVM default,
`e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128` 
@badboy
Contributor
badboy commented Jul 23, 2016

Thanks @eddyb, I'll take a look later today.

@badboy
Contributor
badboy commented Jul 24, 2016

Again, thanks @eddyb. I tracked it down to this commit: 7354055#diff-5bb596c9f589759bef3b0b31f5505d2eR2107
It seems this was a type mismatch for quite some time.
I will fix it in the above code.

@DemiMarie
Contributor

Would it be possible to check with an LLVM clone over HTTPS? Just to make sure that nobody has MITM'd the insecure HTTP clone.

@eddyb
Member
eddyb commented Jul 26, 2016

I was wrong. It's an implementation bug, llvm-mirror/llvm@11bf1ab seems to be the fix. Rebasing on top of master should include it.

@alexcrichton
Member

Ok, I've rebased LLVM and have sent that to dev-bors.

Results will be at http://54.176.156.253/grid as they come in

@eddyb
Member
eddyb commented Jul 26, 2016 edited

MSVC seems to be crashing in a few tests, which are actually assertion failures in debuginfo:

rustc: /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:159:
llvm::LexicalScope* llvm::LexicalScopes::getOrCreateRegularScope(const llvm::DILocalScope*):
Assertion `cast<DISubprogram>(Scope)->describes(MF->getFunction())' failed.

cc @michaelwoerister

EDIT: Backtrace is:

#4  0x00007ffff05502fd in llvm::LexicalScopes::getOrCreateRegularScope (this=0x7fffe45cf038, Scope=0x7fffe547b8c8) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:159
#5  0x00007ffff055010c in llvm::LexicalScopes::getOrCreateLexicalScope (this=0x7fffe45cf038, Scope=0x7fffe547b8c8, IA=0x0) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:136
#6  0x00007ffff0550218 in llvm::LexicalScopes::getOrCreateRegularScope (this=0x7fffe45cf038, Scope=0x7fffe54709e0) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:152
#7  0x00007ffff055010c in llvm::LexicalScopes::getOrCreateLexicalScope (this=0x7fffe45cf038, Scope=0x7fffe54709e0, IA=0x0) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:136
#8  0x00007ffff054fae6 in llvm::LexicalScopes::getOrCreateLexicalScope (this=0x7fffe45cf038, DL=0x7fffe51009a8) at /home/eddy/Projects/rust/src/llvm/include/llvm/CodeGen/LexicalScopes.h:209
#9  0x00007ffff05504b8 in llvm::LexicalScopes::getOrCreateInlinedScope (this=0x7fffe45cf038, Scope=0x7fffe5470b08, InlinedAt=0x7fffe51009a8) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:182
#10 0x00007ffff0550496 in llvm::LexicalScopes::getOrCreateInlinedScope (this=0x7fffe45cf038, Scope=0x7fffe5480760, InlinedAt=0x7fffe51009a8) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:180
#11 0x00007ffff05500f7 in llvm::LexicalScopes::getOrCreateLexicalScope (this=0x7fffe45cf038, Scope=0x7fffe5480760, IA=0x7fffe51009a8) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:133
#12 0x00007ffff054fae6 in llvm::LexicalScopes::getOrCreateLexicalScope (this=0x7fffe45cf038, DL=0x7fffe51de8f0) at /home/eddy/Projects/rust/src/llvm/include/llvm/CodeGen/LexicalScopes.h:209
#13 0x00007ffff054fe8f in llvm::LexicalScopes::extractLexicalScopes (this=0x7fffe45cf038, MIRanges=..., MI2ScopeMap=...) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:102
#14 0x00007ffff054fbd0 in llvm::LexicalScopes::initialize (this=0x7fffe45cf038, Fn=...) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/LexicalScopes.cpp:45
#15 0x00007ffff03a9ef1 in llvm::DebugHandlerBase::beginFunction (this=0x7fffe45cf000, MF=0x7fffe4fc9d20) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/AsmPrinter/DebugHandlerBase.cpp:120
#16 0x00007ffff0426e05 in llvm::CodeViewDebug::beginFunction (this=0x7fffe45cf000, MF=0x7fffe4fc9d20) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp:870
#17 0x00007ffff038c769 in llvm::AsmPrinter::EmitFunctionHeader (this=0x7fffe4ca66d0) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp:602
#18 0x00007ffff038dad4 in llvm::AsmPrinter::EmitFunctionBody (this=0x7fffe4ca66d0) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp:842
#19 0x00007fffef622aff in llvm::X86AsmPrinter::runOnMachineFunction (this=0x7fffe4ca66d0, MF=...) at /home/eddy/Projects/rust/src/llvm/lib/Target/X86/X86AsmPrinter.cpp:70
#20 0x00007ffff05fb432 in llvm::MachineFunctionPass::runOnFunction (this=0x7fffe4ca66d0, F=...) at /home/eddy/Projects/rust/src/llvm/lib/CodeGen/MachineFunctionPass.cpp:60
#21 0x00007ffff101fa75 in llvm::FPPassManager::runOnFunction (this=0x7fffe4dbcec0, F=...) at /home/eddy/Projects/rust/src/llvm/lib/IR/LegacyPassManager.cpp:1523
#22 0x00007ffff101fbec in llvm::FPPassManager::runOnModule (this=0x7fffe4dbcec0, M=...) at /home/eddy/Projects/rust/src/llvm/lib/IR/LegacyPassManager.cpp:1544
#23 0x00007ffff101ff39 in (anonymous namespace)::MPPassManager::runOnModule (this=0x7fffe55aae40, M=...) at /home/eddy/Projects/rust/src/llvm/lib/IR/LegacyPassManager.cpp:1600
#24 0x00007ffff1020607 in llvm::legacy::PassManagerImpl::run (this=0x7fffe4ca8d30, M=...) at /home/eddy/Projects/rust/src/llvm/lib/IR/LegacyPassManager.cpp:1703
#25 0x00007ffff1020813 in llvm::legacy::PassManager::run (this=0x7fffe53daf70, M=...) at /home/eddy/Projects/rust/src/llvm/lib/IR/LegacyPassManager.cpp:1734
#26 0x00007fffef5d4097 in LLVMRustWriteOutputFile (Target=0x7fffe55d4be0, PMR=0x7fffe53daf70, M=0x7fffe4b17c80, path=<optimized out>, FileType=llvm::TargetMachine::CGFT_ObjectFile) at ../rustllvm/PassWrapper.cpp:333
#27 0x00007ffff63c0184 in rustc_trans::back::write::write_output_file::h6cb30f45813227a3 () from /home/eddy/Projects/rust/build/x86_64-unknown-linux-gnu/stage1/bin/../lib/../lib/librustc_trans-250f5d44d5ecf2cd.so
#28 0x00007ffff63c3094 in rustc_trans::back::write::optimize_and_codegen::_$u7b$$u7b$closure$u7d$$u7d$::he595bac4e30c4aa2 ()
@badboy
Contributor
badboy commented Jul 26, 2016

vlad on IRC tried a LLVM 3.9 on Windows last week or so and has some tiny patches in his branch.
More or less just tiny build system fixes, except for vvuk@c18c244

Might this be a fix for above crashes?

@alexcrichton
Member

Nice idea! I've sent that to bors-dev and we'll see what happens

@alexcrichton
Member

Ok, looks like there may be two errors:

for the latter, I got a backtrace:

rustc: ../../../../src/llvm/lib/CodeGen/LexicalScopes.cpp:159: llvm::LexicalScope* llvm::LexicalScopes::getOrCreateRegularScope(const llvm::DILocalScope*): Assertion `cast<DISubprogram>(Scope)->describes(MF->getFunction())' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffed7db700 (LWP 17806)]
0x00007ffff7133c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56      ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff7133c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff7137028 in __GI_abort () at abort.c:89
#2  0x00007ffff712cbf6 in __assert_fail_base (fmt=0x7ffff727d3b8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=assertion@entry=0x7ffff28170f0 "cast<DISubprogram>(Scope)->describes(MF->getFunction())", file=file@entry=0x7ffff2816fb0 "../../../../src/llvm/lib/CodeGen/LexicalScopes.cpp", 
    line=line@entry=159, 
    function=function@entry=0x7ffff28178a0 <llvm::LexicalScopes::getOrCreateRegularScope(llvm::DILocalScope const*)::__PRETTY_FUNCTION__> "llvm::LexicalScope* llvm::LexicalScopes::getOrCreateRegularScope(const llvm::DILocalScope*)") at assert.c:92
#3  0x00007ffff712cca2 in __GI___assert_fail (assertion=0x7ffff28170f0 "cast<DISubprogram>(Scope)->describes(MF->getFunction())", 
    file=0x7ffff2816fb0 "../../../../src/llvm/lib/CodeGen/LexicalScopes.cpp", line=159, 
    function=0x7ffff28178a0 <llvm::LexicalScopes::getOrCreateRegularScope(llvm::DILocalScope const*)::__PRETTY_FUNCTION__> "llvm::LexicalScope* llvm::LexicalScopes::getOrCreateRegularScope(const llvm::DILocalScope*)") at assert.c:101
#4  0x00007ffff16e3430 in llvm::LexicalScopes::getOrCreateRegularScope(llvm::DILocalScope const*) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#5  0x00007ffff16e3340 in llvm::LexicalScopes::getOrCreateRegularScope(llvm::DILocalScope const*) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#6  0x00007ffff16e3eaa in llvm::LexicalScopes::extractLexicalScopes(llvm::SmallVectorImpl<std::pair<llvm::MachineInstr const*, llvm::MachineInstr const*> >&, llvm::DenseMap<llvm::MachineInstr const*, llvm::LexicalScope*, llvm::DenseMapInfo<llvm::MachineInstr const*>, llvm::detail::DenseMapPair<llvm::MachineInstr const*, llvm::LexicalScope*> >&) ()
   from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#7  0x00007ffff16e41f9 in llvm::LexicalScopes::initialize(llvm::MachineFunction const&) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#8  0x00007ffff15ed8ef in llvm::DebugHandlerBase::beginFunction(llvm::MachineFunction const*) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#9  0x00007ffff163de40 in llvm::CodeViewDebug::beginFunction(llvm::MachineFunction const*) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#10 0x00007ffff15e3ae6 in llvm::AsmPrinter::EmitFunctionHeader() () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#11 0x00007ffff15dc5f5 in llvm::AsmPrinter::EmitFunctionBody() () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#12 0x00007ffff1298250 in llvm::X86AsmPrinter::runOnMachineFunction(llvm::MachineFunction&) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#13 0x00007ffff176d013 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#14 0x00007ffff1fa7b03 in llvm::FPPassManager::runOnFunction(llvm::Function&) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#15 0x00007ffff1fa7f2b in llvm::FPPassManager::runOnModule(llvm::Module&) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#16 0x00007ffff1fa823f in llvm::legacy::PassManagerImpl::run(llvm::Module&) () from /home/alex/code/rust2/x86_64-unknown-linux-gnu/stage1/bin/../lib/librustc_llvm-f53fb285.so
#17 0x00007ffff0c36a8d in LLVMRustWriteOutputFile (Target=0x7fffdc85de20, PMR=0x7fffdf30e920, M=0x7fffebaabbc0, path=<optimized out>, FileType=llvm::TargetMachine::CGFT_ObjectFile)
@eddyb
Member
eddyb commented Jul 27, 2016

@alexcrichton It's segfaulting in rustc while compiling librustc, not sure how that could be spurious.
Could be a LLVM assertion though. If only we had those enabled always, we could tell :).

The MSVC one seems identical to the more general debuginfo problem which was supposedly fixed.

@eddyb eddyb commented on the diff Jul 27, 2016
src/librustc_trans/debuginfo/metadata.rs
@@ -68,7 +68,6 @@ pub const UNKNOWN_LINE_NUMBER: c_uint = 0;
pub const UNKNOWN_COLUMN_NUMBER: c_uint = 0;
// ptr::null() doesn't work :(
-pub const NO_FILE_METADATA: DIFile = (0 as DIFile);
pub const NO_SCOPE_METADATA: DIScope = (0 as DIScope);
@eddyb
eddyb Jul 27, 2016 Member

Is it possible NO_SCOPE_METADATA is also wrong now?

@badboy
badboy Jul 27, 2016 edited Contributor

You might be right. As far as I traced it the passed value gets unwrapped (which simply does a reinterpret_cast). Unwrapping a non-existent value in C++ is probably the same kind of bad as it is in Rust, except that in Rust it's properly safe to do so and panic.

@michaelwoerister
Contributor

MSVC seems to be crashing in a few tests, which are actually assertion failures in debuginfo...

I've seen this kind of assertion in the past in the following cases:

  1. When function calls where not assigned a debug location. Then inlining would lead to instructions seemingly coming from another function.
  2. When a bug lead to closures being translated multiple times, leading to multiple debuginfo descriptions for the same LLVM function.

But this seems more likely to be a change in what kind of values LLVM expects in its API?

@alexcrichton alexcrichton added a commit to alexcrichton/rust that referenced this pull request Jul 27, 2016
@alexcrichton alexcrichton mk: Don't pass -msoft-float on mips-gnu
Soon the LLVM upgrade (#34743) will require an updated CMake installation, and
the easiest way to do this was to upgrade the Ubuntu version of the bots to
16.04. This in turn brings in a new MIPS compiler on the linux-cross builder,
which is now from the "official" ubuntu repositories. Unfortunately these
new compilers don't support compiling with the `-msoft-float` flag like we're
currently passing, causing compiles to fail.

This commit removes these flags as it's not clear *why* they're being passed, as
the mipsel targets also don't have it. At least if it's not supported by a
debian default compiler, perhaps it's not too relevant to support?
6833796
@alexcrichton
Member

As an update on this I ran against @bors last night with this tree which included an update to the LLVM submodule that @eddyb was able to track down. This fixes the regression of hyper on auto-win-msvc-64-cargotest.

The final remaining failure is:

rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_passes
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_save_analysis
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_typeck
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_mir
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_trans
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_plugin
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_borrowck
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_driver
rustc: x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustdoc
Segmentation fault (core dumped)
make: *** [x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.rustdoc] Error 139
/buildslave/rust-buildbot/slave/auto-linux-64-debug-opt/build/mk/target.mk:212: recipe for target 'x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.rustdoc' failed
program finished with exit code 2
elapsedTime=10602.248647

@badboy Can you include a few more commits here as well?

@bors
Contributor
bors commented Jul 29, 2016

β˜”οΈ The latest upstream changes (presumably #34956) made this pull request unmergeable. Please resolve the merge conflicts.

badboy and others added some commits Jul 12, 2016
@badboy badboy Upgrade to rust-llvm-2016-07-09 d22a9a2
@badboy badboy [LLVM-3.9] Rename custom methods to Rust-specific ones 7420874
@badboy badboy [LLVM-3.9] Setup the compile unit information immediately
Since LLVM reversed the order of the debug info graphs, we need to have
a compile unit that exists *before* any functions (`DISubprogram`s) are
created. This allows the LLVM debug info builder to automatically link
the functions to the compile unit.
fba1f8f
@badboy badboy [LLVM-3.9] Configure PIE at the module level instead of compilation u…
…nit level

This was deleted here[1] which appears to be replaced by this[2]
which is a new setPIELevel function on the LLVM module itself.

[1]: http://reviews.llvm.org/D19753
[2]: http://reviews.llvm.org/D19671
9e706f9
@badboy badboy [LLVM-3.9] Specify that we are using the legacy interface
LLVM pass manager infrastructure is currently getting rewritten to be
more flexible, but the rewrite isn't complete yet.
6ed5db8
@badboy badboy [LLVM-3.9] Preserve certain functions when internalizing
This makes sure to still use the old way for older LLVM versions.
5b44e10
@badboy badboy [LLVM-3.9] Replace NewArchiveIterator with NewArchiveMember
The new NewArchiveMember is simpler and requires less context,
according to upstream.

This was changed in http://reviews.llvm.org/D21721
8433f9b
@badboy badboy [LLVM-3.9] Update return type for Archive::create dbb4178
@badboy badboy Use relative path to type 12ccff9
@badboy badboy [LLVM-3.9] Increase PIELevel
Previously, we had a PositionIndependentExecutable, now we simply
choose the highest level. This should be equivalent.

🍰
deafab1
@badboy badboy [LLVM-3.9] Maintain backward compatibility in Archiver 1bc0447
@badboy badboy Refactor determining of relocation model into methods 1798c1a
@badboy badboy Upgrade LLVM to include std::thread patch 2bcb2b8
@badboy badboy Upgrade compiler-rt d0e5aa4
@badboy badboy [LLVM-3.9] Use old way of getting next child
This was changed back in
rust-lang/llvm@aacf2fb
f439aee
@badboy badboy Flip LLVM verion check clause 09c3f33
@badboy badboy Update compiler-rt ad262d5
@badboy badboy Upgrade llvm 2c92756
@badboy badboy [LLVM-3.9] Pass correct relocation model flag dc7076b
@badboy badboy Use correct error handling type 079db4f
@badboy badboy Use C type when passing value to LLVM pass
Previously the C type LLVMRelocMode (available as RelocMode in Rust)
was passed as is to the function.
However createTargetMachine expects a Reloc::Model, which is an enum
just one value short.
Additionally, the function was marked as requiring Reloc::Model in the
C code, but RelocMode on the Rust-side.

We now use the correct C type LLVMRelocMode and convert it to an
Optional<Reloc::Model> as expected by the createTargetMachine call the
same the original LLVMCreateTargetMachine function does.
See
https://github.com/llvm-mirror/llvm/blob/c9b262bfbd5b9fb6f10749dba1465569f39bd625/lib/Target/TargetMachineC.cpp#L104-L121

This was found by @eddyb.
2c16e24
@badboy badboy Force check of error
The passed error needs to be checked.
Otherwise it will force an abort when it is deconstructed, but a
success value.
a36595e
@alexcrichton @badboy alexcrichton configure: Fix grep invocation for llvm-mc argument d851428
@alexcrichton @badboy alexcrichton rustc: Fix data-layout for AArch64 targets
Also relax the assertion whenever we have a custom LLVM root as LLVM may
disagree about exact specifics.
e8f7666
@vvuk @badboy vvuk Remove NO_FILE_METADATA; always use unknown_file_metadata instead of …
…passing 0
330dd39
@alexcrichton @badboy alexcrichton rustc: Update LLVM to the LLVM 3.9 release branch
The 3.9 release of LLVM isn't out yet, but this moves us onto that branch to
start tracking it.
75bcda4
@alexcrichton @badboy alexcrichton test: Remove the execution-engine test
We don't actually officially support this at all, and the execution engine
support in LLVM we've had to gut as it's not compiling on MinGW, so just delete
this test for now.
5fa5578
@alexcrichton @badboy alexcrichton Update parsing llvm-config output
Now it prints full paths on MSVC, but we're only interested in path names
0509be1
@alexcrichton @badboy alexcrichton llvm: Remove no longer existent LLVMAddTargetData binding 2492d24
@alexcrichton @badboy alexcrichton test: Fix a test on MSVC
Apparently MSVC now has namespaces in backtraces!
5243072
@badboy badboy [LLVM-3.9] Use llvm-3.9 branch f38762a
@badboy badboy Update LLVM again
7c0cd30
@badboy
Contributor
badboy commented Jul 29, 2016

I just rebased this PR on master, applied the remaining patches and pushed it again.

@eddyb
Member
eddyb commented Jul 29, 2016

@bors r+ p=10

@bors
Contributor
bors commented Jul 29, 2016

πŸ“Œ Commit 7c0cd30 has been approved by eddyb

@bors
Contributor
bors commented Jul 29, 2016

βŒ›οΈ Testing commit 7c0cd30 with merge 554bc72...

@bors bors added a commit that referenced this pull request Jul 29, 2016
@bors bors Auto merge of #34743 - badboy:llvm-upgrade, r=eddyb
LLVM upgrade

As discussed in https://internals.rust-lang.org/t/need-help-with-emscripten-port/3154/46 I'm trying to update the used LLVM checkout in Rust.

I basically took @shepmaster's code and applied it on top (though I did the commits manually, the [original commits have better descriptions](master...avr-rust:avr-support).

With these changes I was able to build rustc. `make check` throws one last error on `run-pass/issue-28950.rs`. Output: https://gist.github.com/badboy/bcdd3bbde260860b6159aa49070a9052

I took the metadata changes as is and they seem to work, though it now uses the module in another step. I'm not sure if this is the best and correct way.

Things to do:

* [x] ~~Make `run-pass/issue-28950.rs` pass~~ unrelated
* [x] Find out how the `PositionIndependentExecutable` setting is now used
* [x] Is the `llvm::legacy` still the right way to do these things?

cc @brson @alexcrichton
554bc72
@bors
Contributor
bors commented Jul 29, 2016

πŸ’” Test failed - auto-win-gnu-32-opt

@alexcrichton
Member

@bors: retry

  • forgot to update the windows bots, now doing so
@alexcrichton
Member

@bors: force

@alexcrichton
Member

@bors: retry

@alexcrichton
Member

@bors: force

@michaelwoerister
Contributor

Use the Force, young Skywalker!

@alexcrichton
Member

@bors: retry force clean

@bors
Contributor
bors commented Jul 29, 2016

βŒ›οΈ Testing commit 7c0cd30 with merge bea0adf...

@bors bors added a commit that referenced this pull request Jul 29, 2016
@bors bors Auto merge of #34743 - badboy:llvm-upgrade, r=eddyb
LLVM upgrade

As discussed in https://internals.rust-lang.org/t/need-help-with-emscripten-port/3154/46 I'm trying to update the used LLVM checkout in Rust.

I basically took @shepmaster's code and applied it on top (though I did the commits manually, the [original commits have better descriptions](master...avr-rust:avr-support).

With these changes I was able to build rustc. `make check` throws one last error on `run-pass/issue-28950.rs`. Output: https://gist.github.com/badboy/bcdd3bbde260860b6159aa49070a9052

I took the metadata changes as is and they seem to work, though it now uses the module in another step. I'm not sure if this is the best and correct way.

Things to do:

* [x] ~~Make `run-pass/issue-28950.rs` pass~~ unrelated
* [x] Find out how the `PositionIndependentExecutable` setting is now used
* [x] Is the `llvm::legacy` still the right way to do these things?

cc @brson @alexcrichton
bea0adf
@bors
Contributor
bors commented Jul 29, 2016

πŸ’” Test failed - auto-linux-64-nopt-t

@alexcrichton
Member

@bors: retry force clean

@bors
Contributor
bors commented Jul 29, 2016

βŒ›οΈ Testing commit 7c0cd30 with merge 21ea9cd...

@bors bors added a commit that referenced this pull request Jul 29, 2016
@bors bors Auto merge of #34743 - badboy:llvm-upgrade, r=eddyb
LLVM upgrade

As discussed in https://internals.rust-lang.org/t/need-help-with-emscripten-port/3154/46 I'm trying to update the used LLVM checkout in Rust.

I basically took @shepmaster's code and applied it on top (though I did the commits manually, the [original commits have better descriptions](master...avr-rust:avr-support).

With these changes I was able to build rustc. `make check` throws one last error on `run-pass/issue-28950.rs`. Output: https://gist.github.com/badboy/bcdd3bbde260860b6159aa49070a9052

I took the metadata changes as is and they seem to work, though it now uses the module in another step. I'm not sure if this is the best and correct way.

Things to do:

* [x] ~~Make `run-pass/issue-28950.rs` pass~~ unrelated
* [x] Find out how the `PositionIndependentExecutable` setting is now used
* [x] Is the `llvm::legacy` still the right way to do these things?

cc @brson @alexcrichton
21ea9cd
@bors
Contributor
bors commented Jul 29, 2016

πŸ’” Test failed - auto-linux-64-opt-rustbuild

@alexcrichton
Member

@bors: retry

@bors bors added a commit that referenced this pull request Jul 29, 2016
@bors bors Auto merge of #34743 - badboy:llvm-upgrade, r=eddyb
LLVM upgrade

As discussed in https://internals.rust-lang.org/t/need-help-with-emscripten-port/3154/46 I'm trying to update the used LLVM checkout in Rust.

I basically took @shepmaster's code and applied it on top (though I did the commits manually, the [original commits have better descriptions](master...avr-rust:avr-support).

With these changes I was able to build rustc. `make check` throws one last error on `run-pass/issue-28950.rs`. Output: https://gist.github.com/badboy/bcdd3bbde260860b6159aa49070a9052

I took the metadata changes as is and they seem to work, though it now uses the module in another step. I'm not sure if this is the best and correct way.

Things to do:

* [x] ~~Make `run-pass/issue-28950.rs` pass~~ unrelated
* [x] Find out how the `PositionIndependentExecutable` setting is now used
* [x] Is the `llvm::legacy` still the right way to do these things?

cc @brson @alexcrichton
8f92cbd
@bors
Contributor
bors commented Jul 29, 2016

βŒ›οΈ Testing commit 7c0cd30 with merge 8f92cbd...

@bors
Contributor
bors commented Jul 29, 2016

πŸ’” Test failed - auto-linux-64-debug-opt

@badboy
Contributor
badboy commented Jul 31, 2016

Hmpf. Is there anyway to get anymore debugable info out of that build?

@eddyb
Member
eddyb commented Jul 31, 2016

@badboy We've tried. Preloading libSegFault.so doesn't work for some reason and Ubuntu has that stupid apport thing that interferes with core dumps. @alexcrichton would have to fix the apport nonsense on the bots before triggering a full build with all the bots on, it's harder to reproduce in isolation.

@tmiasko
Contributor
tmiasko commented Jul 31, 2016

I had similar segfault while building this branch. rustc consistently crashed
while deleting DIBuilder from LLVM. Starting from a clean working directory
fixed it. Given that buildbot reports that "clean-llvm cleaned llvm skipped",
maybe it could also help here.

@eddyb
Member
eddyb commented Jul 31, 2016

@tmiasko That's interesting. Do you have a backtrace? We couldn't get anything, reproduction seemed to need the concurrent builds, or a specific builder, to trigger.

@alexcrichton
Member

@tmiasko did some digging and discovered that https://reviews.llvm.org/D22858 may be relevant (stack trace gathered). I've updated our LLVM submodule with this commit (rebased on the llvm/release_39 branch) and now it's rust-lang/llvm@d1cc489. @badboy wanna try updating to that and give it a go? Builds in dev are running, but we can do in parallel hopefully :)

@badboy badboy Upgrade LLVM once more to get a bugfix
@tmiasko did some digging and discovered that
https://reviews.llvm.org/D22858 may be relevant.
5d1d247
@badboy
Contributor
badboy commented Aug 1, 2016

Done! Upgraded to d1cc489

@eddyb
Member
eddyb commented Aug 1, 2016

@bors r+

@bors
Contributor
bors commented Aug 1, 2016

πŸ“Œ Commit 5d1d247 has been approved by eddyb

@bors
Contributor
bors commented Aug 1, 2016

βŒ›οΈ Testing commit 5d1d247 with merge 2c1612c...

@bors bors added a commit that referenced this pull request Aug 1, 2016
@bors bors Auto merge of #34743 - badboy:llvm-upgrade, r=eddyb
LLVM upgrade

As discussed in https://internals.rust-lang.org/t/need-help-with-emscripten-port/3154/46 I'm trying to update the used LLVM checkout in Rust.

I basically took @shepmaster's code and applied it on top (though I did the commits manually, the [original commits have better descriptions](master...avr-rust:avr-support).

With these changes I was able to build rustc. `make check` throws one last error on `run-pass/issue-28950.rs`. Output: https://gist.github.com/badboy/bcdd3bbde260860b6159aa49070a9052

I took the metadata changes as is and they seem to work, though it now uses the module in another step. I'm not sure if this is the best and correct way.

Things to do:

* [x] ~~Make `run-pass/issue-28950.rs` pass~~ unrelated
* [x] Find out how the `PositionIndependentExecutable` setting is now used
* [x] Is the `llvm::legacy` still the right way to do these things?

cc @brson @alexcrichton
2c1612c
@bors bors merged commit 5d1d247 into rust-lang:master Aug 1, 2016

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
homu Test successful
Details
@badboy
Contributor
badboy commented Aug 1, 2016

Wuhu! πŸŽ‰

@eminence
Contributor
eminence commented Aug 1, 2016

Outstanding work everyone!

@michaelwoerister
Contributor

Epic! :)

@shepmaster
Contributor

Hurrah! Now it's time to rebase the AVR and Emscripten forks and really give it a workout ;-)

@badboy
Contributor
badboy commented Aug 1, 2016

@shepmaster: That's the next step and I'll try to work on it this week

@bstrie
Contributor
bstrie commented Aug 1, 2016

Heroes, every one of you! On with MIR!

@whitequark
Contributor

Looks like this missed the LLVM version check in configure, not sure why it didn't break bots:

--- .../rust/configure  Sat Aug 13 01:03:47 2016
+++ .../rust/configure  Sat Aug 13 01:03:50 2016
@@ -1013,7 +1013,7 @@
     LLVM_VERSION=$($LLVM_CONFIG --version)

     case $LLVM_VERSION in
-        (3.[7-8]*)
+        (3.[7-9]*)
             msg "found ok version of LLVM: $LLVM_VERSION"
             ;;
         (*)
@shepmaster
Contributor

I have a PR out for that - #35594. No idea about the bots.

@spladug spladug added a commit to spladug/rust that referenced this pull request Aug 17, 2016
@spladug spladug Update minimum CMake version in README
The minimum got bumped in the LLVM upgrade of #34743.
4254b31
@eddyb eddyb added a commit to eddyb/rust that referenced this pull request Aug 17, 2016
@eddyb eddyb Rollup merge of #35742 - spladug:readme-cmake-version, r=nikomatsakis
Update minimum CMake version in README

The minimum got bumped in the LLVM upgrade of #34743.
4fe6e11
@eddyb eddyb added a commit to eddyb/rust that referenced this pull request Aug 18, 2016
@eddyb eddyb Rollup merge of #35742 - spladug:readme-cmake-version, r=nikomatsakis
Update minimum CMake version in README

The minimum got bumped in the LLVM upgrade of #34743.
7a27444
@pmatos pmatos pushed a commit to LinkiTools/rust that referenced this pull request Sep 27, 2016
@alexcrichton alexcrichton mk: Don't pass -msoft-float on mips-gnu
Soon the LLVM upgrade (#34743) will require an updated CMake installation, and
the easiest way to do this was to upgrade the Ubuntu version of the bots to
16.04. This in turn brings in a new MIPS compiler on the linux-cross builder,
which is now from the "official" ubuntu repositories. Unfortunately these
new compilers don't support compiling with the `-msoft-float` flag like we're
currently passing, causing compiles to fail.

This commit removes these flags as it's not clear *why* they're being passed, as
the mipsel targets also don't have it. At least if it's not supported by a
debian default compiler, perhaps it's not too relevant to support?
2a1796f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment