New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to LLVM 4.0 #37609

Closed
shepmaster opened this Issue Nov 5, 2016 · 49 comments

Comments

Projects
None yet
@shepmaster
Member

shepmaster commented Nov 5, 2016

The AVR-Rust post continues to track AVR-LLVM, which is tracking upstream LLVM. That means we are running into changes from the future LLVM 4.0. I figured we could open up this issue now to track the changes required to stay compatible.

This is the current list of upgrade hazards. Note that some of the text may be nonsensical as I don't truly grok all the internals of LLVM so some changes are wild stabs in the dark. It's also possible that some failed fix for earlier problems actually caused later problems:

  • DIDescriptorFlags became a real C++ enum and not a bitset (#37857)
  • An EH personality is required for every function that can unwind (#37862)
  • AlwaysInliner pass changed header and constructor name (#37861)
  • createBasicType changed API (#38317)
  • Bitcode/ReaderWriter.h has been renamed / API changed (#38156)
  • powidf2 / powisf2 calling convention changed (LLVM 31685)
  • llvm::AttrBuilder changed API (#37831)
  • DIBuilder::createNameSpace changed API (#38000)
  • LLVMRustPrintPasses prefers StringRef over char * (#38027)
  • OptimizationDiagnostic string type changes (#37982)
  • llvm::StringRef is not guaranteed to be null terminated (#38048)
  • getPassName changed return types (#38195)
  • Archive support changed error handling types (#38196)
  • Teach Rust about an LLVM 4.0 API change for creating debug info
    for global variables (#38295)
  • Check all errors in LLVMRustArchiveIterator API (#38676)
  • Upgrade emscripten (kripken/emscripten-fastcomp#165, kripken/emscripten-fastcomp-clang#13)
  • Alignment now represented with 32 bits instead of 64 (#39529)
  • DIBuilder::createGlobalVariable API changed again (#39528)
  • llvm::DIBuilder::createCompileUnit now takes an llvm::DIFile* instead of a Filename and Directory (#39747)
  • Symbol lookup error when linking dylibs (#39782)
  • Undefined behavior in repr(packed) tests (#40373)
  • Unoptimised builds of compiler_builtins on ARM have references to core::panicking::panic (#40508)

A WIP PR that performs the upgrade! #40123


Some issues whose resolution is believed to be blocked on LLVM 4.0:

@brson

This comment has been minimized.

Contributor

brson commented Nov 9, 2016

Next time we do this upgrade we'll need emscripten to do it to. It would be good if we could pull in a working llvm wasm backend in the same upgrade, but that may not be ready before the end of the year.

@rkruppe

This comment has been minimized.

Contributor

rkruppe commented Nov 10, 2016

For the record, I am also working with a fork of LLVM that is tracking upstream (not AVR-LLVM but similar in spirit) and am starting to work on this now. I'm looking through the AVR-Rust commits and I'd love to coordinate --- feel free to ping me on IRC (rkruppe) or via mail!

@TimNN

This comment has been minimized.

Contributor

TimNN commented Nov 16, 2016

Note that there is a regression reported against llvm 4.0 regarding the powi intrinsic on armhf targets:

https://llvm.org/bugs/show_bug.cgi?id=30543

I'm explicitly mentioning this here, because there were previously problems with it, see #37559 and related issues. Also cc rust-lang-nursery/compiler-builtins#116, an issue about the same problem in the compiler-rt rust port.

@japaric japaric referenced this issue Nov 16, 2016

Open

AVR support #3

2 of 6 tasks complete
@alexcrichton

This comment has been minimized.

Member

alexcrichton commented Nov 16, 2016

To clarify what I think the @TimNN is referring to (correct me if I'm wrong)

  • In LLVM 3.9 and before, the calling convention LLVM used with the powidf2 and powisf2 intrinsics was basically the C calling convention
  • In LLVM 4.0, the backend was corrected to expect these intrinsics to have the aapcs calling convention
  • One person claims that aapcs is expected for powis2f and powid2f, while another cites a different thread claiming that the normal C calling convention should be used.

So it seems that if the LLVM change is not reverted (expected powisf2 to be aapcs) then we need to update our definition.

This is going to be a huge PITA for supporting multiple LLVM versions as well. We need to select the calling convention depending on the LLVM version as well...

@shepmaster

This comment has been minimized.

Member

shepmaster commented Nov 17, 2016

and I'd love to coordinate

@rkruppe Any objections to doing that coordination here on GitHub, or on one of our forks?

@rkruppe

This comment has been minimized.

Contributor

rkruppe commented Nov 17, 2016

I don't have strong preferences for the communication channel.

@rkruppe

This comment has been minimized.

Contributor

rkruppe commented Nov 17, 2016

#37831 should address what is listed as "AttrBuilder API changes" above (the core of the change is in how attributes are represented).

@rkruppe

This comment has been minimized.

Contributor

rkruppe commented Nov 18, 2016

@shepmaster I would prefer to upstream LLVM 4.0 compatibility patches sooner rather than later. That includes my own work obviously, but I'm also looking at the commits I cherry-picked from AVR-Rust. In particular, avr-rust@3e1e4a4 looks like a reasonable simplification even independent of LLVM 4.0 compatibility, and avr-rust@c32f9c1 looks like a nice little self-contained fix. Could you create PRs for those? (If you don't have the time but are otherwise fine with upstreaming, I'd be happy to manage that, giving proper credit of course.)

@shepmaster

This comment has been minimized.

Member

shepmaster commented Nov 18, 2016

@rkruppe Absolutely! I wasn't really sure how much we could expect LLVM 4 to change as it goes towards stable / how much churn the Rust maintainers would want to put up with. I'll get those pulled out and submitted soon. I think I'm also starting to understand some of the silly error handling that the new LLVM does.

@rkruppe

This comment has been minimized.

Contributor

rkruppe commented Nov 18, 2016

@shepmaster Awesome, thanks in advance!

@rkruppe

This comment has been minimized.

Contributor

rkruppe commented Nov 18, 2016

BTW the PR reference for "llvm::AttrBuilder changed API" is wrong, it points at this issue instead of #37831

bors added a commit that referenced this issue Nov 19, 2016

Auto merge of #37831 - rkruppe:llvm-attr-fwdcompat, r=eddyb
[LLVM 4.0] Use llvm::Attribute APIs instead of "raw value" APIs

The latter will be removed in LLVM 4.0 (see llvm-mirror/llvm@4a6fc8b).

The librustc_llvm API remains mostly unchanged, except that llvm::Attribute is no longer a bitflag but represents only a *single* attribute.
The ability to store many attributes in a small number of bits and modify them without interacting with LLVM is only used in rustc_trans::abi and closely related modules, and only attributes for function arguments are considered there.
Thus rustc_trans::abi now has its own bit-packed representation of argument attributes, which are translated to rustc_llvm::Attribute when applying the attributes.

cc #37609

@sanxiyn sanxiyn added the A-LLVM label Nov 22, 2016

bors added a commit that referenced this issue Nov 25, 2016

Auto merge of #37982 - rkruppe:llvm-diagnostic-fwdcompat, r=alexcrichton
[LLVM 4.0] OptimizationDiagnostic FFI forward compatibility

- getMsg() changed to return std::string by-value. Fix: copy the data to a rust String during unpacking.
- getPassName() changed to return StringRef

cc #37609

bors added a commit that referenced this issue Nov 25, 2016

Auto merge of #38000 - rkruppe:llvm-dinamespace-fwdcompat, r=alexcric…
…hton

[LLVM 4.0] Pass new argument ExportSymbol to DIBuilder::createNameSpace

cc #37609
@Kixunil

This comment has been minimized.

Kixunil commented Nov 26, 2016

Does anyone have ETA? I'd like to know when I can use AVR. :)

bors added a commit that referenced this issue Nov 27, 2016

Auto merge of #38027 - rkruppe:llvm-printpasses-fwdcompat, r=alexcric…
…hton

[LLVM 4.0] LLVMRustPrintPasses

Adapt `LLVMRustPrintPasses` to LLVM 4.0 preferring `StringRef` over `char *`

cc #37609

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 10, 2017

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 11, 2017

arielb1 added a commit to arielb1/rust that referenced this issue Mar 11, 2017

arielb1 added a commit to arielb1/rust that referenced this issue Mar 11, 2017

@chneukirchen chneukirchen referenced this issue Mar 15, 2017

Merged

llvm 4.0.0 #5989

7 of 12 tasks complete

@copumpkin copumpkin referenced this issue Mar 27, 2017

Closed

Merge LLVM 4.0 Darwin stdenv #24294

16 of 27 tasks complete

bors added a commit that referenced this issue Apr 8, 2017

Auto merge of #40123 - TimNN:llvm40, r=<try>
[WIP] LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [ ] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [ ] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* `FAIL`: `IMAGE=armhf-gnu` [travis](https://travis-ci.org/rust-lang/rust/jobs/205860539) (looks to be qemu related, or rather the client / server tools)
* `FAIL`: `IMAGE=emscripten` [travis](https://travis-ci.org/rust-lang/rust/jobs/205978867) (with an updated emcc: one bad emcc optimization: kripken/emscripten-fastcomp#176) ~~[travis](https://travis-ci.org/rust-lang/rust/jobs/205860559) (mainly (only?) due to emcc / rust llvm version mismatch)~~
* `FAIL`: `IMAGE=dist-powerpc64-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514546) (llvm unreachable when compiling core: `Invalid PPC CTR loop!
UNREACHABLE executed at .../PPCCTRLoops.cpp:722!`, reported as [upstream bug #32485](https://bugs.llvm.org//show_bug.cgi?id=32485))
* `SUCCESS/TIMEOUT`: `IMAGE=dist-arm-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514540) (passes locally)
* `SUCCESS/TIMEOUT`: `IMAGE=dist-armv7-aarch64-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514541) (passes locally)
* `SUCCESS`: `IMAGE=arm-android` (fixed)
* `SUCCESS`: `IMAGE=cross`
* `SUCCESS`: `IMAGE=linux-tested-targets`
* `SUCCESS`: `IMAGE=dist-android`
* `SUCCESS`: `IMAGE=dist-freebsd` (fixed)
* `SUCCESS`: `IMAGE=dist-fuchsia`
* `SUCCESS`: `IMAGE=dist-mips-linux`
* `SUCCESS`: `IMAGE=dist-mips64-linux`
* `SUCCESS`: `IMAGE=dist-powerpc-linux`
* `SUCCESS`: `IMAGE=dist-s390x-linux-netbsd` (fixed)
* `SUCCESS`: `IMAGE=dist-x86-linux` (fixed)
* `SUCCESS`: `IMAGE=i686-gnu`
* `SUCCESS`: `IMAGE=i686-gnu-nopt`
* `SUCCESS`: `IMAGE=x86_64-gnu-llvm-3.7`
* `SUCCESS`: `IMAGE=x86_64-gnu`
* `SUCCESS`: `IMAGE=x86_64-gnu-aux`
* `SUCCESS`: `IMAGE=x86_64-gnu-debug`
* `SUCCESS`: `IMAGE=x86_64-gnu-nopt`
* `SUCCESS`: `IMAGE=x86_64-gnu-full-bootstrap`
* `SUCCESS`: `IMAGE=x86_64-gnu-distcheck`
* `SUCCESS`: `IMAGE=x86_64-gnu-incremental`

bors added a commit that referenced this issue Apr 14, 2017

Auto merge of #40123 - TimNN:llvm40, r=TimNN
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [ ] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [ ] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* `FAIL`: `IMAGE=armhf-gnu` [travis](https://travis-ci.org/rust-lang/rust/jobs/205860539) (looks to be qemu related, or rather the client / server tools)
* `FAIL`: `IMAGE=emscripten` [travis](https://travis-ci.org/rust-lang/rust/jobs/205978867) (with an updated emcc: one bad emcc optimization: kripken/emscripten-fastcomp#176) ~~[travis](https://travis-ci.org/rust-lang/rust/jobs/205860559) (mainly (only?) due to emcc / rust llvm version mismatch)~~
* `FAIL`: `IMAGE=dist-powerpc64-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514546) (llvm unreachable when compiling core: `Invalid PPC CTR loop!
UNREACHABLE executed at .../PPCCTRLoops.cpp:722!`, reported as [upstream bug #32485](https://bugs.llvm.org//show_bug.cgi?id=32485))
* `SUCCESS/TIMEOUT`: `IMAGE=dist-arm-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514540) (passes locally)
* `SUCCESS/TIMEOUT`: `IMAGE=dist-armv7-aarch64-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514541) (passes locally)
* `SUCCESS`: `IMAGE=arm-android` (fixed)
* `SUCCESS`: `IMAGE=cross`
* `SUCCESS`: `IMAGE=linux-tested-targets`
* `SUCCESS`: `IMAGE=dist-android`
* `SUCCESS`: `IMAGE=dist-freebsd` (fixed)
* `SUCCESS`: `IMAGE=dist-fuchsia`
* `SUCCESS`: `IMAGE=dist-mips-linux`
* `SUCCESS`: `IMAGE=dist-mips64-linux`
* `SUCCESS`: `IMAGE=dist-powerpc-linux`
* `SUCCESS`: `IMAGE=dist-s390x-linux-netbsd` (fixed)
* `SUCCESS`: `IMAGE=dist-x86-linux` (fixed)
* `SUCCESS`: `IMAGE=i686-gnu`
* `SUCCESS`: `IMAGE=i686-gnu-nopt`
* `SUCCESS`: `IMAGE=x86_64-gnu-llvm-3.7`
* `SUCCESS`: `IMAGE=x86_64-gnu`
* `SUCCESS`: `IMAGE=x86_64-gnu-aux`
* `SUCCESS`: `IMAGE=x86_64-gnu-debug`
* `SUCCESS`: `IMAGE=x86_64-gnu-nopt`
* `SUCCESS`: `IMAGE=x86_64-gnu-full-bootstrap`
* `SUCCESS`: `IMAGE=x86_64-gnu-distcheck`
* `SUCCESS`: `IMAGE=x86_64-gnu-incremental`

bors added a commit that referenced this issue Apr 17, 2017

Auto merge of #40123 - TimNN:llvm40, r=TimNN
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 17, 2017

Auto merge of #40123 - TimNN:llvm40, r=TimNN
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 17, 2017

Auto merge of #40123 - TimNN:llvm40, r=TimNN
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 17, 2017

Auto merge of #40123 - TimNN:llvm40, r=arielb1
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 17, 2017

Auto merge of #40123 - TimNN:llvm40, r=arielb1
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 17, 2017

Auto merge of #40123 - TimNN:llvm40, r=arielb1
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 18, 2017

Auto merge of #40123 - TimNN:llvm40, r=TimNN
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 18, 2017

Auto merge of #40123 - TimNN:llvm40, r=TimNN
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 23, 2017

Auto merge of #40123 - TimNN:llvm40, r=TimNN
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 24, 2017

Auto merge of #40123 - TimNN:llvm40, r=TimNN
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.

bors added a commit that referenced this issue Apr 24, 2017

Auto merge of #40123 - TimNN:llvm40, r=alexcrichton
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [x] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

Everything is green 🎉

@bors bors closed this in #40123 Apr 25, 2017

@dsvensson

This comment has been minimized.

dsvensson commented Apr 25, 2017

@shepmaster @dylanmckay So whats the next issue(s) to subscribe to in the race-to-AVR? :)

@dylanmckay

This comment has been minimized.

Contributor

dylanmckay commented Apr 25, 2017

As @shepmaster says on reddit, we can integrate the latest Rust master into the avr-rust compiler. After that, we can spend some time getting libcore compiling for AVR.

Once that is working, we could then start looking into upstreaming the AVR target to Rust.

@shepmaster

This comment has been minimized.

Member

shepmaster commented Apr 25, 2017

So whats the next issue(s) to subscribe to in the race-to-AVR?

AVR-specific issues can be raised / tracked in https://github.com/avr-rust/rust

For example, there are two that I'm aware of right now:

@BatmanAoD

This comment has been minimized.

BatmanAoD commented Apr 26, 2017

#39782 has been closed, so that can be checked off in the top post.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment