-
Notifications
You must be signed in to change notification settings - Fork 12.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rustc: Split Emscripten to a separate codegen backend #47730
Conversation
r? @pnkfelix (rust_highfive has picked a reviewer for you, use r? to override) |
cc @rust-lang/compiler, @rust-lang/infra |
Lots of tidy error. |
885a792
to
2ab5374
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I think I'm mostly happy with this (only reviewing the second commit). Can we check if this will by-default make a clone of rust-lang/rust with ./x.py
run increase by ~700 MB? I know that for me at least .git/modules/src/llvm
is fairly large... I'd like to avoid that being doubled for emscripten. Not a deal breaker though.
.gitmodules
Outdated
[submodule "src/llvm-emscripten"] | ||
path = src/llvm-emscripten | ||
url = https://github.com/rust-lang/llvm | ||
branch = rust-llvm-release-4-0-1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be an Emscripten branch? It seems odd that emscripten has branch =
where other submodules don't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed! I figured that for now it's probably easiest to do the smallest delta possible which is to switch to the same version we have right now. I think if we upgrade in the future though we should try to depend directly on Emscripten's fork of LLVM.
As for the branch =
I think that's because I passed a -b
argument to git submodule add
, but as far as I can tell this is never used when cloning (as the submodule has a revision). It may be used for shallow clones of submodules but I never got that working historically..
src/bootstrap/compile.rs
Outdated
const DEFAULT: bool = true; | ||
|
||
fn should_run(run: ShouldRun) -> ShouldRun { | ||
run.path("src/librustc_trans").krate("rustc_trans") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
krate
here is useless in theory since that crate doesn't have a crate graph really in src/
AFAIK. Though I guess it doesn't hurt... I'd prefer that we didn't have it though.
src/bootstrap/compile.rs
Outdated
run.builder.ensure(CodegenBackend { | ||
compiler: run.builder.compiler(run.builder.top_stage, run.host), | ||
target: run.target, | ||
backend: INTERNER.intern_str("llvm"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should not be hardcoded here... I would expect this sort of thing in src/bootstrap/config.rs
probably.
src/bootstrap/compile.rs
Outdated
} | ||
if max > stamp_mtime { | ||
build.verbose(&format!("updating {:?} as {:?} changed", stamp, max_path)); | ||
} else { | ||
build.verbose(&format!("updating {:?} as deps changed", stamp)); | ||
} | ||
t!(t!(File::create(stamp)).write_all(&new_contents)); | ||
return deps |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: just deps
@@ -318,6 +320,7 @@ impl Config { | |||
config.ignore_git = false; | |||
config.rust_dist_src = true; | |||
config.test_miri = false; | |||
config.rust_codegen_backends = vec![INTERNER.intern_str("llvm")]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, so as I mentioned before I'd expect us to use the first element perhaps from here or something like that in the default backend to compile for in src/bootstrap/compile.rs
CodegenBackend
.
2ab5374
to
0ef3ab7
Compare
Indeed! I made sure to verify that we don't clone LLVM twice (we don't) and the right thing happens on CI (download from github). Now when you do enable emscripten I think that the two llvm submodules probably won't share objects, but I'm not sure there's much we can do about that... |
☔ The latest upstream changes (presumably #47748) made this pull request unmergeable. Please resolve the merge conflicts. |
Sounds good to me. r=me; I agree that it seems unlikely that we can get git to understand that they're fundamentally the same repository. And that may not be true indefinitely. |
0ef3ab7
to
d061f30
Compare
@bors: r=Mark-Simulacrum |
📌 Commit d061f30 has been approved by |
⌛ Testing commit d061f3065f705142a683a4c8b38896d14cdeedb3 with merge ebf5612396edec2dd017b015eca09cfbeadfe3dd... |
💔 Test failed - status-travis |
d061f30
to
b746667
Compare
@bors: r=Mark-Simulacrum |
📌 Commit b746667 has been approved by |
rustc: Upgrade to LLVM 6 The following submodules have been updated for a new version of LLVM: - `src/llvm` - `src/libcompiler_builtins` - transitively contains compiler-rt - `src/dlmalloc` This also updates the docker container for dist-i686-freebsd as the old 16.04 container is no longer capable of building LLVM. The compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without much interesting happening, but the LLVM update here is of particular note. Unlike previous updates I haven't cherry-picked all existing patches we had on top of our LLVM branch as we have a [huge amount][patches4] and have at this point forgotten what most of them are for. Instead I started from the current `release_60` branch in LLVM and only applied patches that were necessary to get our tests working and building. The [current set of custom rustc-specific patches](rust-lang/llvm@f128612...rust-llvm-release-6-0-0) included in this LLVM update are: * rust-lang/llvm@1187443 - this is how we actually implement `cfg(target_feature)` for now and continues to not be upstreamed. While a hazard for SIMD stabilization this commit is otherwise keeping the status quo of a small rustc-specific feature. * rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't upstreamed, notably teaching LLVM about our allocation-related routines (which aren't malloc/free). Once we stabilize the global allocator routines we will likely want to upstream this patch, but for now it seems reasonable to keep it on our fork. * rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in our 32-bit linux container. I'm not really sure why it's necessary but my guess is that it's because of the absolutely ancient glibc that we're using. In any case it's only updating pieces we're not actually using in LLVM so I'm hoping it'll turn out alright. This doesn't seem like something we'll want to upstream.c * rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our i686-freebsd container, I'm not really sure what's going on but we for sure probably don't want to upstream this and otherwise it seems not too bad for now at least. * rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug] which although diagnosed to a particular revision isn't currently fixed upstream (and the bug itself doesn't seem too active). This commit is a partial revert of the suspected cause of this regression (found via a bisection). I'm sort of hoping that this eventually gets fixed upstream with a similar fix (which we can replace in our branch), but for now I'm also hoping it's a relatively harmless change to have. After applying these patches (plus one [backport] which should be [backported upstream][llvm-back]) I believe we should have all tests working on all platforms in our current test suite. I'm like 99% sure that we'll need some more backports as issues are reported for LLVM 6 when this propagates through nightlies, but that's sort of just par for the course nowadays! In any case though some extra scrutiny of the patches here would definitely be welcome, along with scrutiny of the "missing patches" like a [change to pass manager order](rust-lang/llvm@2717444), [another change to pass manager order](rust-lang/llvm@c782feb), some [compile fixes for sparc](rust-lang/llvm@1a83de6), and some [fixes for solaris](rust-lang/llvm@c2bfe0a). [patches4]: rust-lang/llvm@5401fdf...rust-llvm-release-4-0-1 [backport]: rust-lang/llvm@5c54c25 [llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114 [upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096 --- The update to LLVM 6 is desirable for a number of reasons, notably: * This'll allow us to keep up with the upstream wasm backend, picking up new features as they start landing. * Upstream LLVM has fixed a number of SIMD-related compilation errors, especially around AVX-512 and such. * There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed in the LLVM 4 branch we're using. * Overall it's not a great idea to stagnate with our codegen backend! This update is mostly powered by #47730 which is allowing us to update LLVM *independent* of the version of LLVM that Emscripten is locked to. This means that when compiling code for Emscripten we'll still be using the old LLVM 4 backend, but when compiling code for any other target we'll be using the new LLVM 6 target. Once Emscripten updates we may no longer need this distinction, but we're not sure when that will happen! Closes #43370 Closes #43418 Closes #47015 Closes #47683 Closes rust-lang/stdarch#157 Closes rust-lang-nursery/rust-wasm#3
The following submodules have been updated for a new version of LLVM: - `src/llvm` - `src/libcompiler_builtins` - transitively contains compiler-rt - `src/dlmalloc` This also updates the docker container for dist-i686-freebsd as the old 16.04 container is no longer capable of building LLVM. The compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without much interesting happening, but the LLVM update here is of particular note. Unlike previous updates I haven't cherry-picked all existing patches we had on top of our LLVM branch as we have a [huge amount][patches4] and have at this point forgotten what most of them are for. Instead I started from the current `release_60` branch in LLVM and only applied patches that were necessary to get our tests working and building. The current set of custom rustc-specific patches included in this LLVM update are: * rust-lang/llvm@1187443 - this is how we actually implement `cfg(target_feature)` for now and continues to not be upstreamed. While a hazard for SIMD stabilization this commit is otherwise keeping the status quo of a small rustc-specific feature. * rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't upstreamed, notably teaching LLVM about our allocation-related routines (which aren't malloc/free). Once we stabilize the global allocator routines we will likely want to upstream this patch, but for now it seems reasonable to keep it on our fork. * rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in our 32-bit linux container. I'm not really sure why it's necessary but my guess is that it's because of the absolutely ancient glibc that we're using. In any case it's only updating pieces we're not actually using in LLVM so I'm hoping it'll turn out alright. This doesn't seem like something we'll want to upstream.c * rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our i686-freebsd container, I'm not really sure what's going on but we for sure probably don't want to upstream this and otherwise it seems not too bad for now at least. * rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug] which although diagnosed to a particular revision isn't currently fixed upstream (and the bug itself doesn't seem too active). This commit is a partial revert of the suspected cause of this regression (found via a bisection). I'm sort of hoping that this eventually gets fixed upstream with a similar fix (which we can replace in our branch), but for now I'm also hoping it's a relatively harmless change to have. After applying these patches (plus one [backport] which should be [backported upstream][llvm-back]) I believe we should have all tests working on all platforms in our current test suite. I'm like 99% sure that we'll need some more backports as issues are reported for LLVM 6 when this propagates through nightlies, but that's sort of just par for the course nowadays! In any case though some extra scrutiny of the patches here would definitely be welcome, along with scrutiny of the "missing patches" like a [change to pass manager order](rust-lang/llvm@27174447533), [another change to pass manager order](rust-lang/llvm@c782febb7b9), some [compile fixes for sparc](rust-lang/llvm@1a83de63c42), and some [fixes for solaris](rust-lang/llvm@c2bfe0abb). [patches4]: rust-lang/llvm@5401fdf...rust-llvm-release-4-0-1 [backport]: rust-lang/llvm@5c54c252db [llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114 [upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096 --- The update to LLVM 6 is desirable for a number of reasons, notably: * This'll allow us to keep up with the upstream wasm backend, picking up new features as they start landing. * Upstream LLVM has fixed a number of SIMD-related compilation errors, especially around AVX-512 and such. * There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed in the LLVM 4 branch we're using. * Overall it's not a great idea to stagnate with our codegen backend! This update is mostly powered by rust-lang#47730 which is allowing us to update LLVM *independent* of the version of LLVM that Emscripten is locked to. This means that when compiling code for Emscripten we'll still be using the old LLVM 4 backend, but when compiling code for any other target we'll be using the new LLVM 6 target. Once Emscripten updates we may no longer need this distinction, but we're not sure when that will happen! Closes rust-lang#43370 Closes rust-lang#43418 Closes rust-lang#47015 Closes rust-lang#47683 Closes rust-lang/stdarch#157 Closes rust-lang-nursery/rust-wasm#3
rustc: Upgrade to LLVM 6 The following submodules have been updated for a new version of LLVM: - `src/llvm` - `src/libcompiler_builtins` - transitively contains compiler-rt - `src/dlmalloc` This also updates the docker container for dist-i686-freebsd as the old 16.04 container is no longer capable of building LLVM. The compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without much interesting happening, but the LLVM update here is of particular note. Unlike previous updates I haven't cherry-picked all existing patches we had on top of our LLVM branch as we have a [huge amount][patches4] and have at this point forgotten what most of them are for. Instead I started from the current `release_60` branch in LLVM and only applied patches that were necessary to get our tests working and building. The [current set of custom rustc-specific patches](rust-lang/llvm@f128612...rust-llvm-release-6-0-0) included in this LLVM update are: * rust-lang/llvm@1187443 - this is how we actually implement `cfg(target_feature)` for now and continues to not be upstreamed. While a hazard for SIMD stabilization this commit is otherwise keeping the status quo of a small rustc-specific feature. * rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't upstreamed, notably teaching LLVM about our allocation-related routines (which aren't malloc/free). Once we stabilize the global allocator routines we will likely want to upstream this patch, but for now it seems reasonable to keep it on our fork. * rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in our 32-bit linux container. I'm not really sure why it's necessary but my guess is that it's because of the absolutely ancient glibc that we're using. In any case it's only updating pieces we're not actually using in LLVM so I'm hoping it'll turn out alright. This doesn't seem like something we'll want to upstream.c * rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our i686-freebsd container, I'm not really sure what's going on but we for sure probably don't want to upstream this and otherwise it seems not too bad for now at least. * rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug] which although diagnosed to a particular revision isn't currently fixed upstream (and the bug itself doesn't seem too active). This commit is a partial revert of the suspected cause of this regression (found via a bisection). I'm sort of hoping that this eventually gets fixed upstream with a similar fix (which we can replace in our branch), but for now I'm also hoping it's a relatively harmless change to have. After applying these patches (plus one [backport] which should be [backported upstream][llvm-back]) I believe we should have all tests working on all platforms in our current test suite. I'm like 99% sure that we'll need some more backports as issues are reported for LLVM 6 when this propagates through nightlies, but that's sort of just par for the course nowadays! In any case though some extra scrutiny of the patches here would definitely be welcome, along with scrutiny of the "missing patches" like a [change to pass manager order](rust-lang/llvm@2717444), [another change to pass manager order](rust-lang/llvm@c782feb), some [compile fixes for sparc](rust-lang/llvm@1a83de6), and some [fixes for solaris](rust-lang/llvm@c2bfe0a). [patches4]: rust-lang/llvm@5401fdf...rust-llvm-release-4-0-1 [backport]: rust-lang/llvm@5c54c25 [llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114 [upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096 --- The update to LLVM 6 is desirable for a number of reasons, notably: * This'll allow us to keep up with the upstream wasm backend, picking up new features as they start landing. * Upstream LLVM has fixed a number of SIMD-related compilation errors, especially around AVX-512 and such. * There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed in the LLVM 4 branch we're using. * Overall it's not a great idea to stagnate with our codegen backend! This update is mostly powered by #47730 which is allowing us to update LLVM *independent* of the version of LLVM that Emscripten is locked to. This means that when compiling code for Emscripten we'll still be using the old LLVM 4 backend, but when compiling code for any other target we'll be using the new LLVM 6 target. Once Emscripten updates we may no longer need this distinction, but we're not sure when that will happen! Closes #43370 Closes #43418 Closes #47015 Closes #47683 Closes rust-lang/stdarch#157 Closes rust-lang-nursery/rust-wasm#3
The following submodules have been updated for a new version of LLVM: - `src/llvm` - `src/libcompiler_builtins` - transitively contains compiler-rt - `src/dlmalloc` This also updates the docker container for dist-i686-freebsd as the old 16.04 container is no longer capable of building LLVM. The compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without much interesting happening, but the LLVM update here is of particular note. Unlike previous updates I haven't cherry-picked all existing patches we had on top of our LLVM branch as we have a [huge amount][patches4] and have at this point forgotten what most of them are for. Instead I started from the current `release_60` branch in LLVM and only applied patches that were necessary to get our tests working and building. The current set of custom rustc-specific patches included in this LLVM update are: * rust-lang/llvm@1187443 - this is how we actually implement `cfg(target_feature)` for now and continues to not be upstreamed. While a hazard for SIMD stabilization this commit is otherwise keeping the status quo of a small rustc-specific feature. * rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't upstreamed, notably teaching LLVM about our allocation-related routines (which aren't malloc/free). Once we stabilize the global allocator routines we will likely want to upstream this patch, but for now it seems reasonable to keep it on our fork. * rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in our 32-bit linux container. I'm not really sure why it's necessary but my guess is that it's because of the absolutely ancient glibc that we're using. In any case it's only updating pieces we're not actually using in LLVM so I'm hoping it'll turn out alright. This doesn't seem like something we'll want to upstream.c * rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our i686-freebsd container, I'm not really sure what's going on but we for sure probably don't want to upstream this and otherwise it seems not too bad for now at least. * rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug] which although diagnosed to a particular revision isn't currently fixed upstream (and the bug itself doesn't seem too active). This commit is a partial revert of the suspected cause of this regression (found via a bisection). I'm sort of hoping that this eventually gets fixed upstream with a similar fix (which we can replace in our branch), but for now I'm also hoping it's a relatively harmless change to have. After applying these patches (plus one [backport] which should be [backported upstream][llvm-back]) I believe we should have all tests working on all platforms in our current test suite. I'm like 99% sure that we'll need some more backports as issues are reported for LLVM 6 when this propagates through nightlies, but that's sort of just par for the course nowadays! In any case though some extra scrutiny of the patches here would definitely be welcome, along with scrutiny of the "missing patches" like a [change to pass manager order](rust-lang/llvm@27174447533), [another change to pass manager order](rust-lang/llvm@c782febb7b9), some [compile fixes for sparc](rust-lang/llvm@1a83de63c42), and some [fixes for solaris](rust-lang/llvm@c2bfe0abb). [patches4]: rust-lang/llvm@5401fdf...rust-llvm-release-4-0-1 [backport]: rust-lang/llvm@5c54c252db [llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114 [upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096 --- The update to LLVM 6 is desirable for a number of reasons, notably: * This'll allow us to keep up with the upstream wasm backend, picking up new features as they start landing. * Upstream LLVM has fixed a number of SIMD-related compilation errors, especially around AVX-512 and such. * There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed in the LLVM 4 branch we're using. * Overall it's not a great idea to stagnate with our codegen backend! This update is mostly powered by #47730 which is allowing us to update LLVM *independent* of the version of LLVM that Emscripten is locked to. This means that when compiling code for Emscripten we'll still be using the old LLVM 4 backend, but when compiling code for any other target we'll be using the new LLVM 6 target. Once Emscripten updates we may no longer need this distinction, but we're not sure when that will happen! Closes #43370 Closes #43418 Closes #47015 Closes #47683 Closes rust-lang/stdarch#157 Closes rust-lang-nursery/rust-wasm#3
The following submodules have been updated for a new version of LLVM: - `src/llvm` - `src/libcompiler_builtins` - transitively contains compiler-rt - `src/dlmalloc` This also updates the docker container for dist-i686-freebsd as the old 16.04 container is no longer capable of building LLVM. The compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without much interesting happening, but the LLVM update here is of particular note. Unlike previous updates I haven't cherry-picked all existing patches we had on top of our LLVM branch as we have a [huge amount][patches4] and have at this point forgotten what most of them are for. Instead I started from the current `release_60` branch in LLVM and only applied patches that were necessary to get our tests working and building. The current set of custom rustc-specific patches included in this LLVM update are: * rust-lang/llvm@1187443 - this is how we actually implement `cfg(target_feature)` for now and continues to not be upstreamed. While a hazard for SIMD stabilization this commit is otherwise keeping the status quo of a small rustc-specific feature. * rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't upstreamed, notably teaching LLVM about our allocation-related routines (which aren't malloc/free). Once we stabilize the global allocator routines we will likely want to upstream this patch, but for now it seems reasonable to keep it on our fork. * rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in our 32-bit linux container. I'm not really sure why it's necessary but my guess is that it's because of the absolutely ancient glibc that we're using. In any case it's only updating pieces we're not actually using in LLVM so I'm hoping it'll turn out alright. This doesn't seem like something we'll want to upstream.c * rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our i686-freebsd container, I'm not really sure what's going on but we for sure probably don't want to upstream this and otherwise it seems not too bad for now at least. * rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug] which although diagnosed to a particular revision isn't currently fixed upstream (and the bug itself doesn't seem too active). This commit is a partial revert of the suspected cause of this regression (found via a bisection). I'm sort of hoping that this eventually gets fixed upstream with a similar fix (which we can replace in our branch), but for now I'm also hoping it's a relatively harmless change to have. After applying these patches (plus one [backport] which should be [backported upstream][llvm-back]) I believe we should have all tests working on all platforms in our current test suite. I'm like 99% sure that we'll need some more backports as issues are reported for LLVM 6 when this propagates through nightlies, but that's sort of just par for the course nowadays! In any case though some extra scrutiny of the patches here would definitely be welcome, along with scrutiny of the "missing patches" like a [change to pass manager order](rust-lang/llvm@27174447533), [another change to pass manager order](rust-lang/llvm@c782febb7b9), some [compile fixes for sparc](rust-lang/llvm@1a83de63c42), and some [fixes for solaris](rust-lang/llvm@c2bfe0abb). [patches4]: rust-lang/llvm@5401fdf...rust-llvm-release-4-0-1 [backport]: rust-lang/llvm@5c54c252db [llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114 [upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096 --- The update to LLVM 6 is desirable for a number of reasons, notably: * This'll allow us to keep up with the upstream wasm backend, picking up new features as they start landing. * Upstream LLVM has fixed a number of SIMD-related compilation errors, especially around AVX-512 and such. * There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed in the LLVM 4 branch we're using. * Overall it's not a great idea to stagnate with our codegen backend! This update is mostly powered by rust-lang#47730 which is allowing us to update LLVM *independent* of the version of LLVM that Emscripten is locked to. This means that when compiling code for Emscripten we'll still be using the old LLVM 4 backend, but when compiling code for any other target we'll be using the new LLVM 6 target. Once Emscripten updates we may no longer need this distinction, but we're not sure when that will happen! Closes rust-lang#43370 Closes rust-lang#43418 Closes rust-lang#47015 Closes rust-lang#47683 Closes rust-lang/stdarch#157 Closes rust-lang-nursery/rust-wasm#3
rustc: Upgrade to LLVM 6 The following submodules have been updated for a new version of LLVM: - `src/llvm` - `src/libcompiler_builtins` - transitively contains compiler-rt - `src/dlmalloc` This also updates the docker container for dist-i686-freebsd as the old 16.04 container is no longer capable of building LLVM. The compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without much interesting happening, but the LLVM update here is of particular note. Unlike previous updates I haven't cherry-picked all existing patches we had on top of our LLVM branch as we have a [huge amount][patches4] and have at this point forgotten what most of them are for. Instead I started from the current `release_60` branch in LLVM and only applied patches that were necessary to get our tests working and building. The [current set of custom rustc-specific patches](rust-lang/llvm@f128612...rust-llvm-release-6-0-0) included in this LLVM update are: * rust-lang/llvm@1187443 - this is how we actually implement `cfg(target_feature)` for now and continues to not be upstreamed. While a hazard for SIMD stabilization this commit is otherwise keeping the status quo of a small rustc-specific feature. * rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't upstreamed, notably teaching LLVM about our allocation-related routines (which aren't malloc/free). Once we stabilize the global allocator routines we will likely want to upstream this patch, but for now it seems reasonable to keep it on our fork. * rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in our 32-bit linux container. I'm not really sure why it's necessary but my guess is that it's because of the absolutely ancient glibc that we're using. In any case it's only updating pieces we're not actually using in LLVM so I'm hoping it'll turn out alright. This doesn't seem like something we'll want to upstream.c * rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our i686-freebsd container, I'm not really sure what's going on but we for sure probably don't want to upstream this and otherwise it seems not too bad for now at least. * rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug] which although diagnosed to a particular revision isn't currently fixed upstream (and the bug itself doesn't seem too active). This commit is a partial revert of the suspected cause of this regression (found via a bisection). I'm sort of hoping that this eventually gets fixed upstream with a similar fix (which we can replace in our branch), but for now I'm also hoping it's a relatively harmless change to have. After applying these patches (plus one [backport] which should be [backported upstream][llvm-back]) I believe we should have all tests working on all platforms in our current test suite. I'm like 99% sure that we'll need some more backports as issues are reported for LLVM 6 when this propagates through nightlies, but that's sort of just par for the course nowadays! In any case though some extra scrutiny of the patches here would definitely be welcome, along with scrutiny of the "missing patches" like a [change to pass manager order](rust-lang/llvm@2717444), [another change to pass manager order](rust-lang/llvm@c782feb), some [compile fixes for sparc](rust-lang/llvm@1a83de6), and some [fixes for solaris](rust-lang/llvm@c2bfe0a). [patches4]: rust-lang/llvm@5401fdf...rust-llvm-release-4-0-1 [backport]: rust-lang/llvm@5c54c25 [llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114 [upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096 --- The update to LLVM 6 is desirable for a number of reasons, notably: * This'll allow us to keep up with the upstream wasm backend, picking up new features as they start landing. * Upstream LLVM has fixed a number of SIMD-related compilation errors, especially around AVX-512 and such. * There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed in the LLVM 4 branch we're using. * Overall it's not a great idea to stagnate with our codegen backend! This update is mostly powered by #47730 which is allowing us to update LLVM *independent* of the version of LLVM that Emscripten is locked to. This means that when compiling code for Emscripten we'll still be using the old LLVM 4 backend, but when compiling code for any other target we'll be using the new LLVM 6 target. Once Emscripten updates we may no longer need this distinction, but we're not sure when that will happen! Closes #43370 Closes #43418 Closes #47015 Closes #47683 Closes rust-lang/stdarch#157 Closes rust-lang-nursery/rust-wasm#3
The following submodules have been updated for a new version of LLVM: - `src/llvm` - `src/libcompiler_builtins` - transitively contains compiler-rt - `src/dlmalloc` This also updates the docker container for dist-i686-freebsd as the old 16.04 container is no longer capable of building LLVM. The compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without much interesting happening, but the LLVM update here is of particular note. Unlike previous updates I haven't cherry-picked all existing patches we had on top of our LLVM branch as we have a [huge amount][patches4] and have at this point forgotten what most of them are for. Instead I started from the current `release_60` branch in LLVM and only applied patches that were necessary to get our tests working and building. The current set of custom rustc-specific patches included in this LLVM update are: * rust-lang/llvm@1187443 - this is how we actually implement `cfg(target_feature)` for now and continues to not be upstreamed. While a hazard for SIMD stabilization this commit is otherwise keeping the status quo of a small rustc-specific feature. * rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't upstreamed, notably teaching LLVM about our allocation-related routines (which aren't malloc/free). Once we stabilize the global allocator routines we will likely want to upstream this patch, but for now it seems reasonable to keep it on our fork. * rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in our 32-bit linux container. I'm not really sure why it's necessary but my guess is that it's because of the absolutely ancient glibc that we're using. In any case it's only updating pieces we're not actually using in LLVM so I'm hoping it'll turn out alright. This doesn't seem like something we'll want to upstream.c * rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our i686-freebsd container, I'm not really sure what's going on but we for sure probably don't want to upstream this and otherwise it seems not too bad for now at least. * rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug] which although diagnosed to a particular revision isn't currently fixed upstream (and the bug itself doesn't seem too active). This commit is a partial revert of the suspected cause of this regression (found via a bisection). I'm sort of hoping that this eventually gets fixed upstream with a similar fix (which we can replace in our branch), but for now I'm also hoping it's a relatively harmless change to have. After applying these patches (plus one [backport] which should be [backported upstream][llvm-back]) I believe we should have all tests working on all platforms in our current test suite. I'm like 99% sure that we'll need some more backports as issues are reported for LLVM 6 when this propagates through nightlies, but that's sort of just par for the course nowadays! In any case though some extra scrutiny of the patches here would definitely be welcome, along with scrutiny of the "missing patches" like a [change to pass manager order](rust-lang/llvm@27174447533), [another change to pass manager order](rust-lang/llvm@c782febb7b9), some [compile fixes for sparc](rust-lang/llvm@1a83de63c42), and some [fixes for solaris](rust-lang/llvm@c2bfe0abb). [patches4]: rust-lang/llvm@5401fdf...rust-llvm-release-4-0-1 [backport]: rust-lang/llvm@5c54c252db [llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114 [upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096 --- The update to LLVM 6 is desirable for a number of reasons, notably: * This'll allow us to keep up with the upstream wasm backend, picking up new features as they start landing. * Upstream LLVM has fixed a number of SIMD-related compilation errors, especially around AVX-512 and such. * There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed in the LLVM 4 branch we're using. * Overall it's not a great idea to stagnate with our codegen backend! This update is mostly powered by rust-lang#47730 which is allowing us to update LLVM *independent* of the version of LLVM that Emscripten is locked to. This means that when compiling code for Emscripten we'll still be using the old LLVM 4 backend, but when compiling code for any other target we'll be using the new LLVM 6 target. Once Emscripten updates we may no longer need this distinction, but we're not sure when that will happen! Closes rust-lang#43370 Closes rust-lang#43418 Closes rust-lang#47015 Closes rust-lang#47683 Closes rust-lang/stdarch#157 Closes rust-lang-nursery/rust-wasm#3
rustc: Upgrade to LLVM 6 The following submodules have been updated for a new version of LLVM: - `src/llvm` - `src/libcompiler_builtins` - transitively contains compiler-rt - `src/dlmalloc` This also updates the docker container for dist-i686-freebsd as the old 16.04 container is no longer capable of building LLVM. The compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without much interesting happening, but the LLVM update here is of particular note. Unlike previous updates I haven't cherry-picked all existing patches we had on top of our LLVM branch as we have a [huge amount][patches4] and have at this point forgotten what most of them are for. Instead I started from the current `release_60` branch in LLVM and only applied patches that were necessary to get our tests working and building. The [current set of custom rustc-specific patches](rust-lang/llvm@f128612...rust-llvm-release-6-0-0) included in this LLVM update are: * rust-lang/llvm@1187443 - this is how we actually implement `cfg(target_feature)` for now and continues to not be upstreamed. While a hazard for SIMD stabilization this commit is otherwise keeping the status quo of a small rustc-specific feature. * rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't upstreamed, notably teaching LLVM about our allocation-related routines (which aren't malloc/free). Once we stabilize the global allocator routines we will likely want to upstream this patch, but for now it seems reasonable to keep it on our fork. * rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in our 32-bit linux container. I'm not really sure why it's necessary but my guess is that it's because of the absolutely ancient glibc that we're using. In any case it's only updating pieces we're not actually using in LLVM so I'm hoping it'll turn out alright. This doesn't seem like something we'll want to upstream.c * rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our i686-freebsd container, I'm not really sure what's going on but we for sure probably don't want to upstream this and otherwise it seems not too bad for now at least. * rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug] which although diagnosed to a particular revision isn't currently fixed upstream (and the bug itself doesn't seem too active). This commit is a partial revert of the suspected cause of this regression (found via a bisection). I'm sort of hoping that this eventually gets fixed upstream with a similar fix (which we can replace in our branch), but for now I'm also hoping it's a relatively harmless change to have. After applying these patches (plus one [backport] which should be [backported upstream][llvm-back]) I believe we should have all tests working on all platforms in our current test suite. I'm like 99% sure that we'll need some more backports as issues are reported for LLVM 6 when this propagates through nightlies, but that's sort of just par for the course nowadays! In any case though some extra scrutiny of the patches here would definitely be welcome, along with scrutiny of the "missing patches" like a [change to pass manager order](rust-lang/llvm@2717444), [another change to pass manager order](rust-lang/llvm@c782feb), some [compile fixes for sparc](rust-lang/llvm@1a83de6), and some [fixes for solaris](rust-lang/llvm@c2bfe0a). [patches4]: rust-lang/llvm@5401fdf...rust-llvm-release-4-0-1 [backport]: rust-lang/llvm@5c54c25 [llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114 [upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096 --- The update to LLVM 6 is desirable for a number of reasons, notably: * This'll allow us to keep up with the upstream wasm backend, picking up new features as they start landing. * Upstream LLVM has fixed a number of SIMD-related compilation errors, especially around AVX-512 and such. * There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed in the LLVM 4 branch we're using. * Overall it's not a great idea to stagnate with our codegen backend! This update is mostly powered by #47730 which is allowing us to update LLVM *independent* of the version of LLVM that Emscripten is locked to. This means that when compiling code for Emscripten we'll still be using the old LLVM 4 backend, but when compiling code for any other target we'll be using the new LLVM 6 target. Once Emscripten updates we may no longer need this distinction, but we're not sure when that will happen! Closes #43370 Closes #43418 Closes #47015 Closes #47683 Closes rust-lang/stdarch#157 Closes rust-lang-nursery/rust-wasm#3
This commit introduces a separately compiled backend for Emscripten, avoiding
compiling the
JSBackend
target in the main LLVM codegen backend. This buildson the foundation provided by #47671 to create a new codegen backend dedicated
solely to Emscripten, removing the
JSBackend
of the main codegen backend inthe process.
A new field was added to each target for this commit which specifies the backend
to use for translation, the default being
llvm
which is the main backend thatwe use. The Emscripten targets specify an
emscripten
backend instead of themain
llvm
one.There's a whole bunch of consequences of this change, but I'll try to enumerate
them here:
will soon start to drift from the Emscripten submodule, but currently they're
both at the same revision.
This is gated behind a
--enable-emscripten
flag to the configure script. Bydefault users should neither check out the emscripten submodule nor compile
it.
init_repo.sh
script was updated to fetch the Emscripten submodule fromGitHub the same way we do the main LLVM submodule (a tarball fetch).
of targets on CI. We'll only be shipping an Emscripten backend with Tier 1
platforms, though. All cross-compiled platforms will not be receiving an
Emscripten backend yet.
This commit means that when you download the
rustc
package in Rustup for Tier1 platforms you'll be receiving two trans backends, one for Emscripten and one
that's the general LLVM backend. If you never compile for Emscripten you'll
never use the Emscripten backend, so we may update this one day to only download
the Emscripten backend when you add the Emscripten target. For now though it's
just an extra 10MB gzip'd.
Closes #46819