Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to LLVM 10 #67759

Merged
merged 2 commits into from
May 21, 2020
Merged

Update to LLVM 10 #67759

merged 2 commits into from
May 21, 2020

Conversation

nikic
Copy link
Contributor

@nikic nikic commented Dec 31, 2019

LLVM 10 is going to be branched soon, so it's a good time to start finding all those tasty new miscompiles and performance regressions ;)

Status:

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-12-31T15:00:13.5244805Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-12-31T15:00:14.1661198Z ##[command]git config gc.auto 0
2019-12-31T15:00:14.1666994Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-12-31T15:00:14.1671697Z ##[command]git config --get-all http.proxy
2019-12-31T15:00:14.1677709Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67759/merge:refs/remotes/pull/67759/merge
---
2019-12-31T15:07:48.9476380Z    Compiling memoffset v0.5.1
2019-12-31T15:07:49.9020375Z error: failed to run custom build command for `rustc_llvm v0.0.0 (/checkout/src/librustc_llvm)`
2019-12-31T15:07:49.9032390Z 
2019-12-31T15:07:49.9032496Z Caused by:
2019-12-31T15:07:49.9033044Z   process didn't exit successfully: `/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/release/build/rustc_llvm-12ceb52d5dc2c5b7/build-script-build` (exit code: 1)
2019-12-31T15:07:49.9033287Z --- stdout
2019-12-31T15:07:49.9033654Z cargo:rerun-if-env-changed=REAL_LIBRARY_PATH_VAR
2019-12-31T15:07:49.9033853Z cargo:rerun-if-env-changed=REAL_LIBRARY_PATH
2019-12-31T15:07:49.9034087Z cargo:rerun-if-changed=/usr/lib/llvm-7/bin/llvm-config
2019-12-31T15:07:49.9034275Z cargo:rerun-if-env-changed=LLVM_CONFIG
2019-12-31T15:07:49.9034616Z cargo:rustc-cfg=llvm_component="aarch64"
2019-12-31T15:07:49.9034969Z cargo:rustc-cfg=llvm_component="amdgpu"
2019-12-31T15:07:49.9035492Z cargo:rustc-cfg=llvm_component="arm"
2019-12-31T15:07:49.9035914Z cargo:rustc-cfg=llvm_component="asmparser"
2019-12-31T15:07:49.9036624Z cargo:rustc-cfg=llvm_component="bitreader"
2019-12-31T15:07:49.9036814Z cargo:rustc-cfg=llvm_component="bitwriter"
2019-12-31T15:07:49.9037179Z cargo:rustc-cfg=llvm_component="hexagon"
2019-12-31T15:07:49.9037729Z cargo:rustc-cfg=llvm_component="instrumentation"
2019-12-31T15:07:49.9038138Z cargo:rustc-cfg=llvm_component="ipo"
2019-12-31T15:07:49.9038510Z cargo:rustc-cfg=llvm_component="linker"
2019-12-31T15:07:49.9038868Z cargo:rustc-cfg=llvm_component="lto"
2019-12-31T15:07:49.9039205Z cargo:rustc-cfg=llvm_component="mips"
2019-12-31T15:07:49.9039567Z cargo:rustc-cfg=llvm_component="msp430"
2019-12-31T15:07:49.9039905Z cargo:rustc-cfg=llvm_component="nvptx"
2019-12-31T15:07:49.9040241Z cargo:rustc-cfg=llvm_component="powerpc"
2019-12-31T15:07:49.9040605Z cargo:rustc-cfg=llvm_component="sparc"
2019-12-31T15:07:49.9040944Z cargo:rustc-cfg=llvm_component="systemz"
2019-12-31T15:07:49.9041442Z cargo:rustc-cfg=llvm_component="webassembly"
2019-12-31T15:07:49.9041968Z cargo:rustc-cfg=llvm_component="x86"
2019-12-31T15:07:49.9042296Z cargo:rerun-if-changed-env=LLVM_RUSTLLVM
2019-12-31T15:07:49.9042650Z cargo:rerun-if-changed=../rustllvm/ArchiveWrapper.cpp
2019-12-31T15:07:49.9042998Z cargo:rerun-if-changed=../rustllvm/.editorconfig
2019-12-31T15:07:49.9043339Z cargo:rerun-if-changed=../rustllvm/PassWrapper.cpp
2019-12-31T15:07:49.9043696Z cargo:rerun-if-changed=../rustllvm/Linker.cpp
2019-12-31T15:07:49.9044378Z cargo:rerun-if-changed=../rustllvm/rustllvm.h
2019-12-31T15:07:49.9044717Z cargo:rerun-if-changed=../rustllvm/README
2019-12-31T15:07:49.9045335Z cargo:rerun-if-changed=../rustllvm/RustWrapper.cpp
2019-12-31T15:07:49.9045691Z TARGET = Some("x86_64-unknown-linux-gnu")
2019-12-31T15:07:49.9045872Z OPT_LEVEL = Some("2")
2019-12-31T15:07:49.9046141Z HOST = Some("x86_64-unknown-linux-gnu")
2019-12-31T15:07:49.9046336Z CXX_x86_64-unknown-linux-gnu = Some("sccache c++")
2019-12-31T15:07:49.9046578Z CXXFLAGS_x86_64-unknown-linux-gnu = Some("-ffunction-sections -fdata-sections -fPIC -m64")
2019-12-31T15:07:49.9046625Z CRATE_CC_NO_DEFAULTS = None
2019-12-31T15:07:49.9046666Z DEBUG = Some("false")
2019-12-31T15:07:49.9046722Z CARGO_CFG_TARGET_FEATURE = Some("fxsr,mmx,sse,sse2")
2019-12-31T15:07:49.9048439Z running: "sccache" "sccache" "c++" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I/usr/lib/llvm-7/include" "-std=c++0x" "-fuse-ld=gold" "-Wl,--no-keep-files-mapped" "-Wl,--no-map-whole-files" "-fPIC" "-fvisibility-inlines-hidden" "-Werror=date-time" "-std=c++11" "-Wall" "-Wextra" "-Wno-unused-parameter" "-Wwrite-strings" "-Wcast-qual" "-Wno-missing-field-initializers" "-pedantic" "-Wno-long-long" "-Wno-maybe-uninitialized" "-Wdelete-non-virtual-dtor" "-Wno-comment" "-ffunction-sections" "-fdata-sections" "-O2" "-DNDEBUG" "-fno-exceptions" "-D_GNU_SOURCE" "-D__STDC_CONSTANT_MACROS" "-D__STDC_FORMAT_MACROS" "-D__STDC_LIMIT_MACROS" "-DLLVM_COMPONENT_AARCH64" "-DLLVM_COMPONENT_AMDGPU" "-DLLVM_COMPONENT_ARM" "-DLLVM_COMPONENT_ASMPARSER" "-DLLVM_COMPONENT_BITREADER" "-DLLVM_COMPONENT_BITWRITER" "-DLLVM_COMPONENT_HEXAGON" "-DLLVM_COMPONENT_INSTRUMENTATION" "-DLLVM_COMPONENT_IPO" "-DLLVM_COMPONENT_LINKER" "-DLLVM_COMPONENT_LTO" "-DLLVM_COMPONENT_MIPS" "-DLLVM_COMPONENT_MSP430" "-DLLVM_COMPONENT_NVPTX" "-DLLVM_COMPONENT_POWERPC" "-DLLVM_COMPONENT_SPARC" "-DLLVM_COMPONENT_SYSTEMZ" "-DLLVM_COMPONENT_WEBASSEMBLY" "-DLLVM_COMPONENT_X86" "-DNDEBUG" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/rustc_llvm-20b5b3b7a5d2cc79/out/../rustllvm/PassWrapper.o" "-c" "../rustllvm/PassWrapper.cpp"
2019-12-31T15:07:49.9049232Z cargo:warning=../rustllvm/PassWrapper.cpp: In function 'LLVMRustThinLTOData* LLVMRustCreateThinLTOData(LLVMRustThinLTOModule*, int, const char**, int)':
2019-12-31T15:07:49.9049623Z cargo:warning=../rustllvm/PassWrapper.cpp:866:19: error: 'make_unique' is not a member of 'std'
2019-12-31T15:07:49.9049810Z cargo:warning=   auto Ret = std::make_unique<LLVMRustThinLTOData>();
2019-12-31T15:07:49.9049886Z cargo:warning=                   ^~~~~~~~~~~
2019-12-31T15:07:49.9050168Z cargo:warning=../rustllvm/PassWrapper.cpp:866:19: note: suggested alternative: '__unique'
2019-12-31T15:07:49.9050452Z cargo:warning=   auto Ret = std::make_unique<LLVMRustThinLTOData>();
2019-12-31T15:07:49.9050543Z cargo:warning=                   ^~~~~~~~~~~
2019-12-31T15:07:49.9050602Z cargo:warning=                   __unique
2019-12-31T15:07:49.9050959Z cargo:warning=../rustllvm/PassWrapper.cpp:866:50: error: expected primary-expression before '>' token
2019-12-31T15:07:49.9051140Z cargo:warning=   auto Ret = std::make_unique<LLVMRustThinLTOData>();
2019-12-31T15:07:49.9051821Z cargo:warning=                                                  ^
2019-12-31T15:07:49.9052189Z cargo:warning=../rustllvm/PassWrapper.cpp:866:52: error: expected primary-expression before ')' token
2019-12-31T15:07:49.9052365Z cargo:warning=   auto Ret = std::make_unique<LLVMRustThinLTOData>();
2019-12-31T15:07:49.9052461Z cargo:warning=                                                    ^
2019-12-31T15:07:49.9052780Z cargo:warning=../rustllvm/PassWrapper.cpp: In function 'LLVMRustThinLTOBuffer* LLVMRustThinLTOBufferCreate(LLVMModuleRef)':
2019-12-31T15:07:49.9053209Z cargo:warning=../rustllvm/PassWrapper.cpp:1108:19: error: 'make_unique' is not a member of 'std'
2019-12-31T15:07:49.9053555Z cargo:warning=   auto Ret = std::make_unique<LLVMRustThinLTOBuffer>();
2019-12-31T15:07:49.9053845Z cargo:warning=                   ^~~~~~~~~~~
2019-12-31T15:07:49.9054388Z cargo:warning=../rustllvm/PassWrapper.cpp:1108:19: note: suggested alternative: '__unique'
2019-12-31T15:07:49.9054442Z cargo:warning=   auto Ret = std::make_unique<LLVMRustThinLTOBuffer>();
2019-12-31T15:07:49.9054634Z cargo:warning=                   ^~~~~~~~~~~
2019-12-31T15:07:49.9054713Z cargo:warning=                   __unique
2019-12-31T15:07:49.9055027Z cargo:warning=../rustllvm/PassWrapper.cpp:1108:52: error: expected primary-expression before '>' token
2019-12-31T15:07:49.9055229Z cargo:warning=   auto Ret = std::make_unique<LLVMRustThinLTOBuffer>();
2019-12-31T15:07:49.9055309Z cargo:warning=                                                    ^
2019-12-31T15:07:49.9055655Z cargo:warning=../rustllvm/PassWrapper.cpp:1108:54: error: expected primary-expression before ')' token
2019-12-31T15:07:49.9055856Z cargo:warning=   auto Ret = std::make_unique<LLVMRustThinLTOBuffer>();
2019-12-31T15:07:49.9055947Z cargo:warning=                                                      ^
2019-12-31T15:07:49.9056032Z cargo:warning=../rustllvm/PassWrapper.cpp: At global scope:
2019-12-31T15:07:49.9056455Z cargo:warning=../rustllvm/PassWrapper.cpp:838:1: warning: 'const llvm::GlobalValueSummary* getFirstDefinitionForLinker(const GlobalValueSummaryList&)' defined but not used [-Wunused-function]
2019-12-31T15:07:49.9056690Z cargo:warning= getFirstDefinitionForLinker(const GlobalValueSummaryList &GVSummaryList) {
2019-12-31T15:07:49.9057032Z cargo:warning= ^~~~~~~~~~~~~~~~~~~~~~~~~~~
2019-12-31T15:07:49.9057149Z 
2019-12-31T15:07:49.9057374Z --- stderr
2019-12-31T15:07:49.9057401Z 
2019-12-31T15:07:49.9058329Z 
2019-12-31T15:07:49.9058329Z 
2019-12-31T15:07:49.9060058Z error occurred: Command "sccache" "sccache" "c++" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I/usr/lib/llvm-7/include" "-std=c++0x" "-fuse-ld=gold" "-Wl,--no-keep-files-mapped" "-Wl,--no-map-whole-files" "-fPIC" "-fvisibility-inlines-hidden" "-Werror=date-time" "-std=c++11" "-Wall" "-Wextra" "-Wno-unused-parameter" "-Wwrite-strings" "-Wcast-qual" "-Wno-missing-field-initializers" "-pedantic" "-Wno-long-long" "-Wno-maybe-uninitialized" "-Wdelete-non-virtual-dtor" "-Wno-comment" "-ffunction-sections" "-fdata-sections" "-O2" "-DNDEBUG" "-fno-exceptions" "-D_GNU_SOURCE" "-D__STDC_CONSTANT_MACROS" "-D__STDC_FORMAT_MACROS" "-D__STDC_LIMIT_MACROS" "-DLLVM_COMPONENT_AARCH64" "-DLLVM_COMPONENT_AMDGPU" "-DLLVM_COMPONENT_ARM" "-DLLVM_COMPONENT_ASMPARSER" "-DLLVM_COMPONENT_BITREADER" "-DLLVM_COMPONENT_BITWRITER" "-DLLVM_COMPONENT_HEXAGON" "-DLLVM_COMPONENT_INSTRUMENTATION" "-DLLVM_COMPONENT_IPO" "-DLLVM_COMPONENT_LINKER" "-DLLVM_COMPONENT_LTO" "-DLLVM_COMPONENT_MIPS" "-DLLVM_COMPONENT_MSP430" "-DLLVM_COMPONENT_NVPTX" "-DLLVM_COMPONENT_POWERPC" "-DLLVM_COMPONENT_SPARC" "-DLLVM_COMPONENT_SYSTEMZ" "-DLLVM_COMPONENT_WEBASSEMBLY" "-DLLVM_COMPONENT_X86" "-DNDEBUG" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/rustc_llvm-20b5b3b7a5d2cc79/out/../rustllvm/PassWrapper.o" "-c" "../rustllvm/PassWrapper.cpp" with args "c++" did not execute successfully (status code exit code: 1).
2019-12-31T15:07:49.9060298Z 
2019-12-31T15:07:49.9060320Z 
2019-12-31T15:07:49.9061220Z warning: build failed, waiting for other jobs to finish...
2019-12-31T15:07:50.0699244Z error: build failed
---
2019-12-31T15:07:50.8023825Z   local time: Tue Dec 31 15:07:50 UTC 2019
2019-12-31T15:07:50.8023889Z   network time: Tue, 31 Dec 2019 15:07:50 GMT
2019-12-31T15:07:50.8023940Z == end clock drift check ==
2019-12-31T15:07:52.2523357Z 
2019-12-31T15:07:52.2583057Z ##[error]Bash exited with code '1'.
2019-12-31T15:07:52.2606973Z ##[section]Starting: Checkout
2019-12-31T15:07:52.2608333Z ==============================================================================
2019-12-31T15:07:52.2608377Z Task         : Get sources
2019-12-31T15:07:52.2608427Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@nikic
Copy link
Contributor Author

nikic commented Dec 31, 2019

First optimization regression, eventually leading to failure to eliminate bounds checks: https://bugs.llvm.org/show_bug.cgi?id=44419

@nikic
Copy link
Contributor Author

nikic commented Jan 1, 2020

Second optimization regression: https://bugs.llvm.org/show_bug.cgi?id=44423

@nikic
Copy link
Contributor Author

nikic commented Jan 2, 2020

src/test/codegen/issue-45222.rs shows a regression in optimization of the notoriously fickle RangeInclusive.

While previously

bb10:                                             ; preds = %bb3.i.i, %bb6
  %iter1.sroa.5.0 = phi i64 [ 100000, %bb6 ], [ %spec.select, %bb3.i.i ]
  %iter1.sroa.9.0 = phi i8 [ 2, %bb6 ], [ %6, %bb3.i.i ]
  %count.1 = phi i64 [ %count.0, %bb6 ], [ %8, %bb3.i.i ]
  %2 = icmp eq i8 %iter1.sroa.9.0, 2
  %3 = and i8 %iter1.sroa.9.0, 1
  %phitmp.i.i.i = icmp eq i8 %3, 0
  %or.cond = or i1 %2, %phitmp.i.i.i
  br i1 %or.cond, label %bb3.i.i, label %bb2.loopexit

bb3.i.i:                                          ; preds = %bb10
  %4 = icmp ne i64 %iter1.sroa.5.0, 0
  %5 = xor i1 %4, true
  %6 = zext i1 %5 to i8
  %7 = add i64 %iter1.sroa.5.0, -1
  %spec.select = select i1 %4, i64 %7, i64 0
  %8 = add i64 %count.1, %iter1.sroa.5.0
  br label %bb10

was rotated to

bb3.i.i:                                          ; preds = %bb6, %bb3.i.i
  %count.159 = phi i64 [ %count.0, %bb6 ], [ %6, %bb3.i.i ]
  %iter1.sroa.5.058 = phi i64 [ 100000, %bb6 ], [ %spec.select, %bb3.i.i ]
  %2 = icmp ne i64 %iter1.sroa.5.058, 0
  %3 = xor i1 %2, true
  %4 = zext i1 %3 to i8
  %5 = add i64 %iter1.sroa.5.058, -1
  %spec.select = select i1 %2, i64 %5, i64 0
  %6 = add i64 %count.159, %iter1.sroa.5.058
  %7 = icmp eq i8 %4, 2
  %8 = and i8 %4, 1
  %phitmp.i.i.i = icmp eq i8 %8, 0
  %or.cond = or i1 %7, %phitmp.i.i.i
  br i1 %or.cond, label %bb3.i.i, label %bb2.loopexit

we now get

bb10:                                             ; preds = %bb3.i.i, %bb6
  %iter1.sroa.5.0 = phi i64 [ 100000, %bb6 ], [ %spec.select, %bb3.i.i ]
  %iter1.sroa.9.0 = phi i8 [ 2, %bb6 ], [ %3, %bb3.i.i ]
  %count.1 = phi i64 [ %count.0, %bb6 ], [ %4, %bb3.i.i ]
  switch i8 %iter1.sroa.9.0, label %bb2.loopexit [
    i8 2, label %bb3.i.i
    i8 0, label %bb3.i.i
  ]

bb3.i.i:                                          ; preds = %bb10, %bb10
  %2 = icmp ne i64 %iter1.sroa.5.0, 0
  %_10.i.i = xor i1 %2, true
  %3 = zext i1 %_10.i.i to i8
  %_5.0.i.i.i.i = add i64 %iter1.sroa.5.0, -1
  %spec.select = select i1 %2, i64 %_5.0.i.i.i.i, i64 0
  %4 = add i64 %count.1, %iter1.sroa.5.0
  br label %bb10

which does not get rotated, and thus not simplified subsequently.

The reason why we get a switch here (created by SimplifyCFG) is that CVP eliminated a masking operation, namely instruction %5 in https://llvm.godbolt.org/z/cCuRtG. The original optimization does get applied at -C opt-level=3 -- ironically, the reason is that argument promotion results in a loss of !range metadata, preventing the CVP optimization...

I'm not yet sure how to address this issue.

@nikic
Copy link
Contributor Author

nikic commented Jan 2, 2020

Actually, CVP might be both the problem and the solution here ... if we relax https://github.com/llvm/llvm-project/blob/a58da1a2ff039dd3bb4c43db3919995cf4a74cc7/llvm/lib/Transforms/Scalar/CorrelatedValuePropagation.cpp#L315 to allow phis, then condition %4 can be folded as well, which will likely avoid the followup issues.

@nikic
Copy link
Contributor Author

nikic commented Jan 3, 2020

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Jan 3, 2020

⌛ Trying commit d6773bf with merge d5ca0b366ee2e90c658ca0334740b4907ac1ce69...

@rust-highfive
Copy link
Collaborator

The job dist-x86_64-linux-alt of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-01-03T22:40:54.9106810Z    Compiling rustc-std-workspace-core v1.99.0 (/checkout/src/tools/rustc-std-workspace-core)
2020-01-03T22:41:01.7062662Z error: failed to run custom build command for `profiler_builtins v0.0.0 (/checkout/src/libprofiler_builtins)`
2020-01-03T22:41:01.7062812Z 
2020-01-03T22:41:01.7062911Z Caused by:
2020-01-03T22:41:01.7063662Z   process didn't exit successfully: `/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/release/build/profiler_builtins-79d44dd7732723e3/build-script-build` (exit code: 1)
2020-01-03T22:41:01.7064278Z --- stdout
2020-01-03T22:41:01.7064543Z TARGET = Some("x86_64-unknown-linux-gnu")
2020-01-03T22:41:01.7064615Z OPT_LEVEL = Some("2")
2020-01-03T22:41:01.7064857Z HOST = Some("x86_64-unknown-linux-gnu")
2020-01-03T22:41:01.7065439Z CC_x86_64-unknown-linux-gnu = Some("sccache clang")
2020-01-03T22:41:01.7066383Z CFLAGS_x86_64-unknown-linux-gnu = Some("-ffunction-sections -fdata-sections -fPIC --target=x86_64-unknown-linux-gnu -fdebug-prefix-map=/checkout=/rustc/d5ca0b366ee2e90c658ca0334740b4907ac1ce69")
2020-01-03T22:41:01.7066528Z CRATE_CC_NO_DEFAULTS = None
2020-01-03T22:41:01.7066600Z DEBUG = Some("true")
2020-01-03T22:41:01.7067755Z running: "sccache" "sccache" "clang" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=x86_64-unknown-linux-gnu" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=x86_64-unknown-linux-gnu" "-fdebug-prefix-map=/checkout=/rustc/d5ca0b366ee2e90c658ca0334740b4907ac1ce69" "-fno-builtin" "-fvisibility=hidden" "-fomit-frame-pointer" "-ffreestanding" "-DVISIBILITY_HIDDEN" "-DCOMPILER_RT_HAS_UNAME=1" "-DCOMPILER_RT_HAS_FCNTL_LCK=1" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/build/profiler_builtins-3fff06880a6b0050/out/GCDAProfiling.o" "-c" "/checkout/src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c"
2020-01-03T22:41:01.7068350Z cargo:warning=In file included from /checkout/src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c:62:
2020-01-03T22:41:01.7068756Z cargo:warning=/checkout/src/llvm-project/compiler-rt/lib/profile/InstrProfiling.h:16:10: fatal error: 'profile/InstrProfData.inc' file not found
2020-01-03T22:41:01.7068907Z cargo:warning=#include "profile/InstrProfData.inc"
2020-01-03T22:41:01.7069008Z cargo:warning=         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-01-03T22:41:01.7069083Z cargo:warning=1 error generated.
2020-01-03T22:41:01.7069361Z 
2020-01-03T22:41:01.7069562Z --- stderr
2020-01-03T22:41:01.7069596Z 
2020-01-03T22:41:01.7069627Z 
2020-01-03T22:41:01.7069627Z 
2020-01-03T22:41:01.7071484Z error occurred: Command "sccache" "sccache" "clang" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=x86_64-unknown-linux-gnu" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=x86_64-unknown-linux-gnu" "-fdebug-prefix-map=/checkout=/rustc/d5ca0b366ee2e90c658ca0334740b4907ac1ce69" "-fno-builtin" "-fvisibility=hidden" "-fomit-frame-pointer" "-ffreestanding" "-DVISIBILITY_HIDDEN" "-DCOMPILER_RT_HAS_UNAME=1" "-DCOMPILER_RT_HAS_FCNTL_LCK=1" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/build/profiler_builtins-3fff06880a6b0050/out/GCDAProfiling.o" "-c" "/checkout/src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c" with args "clang" did not execute successfully (status code exit code: 1).
2020-01-03T22:41:01.7072062Z 
2020-01-03T22:41:01.7072113Z 
2020-01-03T22:41:01.7072467Z warning: build failed, waiting for other jobs to finish...
2020-01-03T22:41:03.4272429Z error: build failed
---
2020-01-03T22:41:03.4396848Z   local time: Fri Jan  3 22:41:03 UTC 2020
2020-01-03T22:41:03.9739354Z   network time: Fri, 03 Jan 2020 22:41:03 GMT
2020-01-03T22:41:03.9742186Z == end clock drift check ==
2020-01-03T22:41:07.4513425Z 
2020-01-03T22:41:07.4632411Z ##[error]Bash exited with code '1'.
2020-01-03T22:41:07.4674659Z ##[section]Starting: Checkout
2020-01-03T22:41:07.4677067Z ==============================================================================
2020-01-03T22:41:07.4677161Z Task         : Get sources
2020-01-03T22:41:07.4677264Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Jan 3, 2020

💔 Test failed - checks-azure

@bors bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jan 3, 2020
@nikic
Copy link
Contributor Author

nikic commented Jan 3, 2020

@bors try

@bors
Copy link
Contributor

bors commented Jan 3, 2020

⌛ Trying commit 26bf168 with merge 97588ae...

bors added a commit that referenced this pull request Jan 3, 2020
[WIP] Update to LLVM 10

LLVM 10 is going to be branched soon, so it's a good time to start finding all those tasty new miscompiles and performance regressions ;)

r? @ghost
@bors
Copy link
Contributor

bors commented Jan 4, 2020

☀️ Try build successful - checks-azure
Build commit: 97588ae (97588aeda139309169b11654fc809e1ac5fd246c)

@rust-timer
Copy link
Collaborator

Queued 97588ae with parent c5840f9, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit 97588ae, comparison URL.

@nikic
Copy link
Contributor Author

nikic commented Jan 4, 2020

Those are some pretty hefty compile-time regressions. Some local numbers for the largest diffs in LLVM timings on clap-rs (via this script):

Global Variable Optimizer #2                                   0.018s   0.115s   0.097s
Loop Sink                                                      0.118s   0.002s  -0.116s
Function Integration/Inlining                                  1.278s   1.160s  -0.117s
Loop Strength Reduction                                        ------   0.124s   0.124s
Branch Probability Analysis                                    0.011s   0.139s   0.128s
Memory SSA #3                                                  ------   0.132s   0.132s
Memory SSA #2                                                  ------   0.147s   0.147s
X86 DAG->DAG Instruction Selection                             1.087s   1.363s   0.277s
Global Variable Optimizer                                      0.087s   0.506s   0.419s
Total                                                         14.547s  16.280s   1.733s

Haven't tested how stable these numbers are, but we clearly have a big regression in "Global Variable Optimizer" (first time I hear about this pass), some regression in X86 ISel (this tends to get slower with each release, but this seems an unusually large jump) and we now have a lot more MemorySSA runs. Previously there was just one, now there are 5 (!!). Not sure if this is an analysis preservation fail or supposed to be that way.

@nikic
Copy link
Contributor Author

nikic commented Jan 4, 2020

Looks like the five MemorySSA runs are expected, as MemorySSA is now enabled for loop passes: https://reviews.llvm.org/D58311

The GlobalOpt regression is likely https://reviews.llvm.org/D68298 and hopefully easy to address with a revert.

@nikic
Copy link
Contributor Author

nikic commented Jan 4, 2020

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Jan 4, 2020

⌛ Trying commit 6011237 with merge 1b6633b46cd7705ec813cfd1cd7083e92f9da903...

@bors
Copy link
Contributor

bors commented Jan 4, 2020

☀️ Try build successful - checks-azure
Build commit: 1b6633b46cd7705ec813cfd1cd7083e92f9da903 (1b6633b46cd7705ec813cfd1cd7083e92f9da903)

@rust-timer
Copy link
Collaborator

Queued 1b6633b46cd7705ec813cfd1cd7083e92f9da903 with parent 79cf5e4, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit 1b6633b46cd7705ec813cfd1cd7083e92f9da903, comparison URL.

@nikic
Copy link
Contributor Author

nikic commented Jan 4, 2020

Okay, that improved things a bit, reclaiming about 5% in the best case. Still pretty bad compile-time regression overall.

@nikic
Copy link
Contributor Author

nikic commented Jan 4, 2020

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@nikomatsakis
Copy link
Contributor

Sorry, I'm not clear on this, but do we have perf results from this change? It seems like we ought to check back and discuss, perhaps at the next compiler team triage meeting. A quick glance at perf.rust-lang.org didn't show anything yet, but it may not have taken into account the effects of this PR.

cc @rust-lang/wg-prioritization -- can we add this to our next agenda to check back in?

@ecstatic-morse
Copy link
Contributor

ecstatic-morse commented May 22, 2020

Personally, I would be happy to spend 10% longer compiling at -C opt-level=3 if there were an improvement in runtime performance, since you can always set a lower optimization level. However, non-codegened full builds seem to be about the same for real-world crates, or in the case of keccak and inflate even slower. This means rustc isn't really benefiting from whatever additional optimizations LLVM is doing. It would be nice to know whether this is true for the ecosystem as well.

@nnethercote
Copy link
Contributor

nnethercote commented May 22, 2020

Final perf results from the landing are here.

Lots of red. Interestingly, check builds (which don't run the LLVM backend) are a little improved. Which suggests that the quality of the code generated by LLVM 10 is a little better, at least for rustc itself. (Edit: as @ecstatic-morse said, the gains are mostly in the short-running benchmarks; the more realistic benchmarks see little change.)

@cuviper
Copy link
Member

cuviper commented May 22, 2020

The net results compared to 45d050c (git merge-base beta master) don't look so dire. The wild outliers make me wonder if this is really a valid comparison though.

https://perf.rust-lang.org/compare.html?start=45d050cde277b22a755847338f2acc2c7b834141&end=82911b3bba76e73afe2881b732fe6b0edb35d5d3&stat=instructions%3Au

@nnethercote
Copy link
Contributor

The script-servo-opt-incr-patched outlier is a known problem where slight changes in codegen boundaries have major effects on that benchmark. See rust-lang/compiler-team#281 for more details.

The net results compared to 45d050c (git merge-base beta master) don't look so dire.

Is that comparing the nightly against the most recent beta? It won't look so dire because we've made a bunch of speedups recently, particularly for debug builds. If you filter down to opt results only, things look a lot worse.

And compare it against this, which is the comparison of that beta against the revision just before the LLVM 10 upgrade. Again, filter down to opt only and we were doing great there.

Or just look at the individual opt benchmark graphs on perf.rust-lang.org, it's pretty clear there. (Ignore the summary, that's misleading because script-servo utterly dominates it because it's by far the longest-running benchmark.)

@cuviper
Copy link
Member

cuviper commented May 22, 2020

I don't doubt at all that this PR slowed things down. I was trying to see what the overall story from 1.44 to 1.45 will look like, in spite of this.

@nnethercote
Copy link
Contributor

Sorry, I'm not clear on this, but do we have perf results from this change? It seems like we ought to check back and discuss, perhaps at the next compiler team triage meeting. A quick glance at perf.rust-lang.org didn't show anything yet, but it may not have taken into account the effects of this PR.

cc @rust-lang/wg-prioritization -- can we add this to our next agenda to check back in?

@nikomatsakis: Any update here? The current dev cycle is almost over and the upgrade to LLVM 10 is going to cause a sizeable perf regression without any upside that I'm aware of.

@cuviper
Copy link
Member

cuviper commented Jun 1, 2020

There's a maintenance upside in bugs that may be fixed in LLVM 10, without Rust having to encounter them and try to backport fixes. For instance, I just got done dealing with an LLVM 9 miscompilation in s390x, rhbz1837660, which was thankfully already fixed in LLVM 10. Of course there's a risk of LLVM regressions too, even apart from performance, but let's hope for forward progress.

@tmandry
Copy link
Member

tmandry commented Jun 2, 2020

Personally I think our stance in general should be to accept slower opt builds, especially if it results in better code generation (anything else is swimming against the tide for LLVM). For major regressions and regressions to debug builds, we ideally should report to upstream and work with them to identify the cause.

For this change specifically, most regressions are for opt builds, but there are some for debug builds. But sticking to a >6mo old version of LLVM is going to cause a lot more pain than it would solve at this point, IMO. Conversely, rolling to new versions of LLVM more frequently makes it far easier to catch and identify regressions like this.

Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Jun 13, 2020
Rename "cyclone" to "apple-a7" per changes in upstream LLVM

It looks like they intended to keep "cyclone" as a legacy option, but removed it from the list of subtarget features. This created a flood of warnings when targeting aarch64-apple-ios, and probably also created incorrectly optimized artifacts.

See:
https://reviews.llvm.org/D70779
https://reviews.llvm.org/D70779#C1703593NL568

LLVM 10 merged into master at:
rust-lang#67759
RalfJung added a commit to RalfJung/rust that referenced this pull request Jun 13, 2020
Rename "cyclone" to "apple-a7" per changes in upstream LLVM

It looks like they intended to keep "cyclone" as a legacy option, but removed it from the list of subtarget features. This created a flood of warnings when targeting aarch64-apple-ios, and probably also created incorrectly optimized artifacts.

See:
https://reviews.llvm.org/D70779
https://reviews.llvm.org/D70779#C1703593NL568

LLVM 10 merged into master at:
rust-lang#67759
RalfJung added a commit to RalfJung/rust that referenced this pull request Jun 13, 2020
Rename "cyclone" to "apple-a7" per changes in upstream LLVM

It looks like they intended to keep "cyclone" as a legacy option, but removed it from the list of subtarget features. This created a flood of warnings when targeting aarch64-apple-ios, and probably also created incorrectly optimized artifacts.

See:
https://reviews.llvm.org/D70779
https://reviews.llvm.org/D70779#C1703593NL568

LLVM 10 merged into master at:
rust-lang#67759
RalfJung added a commit to RalfJung/rust that referenced this pull request Jun 13, 2020
Rename "cyclone" to "apple-a7" per changes in upstream LLVM

It looks like they intended to keep "cyclone" as a legacy option, but removed it from the list of subtarget features. This created a flood of warnings when targeting aarch64-apple-ios, and probably also created incorrectly optimized artifacts.

See:
https://reviews.llvm.org/D70779
https://reviews.llvm.org/D70779#C1703593NL568

LLVM 10 merged into master at:
rust-lang#67759
RalfJung added a commit to RalfJung/rust that referenced this pull request Jun 13, 2020
Rename "cyclone" to "apple-a7" per changes in upstream LLVM

It looks like they intended to keep "cyclone" as a legacy option, but removed it from the list of subtarget features. This created a flood of warnings when targeting aarch64-apple-ios, and probably also created incorrectly optimized artifacts.

See:
https://reviews.llvm.org/D70779
https://reviews.llvm.org/D70779#C1703593NL568

LLVM 10 merged into master at:
rust-lang#67759
RalfJung added a commit to RalfJung/rust that referenced this pull request Jun 13, 2020
Rename "cyclone" to "apple-a7" per changes in upstream LLVM

It looks like they intended to keep "cyclone" as a legacy option, but removed it from the list of subtarget features. This created a flood of warnings when targeting aarch64-apple-ios, and probably also created incorrectly optimized artifacts.

See:
https://reviews.llvm.org/D70779
https://reviews.llvm.org/D70779#C1703593NL568

LLVM 10 merged into master at:
rust-lang#67759
RalfJung added a commit to RalfJung/rust that referenced this pull request Jun 15, 2020
Rename "cyclone" to "apple-a7" per changes in upstream LLVM

It looks like they intended to keep "cyclone" as a legacy option, but removed it from the list of subtarget features. This created a flood of warnings when targeting aarch64-apple-ios, and probably also created incorrectly optimized artifacts.

See:
https://reviews.llvm.org/D70779
https://reviews.llvm.org/D70779#C1703593NL568

LLVM 10 merged into master at:
rust-lang#67759
RalfJung added a commit to RalfJung/rust that referenced this pull request Jun 15, 2020
Rename "cyclone" to "apple-a7" per changes in upstream LLVM

It looks like they intended to keep "cyclone" as a legacy option, but removed it from the list of subtarget features. This created a flood of warnings when targeting aarch64-apple-ios, and probably also created incorrectly optimized artifacts.

See:
https://reviews.llvm.org/D70779
https://reviews.llvm.org/D70779#C1703593NL568

LLVM 10 merged into master at:
rust-lang#67759
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.