Add Polly support.

[DiamondLovesYou]

Use can be triggered via -Z polly, when rustc uses an LLVM which includes polly.

Force LLVM rebuild on buildbots.

[rust-highfive]

r? @estebank

(rust_highfive has picked a reviewer for you, use r? to override)

[rust-highfive]

Warning

• These commits modify submodules.

[DiamondLovesYou]

r? @kennytm @ollie27

[kennytm]

Previous discussion: #50044.

[kennytm]

@bors r+

[bors]

📌 Commit 27586fd has been approved by kennytm

[kennytm]

cc #39884

[bors]

⌛️ Testing commit 27586fd with merge 15d620c...

[bors]

💔 Test failed - status-appveyor

[kennytm]

It caused a bunch on undefined references when linking lib/LLVMPolly.dll (dll?) on Windows (i686-mingw-1).

[00:12:16] [2027/2254] Linking CXX shared module lib\LLVMPolly.dll
[00:12:16] FAILED: lib/LLVMPolly.dll 
[00:12:16] cmd.exe /C "cd . && C:\projects\rust\sccache.exe g++.exe  -ffunction-sections -fdata-sections -m32 -fno-omit-frame-pointer -Werror=date-time -std=gnu++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fno-common -Woverloaded-virtual -Wno-long-long -Wall -W -Wno-unused-parameter -Wwrite-strings -fno-exceptions -fno-rtti  -O2   -shared -o lib\LLVMPolly.dll -Wl,--major-image-version,0,--minor-image-version,0 tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/DependenceInfo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/PolyhedralInfo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopDetection.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopDetectionDiagnostic.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopInfo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopGraphPrinter.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopPass.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/PruneUnprofitable.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/BlockGenerators.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/IslAst.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/IslExprBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/IslNodeBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/CodeGeneration.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/LoopGenerators.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/IRBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/Utils.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/RuntimeDebugBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/CodegenCleanup.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/PerfMonitor.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Exchange/JSONExporter.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/GICHelper.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/SCEVAffinator.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/SCEVValidator.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/RegisterPasses.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/ScopHelper.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/ScopLocation.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/ISLTools.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/DumpModulePass.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/VirtualInstruction.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/External/JSON/json_reader.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/External/JSON/json_value.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/External/JSON/json_writer.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/Canonicalization.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/CodePreparation.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/DeadCodeElimination.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/ScheduleOptimizer.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/FlattenSchedule.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/FlattenAlgo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/ForwardOpTree.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/DeLICM.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/ZoneAlgo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/Simplify.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/MaximalStaticExpansion.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/RewriteByReferenceParameters.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/ScopInliner.cpp.obj tools/polly/lib/CMakeFiles/LLVMPolly.dir/Polly.cpp.obj  lib/libPollyISL.a -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 && cd ."
[00:12:16] tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/DependenceInfo.cpp.obj:DependenceInfo.cpp:(.text$__tcf_2+0x11): undefined reference to `vtable for llvm::cl::Option'
...
[00:12:16] tools/polly/lib/CMakeFiles/LLVMPolly.dir/Polly.cpp.obj:Polly.cpp:(.text.startup._GLOBAL__sub_I_Polly.cpp+0x7): undefined reference to `llvm::PassRegistry::getPassRegistry()'
[00:12:16] collect2.exe: error: ld returned 1 exit status

[DiamondLovesYou]

Huh, that's odd. Rust shouldn't ever try to link polly dynamically, due to the fact that it's in LLVM*.* form and not libLLVM*.* form. So we could, as a last resort, just not build Polly when the host is Windows. I don't develop on windows normally (or at all really); let me setup my gaming windows VM to try and reproduce this. It would be unfortunate to have to disable this for Windows devs.

[bors]

☔️ The latest upstream changes (presumably #50955) made this pull request unmergeable. Please resolve the merge conflicts.

[bors]

☔️ The latest upstream changes (presumably #54021) made this pull request unmergeable. Please resolve the merge conflicts.

[DiamondLovesYou]

K, for future reference: on Msys2, use mingw-w64-x86_64-python2, not python2. Finally got LLVM to configure correctly.

[DiamondLovesYou]

Patch to make Polly not try to build the loadable modules on all Windows platforms: https://reviews.llvm.org/D51904.

@mati865 I made a minor change: I used LLVM_ENABLE_PLUGINS instead of checking platform specific booleans.

[mati865]

@DiamondLovesYou MSYS2 builds LLVM with LLVM_BUILD_LLVM_DYLIB so it has LLVM_ENABLE_PLUGINS enabled and Polly doesn't build.

As long as Rust doesn't enable BUILD_SHARED_LIBS or LLVM_BUILD_LLVM_DYLIB for LLVM Polly should build on Windows.

[DiamondLovesYou]

@mati865 Okay, I dug in and managed to (just now) fix the original issue. I built LLVM with LLVM_BUILD_LLVM_DYLIB && LLVM_LINK_LLVM_DYLIB on Msys/mingw/whatnot without disabling LLVMPolly!

Sadly, using LINK_POLLY_INTO_TOOLS and LLVM_LINK_LLVM_DYLIB together breaks some tools as Polly is linked into the dylib already.

[mati865]

@DiamondLovesYou nice work but maybe you are going too deep? This is pretty much uncharted territory.
Ofc having it fully working everwhere would be nice but it's a lot of effort and time.
This PR seems to be already on pair with LLVM with Polly working on UNIX like platforms and not regress on Windows.

[DiamondLovesYou]

@mati865 Maybe, but it doesn't matter now.

New patch: https://reviews.llvm.org/D51963.

[bors]

☔️ The latest upstream changes (presumably #54146) made this pull request unmergeable. Please resolve the merge conflicts.

[DiamondLovesYou]

@mati865 https://reviews.llvm.org/D51984 and https://reviews.llvm.org/D51986

[bemeurer]

@DiamondLovesYou You're an absolute hero for pushing this through, thank you so much! Hope to see those Polly patches landing soon.

[DiamondLovesYou]

Thanks! Hopefully I can actually get it in; a lot of platform specific issues are coming up, which are breaking check-all in different ways for the BUILD_SHARED_LIBS && LLVM_LINK_LLVM_DYLIB && platform combinations.

[matthiaskrgr]

Perhaps rust-lang should have its own fork of this repo?
I could imagine that if upstream polly gets synced with upstream llvm but rustc uses rust-lang-llvm with llvm upstream polly, there might be conflicts between let's say polly 8.0 and llvm 7.0 (the rustc-lang fork).

[kennytm]

We could track the release_70 branch instead of master if version incompatibility is the only concern.

[bors]

☔️ The latest upstream changes (presumably #55230) made this pull request unmergeable. Please resolve the merge conflicts.

[Centril]

Ping from triage, @kennytm: should this PR be closed? @DiamondLovesYou any updates?

[kennytm]

@Centril this was blocked by several patched on LLVM. If OP did not have any updates this could be closed as S-blocked-closed.

[Centril]

Alright; closing as S-blocked-closed for the time being then.