Include llvm-link into Windows distribution

[polkovnikov]

Bugzilla Link	51850
Version	unspecified
OS	Windows NT
CC	@xgupta

Extended Description

There is no llvm-link.exe inside Windows distribution, for example in this one https://github.com/llvm/llvm-project/releases/download/llvmorg-13.0.0-rc2/LLVM-13.0.0-rc2-win64.exe.

I don't know how to join two .bc files into one, like I did with llvm-link on Linux:

llvm-link a.bc b.bc -o ab.bc

Maybe you can suggest at least a workaround with those tools that are included.

Also would be nice to have llvm-as.exe inside package so to be able to compile .ll text assembly to .bc bitcode.

[xgupta]

packaging of binary is done for end users and end users don't need llvm-link and llvm-as as they are llvm developers tools. If you need them then best way is to build them yourself.

[RyanGlScott]

I'm not quite sure I understand the distinction between "end user tools" and "developer tools", especially since the Linux and macOS binary distributions include both kinds of tools, but the Windows binary distributions only include the former.

Practically speaking, the omission of tools like llvm-link from Windows binary distributions makes it impossible to test an application that relies of llvm-link in the backend in CI. Building LLVM on every CI job isn't feasible because of how long it would take.

[xgupta]

Hmm, Then someone from Windows release team can comment correctly on this.

CC @zmodem

[zmodem]

I'm not quite sure I understand the distinction between "end user tools" and "developer tools"

The focus of the Windows installer has been to ship an llvm-based toolchain: clang, lld, sanitizer libraries etc.

For LLVM developers, or developers using LLVM as library to build other things, the best recommendation has been to build it themselves in a configuration that works for them.

With that said, the set of tools that are included (what's in LLVM_TOOLCHAIN_TOOLS) keep expanding. If there's a use case for having llvm-link in there, it could be added. What are you using it for?

[RyanGlScott]

If there's a use case for having llvm-link in there, it could be added. What are you using it for?

We are using llvm-link as part of the Crux verification tool, which performs verification of LLVM code by parsing bitcode files. In order to support C++ code (as well as other projects where there are non-trivial dependencies), we first link together all of the dependencies into a single .bc file with llvm-link and then analyze that file.

Unfortunately, it is extremely difficult to use Crux on Windows at the moment due to how hard it is to get a working version of llvm-link on that platform. To my knowledge, the only way to do so is via MSYS2, but that uses a rolling-release model where you can only ever use the latest LLVM release at a given moment. Ideally, we would be able to use llvm-link from official LLVM downloads so that we can depend on a stable version.

---

While the discussion above is about llvm-link, it is worth noting that we also use other LLVM tools in Crux's test suites that are also not included on Windows, such as llvm-as and llvm-dis. It would be ideal if we could get all of those sorts of tools in Windows bindists as well.

[zmodem]

How is Crux distributed? It sounds like maybe it should be providing llvm-link, llvm-as, llvm-dis etc. on Windows?

[RyanGlScott]

How is Crux distributed?

Crux is distributed as bindists here.

It sounds like maybe it should be providing llvm-link, llvm-as, llvm-dis etc. on Windows?

Yes, that might be wise. But where are we to obtain llvm-link et al. in the first place? As I mentioned above, there are no reliable methods for downloading stable versions of these LLVM tools on Windows, as the official Windows LLVM bindists do not provide them. And we really, really don't want to be in the business of building our own copies of multiple LLVM releases, hosting them somewhere, and maintaining these download pages into perpetuity. After all, that is exactly what this page is doing already—why duplicate the effort?

Life would be so, so much easier if the official LLVM bindists for Windows just included the same tools as every other operating system does.

[Boscop]

I would also really appreciate if the windows binaries also include llvm-link, llvm-as and llvm-dis!

@RyanGlScott Btw, did you manage to build them or do you know where I can find Windows builds of these executables? (Ideally for llvm 15)

[RyanGlScott]

did you manage to build them or do you know where I can find Windows builds of these executables?

The closest thing that I have found are MSYS2's builds of LLVM. For example, MinGW-w64–based builds of LLVM are hosted here, and they also provide Clang-based and UCRT-based builds. But as noted above, MSYS2 uses a rolling-release distribution model, so there is no guarantee that these will be available going forward. Aside from that, these builds aren't very convenient to use in isolation, as they dynamically link against several other MSYS2 libraries that must be downloaded and placed alongside the executables.

The second-closest thing that I have found is this repo. This actually does provide LLVM bindists for Windows that include all of the tools that the official LLVM downloads lack, such as llvm-as and llvm-link. Sadly, this repo no longer appears to be maintained, as its last release was an LLVM 13 build.

[Boscop]

Thanks! When to use the mingw-w64-based vs clang-based vs UCRT-based executables?

I forked the most updated fork of this repo https://github.com/vovkos/llvm-package-windows and adjusted the LLVM version number to 15.0.7, hoping that it would still work, but apparently not:
https://github.com/Boscop/llvm-package-windows/actions/runs/4669816368/jobs/8268772718#step:4:236

CMake Error at cmake/modules/CMakeLists.txt:1 (include):
  include could not find requested file:

    ExtendPath


CMake Error at cmake/modules/CMakeLists.txt:3 (include):
  include could not find requested file:

    FindPrefixFromConfig


CMake Error at cmake/modules/CMakeLists.txt:116 (find_prefix_from_config):
  Unknown CMake command "find_prefix_from_config".


-- Configuring incomplete, errors occurred!
MSBUILD : error MSB1009: Project file does not exist.

Any idea what changed since version 14 / how the script needs to be adapted?

---

Also, when I tried to build it locally on Win 10, it didn't work either (in VS dev console 64-bit):

D:\llvm-15.0.7.src\build>set LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON

D:\llvm-15.0.7.src\build>cmake -GNinja -DCMAKE_BUILD_TYPE=Release ..
[...]
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:327 (include):
  include could not find requested file:

    GNUInstallPackageDir


CMake Error at cmake/modules/CheckCompilerVersion.cmake:44 (message):
  Host Visual Studio version should be at least 19.27 because LLVM will soon
  use new C++ features which your toolchain version doesn't support.  Your
  version is 19.24.28316.0.  You can temporarily opt out using
  LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN, but very soon your toolchain won't be
  supported.
Call Stack (most recent call first):
  cmake/modules/CheckCompilerVersion.cmake:52 (check_compiler_version)
  cmake/config-ix.cmake:15 (include)
  CMakeLists.txt:776 (include)


-- Configuring incomplete, errors occurred!

And I'm also wondering why I still got the error that mentions LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN even though I set it to ON. (I have VS 2019 and would prefer to build with that if possible.)

[RyanGlScott]

When to use the mingw-w64-based vs clang-based vs UCRT-based executables?

The differences are explained in this MSYS2 page. It mostly comes down to whether you want to link against msvcrt or ucrt (two popular C runtimes on Windows), and whether you want to use a GCC- or LLVM-based toolchain. If these considerations aren't important to you, then the MinGW-w64 version (which uses a GCC-based toolchain to link against msvcrt) is a reasonable default.

I'm afraid I can't help you on the build errors. (This is another reason that it would be convenient for the official LLVM bindists to offer these tools, as "just building them" is rarely straightforward.)

[zmodem]

I looked into this some more.

The latest 64-bit installer (https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.2/LLVM-16.0.2-win64.exe) is 293 MB and expands to 2.5 GB. I believe focusing on shipping only the toolchain parts in order to keep these sizes down is still the right thing to do.

We could however consider shipping a separate package that includes everything (we're building all those binaries anyway), but it's not as easy as just building another installer with -DLLVM_INSTALL_TOOLCHAIN_ONLY=OFF since we run into size limitations with NSIS (it seems the 3.6 GB or so of binaries is larger than what it allows). Perhaps for the "everything" package we could just have a zip file. Anyway, to do this I'd welcome patches to the script: https://github.com/llvm/llvm-project/blob/main/llvm/utils/release/build_llvm_release.bat

[RyanGlScott]

I see, thanks for looking into this!

I'm not very familiar with how LLVM's packaging works. What part of the build_llvm_release.bat script is responsible for choosing NSIS? And what would need to be tweaked to change this to .zip? I'm vaguely aware of CMake's ability to configure how something is packaged, but I'm not quite what the best way to override this would be.