Unifying Windows vs Linux build steps

[gwihlidal]

It may be a bit early for this, but it seems like a brief discussion around unifying the build steps between Windows and Linux would be nice.

Personally, I really dislike how the Windows version of DXC is configured and built - it deviates heavily from standard LLVM build practices, and even how cmake should be invoked. The Linux porting work makes building DXC (for Linux and OSX) much cleaner and obvious, and allowing for the Windows build to be configured and compiled in the same manner (i.e. cmake, ninja, etc..) would be fantastic.

Currently, trying to invoke a build on Windows using the Linux instructions results in the following:

PS H:\Repositories\dxc_test1\DirectXShaderCompiler\build> cmake ../ -GNinja -DCMAKE_BUILD_TYPE=Release $(cat ../utils/cmake-predefined-
config-params)
-- The C compiler identification is Clang 6.0.0
CMake Error at C:/Program Files/CMake/share/cmake-3.10/Modules/CMakeDetermineCompilerId.cmake:771 (message):
  The Clang compiler tool

    "C:/Program Files/LLVM/bin/clang.exe"

  targets the MSVC ABI but has a GNU-like command-line interface.  This is
  not supported.  Use 'clang-cl' instead, e.g.  by setting 'CC=clang-cl' in
  the environment.  Furthermore, use the MSVC command-line environment.
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.10/Modules/CMakeDetermineCCompiler.cmake:113 (CMAKE_DIAGNOSE_UNSUPPORTED_CLANG)
  CMakeLists.txt:28 (project)


CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
See also "H:/Repositories/dxc_test1/DirectXShaderCompiler/build/CMakeFiles/CMakeOutput.log".

Is there any idea (or previous discussion) on the effort or feasibility of allowing the Windows compilation to work the same as Linux/OSX? This would also be very helpful when bootstrapping FFI bindings to DXC from other languages.

Thank you!
Graham

[gwihlidal]

Btw, clang is specified due to a local LLVM windows build. I removed it and also launched cmake from a VS 2017 dev prompt. I progressed further, but ended up hitting:

-- Found DIASDK: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/DIA SDK/lib/diaguids.lib
CMake Error at C:/Program Files/CMake/share/cmake-3.10/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find TAEF (missing: TAEF_COMMON_LIBRARY)
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.10/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindTAEF.cmake:82 (find_package_handle_standard_args)
  tools/clang/unittests/HLSL/CMakeLists.txt:3 (find_package)


-- Configuring incomplete, errors occurred!

[pow2clk]

This is something I've thought about. Personally, I'd like to be able to type cmake .. and have it "just work" unless you want to diverge from the default options. However, I've only discussed this with the spider that lives behind my desk. Off the record, I've seen some shade thrown about the Windows build system, but I don't know how the authors of hctbuild would feel about eliminating it. I'm assuming it was created out of disdain for cmake customizations.

[gwihlidal]

It definitely feels incorrect the way it's done on Windows, to be completely blunt. This is especially true when considering that Microsoft developed vcpkg on top of cmake, and having a dxc portfile in vcpkg would make building and integrating dxc a breeze.

[gwihlidal]

I listed more about the desire for dxc + vcpkg a while ago in microsoft#715

I wouldn't even mind maintaining the portfile, but I would want to start from a dxc codebase that cleanly compiles with standard cmake first :)

[antiagainst]

Hey @gwihlidal: I'm also a fan of clean build process using well-known standard procedure. So you can see that we do not have any script wrapping around the Linux/macOS build process. (Or at least yet. ;-P) We are striking for plain cmake + ninja, but some additional options are requested to configure DXC. They are put in the cmake-predefined-config-params file. Merging them fully into the existing cmake configurations may be doable, but it seems very intrusive. However, we haven't looked into it very carefully, so still an option.

I'm not aware of how vcpkg works; I assume it can take additional cmake options when invoking cmake? Then you can just copy what is in cmake-predefined-config-params. Kinda annoying and duplicating configuration facts again, but we are unlikely to change these cmake options. So they are stable at least.

Regarding the specific error involving TAEF: it is required for building and running tests. You probably want to run utils\hct\hctgettaef.py to download it. Another thing that may work is -DHLSL_INCLUDE_TESTS=OFF.

[gwihlidal]

Thanks for the info, @antiagainst - I will try what you suggested.

Regarding vcpkg, yes you can customize all aspects of cmake, apply source level patches, and other processes needed to build for the specified "triplet" (i.e. x64, static, windows).

In fact, vcpkg is the only reliable way I've been able to compile gRPC and protobuf on Windows :)

Looking at the portfile for gRPC, you can see quite of a bit of customization to the build without needing to pollute the original gRPC repository:
https://github.com/Microsoft/vcpkg/blob/master/ports/grpc/portfile.cmake

It also applies some patches for disabling c# and fixing up uwp in this specific case: https://github.com/Microsoft/vcpkg/tree/master/ports/grpc

I feel that vcpkg would be a perfect fit for reliably building dxc for windows, and now linux\osx with the recent support added to vcpkg to make it cross platform. I have a ton of experience with vcpkg now, so I'm happy to answer any questions about it!