Skip to content

Commit

Permalink
configure: Start adding MSVC support
Browse files Browse the repository at this point in the history 
This commit starts to add MSVC support to the ./configure script to enable the
build system to detect and build an MSVC target with the cl.exe compiler and
toolchain. The primary change here is a large sanity check when an MSVC target
is requested (and currently only `x86_64-pc-windows-msvc` is recognized).

When building an MSVC target, the configure script either requires the
`--msvc-root` argument or for `cl.exe` to be in `PATH`. It also requires that if
in the path `cl.exe` is the 64-bit version of the compiler.

Once detected the configure script will run the `vcvarsall.bat` script provided
by Visual Studio to learn about the `INCLUDE` and `LIB` variables needed by the
`cl.exe` compiler to run (the default include/lib paths for the
compiler/linker). These variables are then reexported when running `make` to
ensure that our own compiles are running the same toolchain.

The purpose of this detection and environment variable scraping is to avoid
requiring the build itself to be run inside of a `cmd.exe` shell but rather
allow it to run in the currently expected MinGW/MSYS shell.
  • Loading branch information
@alexcrichton
alexcrichton committed May 19, 2015
1 parent ee258c5 commit 7cf0b17
Show file tree
Hide file tree
Showing 2 changed files with 83 additions and 11 deletions.
65 changes: 60 additions & 5 deletions configure
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1084,6 +1084,65 @@ do
err "musl libc $CFG_MUSL_ROOT/lib/libc.a not found"
fi
;;

x86_64-*-msvc)
# Currently the build system is not configured to build jemalloc
# with MSVC, so we omit this optional dependency.
step_msg "targeting MSVC, disabling jemalloc"
CFG_DISABLE_JEMALLOC=1
putvar CFG_DISABLE_JEMALLOC

# There are some MSYS python builds which will auto-translate
# windows-style paths to MSYS-style paths in Python itself.
# Unfortunately this breaks LLVM's build system as somewhere along
# the line LLVM prints a path into a file from Python and then CMake
# later tries to interpret that path. If Python prints a MSYS path
# and CMake tries to use it as a Windows path, you're gonna have a
# Bad Time.
#
# Consequently here we try to detect when that happens and print an
# error if it does.
if $CFG_PYTHON -c 'import sys; print sys.argv[1]' `pwd` | grep '^/'

This comment has been minimized.

Sign in to view

Copy link
@vadimcn

vadimcn Jun 1, 2015

Contributor

@alexcrichton: My configure is failing this check, but it doesn't look like python is changing the path, but rather that pwd prints it in the unix format to begin with.

This comment has been minimized.

Sign in to view

Copy link
@alexcrichton

alexcrichton Jun 1, 2015

Author Member

Oh bah, this is relying being run in a MSYS shell so the Unix path being printed out by pwd is translated to a Windows path by the shell (passed as an argument to Python), and then ensures that it's not re-translated by python again. I wonder if this could perhaps just be a hardcoded C:\foo argument?

This comment has been minimized.

Sign in to view

Copy link
@vadimcn

vadimcn Jun 1, 2015

Contributor

So it's relying on this? Hmm, that might explain it. So far I've bee using msys2's version of python while building Rust, and it worked without problems. I take it, you are using non-msys python?
What about cmake? msys or non-msys?

I would encourage you to document the environment required to build msvc target. :-) Clearly, it isn't what it says here.

This comment has been minimized.

Sign in to view

Copy link
@alexcrichton

alexcrichton Jun 2, 2015

Author Member

I take it, you are using non-msys python?

There are two MSYS pythons, and there's also a non-MSYS python (e.g. download directly). This is attempting to detect the one broken MSYS python, but all others should work.

What about cmake? msys or non-msys?

Both should work just fine

I would encourage you to document the environment required to build msvc target. :-)

I'd prefer to actually just get it to work everywhere, as if it's not working it's definitely a bug! After some more testing it appears the problem here is a little more subtle than I originally thought:

  • Of the three pythons, /usr/bin/python, /mingw64/bin/python, C:/PythonXX/python, all work.
  • If sh runs /usr/bin/python, it does not work

The problem is that when sh runs /usr/bin/python with an argument of a unix path, it does not auto-translate the argument to Windows path, unlike all other configurations. As to why, I have no idea!

Can you try modifying this check to see if this works for you?

if /bin/sh -c "$CFG_PYTHON -c 'import sys; print sys.argv[1]' /foo"  | grep '^/'

This comment has been minimized.

Sign in to view

Copy link
@vadimcn

vadimcn Jun 2, 2015

Contributor

The problem is that when sh runs /usr/bin/python with an argument of a unix path, it does not auto-translate the argument to Windows path, unlike all other configurations. As to why, I have no idea!

I think the above link actually explains why: /usr/bin/python depends on msys-2.0.dll, and is therefore considered a native posix program, which gets the /c/foo/bar version.

Do you know, at what stage do the posix paths break cmake? Can we use cygpath -w <path> to convert path explicitly?

This comment has been minimized.

Sign in to view

Copy link
@alexcrichton

alexcrichton Jun 2, 2015

Author Member

Ah yep, that'd do it! Unfortunately if MSYS does not translate a Unix path to a Windows path for the python-in-use, it will end up leaking into LLVM's build system as a Unix path, and that path is then emitted in turn to a file which is later read by CMake. CMake then tries to read a file at a Unix path, which definitely doesn't exist, so it chokes.

The specific path taken is:

I'm not 100% sure, but it seems that the current directory of CMake is Unix-style due to being run from MSYS (maybe?) and this then leaks into python via __file__, and eventually leaks into CMake as well.

If there's a way to force windows style paths all the way, then that'd be great! Unfortunately last time I tried that I couldn't figure out a way to do so :\

This comment has been minimized.

Sign in to view

Copy link
@vadimcn

vadimcn Jun 2, 2015

Contributor

CMake probably isn't at fault here. It seems that the act of importing a module in msys python is what causes the conversion of path into unix format:
c:\rust\foo.py: print(__file__); import bar
c:\rust\bar.py: print(__file__)

C:\> python2.7 c:\rust\foo.py
c:\rust\foo.py
/c/rust/bar.py

I don't think there is a way to force windows paths in msys, but here's something that sort of works:

C:\> mount c:\rust /rust
mount: warning - /rust does not exist.
C:\> python2.7 c:\rust\foo.py
c:\rust\foo.py
/rust/bar.py

Apparently the /rust mount point wins over /c/rust when converting c:\rust to unix style.
/rust/... can be fed to most native windows programs, including cmake, without problems; they just interpret it as a current drive-relative path.

Thoughts?

This comment has been minimized.

Sign in to view

Copy link
@alexcrichton

alexcrichton Jun 2, 2015

Author Member

I'd probably err on the side of avoiding the mount, but being able to detect a MSYS python through the imports reliably sounds like it's more beneficial than this check today. We could include some src/etc scripts which just print out __file__ and detect if a unix path comes out?

We could also probably improve this error message (e.g. suggest an alternate python package)
then
err "python is silently translating windows paths to MSYS paths \
and the build will fail if this python is used.\n\n \
Either an official python install must be used or an \
alternative python package in MinGW must be used."
fi

# MSVC requires cmake because that's how we're going to build LLVM
probe_need CFG_CMAKE cmake

# Use the REG program to figure out where VS is installed
# We need to figure out where cl.exe and link.exe are, so we do some
# munging and some probing here. We also look for the default
# INCLUDE and LIB variables for MSVC so we can set those in the
# build system as well.
install=$(reg QUERY \
'HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\12.0' \
-v InstallDir)
need_ok "couldn't find visual studio install root"
CFG_MSVC_ROOT=$(echo "$install" | grep InstallDir | sed 's/.*REG_SZ[ ]*//')
CFG_MSVC_ROOT=$(dirname "$CFG_MSVC_ROOT")
CFG_MSVC_ROOT=$(dirname "$CFG_MSVC_ROOT")
CFG_MSVC_CL="${CFG_MSVC_ROOT}/VC/bin/amd64/cl.exe"
CFG_MSVC_LIB="${CFG_MSVC_ROOT}/VC/bin/amd64/lib.exe"
CFG_MSVC_LINK="${CFG_MSVC_ROOT}/VC/bin/amd64/link.exe"

vcvarsall="${CFG_MSVC_ROOT}/VC/vcvarsall.bat"
CFG_MSVC_INCLUDE_PATH=$(cmd /c "\"$vcvarsall\" amd64 && cmd /c echo %INCLUDE%")
need_ok "failed to learn about MSVC's INCLUDE"
CFG_MSVC_LIB_PATH=$(cmd /c "\"$vcvarsall\" amd64 && cmd /c echo %LIB%")
need_ok "failed to learn about MSVC's LIB"

putvar CFG_MSVC_ROOT
putvar CFG_MSVC_CL
putvar CFG_MSVC_LIB
putvar CFG_MSVC_LINK
putvar CFG_MSVC_INCLUDE_PATH
putvar CFG_MSVC_LIB_PATH
;;

*)
;;
esac
Expand Down Expand Up @@ -1125,6 +1184,7 @@ do
do
make_dir $t/rt/stage$s
make_dir $t/rt/jemalloc
make_dir $t/rt/compiler-rt
for i in \
isaac sync test \
arch/i386 arch/x86_64 arch/arm arch/aarch64 arch/mips arch/powerpc
Expand Down Expand Up @@ -1496,11 +1556,6 @@ do
putvar $CFG_LLVM_INST_DIR
done

# Munge any paths that appear in config.mk back to posix-y
cp config.tmp config.tmp.bak
sed -e 's@ \([a-zA-Z]\):[/\\]@ /\1/@g;' <config.tmp.bak >config.tmp
rm -f config.tmp.bak

msg
copy_if_changed ${CFG_SRC_DIR}Makefile.in ./Makefile
move_if_changed config.tmp config.mk
Expand Down
29 changes: 23 additions & 6 deletions mk/cfg/x86_64-pc-windows-msvc.mk
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# x86_64-pc-windows-msvc configuration
CC_x86_64-pc-windows-msvc=cl
LINK_x86_64-pc-windows-msvc=link
CXX_x86_64-pc-windows-msvc=cl
CPP_x86_64-pc-windows-msvc=cl
AR_x86_64-pc-windows-msvc=llvm-ar
CC_x86_64-pc-windows-msvc="$(CFG_MSVC_CL)" -nologo
LINK_x86_64-pc-windows-msvc="$(CFG_MSVC_LINK)" -nologo
CXX_x86_64-pc-windows-msvc="$(CFG_MSVC_CL)" -nologo
CPP_x86_64-pc-windows-msvc="$(CFG_MSVC_CL)" -nologo
AR_x86_64-pc-windows-msvc="$(CFG_MSVC_LIB)" -nologo
CFG_LIB_NAME_x86_64-pc-windows-msvc=$(1).dll
CFG_STATIC_LIB_NAME_x86_64-pc-windows-msvc=$(1).lib
CFG_LIB_GLOB_x86_64-pc-windows-msvc=$(1)-*.dll
Expand All @@ -21,4 +21,21 @@ CFG_UNIXY_x86_64-pc-windows-msvc :=
CFG_LDPATH_x86_64-pc-windows-msvc :=
CFG_RUN_x86_64-pc-windows-msvc=$(2)
CFG_RUN_TARG_x86_64-pc-windows-msvc=$(call CFG_RUN_x86_64-pc-windows-msvc,,$(2))
CFG_GNU_TRIPLE_x86_64-pc-windows-msvc := x86_64-pc-windows-msvc
CFG_GNU_TRIPLE_x86_64-pc-windows-msvc := x86_64-pc-win32

# These two environment variables are scraped by the `./configure` script and
# are necessary for `cl.exe` to find standard headers (the INCLUDE variable) and
# for `link.exe` to find standard libraries (the LIB variable).
ifdef CFG_MSVC_INCLUDE_PATH
export INCLUDE := $(CFG_MSVC_INCLUDE_PATH)
endif
ifdef CFG_MSVC_LIB_PATH
export LIB := $(CFG_MSVC_LIB_PATH)
endif

# Unfortunately `link.exe` is also a program in `/usr/bin` on MinGW installs,
# but it's not the one that we want. As a result we make sure that our detected
# `link.exe` shows up in PATH first.
ifdef CFG_MSVC_LINK
export PATH := $(CFG_MSVC_ROOT)/VC/bin/amd64:$(PATH)
endif

0 comments on commit 7cf0b17

Please sign in to comment.