Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSVC resource compiler `rc.exe` not found unless on PATH during `xmake config` #225

Closed
rivy opened this Issue Oct 11, 2018 · 15 comments

Comments

Projects
None yet
2 participants
@rivy
Copy link
Contributor

commented Oct 11, 2018

The MSVC resource compiler (rc) is not found unless it is already on PATH when xmake config is executed. This is not the case for the MSVC compiler (cl) which is found when on PATH or when not on PATH and in a default location.

C:\...\hello.c>ver
Microsoft Windows [Version 10.0.17134.320]

C:\...\hello.c>xmake --version
xmake v2.2.1.201806212335, A cross-platform build utility based on Lua
Copyright (C) 2015-2018 Ruki Wang, tboox.org, xmake.io
Copyright (C) 2005-2015 Mike Pall, luajit.org
                         _
    __  ___ __  __  __ _| | ______
    \ \/ / |  \/  |/ _  | |/ / __ \
     >  <  | \__/ | /_| |   <  ___/
    /_/\_\_|_|  |_|\__ \|_|\_\____|
                         by ruki, tboox.org

      Manual: https://xmake.io/#/home
      Donate: https://xmake.io/pages/donation.html#donate

C:\...\hello.c>cl
'cl' is not recognized as an internal or external command,
operable program or batch file.

C:\...\hello.c>rc
'rc' is not recognized as an internal or external command,
operable program or batch file.

C:\...\hello.c>cat xmake.lua
target("hello.c")
    set_kind("binary")
    add_files("src/*.c")
    add_files("src/*.rc")

C:\...\hello.c>xmake
checking for the architecture ... x64
checking for the Microsoft Visual Studio (x64) version ... 2015
warning: cannot match target(hello.c).add_files("src\*.rc") at ...\hello.c\xmake.lua:4
[00%]: compiling.release src\main.c
[100%]: linking.release hello.c.exe

C:\...\hello.c>touch src\main.rc

C:\...\hello.c>xmake
error: cannot get program for mrc

I'd be happy to fix and submit a PR, but, although I've looked over the code for the last couple of hours, I haven't been able to grasp the configuration and search logic yet.

@waruqi

This comment has been minimized.

Copy link
Member

commented Oct 12, 2018

You can run the following comand to clean the configuration cache and force to re-detect vs toolchains, then let me see the output info.

$ xmake f -c 
$ xmake -v

Or you can run $ xmake l detect.sdks.find_vstudio. I want to see the vs environment info.

@rivy

This comment has been minimized.

Copy link
Contributor Author

commented Oct 12, 2018

Here you go...

C:\...\hello.c>xmake f -c
checking for the architecture ... x64
checking for the Microsoft Visual Studio (x64) version ... 2015

C:\...\hello.c>xmake -v
configure
{
    arch = x64
,   ccache = true
,   clean = true
,   plat = windows
,   kind = static
,   buildir = build
,   vs = 2015
,   mode = release
,   host = windows
}

error: runv(rc.exe -?) failed(-1)!
checking for the rc.exe ... no
checking for the resource compiler (mrc: rc.exe) ... no
checking for the cl.exe ... ok
checking for the c compiler (cc) ... cl.exe
checking for the ccache ... no
checking for the resource compiler (mrc: rc.exe) ... no
error: cannot get program for mrc

I'm attaching the much longer [xmake l detect.sdks.find_vstudio output] (https://github.com/tboox/xmake/files/2471534/output.xmake.l.detect.sdks.find_vstudio.txt) for your review, as well.

While looking further into this, I realized that I'd hit a problem with a missing rc earlier this year. Apparently, MS left out rc from the Windows Kit 10 "bin" directory, which is the one pushed into the PATH by the VC14 vcvarsall. A solution is to append an "8.1" argument when running the VC14 vcvarsall batch file causing the batch to use the Windows Kit 8.1 instead, eg:

C:\>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.BAT" x86 8.1
C:\>:: or
C:\>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.BAT" x64 8.1

I have a custom local vcvars script which automatically fixes it, so I had forgotten it was a problem.

Here are some references for the issue and the repair:

[1] https://stackoverflow.com/questions/14372706/visual-studio-cant-build-due-to-rc-exe/46166632#46166632 @@ https://archive.is/6HPIG#19.25%

[2] https://stackoverflow.com/questions/43847542/rc-exe-no-longer-found-in-vs-2015-command-prompt/45319119 @@ https://archive.is/GuODI

I also noticed that, unless pre-setup and on the PATH via vcvars, xmake wasn't picking up on my more recent MSVC installation (VC2017), which was installed via the MS "BuildTools" installer. These are possible paths for VC2017 (and the earlier VS15) that you might want to add to the xmake search for vcvarsall:

VC2017 potential paths

  • "%ProgramFiles(x86)%\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build"
  • "%ProgramFiles(x86)%\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build"
  • "%ProgramFiles(x86)%\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build"
  • "%ProgramFiles(x86)%\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build"
  • "%ProgramFiles(x86)%\Microsoft Visual Studio\2017\Common7\IDE\VC"

VS15 potential paths

  • "%ProgramFiles(x86)%\Microsoft Visual Studio\VS15\Enterprise\VC\Auxiliary\Build"
  • "%ProgramFiles(x86)%\Microsoft Visual Studio\VS15\Professional\VC\Auxiliary\Build"
  • "%ProgramFiles(x86)%\Microsoft Visual Studio\VS15\Community\VC\Auxiliary\Build"
  • "%ProgramFiles(x86)%\Microsoft Visual Studio\VS15\BuildTools\VC\Auxiliary\Build"
  • "%ProgramFiles(x86)%\Microsoft Visual Studio\VS15\Common7\IDE\VC"
@waruqi

This comment has been minimized.

Copy link
Member

commented Oct 12, 2018

Thank you for your information, I written a fix script to fix find_vstudio.lua.

You can use find_vstudio.lua.zip to replace your local script and try it again.

The patch code:

diff --git a/xmake/modules/detect/sdks/find_vstudio.lua b/xmake/modules/detect/sdks/find_vstudio.lua
index ae011d9d..7cf4dca1 100644
--- a/xmake/modules/detect/sdks/find_vstudio.lua
+++ b/xmake/modules/detect/sdks/find_vstudio.lua
@@ -107,6 +107,24 @@ function _load_vcvarsall(vcvarsall, arch)
         variables["UCRTVersion"] = UCRTVersion
     end
 
+    -- fix rc.exe missing issues
+    --
+    -- @see https://github.com/tboox/xmake/issues/225
+    -- https://stackoverflow.com/questions/43847542/rc-exe-no-longer-found-in-vs-2015-command-prompt/45319119
+    --
+    -- patch sdk bin directory to path environment
+    --
+    -- .e.g C:\Program Files (x86)\Windows Kits\10\bin\10.0.17134.0\x64
+    --
+    local WindowsSdkDir = variables["WindowsSdkDir"]
+    if WindowsSdkDir and WindowsSDKVersion then
+        local pathes = variables["path"]
+        local bindir = path.join(WindowsSdkDir, "bin", WindowsSDKVersion, arch)
+        if os.isdir(bindir) and pathes then
+            variables["path"] = pathes .. ';' .. bindir
+        end
+    end
+
     -- ok
     return variables
 end

I also noticed that, unless pre-setup and on the PATH via vcvars, xmake wasn't picking up on my more recent MSVC installation (VC2017), which was installed via the MS "BuildTools" installer. These are possible paths for VC2017 (and the earlier VS15) that you might want to add to the xmake search for vcvarsall:

The detection result will be cached in project directory, you can leave the project directory with xmake.lua and run xmake l detect.sdks.find_vstudio to re-detect vs2017 if it was installed recently.

If still not found, you can attempt to add a vs2017 registry path with vcvarsall.bat to find_vstudio.lua.

local pathes = 
{
    format("$(reg HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\VisualStudio\\SxS\\VS7;%s)\\VC", version),
    format("$(reg HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\VisualStudio\\SxS\\VS7;%s)\\VC7\\bin", version),
    format("$(reg HKEY_LOCAL_MACHINE\\SOFTWARE\\Wow6432Node\\Microsoft\\VisualStudio\\SxS\\VS7;%s)\\VC", version),
    format("$(reg HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\VisualStudio\\SxS\\VS7;%s)\\VC\\Auxiliary\\Build", version),
    format("$(reg HKEY_LOCAL_MACHINE\\SOFTWARE\\Wow6432Node\\Microsoft\\VisualStudio\\SxS\\VS7;%s)\\VC\\Auxiliary\\Build", version),
    format("$(env %s)\\..\\..\\VC", vsenvs[version] or "")
}

rivy added a commit to rivy/xmake that referenced this issue Oct 13, 2018

fix MSVC/VS search for MSVC2017
* wip
  - pending portable-VC6 inclusion?
  - improve commit discussion
  - update git discussion @ xmake-io#225

.# Discussion

MSVC > 15.0 no longer sets any registry variables for location purposes.

https://docs.microsoft.com/en-us/visualstudio/install/tools-for-managing-visual-studio-instances?view=vs-2017

https://github.com/Microsoft/vswhere/wiki/Installing
https://github.com/Microsoft/vswhere/blob/master/README.md @@ https://archive.is/mEmdu

`vswhere` is included with all versions >= 15.2
* location is guaranteed

I had to use os.runv(...) b/c of issues with internal spaces and '[', ')' with the vswhere command
@waruqi

This comment has been minimized.

Copy link
Member

commented Oct 14, 2018

I rebuild the installer to attempt to fix the missing rc.exe problem, you can try it again. https://github.com/tboox/xmake/releases/download/v2.2.1/xmake-v2.2.2-dev.exe

the patch commit: 65f9897

rivy added a commit to rivy/xmake that referenced this issue Oct 14, 2018

fix detection strategy for MSVC/VS2017+
* fixes the MSVC/VS 2017 detection problem discussed within <xmake-io#225>

.# Discussion

MSVC/VS versions >= 15.0 will no longer be setting registry entries for tool location
purposes. But `vswhere` (included with MSVC/VS versions >= 15.2, and placed in a
guaranteed location) can instead be used to find the needed path.

`os.runv(...)` is used as the tool to gather `vswhere` output instead of `($shell ...)`
in order to avoid some `($shell ...)` limitations. Commands containing white space and/or
special characters (eg, '[', ')') are difficult or impossible to pass through the current
`($shell ...)` implementation, causing application exceptions. Both white space and the
noted special characters are needed to construct the required `vswhere` command.

ref: <https://github.com/Microsoft/vswhere/blob/master/README.md> @@ <https://archive.is/mEmdu>
@rivy

This comment has been minimized.

Copy link
Contributor Author

commented Oct 14, 2018

I spent some time with the code and investigating MSVC/VS2017+ detection for the last couple of days.

I created a patch that detects it per the vswhere command, finishing a more polished patch last night (which I just pushed: rivy@e55d6f8). The commit message contains a lot of reference info.

I also just saw that you pushed some code related to it as well this morning. I was still having issues with missing rc, so I'll look at your newest code to see if its working.

Thanks for your attention.

rivy added a commit to rivy/xmake that referenced this issue Oct 14, 2018

fix detection strategy for MSVC/VS2017+
* fixes the MSVC/VS 2017 detection problem discussed within <xmake-io#225>

.# Discussion

MSVC/VS versions >= 15.0 will no longer be setting registry entries for tool location
purposes. But `vswhere` (included with MSVC/VS versions >= 15.2, and placed in a
guaranteed location) can instead be used to find the needed path.

`os.runv(...)` is used as the tool to gather `vswhere` output instead of `($shell ...)`
in order to avoid some `($shell ...)` limitations. Commands containing white space and/or
special characters (eg, '[', ')') are difficult or impossible to pass through the current
`($shell ...)` implementation, causing application exceptions. Both white space and the
noted special characters are needed to construct the required `vswhere` command.

ref: <https://github.com/Microsoft/vswhere/blob/master/README.md> @@ <https://archive.is/mEmdu>
@rivy

This comment has been minimized.

Copy link
Contributor Author

commented Oct 14, 2018

I fixed up the detection of MSVC/VS 2017+, including the use of your find_vswhere() function, and created a PR for it (see #228).

Now that that is cleaned up, I'll get back to looking at the initial missing rc issue.

rivy added a commit to rivy/xmake that referenced this issue Oct 14, 2018

fix detection strategy for MSVC/VS2017+
* fixes the MSVC/VS 2017 detection problem discussed within <xmake-io#225>

.# Discussion

MSVC/VS versions >= 15.0 will no longer be setting registry entries for tool location
purposes. But `vswhere` (included with MSVC/VS versions >= 15.2, and placed in a
guaranteed location) can instead be used to find the needed path.

`os.runv(...)` is used as the tool to gather `vswhere` output instead of `($shell ...)`
in order to avoid some `($shell ...)` limitations. Commands containing white space and/or
special characters (eg, '[', ')') are difficult or impossible to pass through the current
`($shell ...)` implementation, causing application exceptions. Both white space and the
noted special characters are needed to construct the required `vswhere` command.

ref: <https://github.com/Microsoft/vswhere/blob/master/README.md> @@ <https://archive.is/mEmdu>
@rivy

This comment has been minimized.

Copy link
Contributor Author

commented Oct 15, 2018

It's working correctly on my machines for all of my currently in-use versions of MSVC/VS (6.0/1998, 8.0/2005, 11.0/2012, 14.0/2015, and 15.0/2017).

Just FYI, to my knowledge, the issue only affected the 14.0/2015 version, so the fix might be overly broad, negatively affecting other versions. But, given the fix being in _load_vcvarsall(), I don't see an easy way to limit the PATH change to just the 14.0/2015 version. And, as I said, it seems to be working without issue for all versions that I have right now.

@waruqi

This comment has been minimized.

Copy link
Member

commented Oct 15, 2018

I moved the fix patch code to find_rc.lua. You can try it again. 254c408
https://github.com/tboox/xmake/releases/download/v2.2.1/xmake-v2.2.2-dev.exe

@waruqi waruqi added the improvement label Oct 16, 2018

@waruqi waruqi added this to the v2.2.2 milestone Oct 16, 2018

@waruqi

This comment has been minimized.

Copy link
Member

commented Oct 16, 2018

I moved the fix patch code to find_rc.lua. You can try it again. 254c408
https://github.com/tboox/xmake/releases/download/v2.2.1/xmake-v2.2.2-dev.exe

Is this ok?

@rivy

This comment has been minimized.

Copy link
Contributor Author

commented Oct 16, 2018

I look at it again tomorrow.

@rivy

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2018

"v2.2.2" is working just fine finding the resource compiler for MSVC/VS 14.0/2015.

I am getting an error message when I run xmake verbosely:

C:>cl
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24234.1 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

usage: cl [ option... ] filename... [ /link linkoption... ]

C:>rc
'rc' is not recognized as an internal or external command,
operable program or batch file.

C:>>type xmake.lua
target("hello")
    set_kind("binary")
    add_files("src/*.c")
    add_files("src/*.rc")

C:>dir src
 Volume in drive C has no label.
 Volume Serial Number is 8A0D-1F81

 Directory of C:\Users\Roy\OneDrive\Projects\eg.xmake\hello\src

10/11/2018  11:50 PM    <DIR>          .
10/11/2018  11:50 PM    <DIR>          ..
09/13/2017  08:12 AM               100 main.c
10/11/2018  12:13 AM             1,296 main.rc
               2 File(s)          1,396 bytes
               2 Dir(s)  613,415,030,784 bytes free

C:>>xmake clean --all & xmake -v
checking for the architecture ... x64
checking for the cl.exe ... ok
checking for the Microsoft Visual Studio (x64) version ... 2015
configure
{
    arch = x64
,   ccache = true
,   mode = release
,   host = windows
,   kind = static
,   buildir = build
,   vs = 2015
,   clean = true
,   plat = windows
}

error: runv(rc.exe -?) failed(-1)!
checking for the rc.exe ... C:\Program Files (x86)\Windows Kits\10\bin\10.0.17134.0\x64\rc.exe
checking for the resource compiler (mrc) ... rc.exe
checking for the cl.exe ... ok
checking for the c compiler (cc) ... cl.exe
checking for the ccache ... no
[00%]: compiling.release src\main.rc
"C:\\Program Files (x86)\\Windows Kits\\10\\bin\\10.0.17134.0\\x64\\rc.exe" -Fobuild\\.objs\\hello\\windows\\x64\\release\\src\\main.rc.obj src\\main.rc
Microsoft (R) Windows (R) Resource Compiler Version 10.0.10011.16384
Copyright (C) Microsoft Corporation.  All rights reserved.
checking for the flags(cl.exe) -Ox -fp:fast ... ok
[50%]: compiling.release src\main.c
cl.exe -c -Ox -fp:fast -nologo -Fobuild\\.objs\\hello\\windows\\x64\\release\\src\\main.c.obj src\\main.c
checking for the link.exe ... ok
checking for the linker (ld) ... link.exe
[100%]: linking.release hello.exe
link.exe -nologo -dynamicbase -nxcompat -machine:x64 -out:build\\windows\\x64\\release\\hello.exe build\\.objs\\hello\\windows\\x64\\release\\src\\main.rc.obj build\\.objs\\hello\\windows\\x64\\release\\src\\main.c.obj 

It still compiles correctly, but I would think that error: runv(rc.exe -?) failed(-1)! should be at most a warning or not displayed at all.

@waruqi

This comment has been minimized.

Copy link
Member

commented Oct 18, 2018

xmake will attempt to run rc.exe /? to detect it first. If fails, it will continue to run C:\Program Files (x86)\Windows Kits\10\bin\10.0.17134.0\x64\rc.exe /? again.

Finally, we found rc.exe, so it's nothing. We need not display the previous detection error info.

@rivy

This comment has been minimized.

Copy link
Contributor Author

commented Oct 18, 2018

Yes, I would suppress that error message unless the resource compiler isn't detected at all. I'm still not well versed in the code path or I would have submitted a patch to suppress the message.

@waruqi

This comment has been minimized.

Copy link
Member

commented Oct 18, 2018

In the default compilation situation this error message has been suppressed, you need only remove -v argument to compile. $ xmake

So we don't need to make any changes.

And you can run xmake -r instead of xmake clean --all & xmake -v

@rivy

This comment has been minimized.

Copy link
Contributor Author

commented Nov 2, 2018

I think this can be marked completed and closed.

The error during normal tool search operation was bothering me, so I made a change for your consideration to suppress it (see PR #247).

@waruqi waruqi closed this Nov 2, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.