Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build as administrator is required on windows (sometimes?) #4440

Open
koubaa opened this issue Nov 1, 2023 · 18 comments
Open

Build as administrator is required on windows (sometimes?) #4440

koubaa opened this issue Nov 1, 2023 · 18 comments
Labels
MSVC Microsoft Visual C++ Support

Comments

@koubaa
Copy link

koubaa commented Nov 1, 2023

  • I tried to submit the bug on the mailing list on 5/12/23 but looking at the archives it is not there. So I am posting here.
    SCons version: 4.0.1, installed via pip from pypi
    Python 3.10 from python.org
    Windows 10, Visual Studio 2019.
def GetVCVarsPath():¬
    vswhere_path = os.path.expandvars('%ProgramFiles(x86)%/Microsoft Visual Studio/Installer/vswhere.exe'args = [vswhere_path'-format', 'json''-utf8''-version', '[16.0,)'] # VS2019+¬
    # Prefer older Visual Studio versions over newer ones¬
    for cand in sorted(json.loads(check_output(args).decode('utf-8')),¬
                       key=lambda cand: cand['installationVersion']):¬
        install_path = cand['installationPath'vcvars_path = os.path.join(install_path'VC''Auxiliary''Build''vcvars64.bat'if os.path.exists(vcvars_path):¬
            return vcvars_path

We use MSVC_USE_SCRIPT=GetVCVarsPath(), function defined above. Users might also have Visual Studio 2022 installed, but this build requires VS2019.

Describe the bug
I set up a scons-based build system at my organization. Some users have complained to me that they can only run the build as administrator. I am not able to reproduce that problem myself. Here is the error message that the users who are affected get when they build as a non-admin.

cl : Command line error D8050 : cannot execute 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.20.27508\bin\HostX64\x64\c1xx.dll': failed to get command line into debug records
scons: building terminated because of errors.

I can try to get a minimal reproducible example, but apparently any c++ scons build is affected for these users. We use Visual Studio 2019

scons is invoked from python in a virtual environment where scons is installed as a subprocess. The executable in the subprocess is determined using the result of the subprocess where scons.

@mwichmann
Copy link
Collaborator

Well, the error is a compiler error, not an SCons error - SCons has found and executed a compiler binary. It seems it's trying to create a file somewhere that it hasn't the rights to, which is why running with admin rights helps. I'd be looking at the settings of environment variables TEMP and TMP in the places where this is causing trouble to see if there are any clues there. SCons does fetch those from the environment (os.environ) and put them into the execution environment so they'll be available to the processes that are kicked off by SCons.

There are other ways to specify the version you want to use, though those have evolved somewhat since 4.0, so you shouldn't have to resort to your own call to vswhere (scons is going to do that anyway) but that's not the problem since you've shown it's finding it already, since 14.2x is the 2019 series.

scons is invoked from python in a virtual environment where scons is installed as a subprocess. The executable in the subprocess is determined using the result of the subprocess where scons.

Out of curiosity, why make it so complicated?

@jcbrill any thoughts here?

@koubaa
Copy link
Author

koubaa commented Nov 1, 2023

Out of curiosity, why make it so complicated?

@mwichmann I set it up to just run scons from the shell in the appropriate python environment. At some point I took on other responsibilities and someone else made that change, and I don't know why, unfortunately. But I am still asked for help by users when the build system doesn't work since I created it initially.

@jcbrill
Copy link
Contributor

jcbrill commented Nov 1, 2023

Not much to add yet.

This thread seems to be similar:
https://developercommunity.visualstudio.com/t/binhostx86x86c1xxdll/314749 (emphasis added)

In my case I was running cl.exe via SCons. Scons strips the environment! So you need to tell Scons to pass through TMP from up-level environment, or just set it to something in the sconscipt.

To verify that this indeed is the case, create cl.bat somewhere in path with content that conststs of "set". This will print the environment variables. In my case TMP wasn't set to anything (and apparently cl.exe defaults to %systemroot% and fails with access denied -- that's why one of the solutions is to have cl run as admin)

At present, it sounds like a Windows environment issue.

In addition to checking that the TEMP and TMP environment variables are defined and point to folders that exist, one user experienced this error with VS2017 when the USERPROFILE environment variable was not defined.

https://stackoverflow.com/questions/52152923/vs2017-build-returns-d8050-failed-to-get-command-line-into-debug-records
https://stackoverflow.com/questions/26547214/cl-exe-returing-error-code-d8050

@mwichmann
Copy link
Collaborator

mwichmann commented Nov 1, 2023

SCons made this change:

  • The %TEMP% and %TMP% external environment variables are now propagated automatically to the command execution environment on Windows systems.

way back in

RELEASE 0.96.90 - Tue, 15 Feb 2005

The change to automatically propagate %USERPROFILE% into the execution environment came much later, in 2021 (more than a year after 4.0), so it's possible that is a part of the issue you're seeing.

@jcbrill
Copy link
Contributor

jcbrill commented Nov 1, 2023

USERPROFILE does not appear to be imported from the environment until version 4.4.0.

@mwichmann
Copy link
Collaborator

USERPROFILE does not appear to be imported from the environment until version 4.4.0.

Yes, I think it missed the 4.3.0 release by a couple of weeks.

diff --git a/SCons/Platform/win32.py b/SCons/Platform/win32.py
index eeb2abff4..990794f96 100644
--- a/SCons/Platform/win32.py
+++ b/SCons/Platform/win32.py
@@ -381,7 +381,7 @@ def generate(env):
     # for SystemDrive because it's related.
     #
     # Weigh the impact carefully before adding other variables to this list.
-    import_env = ['SystemDrive', 'SystemRoot', 'TEMP', 'TMP' ]
+    import_env = ['SystemDrive', 'SystemRoot', 'TEMP', 'TMP', 'USERPROFILE']
     for var in import_env:
         v = os.environ.get(var)
         if v:

@jcbrill
Copy link
Contributor

jcbrill commented Nov 1, 2023

Typical TEMP and TMP User Variables definitions are:

TEMP=%USERPROFILE%\AppData\Local\Temp
TMP=%USERPROFILE%\AppData\Local\Temp

Typical TEMP and TMP System Variables definitions are:

TEMP=%SystemRoot%\TEMP
TMP=%SystemRoot%\TEMP

If the User Variable TEMP and TMP definitions are missing, there might be a permissions problem with the temp folder off the system root (as mentioned in the linked posts) which would explain why it works with admin privileges.

@bdbaddog
Copy link
Contributor

bdbaddog commented Nov 1, 2023

Try upgrading to latest SCons...

@jcbrill
Copy link
Contributor

jcbrill commented Nov 1, 2023

The windows environment issue experienced is reproducible when both TEMP and TMP are undefined using scons 4.0.1 and VS2019.

If either TEMP or TMP are defined and point to a valid folder the build will succeed. USERPROFILE does not matter.

FAILS from a "normal" windows command-line prompt without TEMP and TMP:

K:\SCons\test-scons-4440>set TEMP=
K:\SCons\test-scons-4440>set TMP=
K:\SCons\test-scons-4440>set USERPROFILE=

K:\SCons\test-scons-4440>%PYEXE% ../scons-4.0.1/scripts/scons.py
scons: Reading SConscript files ...

Computer Name: VS64-DEV-W11
SCons Root: Unknown
Installed Versions: ['14.2', '14.1', '14.0', '12.0', '11.0', '10.0', '9.0', '8.0', '7.1', '7.0', '6.0']
Default Tools: ['default', 'mslink', 'msvc', 'gfortran', 'masm', 'mslib', 'dmd', 'msvs', 'midl', 'filesystem', 'jar', 'javac', 'rmic', 'tar', 'zip', 'textfile']

Build: _build-vs64-dev-w11-0001 {'MSVC_VERSION': '14.2', 'MSVC_USE_SCRIPT': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Auxiliary\\Build\\vcvars64.bat'}

scons: done reading SConscript files.
scons: Building targets ...
cl /Fo_build-vs64-dev-w11-0001\hello.obj /c src\hello.cpp /TP /MDd /EHsc /ZI /FS
Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30151 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

hello.cpp
cl : Command line error D8050 : cannot execute 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\c1xx.dll': failed to get command line into debug records
scons: *** [_build-vs64-dev-w11-0001\hello.obj] Error 2
scons: building terminated because of errors.

K:\SCons\test-scons-4440>

PASSES from an admin windows command prompt without TEMP and TMP:

J:\SCons\test-scons-4440>set TEMP=
J:\SCons\test-scons-4440>set TMP=
J:\SCons\test-scons-4440>set USERPROFILE=

J:\SCons\test-scons-4440>%PYEXE% ../scons-4.0.1/scripts/scons.py
scons: Reading SConscript files ...

Computer Name: VS64-DEV-W11
SCons Root: Unknown
Installed Versions: ['14.2', '14.1', '14.0', '12.0', '11.0', '10.0', '9.0', '8.0', '7.1', '7.0', '6.0']
Default Tools: ['default', 'mslink', 'msvc', 'gfortran', 'masm', 'mslib', 'dmd', 'msvs', 'midl', 'filesystem', 'jar', 'javac', 'rmic', 'tar', 'zip', 'textfile']

Build: _build-vs64-dev-w11-0001 {'MSVC_VERSION': '14.2', 'MSVC_USE_SCRIPT': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Auxiliary\\Build\\vcvars64.bat'}

scons: done reading SConscript files.
scons: Building targets ...
cl /Fo_build-vs64-dev-w11-0001\hello.obj /c src\hello.cpp /TP /MDd /EHsc /ZI /FS
Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30151 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

hello.cpp

<...REMOVED...>

link /DEBUG /OUT:_build-vs64-dev-w11-0001\hello.exe kernel32.lib user32.lib _build-vs64-dev-w11-0001\hello.obj _build-vs64-dev-w11-0001\hello01.obj _build-vs64-dev-w11-0001\hello02.obj _build-vs64-dev-w11-0001\hello03.obj _build-vs64-dev-w11-0001\hello04.obj _build-vs64-dev-w11-0001\hello05.obj _build-vs64-dev-w11-0001\hello06.obj _build-vs64-dev-w11-0001\hello07.obj _build-vs64-dev-w11-0001\hello08.obj _build-vs64-dev-w11-0001\hello09.obj _build-vs64-dev-w11-0001\hello10.obj _build-vs64-dev-w11-0001\hello11.obj _build-vs64-dev-w11-0001\hello12.obj _build-vs64-dev-w11-0001\hello13.obj _build-vs64-dev-w11-0001\hello14.obj _build-vs64-dev-w11-0001\hello15.obj _build-vs64-dev-w11-0001\hello16.obj _build-vs64-dev-w11-0001\hello17.obj _build-vs64-dev-w11-0001\hello18.obj _build-vs64-dev-w11-0001\hello19.obj _build-vs64-dev-w11-0001\hello20.obj
Microsoft (R) Incremental Linker Version 14.29.30151.0
Copyright (C) Microsoft Corporation.  All rights reserved.

scons: done building targets.

J:\SCons\test-scons-4440>

PASSES from a "normal" command-line prompt with either TEMP or TMP:

K:\SCons\test-scons-4440>set TMP=
K:\SCons\test-scons-4440>set TEMP=%SYSTEMROOT%\temp
K:\SCons\test-scons-4440>set USERPROFILE=

K:\SCons\test-scons-4440>%PYEXE% ../scons-4.0.1/scripts/scons.py
scons: Reading SConscript files ...

Computer Name: VS64-DEV-W11
SCons Root: Unknown
Installed Versions: ['14.2', '14.1', '14.0', '12.0', '11.0', '10.0', '9.0', '8.0', '7.1', '7.0', '6.0']
Default Tools: ['default', 'mslink', 'msvc', 'gfortran', 'masm', 'mslib', 'dmd', 'msvs', 'midl', 'filesystem', 'jar', 'javac', 'rmic', 'tar', 'zip', 'textfile']

Build: _build-vs64-dev-w11-0001 {'MSVC_VERSION': '14.2', 'MSVC_USE_SCRIPT': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Auxiliary\\Build\\vcvars64.bat'}

scons: done reading SConscript files.
scons: Building targets ...
cl /Fo_build-vs64-dev-w11-0001\hello.obj /c src\hello.cpp /TP /MDd /EHsc /ZI /FS
Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30151 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

hello.cpp

<...REMOVED...>

link /DEBUG /OUT:_build-vs64-dev-w11-0001\hello.exe kernel32.lib user32.lib _build-vs64-dev-w11-0001\hello.obj _build-vs64-dev-w11-0001\hello01.obj _build-vs64-dev-w11-0001\hello02.obj _build-vs64-dev-w11-0001\hello03.obj _build-vs64-dev-w11-0001\hello04.obj _build-vs64-dev-w11-0001\hello05.obj _build-vs64-dev-w11-0001\hello06.obj _build-vs64-dev-w11-0001\hello07.obj _build-vs64-dev-w11-0001\hello08.obj _build-vs64-dev-w11-0001\hello09.obj _build-vs64-dev-w11-0001\hello10.obj _build-vs64-dev-w11-0001\hello11.obj _build-vs64-dev-w11-0001\hello12.obj _build-vs64-dev-w11-0001\hello13.obj _build-vs64-dev-w11-0001\hello14.obj _build-vs64-dev-w11-0001\hello15.obj _build-vs64-dev-w11-0001\hello16.obj _build-vs64-dev-w11-0001\hello17.obj _build-vs64-dev-w11-0001\hello18.obj _build-vs64-dev-w11-0001\hello19.obj _build-vs64-dev-w11-0001\hello20.obj
Microsoft (R) Incremental Linker Version 14.29.30151.0
Copyright (C) Microsoft Corporation.  All rights reserved.

scons: done building targets.

K:\SCons\test-scons-4440>

Edits:

  • fixed environment variable setting in listing 3.
  • using embedded python 3.6

@jcbrill
Copy link
Contributor

jcbrill commented Nov 1, 2023

Try upgrading to latest SCons...

If TEMP and TMP are both undefined, the latest SCons with VS2022 will fail in the same manner.

@bdbaddog
Copy link
Contributor

bdbaddog commented Nov 1, 2023

Try upgrading to latest SCons...

If TEMP and TMP are both undefined, the latest SCons with VS2022 will fail in the same manner.

I don't see that he said those aren't defined..?
Regardless 4.0.1 is old. Always try with newest SCons before reporting a defect.

@mwichmann
Copy link
Collaborator

I don't see that he said those aren't defined..?

That was us running off and speculating on possible causes, and now testing (thanks @jcbrill for confirming which case it - likely - is) based on some other instances of similar.

@jcbrill
Copy link
Contributor

jcbrill commented Nov 1, 2023

Research notes thus far based on the issue report above, search results, msvc documentation, and command-line experiments for posterity.

There appears to be a confluence of events based on the windows environment and an msvc debug build that can result in build failures:

  • TEMP and TMP undefined.
  • msvc debug build.

This is not specific to SCons and can be reproduced in a windows command-line environment. The behavior experienced is exactly the same as described in the second and third items below.

Background information:

  • Build as administrator is required on windows (sometimes?) #4440 (comment) (emphasis added)

    ... Some users have complained to me that they can only run the build as administrator. I am not able to reproduce that problem myself. Here is the error message that the users who are affected get when they build as a non-admin.
    ...
    cl : Command line error D8050 : cannot execute ... : failed to get command line into debug records

  • https://developercommunity.visualstudio.com/t/binhostx86x86c1xxdll/314749#T-N531583 (emphasis added)

    my %tmp% is fine. Procmon trace shows that cl.exe tries to open some c:\windows{GUID} for write access -- and then fails.

  • https://developercommunity.visualstudio.com/t/binhostx86x86c1xxdll/314749#T-N531583 (emphasis added)

    In my case I was running cl.exe via SCons. Scons strips the environment! So you need to tell Scons to pass through TMP from up-level environment, or just set it to something in the sconscipt.

    To verify that this indeed is the case, create cl.bat somewhere in path with content that conststs of "set". This will print the environment variables. In my case TMP wasn't set to anything (and apparently cl.exe defaults to %systemroot% and fails with access denied -- that's why one of the solutions is to have cl run as admin)

  • https://learn.microsoft.com/en-us/cpp/error-messages/tool-errors/command-line-error-d8049?view=msvc-160 (emphasis added)

    Command-line error D8049
    ...
    To resolve this issue
    Use shorter paths for your include and build directories. Install your library include headers in directories that have shorter paths, and use short paths to your projects' intermediate and destination build directories. IF you alias paths, use short aliases. Set %TMP% and %TEMP% to directories that have shorter paths.

While there does not appear to be specific documentation for D8050, D8049 seems to imply the usage of the windows temp folders as a source for populating the debug records.

Test methodology for windows environment without SCons for failed build:

  • set TEMP=
  • set TMP=
  • <execute VS2022 vcvars64.bat>
  • <start procmon64.exe>
  • manually invoke compiler cl /c /MDd /EHsc /ZI /FS hello.cpp
  • <stop procmon64.exe>

Procmon excerpts for a failed build:

5:15:07.3381547 PM cl.exe  8208  CreateFile  C:\Windows\{9BB928B8-7056-4D5E-B84A-DCB45C12CB49}  ACCESS DENIED  Desired Access: Generic Write, Read Attributes, ...

The attempt to create a file for writing in C:\Windows\{9BB928B8-7056-4D5E-B84A-DCB45C12CB49} fails due to access restrictions for a "normal" user which results in the build failure message:

cl : Command line error D8050 : cannot execute ... : failed to get command line into debug records

Test methodology for admin windows environment without SCons for successful build:

  • set TEMP=
  • set TMP=
  • <execute VS2022 vcvars64.bat>
  • <start procmon64.exe>
  • manually invoke compiler cl /c /MDd /EHsc /ZI /FS hello.cpp
  • <stop procmon64.exe>

Procmon excerpts from a successful build:

5:16:33.1787847 PM cl.exe  7900  CreateFile  C:\Windows\{23FA6196-BD74-4313-94D3-B1C0C6920F98}  SUCCESS  Desired Access: Generic Write, Read Attributes, ..., OpenResult: Created
5:16:33.1792722 PM cl.exe  7900  WriteFile   C:\Windows\{23FA6196-BD74-4313-94D3-B1C0C6920F98}  SUCCESS  Offset: 0, Length: 852, Priority: Normal
5:16:33.1796428 PM cl.exe  7900  WriteFile   C:\Windows\{23FA6196-BD74-4313-94D3-B1C0C6920F98}  SUCCESS  Offset: 852, Length: 120
5:16:33.1797237 PM cl.exe  7900  CloseFile   C:\Windows\{23FA6196-BD74-4313-94D3-B1C0C6920F98}  SUCCESS	
...
5:16:33.2503249 PM mspdbsrv.exe  692  Process Start  SUCCESS
Parent PID: 7900,
Command line: mspdbsrv.exe -start -spawn,
Current directory: C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\HostX64\x64\,
Environment:
  ...
  ENC_CL=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\HostX64\x64\cl.exe
  ENC_CMD=3cb C:\WINDOWS\{23FA6196-BD74-4313-94D3-B1C0C6920F98}
  ENC_CWD=J:\SCons\test-scons-4440\src
  ENC_PDB=J:\SCons\test-scons-4440\src\vc140.pdb
  ENC_SRC=hello.cpp
  ...
...

The attempt to create a file for writing in C:\Windows\{23FA6196-BD74-4313-94D3-B1C0C6920F98} is successful and the file is populated.

The mspdbsrv.exe program is launched and includes environment variables for the temporary files (e.g., ENC_*).

The environment variable ENC_CMD is likely the command line that is referred to in the error message above.

This succeeds in an admin command interpreter due to the ability to read/write into the %SYSTEMROOT% folder.

@bdbaddog
Copy link
Contributor

bdbaddog commented Nov 2, 2023

Seems like the user should FIRST try upgrading to latest SCons which we know has better MSVC logic and then we can spend any time trying to debug their issue..
We don't generally provide support to versions which are not the most current.

@koubaa
Copy link
Author

koubaa commented Apr 2, 2024

@jcbrill @bdbaddog @mwichmann
Sorry for the late reply here. The original user who reported the issue had issues reproducing the issue. Another user recently reported the same issue. They have %TMP%, %TEMP%, and %USERPROFILE% set and have tried SCons 4.0.1, 4.5.3, and 4.7.0 in a virtual environment. and can still reproduce the problem with all three versions of SCons.

@mwichmann mwichmann added the MSVC Microsoft Visual C++ Support label Apr 2, 2024
@bdbaddog
Copy link
Contributor

bdbaddog commented Apr 2, 2024

@jcbrill @bdbaddog @mwichmann Sorry for the late reply here. The original user who reported the issue had issues reproducing the issue. Another user recently reported the same issue. They have %TMP%, %TEMP%, and %USERPROFILE% set and have tried SCons 4.0.1, 4.5.3, and 4.7.0 in a virtual environment. and can still reproduce the problem with all three versions of SCons.

Are they running these from powershell or cmd.exe shell?
By virtual environment, we're talking about python's venv, or virtualenv, and not something else right?
Is this being run in docker, or similar?

We have seem weird issues in the past where anti-virus could impact such things.

So the original user no longer has the issue, but another user does and it's currently reproducible?
How long has their machine been up? Can they try rebooting and see if it goes away?

Can you share the most minimal SConstruct which reproduces this?

env=Environment()
env.SharedObject('hello.c', SHCXXFLAGS="$SHCXXFLAGS /MDd /EHsc /ZI /FS")

Would that do it?

@jcbrill
Copy link
Contributor

jcbrill commented Apr 2, 2024

The installed Visual Studio version number might be helpful as well (e.g., 16.1, 16.2).

Is vs2019 updated to the latest version?

@jcbrill
Copy link
Contributor

jcbrill commented Apr 3, 2024

cl : Command line error D8050 : cannot execute 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.20.27508\bin\HostX64\x64\c1xx.dll': failed to get command line into debug records scons: building terminated because of errors.

The original error message posted above in Nov 2023 shows toolset version 14.20.27508 which is a very early 2019 toolset and possibly the very first toolset.

The latest toolset version for 2019 is 14.29.30133.

Have updates to VS2019 been applied after the initial installation?

Note that the v142 tools can be used in VS2022 by using the MSVC_TOOLSET_VERSION construction variable: Environment(MSVC_VERSION='14.3', MSVC_TOOLSET_VERSION='14.2', ...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MSVC Microsoft Visual C++ Support
Projects
None yet
Development

No branches or pull requests

4 participants