MSVC Version 14.1 not found with 14.2 installed #3664
Comments
|
Just as a "note to ourselves" - this failure isn't a surprise, as things are currently constituted. So when asking for 14.1, we use the table to ask |
|
Just going to dump some research data here. On a setup with Build Tools 19 installed, we start with the 14.26 compiler. Used the setup tool to add 14.25 and 14.16 compilers to this. The list of |
|
Okay, I have a simple enough bit of code which detects the msvc versions installed under a given product, but the structure of def _findall_vctools_files(startpath):
"""Extract versions of msvc available for a given product
Given a path to a 2017-or-later install (where multi-version was enabled),
look for Microsoft.VCToolsVersion*.txt files and extract the version
string from them (we could also parse the .props file but this seems easier)
Args:
startpath: starting path
Returns:
list: the versions found, sorted in descending numerical order.
"""
import glob
vcversions = []
for vfile in glob.glob(r'%s/**/Microsoft.VCToolsVersion*txt' % startpath, recursive=True):
with open(vfile, 'r') as f:
vcversions.append(f.readlines()[0].strip())
vcversions = sorted(set(vcversions), reverse=True) # trim any dupes
debug('_findall_vctools_files(): located %s' % str(vcversions))
return vcversions |
|
Let me know if I can help provide any information or test anything! Thanks 😃 |
|
Might be a bit... trying to think about how to set this up. |
|
Just another note to ourselves. Informational only. This appears to be a known problem that falls through the cracks between the VC++ development team and vswhere. There was an issue filed for vswhere concerning this very problem that appears to have been opened and closed on March 30, 2020. The same issue was reported to the VC++ team much earlier in October 2019. Of interest is the distinction between a "product" and a "feature":
Historical reference:
|
|
After stepping through the logic in |
|
From the vswhere issue listed above. Sugguestion that the following would work
|
|
FYI, the MSVS batch file code for processing the @REM Support the VS 2015 Visual C++ Toolset
if "%__VCVARS_VERSION%" == "14.0" (
goto :vcvars140_version
)
@REM If VCVARS_VERSION was not specified, then default initialize the environment
if "%__VCVARS_VERSION%" == "" (
goto :check_platform
)
:check_vcvars_ver_exists
@REM If we've reached this point, we've detected an override of the toolset version.
@REM Check if full version was provided and the target directory exists. If so, we can proceed to environment setup.
if EXIST "%VSINSTALLDIR%\VC\Tools\MSVC\%__VCVARS_VERSION%" (
goto :check_platform
)
@REM Check if a partial version was provided (e.g. MAJOR.MINOR only). In this case,
@REM select the first directory we find that matches that prefix.
set __VCVARS_VER_TMP=
setlocal enableDelayedExpansion
for /F %%a IN ('dir "%VSINSTALLDIR%\VC\Tools\MSVC\" /b /ad-h /o-n') DO (
set __VCVARS_DIR=%%a
set __VCVARS_DIR_REP=!__VCVARS_DIR:%__VCVARS_VERSION%=_vcvars_found!
if "!__VCVARS_DIR!" NEQ "!__VCVARS_DIR_REP!" (
set "__VCVARS_VER_TMP=!__VCVARS_DIR!"
goto :check_vcvars_ver_exists_end
)
)
:check_vcvars_ver_exists_end If a full version number (e.g., "14.NN.NNNNN") is not passed via vcvars_ver, an attempt is made to make a partial match of the user's version number and the folder names under VC\Tools\MSVC. The vc\tools\msvc directory contents are sorted by reverse name order. If the user's vcvars_ver argument matches ANY part of the folder name then it is considered to found. In Bill's example above, using "-vcvars_ver=14.1" should work just as well and will match the "largest" version of 14.1X installed (i.e., "14.1X.nnnnn"). Due to the code above, caution should be exercised when passing a version number as a mistake may very well match: "-vcvars_ver=26" will match "14.26.28801". |
|
I have not had time to test the following yet. A quick-and-dirty hack would be to add an environment setting (e.g., "MSVC_VCVARS_VER") that will be passed to the vcvarsall batch file via " In the msvc_find_valid_batch_script method in vc.py , an argument list is already being constructed. It might be possible to add the " # Get just version numbers
maj, min = msvc_version_to_maj_min(version)
# VS2015+
if maj >= 14:
if env.get('MSVC_UWP_APP') == '1':
# Initialize environment variables with store/universal paths
arg += ' store'
# VS2017+
if maj >= 14.1:
vcvars_ver = env.get('MSVC_VCVARS_VER')
if vcvars_ver:
vcvars_ver = vcvars_ver.strip()
if vcvars_ver:
arg += ' -vcvars_ver=' + vcvars_verUsage would be something along the lines of: Environment(MSVC_VERSION="14.2", MSVC_VCVARS_VER="14.1")If there is a problem with user's version string, the batch file execution should fail. While this is quick-and-dirty and may work, there may be deeper issues involving the cache. In the long run, a second variable may be necessary: A user that has both 2017 and 2019 installed, may desire to use 2017 rather than the 2019 build tools for 2017. In this case specifying a |
|
Proof-of-concept code for supporting specific msvc toolset versions is shown below and attached. Testing has been limited. An environment variable " The motivating example from above would be specified as follows for MSVC 2019: env = Environment(MSVC_VERSION="14.2", MSVC_SPECIFIC_VERSION="14.1")
env.Program("hello", "hello.cpp")A patch of the modified version of vc.py from the scons master is attached in scons-master-msvcver-patch.zip. The modified version of vc.py from the scons master is attached in scons-master-msvcver-souce.zip along with a test folder containing a minimal SConstruct file used for testing. The implementation:
New code: # Specific toolset version support for 14.1 (VS2017) and later
# Store toolset version list by (host,target) for each _VCVER version > 14
_VCVER_SPECIFIC_VERSIONS = {}
# Map MSVC host/target directory names to (host,target) specification names
_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14 = {}
# Mapping of directory names to specification names done once during initialization
__SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14 = False
def _find_all_cl_in_vc_dir(env, vc_dir, msvc_version):
"""
Find the locations of all cl.exe installed by Visual Studio/VCExpress.
For versions 2017 and later, store individual toolset versions for each
(host, target) pair discovered. Save the vcvars_ver argument as well.
"""
global __SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14
ver_num = float(get_msvc_version_numeric(msvc_version))
if ver_num > 14:
if not __SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14:
# map msvc host/target directory names to (host,target) specification names
for vc_target_pair, vc_dir_pair inHOST_TARGET_TO_CL_DIR_GREATER_THAN_14.items():
# case insensitive store and lookup
_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_dir_pair[0].lower()] = vc_target_pair[0]
_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_dir_pair[1].lower()] = vc_target_pair[1]
__SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14 = True
vc_tool_d = {}
# expected folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST\TARGET\cl.exe
vc_msvc_path = os.path.join(vc_dir, 'Tools', 'MSVC')
for vc_specific_version in os.listdir(vc_msvc_path):
vc_tool_path = os.path.join(vc_msvc_path, vc_specific_version)
if not os.path.isdir(vc_tool_path): continue
# folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX
debug('vc_specific_version: ' + vc_specific_version)
vc_bin_path = os.path.join(vc_tool_path, 'bin')
if not os.path.exists(vc_bin_path): continue
for vc_host_dir in os.listdir(vc_bin_path):
vc_host_path = os.path.join(vc_bin_path, vc_host_dir)
if not os.path.isdir(vc_host_path): continue
# folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST
debug('vc_host_dir: ' + vc_host_dir)
for vc_target_dir in os.listdir(vc_host_path):
vc_target_path = os.path.join(vc_host_path, vc_target_dir)
if not os.path.isdir(vc_target_path): continue
# folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST\TARGET
debug('vc_target_dir: ' + vc_target_dir)
cl_tool_path = os.path.join(vc_target_path, _CL_EXE_NAME)
if os.path.exists(cl_tool_path):
# map directory names to specication names
vc_host_target_name = _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_host_dir.lower()]
vc_target_target_name = _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_target_dir.lower()]
# folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST\TARGET\cl.exe
debug('found cl.exe: %s\\%s\\%s (%s, %s, %s)' % (
vc_specific_version, vc_host_dir, vc_target_dir,
vc_specific_version, vc_host_target_name, vc_target_target_name)
)
# register specific version for target specification (host,target)
key = (vc_host_target_name, vc_target_target_name)
payload = (vc_specific_version, ' -vcvars_ver=' + vc_specific_version)
vc_tool_d.setdefault(key,[]).append(payload)
# sort version number lists in descending order
for vc_target_spec in vc_tool_d.keys():
vc_tool_d[vc_target_spec] = sorted(vc_tool_d[vc_target_spec], key = lambda x: x[0], reverse=True)
# register tool versions dictionary
_VCVER_SPECIFIC_VERSIONS[msvc_version] = vc_tool_d
return None
def _find_specific_version(env, msvc_version, vc_host, vc_target):
"""
Find a specific toolset version.
Returns vcvars_ver argument string or None
"""
vc_specific_version = env.get("MSVC_SPECIFIC_VERSION", "").strip()
if not vc_specific_version:
return None
if vc_specific_version == "14.0":
# "14.0" is handled as a special case in vcvars.bat
vcvars_ver = ' -vcvars_ver=' + vc_specific_version
debug('vcvars_ver match found:%s %s' % (vc_specific_version, vcvars_ver))
return vcvars_ver
try:
vc_tool_d = _VCVER_SPECIFIC_VERSIONS[msvc_version]
except KeyError:
debug('_VCVER_SPECIFIC_VERSIONS lookup failed:%s' % msvc_version)
return None
try:
vc_tool_list = vc_tool_d[(vc_host, vc_target)]
except KeyError:
debug('vc_tool_d lookup failed:%s (%s,%s)' % (msvc_version, vc_host, vc_target))
return None
for version_t in vc_tool_list:
specific_version, vcvars_ver = version_t
if specific_version.startswith(vc_specific_version):
debug('vcvars_ver match found:%s %s' % (vc_specific_version, vcvars_ver))
return vcvars_ver
debug('vcvars_ver lookup failed:%s (%s,%s) version %s' % (msvc_version, vc_host, vc_target, vc_specific_version))
return NoneHook into # make sure the cl.exe exists meaning the tool is installed
if ver_num > 14:
# find all vc toolset versions for 2017 and later
_find_all_cl_in_vc_dir(env, vc_dir, msvc_version)
# 2017 and newer allowed multiple versions of the VC toolset to be
# installed at the same time. This changes the layout.
# Just get the default tool version for now
#TODO: support setting a specific minor VC version
default_toolset_file = os.path.join(vc_dir, _VC_TOOLS_VERSION_FILE)Hook into # Get just version numbers
maj, min = msvc_version_to_maj_min(version)
# VS2015+
if maj >= 14:
if env.get('MSVC_UWP_APP') == '1':
# Initialize environment variables with store/UWP paths
arg = (arg + ' store').lstrip()
# VS2017+
if maj >= 14 and min >= 1:
vcvars_ver = _find_specific_version(env, version, host_platform, tp)
if vcvars_ver:
arg = (arg + vcvars_ver).lstrip()scons-master-msvcver-patch.zip |
|
@jcbrill - please make a PR. large patches in issues are hard to manage/comment on/etc.. We can refine and/or at least pull your changes from your branch and continue work on them much more easily from a PR. |
|
We might want to resurrect the MSVS_VERSION var, which was deprecated (right now forced to be the same as MSVC_VERSION). |
|
Are there legacy issues with existing SConstruct/SConscript files if MSVS_VERSION is resurrected and the definition of MSVC_VERSION changes/expands? |
|
Honestly, no clue. It feels like letting MSVC_VERSION also take a more specific version string wouldn't hurt anything, here's the current claim: What turning MSVS_VERSION back to relevant and allowed to specify a required VS version would do - it might affect things. I think it might still have a meaning in the vs module, the one that is used to generate project files but not actually hunt for usable compilers. It was just a thought :) |
|
Note to self (or rather to create a github xref)... tried to capture the detection flow in a README file added by #3690 (unmerged as of this writing). Would want to update that if the above changes are added. |
|
A challenge with the current MSVC_VERSION definition is that "14.2", "14.1" and "14.1Exp" describe an msvs installation rather than an msvc toolset (14.16.27023) which can be present in all three. For a user with all three versions (2019, 2017, 2017Exp) installed, an msvc version of 14.16.27023 is not unique. If a user specifies 14.16, should SCons use the 14.1, 14.1Exp, or 14.2 toolset? My guess is that some users will want to explicitly specify the msvs installation and toolset. Perhaps when migrating from an older version of msvs to a newer version of msvs and making sure the build processes are identical. Or perhaps a client/customer only has the express version installed and a developer would like to make sure there are no build issues between the two. Still thinking about the problem... |
|
I apologize in advance as I am time limited for the rest of the day today and tomorrow morning. Notes to self while it is still fresh that a bug report that needs to be issued. During initial testing of the proposed changes above, and after installing 14.1Exp, a few issues were discovered. There are at least three problems with the msvc detection in the master and one of them is in 3.1.2:
_VCVER_TO_VSWHERE_VER = {
'14.2' : [ '-version', '[16.0, 17.0)' ],
'14.1' : [ '-version', '[15.0, 16.0)' ],
'14.1Exp' : [ '-products', 'Microsoft.VisualStudio.Product.WDExpress' ],
}
...
vswhere_cmd = [vswhere_path] + vswhere_ver + ['-property', 'installationPath']
# quick test for bin directory
cl_path = os.path.join(vc_dir, 'bin', _CL_EXE_NAME)
if os.path.exists(cl_path):
debug('get_installed_vcs ' + _CL_EXE_NAME + ' found %s' % cl_path)
return TrueThis is somewhat embarrassing: as recently as last week I thought I was doing c preprocessor tests against 6.0. Vintage 32-bit windows 7 test box (circa 2007) with the following now discovered by locally "modified" Scons: There are no boxes here with git installed nor anyone with git experience. Minor patches for the above can be provided quickly. In the short term, pull request(s) a little longer... |
|
The issue with MSVC 6.0 mentioned above is a false negative:
As described above, 14.1 Express will not work as intended in the master and the obvious fix will lead to a less-obvious problem with the vswhere arguments. The introduction of 14.1 Express also makes the original issue report a little more thorny as now there are at least three msvs installations that support the 14.1 toolset. @bdbaddog or @mwichmann Should the 14.1 Express and msvc 6 false negative issues go through the mailing list or is it OK to file an issue directly? Joe |
|
For me, the 14.1Express support was a recent addition, and so "not fully debugged" isn't that surprising. My tests were on a VM that has only express, so I wouldn't have seen the overlap problems. Seems like including fixes for that as part of the other "finding" work is fine - but will wait for @bdbaddog to weigh in. I don't have disk space to test more combinations - my native Windows box is full (and old/slow), and the drive holding the VMs is also full - including 1TB of Windows VMs that do nothing but test scons combinations :) |
|
I have attached a second prototype of specific version support. This is pre-alpha quality lacking adequate testing, documentation and error checking. It may very well only work on my computer. This prototype accepts a specific toolset version via the MSVC_VERSION variable. Internally, the specific version is converted to the existing _VCVER format for the existing code. This very well may break code outside of vc.py (e.g., test code) expecting the current definitions. Probably a better implementation is to define an internal-user variable with the specific version. An additional variable was added "MSVC_PRODUCT" to specify the installed version when there are choices. For example, when 14.1 and 14.2 are both installed, a MSVC_VERSION="14.1" will use the 14.1 installation. With MSVC_VERSION="14.1" and MSVC_PRODUCT="14.2", the "14.2" installation will use the 14.16 tools. This is a bit tricky as early on the MSVC_VERSION need to be "rewritten". This version will also use installed 14.2 for MSVC_VERSION="14.1" when there are no instances of 14.1 installed which is the point of this current issue. The desire to expand the definition of MSVC_VERSION has been satisfied. Trying to adhere to the "Law of Least Astonishement", if the product for the MSVC_VERSION is installed that will be used. It can be over-ridden via the MSVC_PRODUCT variable. Due to the nature of the implementation, there is experimental support for choosing the product installation type: Ent (Enterprise), Pro (Professional), Com (Community), Exp (Express), and BT (BuildTools). Examples are shown below. vswhere is run once for all installations and the json output is processed for fields of interest. Unfortunately, since this is done so early I don't believe a user's vswhere variable will have any effect. Three exceptions were added and raised for invalid data. As the specific version information is passed as an argument to the vcvars batch files, ignoring any errors can/will result in a successful compilation with possibly a different (e.g., default) toolset than the user specification. For now, the exceptions have been useful in identifying corner cases. Support for 14.0 with the later tools is precarious at best. The Express version of 14.1 makes the problem more annoyingly difficult. scons-master-msvcver-source-2.zip Example test configurations: env = Environment() # 14.2 Community
# 14.2 Installed [Community]
#env = Environment(MSVC_VERSION="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.26") # 14.2 Community
#env = Environment(MSVC_VERSION="14.26.28801") # 14.2 Community
#env = Environment(MSVC_VERSION="14.25") # 14.2 Community
#env = Environment(MSVC_VERSION="14.25.28610") # 14.2 Community
#env = Environment(MSVC_VERSION="14.20") # 14.2 Community
#env = Environment(MSVC_VERSION="14.20.27508") # 14.2 Community
#env = Environment(MSVC_VERSION="14.27") # 14.2 Community: MSVCProductNotFound exception
# 14.1 [Fake] Not Installed [Community or Express]
#env = Environment(MSVC_VERSION="14.1") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27023") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27024") # 14.2 Community: MSVCProductNotFound exception
# 14.1 Installed [Community, Express]
#env = Environment(MSVC_VERSION="14.1") # 14.1 Community [Ent->Pro->Com->Exp->BT]
#env = Environment(MSVC_VERSION="14.16") # 14.1 Community
#env = Environment(MSVC_VERSION="14.16.27") # 14.1 Community
#env = Environment(MSVC_VERSION="14.16.27023") # 14.1 Community
#env = Environment(MSVC_VERSION="14.15") # 14.1 Community
#env = Environment(MSVC_VERSION="14.15.26726") # 14.1 Community
#env = Environment(MSVC_VERSION="14.14") # 14.1 Community
#env = Environment(MSVC_VERSION="14.14.26428") # 14.1 Community
#env = Environment(MSVC_VERSION="14.16.27024") # 14.1 Community: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1") # 14.1 Community
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Exp") # 14.1 Express
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Com") # 14.1 Community
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Ent") # 14.1 Enterprise: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Pro") # 14.1 Professional: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1BT") # 14.1 BuildTools: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2Com") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16", MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27", MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27023", MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27024", MSVC_PRODUCT="14.2") # 14.2 Community: MSVCProductNotFound exception
# 14.1 Express
#env = Environment(MSVC_VERSION="14.1Exp") # 14.1 Express
#env = Environment(MSVC_VERSION="14.16Exp") # 14.1 Express
#env = Environment(MSVC_VERSION="14.16.27Exp") # 14.1 Express
#env = Environment(MSVC_VERSION="14.16.27023Exp") # 14.1 Express
#env = Environment(MSVC_VERSION="14.16.27024Exp") # 14.1 Express: MSVCProductNotFound exception
# 14.0 [Fake] Not Installed
#env = Environment(MSVC_VERSION="14.0") # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2") # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Com") # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1") # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Com") # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Exp") # 14.1 Express -> 14.0
# 14.0 Installed
#env = Environment(MSVC_VERSION="14.0") # 14.0 Community
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.0") # 14.0 Community
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.0Exp") # 14.0 Community: MSVCProductInvalid [conflict]
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1") # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Com") # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Exp") # 14.1 Express -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2") # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Com") # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Ent") # 14.1 Enterprise: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Pro") # 14.1 Professional: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1BT") # 14.1 BuildTools: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Ent") # 14.2 Enterprise: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Pro") # 14.2 Professional: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools: MSVCProductNotFound exception |
|
Once the 2019 build tools were actually installed, a problem with identifying "synthetic" versions (e.g., Enterprise, Professional, Community, BuildTools) when there are multiple versions installed was revealed. The example configurations below actually address the problem in the original post. The MSVC_PRODUCT="14.2BT" may not be necessary if there is only one instance of 14.2. This has not been tested yet. Will move the source to git soon ... scons-master-msvcver-source-3.zip Example test configurations: env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.2", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.26", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.26.28801", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.25", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.25.28610", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.16", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.16.27023", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools -> 14.0 |
|
I have a "working" prototype version of vc.py that unifies both #3265 and #3664. The prototype is not production ready but does illustrate that both #3265 and #3664 can be addressed with the same implementation. There is a great deal of new code that needs to be fully tested, documented, and modified based on feedback. The ramifications to code outside of vc.py are unknown. If desired, A PR can be issued to move this discussion. The reality is that integration is likely a ways off in the future. #3265 is an enhancement to allow a specific version of an msvc toolset. In #3265, there was a suggestion to allow specifying a specific version via the existing environment variable MSVC_VERSION. #3664 (this issue) involves detecting a toolset that is installed in a later product version (i.e., the 14.16 toolset via the 14.2 product line). In this case, the 14.1 toolset via the 14.2 Build Tools. The prototype implementation is "wired-in" to the existing code base in a few select locations. The existing code base continues to operate on an MSVC_VERSION that is compatible with the _VCVER version list (e.g., "14.2"). A single internal environment variable and dictionary was added to maintain the specific version and product state. There is a single hard-coded variable at the top of the source file that selectively enables/disables the specific version support. When disabled, the functionality should be exactly the same as the current master. The hooks into the existing code were intentionally minimized to ease updating from the master. The bulk of the new code was added at the bottom of the file. The MSVC_VERSION format was expanded to allow:
Without a product qualifier (e.g., "BT"), there is a subtle but significant change semantically: the product selected will be the newest/latest toolset that matches the user specification within installed toolsets for a given (host,target) combination. Effectively, without a product qualifier, any product type could be returned based on the installed toolsets. This only affects multiple installations of the same product version (e.g., 14.1 and 14.Exp and/or 14.2Com and 14.2BT). This means that a user does not need to specify "14.1Exp" explicitly if there is a single 14.1 product installation. On the flip side, it also means that "14.1Exp" may be returned for a "14.1" request. In local testing, "14.1Exp" was returned for an "arm" target. In the local installations of 14.1 Community and 14.1 Express, the "newest" toolset for an arm target was in the 14.1 Express installation as it was installed recently. With a product qualifier, a specific product version and type is used. The motivating example for the current issue, is that no 14.1 products are installed and the 14.1 toolset is not found within the 14.2 Build Tools installation. In the prototype implementation, "MSVC_VERSION=14.1" will return an installed version of 14.1 if it exists or an installed version of 14.2 if the 14.16 toolset is installed for the host/target combination. There are two attachments:
The internal version tables (all installed and by version), internal toolset dictionary state at four locations, and select SConstruct environment variables are displayed. I realize everyone is busy trying get 4.0 out the door... |
|
Yes. Please push a PR mark it "[WIP]" in the title.
…On Wed, Jun 24, 2020 at 10:49 AM Joseph Brill ***@***.***> wrote:
I have a "working" prototype version of *vc.py* that unifies both #3265
<#3265> and #3664
<#3664>.
The prototype is not production ready but does illustrate that both #3265
<#3265> and #3664
<#3664> can be addressed with the
same implementation. There is a great deal of new code that needs to be
fully tested, documented, and modified based on feedback. The ramifications
to code outside of *vc.py* are unknown.
*If desired, A PR can be issued to move this discussion. The reality is
that integration is likely a ways off in the future.*
#3265 <#3265> is an enhancement to
allow a specific version of an msvc toolset. In #3265
<#3265>, there was a suggestion to
allow specifying a specific version via the existing environment variable
MSVC_VERSION.
#3664 <#3664> (this issue) involves
detecting a toolset that is installed in a later product version (i.e., the
14.16 toolset via the 14.2 product line). In this case, the 14.1 toolset
via the 14.2 Build Tools.
The prototype implementation is "wired-in" to the existing code base in a
few select locations. The existing code base continues to operate on an
MSVC_VERSION that is compatible with the _VCVER version list (e.g.,
"14.2"). A single internal environment variable and dictionary was added to
maintain the specific version and product state.
There is a single hard-coded variable at the top of the source file that
selectively enables/disables the specific version support. When disabled,
the functionality should be exactly the same as the current master. The
hooks into the existing code were intentionally minimized to ease updating
from the master. The bulk of the new code was added at the bottom of the
file.
The MSVC_VERSION format was expanded to allow:
- specifying an individual toolset at varying degrees of precision:
- "14.1"
- "14.16"
- "14.16.27"
- "14.16.27023"
- specifying individual product types (attached to a specific toolset
*or* a product):
- Enterprise: "14.26Ent"
- Professional: "14.2Pro"
- Community: "14.26.27Com"
- Build Tools: "14.2BT"
- Express: "14.1Exp"
- a right assignment operator ("->") that maps a specific toolset to a
defined product version with an optional product type:
- "14.0->14.2" use the the 14.0 toolset via a 2019 installation
- "14.1->14.2BT" use the 14.1 toolset via a 2019 Build Tools
installation
- "14.1->14.2Com" use the 14.1 toolset via a 2019 Community
installation
Without a product qualifier (e.g., "BT"), there is a *subtle but
significant* change semantically: the product selected will be the
newest/latest toolset that matches the user specification within installed
toolsets for a given (host,target) combination. Effectively, without a
product qualifier, any product type could be returned based on the
installed toolsets. This only affects multiple installations of the same
product version (e.g., 14.1 and 14.Exp and/or 14.2Com and 14.BT).
This means that a user does not need to specify "14.1Exp" explicitly if
there is a single 14.1 product installation. On the flip side, it also
means that "14.1Exp" may be returned for a "14.1" request. In local
testing, "14.1Exp" was returned for an "arm" target. In the local
installations of 14.1 Community and 14.1 Express, the "newest" toolset for
an arm target was in the 14.1 Express installation as it was installed
recently.
With a product qualifier, a specific product version and type is used.
The motivating example for the current issue, is that no 14.1 products are
installed and the 14.1 toolset is not found within the 14.2 Build Tools
installation. In the prototype implementation, "MSVC_VERSION=14.1" will
return an installed version of 14.1 if it exists or an installed version of
14.2 if the 14.16 toolset is installed for the host/target combination.
There are two attachments:
1. msvc-specific-142.txt contains edited testing output for the
following SConstruct specification:
# 14.1 is not installed, 14.2 Build Tools is installed
env = Environment(MSVC_VERSION="14.1")
2. msvc-specific-141-142.txt contains edited testing output for the
following SConstruct specifications:
# 14.1 Installed [Community, Express], 14.2 Installed [Community, BuildTools]
env1 = Environment(MSVC_VERSION="14.1")
env2 = Environment(MSVC_VERSION="14.16.27Exp") # 14.16.27->14.1Exp
env3 = Environment(MSVC_VERSION="14.1->14.2")
env4 = Environment(MSVC_VERSION="14.1->14.2BT")
The internal version tables (all installed and by version), internal
toolset dictionary state at four locations, and select SConstruct
environment variables are displayed.
I realize everyone is busy trying get 4.0 out the door...
msvc-specific-142.txt
<https://github.com/SCons/scons/files/4827072/msvc-specific-142.txt>
msvc-specific-141-142.txt
<https://github.com/SCons/scons/files/4827073/msvc-specific-141-142.txt>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3664 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAUCNB5IL26H5KMOZWX3ZTRYI4DFANCNFSM4NHLQUDA>
.
|
|
For posterity. It has been deemed that this issue is not a bug and is the preferred behavior based on the response in #3717 (comment):
Curiously, the behavior deemed unacceptable is this reason for this issue and was laid out very clearly above prior to being requested in a PR.
Which is exactly the "bug" in the current issue. Despite the fact that MSVC 2017 and MSVC 2019 are toolset oriented (i.e., a MSVC 2019 installation is a container for 14.1X, 14.2X, and to a lesser extent 14.0 toolsets), the desire is that the MSVC detection remain product oriented. Prior to MSVS 2017, there was a 1:1 mapping between the msvc product and the msvc toolset. The toolset might be upgraded in an update or service pack but there was no way to select a specific toolset as side-by-side installations were not supported. With MSVC 2017 and MSVC 2019, an SCons request of "14.1" or "14.2" is a toolset request. The current msvc behavior uses the default toolset which is generally the newest installed toolset version when the "--vcvars_ver" argument is not specified for the vcvars batch file. For example, "14.1" (MSVC 2017) is likely to return "14.16.27023". Effectively, the product request is a toolset request similar to "find the newest toolset version that startswith('14.1')". However, "14.2" (MSVC 2019) will return a toolset based on the last update which may vary across users and computers. The actual toolset version used for 2017 and 2019 is based on end-user's installation options and frequency of updates. While two different users may be using the same product (e.g., 2019) they may not be using the same toolset. Similarly, the installed toolset version is not necessary the same for all host/target combinations in a given installation. The current implementation will use a default toolset within a product across different users and environments, which may not be the same, but will not use a toolset across products. For whatever reason (e.g., enterprise and/or customer requirements), if a build depends on the 14.1X toolset it hardly seems as important in which product container it is installed (given a simple selection rule). The same issue will arise when the next MSVC product is released and end-users desire to use the 14.2X toolset combination. There are two schools of thought:
A fundamental shift from a product-centric view to a toolset-centric view for the msvc tools would go a long way in supporting the msvc tools in the future and reduce the burden on an end-user. The point of this particular issue is that a toolset is desired and not a necessarily product/toolset.
If the selected product for the requested toolset is supported by SCons, this principle should hold. The binary tools may be different. The binary tools are likely already different within the same product across users and computers. It does not seem to be any different than the same build scripts using an installed version of mingw gcc 9.3 on one computer and using an installed version of mingw gcc 8.3 on another computer simply due to a lag in updates. While this is certainly a noble tenet, in practice I'm not sure this always holds across two or more users in a windows environment or even across two or more windows boxes. For example, the addition of cygwin to the default tools path caused some local builds to fail. The solution was to explicitly specify the tools used thus bypassing the effect of cygwin being added to the path. However, if the same build scripts were developed on a box without cygwin installed and sent to a customer who had cygwin installed the build would have failed. |
|
@jcbrill - No it has not been ruled that this is not a bug. Please stop overreacting. |
|
@bdbaddog I apologize if you believe that I'm overreacting. It is impossible to tell how much thought you may have put into supporting this issue and specific versions in general from one or two line comments here and there. I am merely trying to document some hard earned insight for your benefit in the future and for others who may attempt a solution. |
|
@bryanwweber With the recent merge of #4174, there are now two methods in the main branch which allow an installed toolset to be used for a build. However, there are limitations... At present, there are two pieces of information required in order to use a specific toolset version: the msvc product (i.e., The msvc toolset version is not used during selection. If The internal ms readme file ( In particular, the default msvc toolset version section as discussed above may be relevant: Examples: With an exception raised when the msvc toolset version is not found, it is possible to "query" by msvc version and msvc toolset version in user-space for a desired toolset. Since this appears useful from an end-user standpoint, an experimental function The docstring is: This function takes a version, either an msvc version or an msvc toolset version, and returns the associated msvc version and msvc toolset version. By default, it prefers newer versions of Visual Studio. Examples: Any and all feedback, positive and negative, is appreciated. I am happy to field any questions. |
|
Thanks @jcbrill I'll try it out on our GitHub Actions runs and see how it goes! |
Any updates on this and/or using 4.4.0? |
|
I've not been able to get this to work on GH hosted runners. Unfortunately, I don't have a local Windows machine for testing. For reference, I'm working in this PR: Cantera/cantera#1392, I'm using SCons 4.4.0, and the software on the
The specific changes I'm making are in the Environment(MSVC_VERSION="14.3")but SCons couldn't find that version of MSVC (makes sense, 2019 is installed). I also tried Environment(MSVC_VERSION="14.2", MSVC_TOOLSET_VERSION="14.3")but SCons complained that the toolset version cannot be greater than the * I'm using |
|
The following was done rather quickly, I apologize in advance for any erroneous information. Unless I'm missing something, the v143 (14.3x) toolsets are not available in a 2019 installation. The table on the windows-2019 image page seems to indicate that the v142, v141, and v140 toolsets are installed in VS2019. It looks like the 14.3X runtimes are installed as well. This is consistent with the SCons message in the first log: Based on the installation table for the windows-2019 image and the error message above, the message appears to be consistent/correct. VS2019 ('14.2') is detected and VS2015 ('14.0') is detected due to the v140 build tools being installed from VS2019 which is found independently via the registry. If the above is correct, then neither The following should work on the windows-2019 image for VS2019;
Based on the table linked above, the windows-2022 image for VS2022 appears to contain the v143, v142, and v141 build tools. The following should work on the windows-2022 image for VS2022:
If possible, try I hope this helps. |
|
Thanks @jcbrill I totally misread that table in the GitHub docs. I set it up the way you suggested on both their windows-2019 and windows-2022 images, and things seem to be working exactly the way I'd expect! https://github.com/Cantera/cantera/actions/runs/3094836889 This is great news for SCons and I think this issue is resolved as far as the initial report is concerned. I'll let you or @bdbaddog decide whether there's still more action items from this issue and whether those are being tracked here or elsewhere, to actually close this issue. Thanks again for the help! |
|
May be unrelated, but my setup is still failing the |
@mwichmann I will start investigating (time limited today though). |
|
@mwichmann The Any chance there was a msvc/toolset update and the cache is out-of-date? |
|
of course... but weren't we somewhat immune to that now? |
|
still fails if I move aside the old cache |
Yes, but it is not foolproof. The "VCToolsInstallDir" folder is checked for existence (if the list is not empty). It may be possible that the folder exists but is empty. Unlikely, but possible.
groan. Could you try running just the MSVC_USE_SETTINGS test from the scons root at your convenience? Set debug log, cache file, and run problematic test: and post the debug log and cache file: |
|
@mwichmann Is it possible there is a compiler executable ('cl.exe') found on the windows system path? One of the tests in MSVC_USE_SETTINGS passes an empty dictionary and expects a warning to be issued. As we discussed in another thread, the If 'cl.exe' is not in the generated environment but IS on the system path, the warning will not be issued and the test would fail as the expected warning text is not produced. |
|
yeah we've been talking about the mismatch between windows path and scons path elsewhere too - I'm having trouble with the D compilers on Windows, as well as Intel Fortran, because of that. |
|
I just tried a test by running the windows batch file from the command prompt and then running the single test: |
|
Here are the two files... the cache I had to call .txt instead because of github's silly limitations. And yes, that looks like the same error. |
|
I think a cl.exe was found on the system path. The log contains the following: If no cl.exe is found the log should look like: Which triggers the warning expected by the test. In SCons\Tool\MSCommon\vc.py (starting at line 1530), try changing the search function from: to: The test should pass. |
|
Prior discussion concerning cl.exe found on system path but not in environment starting here: #4312 (comment) |
yes - I just looked over at the screen and it seems I opened a "Developer Command Prompt for VS 2022" there, rather than my normal bare cmd prompt. The test passes if I do the latter. |
|
... needless to say, that shouldn't affect the test pass/fail status... Seems like msvc tool does plenty of work now to find legitimate copies in the way we want, and we should probably stop doing what we're doing with that find_program_path call. |
I agree. I'm not convinced that It is possible the user modified the passed in environment (e.g., The check should probably use only the generated dictionary PATH elements rather than the resultant My $0.02 cents. Edit: add literal text for readability. |
|
how do we solve this now? |
|
There was a lot of discussion here, which makes for some messy reading, but the original poster did comment that things seemed to be working eventually - see #3664 (comment) If you're having problems finding the compiler, it would be useful to get a few more details... |
Thanks! |
|
Some additional examples of using an older toolset are also here: #4412 (comment) |
Describe the bug
I have VS2019 installed, with the VC++ 14.2, 14.1, and 14.0 build tools selected in the installer. When
MSVC_VERSION="14.1"is specified, SCons cannot find the 14.1 build tools. WhenMSVC_VERSION="14.2"or14.0are specified, SCons works as expected.Required information
sconsFurther information:
MSVC_VERSION="14.2", SCons generates this command forvswhere.exe:vswhere.exe -products * -version "[16.0, 17.0)" -property installationPathand the output when run on the command line is:C:\Program Files (x86)\Microsoft Visual Studio\2019\EnterpriseMSVC_VERSION="14.1", SCons generates this command:vswhere.exe -products * -version "[15.0, 16.0)" -property installationPathand there is no output when run on the command lineC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.16.27023vswhere --products * -format jsonis:The text was updated successfully, but these errors were encountered: