Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WinError 1 Incorrect function when creating env on shared drive #1706

Closed
jaraco opened this issue Oct 20, 2020 · 10 comments
Closed

WinError 1 Incorrect function when creating env on shared drive #1706

jaraco opened this issue Oct 20, 2020 · 10 comments
Labels
bug:normal affects many people or has quite an impact bug:upstream something does not behave as it should, but can't or shouldn't be fixed in tox, but in a dependency

Comments

@jaraco
Copy link

jaraco commented Oct 20, 2020

I'm attempting to build a project on my Windows VM mounted against the source on my macOS host, but when I do, the env fails to be created:

importlib_resources bugfix/bpo-41490 # tox -e py35-win                                                                                                                    
py35-win create: \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win                                                                          
ERROR: invocation failed (exit code 1), logfile: \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\log\py35-win-0.log                       
=============================================================================== log start =============================================================================== 
OSError: [WinError 1] Incorrect function: 'C:\\Python35\\python3.dll' -> '\\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35-win\\Script
s\\python3.dll'                                                                                                                                                           
                                                                                                                                                                          
================================================================================ log end ================================================================================ 
ERROR: InvocationError for command 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --python 'C:\Python35\python.exe' py35-win (exit
ed with code 1)                                                                                                                                                           
________________________________________________________________________________ summary ________________________________________________________________________________ 
ERROR:   py35-win: InvocationError for command 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --python 'C:\Python35\python.exe' py
35-win (exited with code 1)                                                                                                                                               

I am, however, able to create the venv manually.

importlib_resources bugfix/bpo-41490 # rm -rf .tox/py35-win                                                                                                               
importlib_resources bugfix/bpo-41490 # ~/.local/pipx/venvs/tox/Scripts/python -m virtualenv .tox/py35-win                                                                 
created virtual environment CPython3.9.0.final.0-64 in 19278ms                                                                                                            
  creator CPython3Windows(dest=\\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win, clear=False, global=False)                                
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=C:\Users\jaraco\AppData\Local\pypa\virtualenv)                   
    added seed packages: pip==20.2.3, setuptools==50.3.1, wheel==0.35.1                                                                                                   
  activators BashActivator,BatchActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator                                                                
importlib_resources bugfix/bpo-41490 # .tox/py35-win/Scripts/python -c "print('hello world')"                                                                             
hello world                                                                                                                                                               

It seems something about how tox is attempting to create the virtualenv is triggering the failure.

Verbose:

importlib_resources bugfix/bpo-41490 # tox -rvv -e py35-win                                                                                                               
using tox.ini: C:\Users\jaraco\p\importlib_resources\tox.ini (pid 3744)                                                                                                   
using tox-3.20.1 from c:\users\jaraco\.local\pipx\venvs\tox\lib\site-packages\tox\__init__.py (pid 3744)                                                                  
skipping sdist step                                                                                                                                                       
PEP-514 violation in Windows Registry at HKEY_CURRENT_USER/PythonCore/3.9/InstallPath error: missing                                                                      
PEP-514 violation in Windows Registry at HKEY_LOCAL_MACHINE/PythonCore/2.7/InstallPath error: missing                                                                     
PEP-514 violation in Windows Registry at HKEY_LOCAL_MACHINE/PythonCore/3.4/InstallPath error: missing                                                                     
PEP-514 violation in Windows Registry at HKEY_LOCAL_MACHINE/PythonCore/3.6/InstallPath error: missing                                                                     
py35-win uses C:\Python35\python.exe                                                                                                                                      
py35-win start: getenv C:\Users\jaraco\p\importlib_resources\.tox\py35-win                                                                                                
py35-win cannot reuse: -r flag                                                                                                                                            
py35-win create: C:\Users\jaraco\p\importlib_resources\.tox\py35-win                                                                                                      
setting PATH=C:\Users\jaraco\p\importlib_resources\.tox\py35-win\Scripts;C:\Python35\Scripts\;C:\Python35\;C:\Python39\Scripts\;C:\Python39\;C:\Program Files\Python39\Scr
ipts\;C:\Program Files\Python39\;C:\Python38\Scripts\;C:\Python38\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\
ProgramData\chocolatey\bin;C:\Program Files\dotnet\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Users\jaraco\AppData\Local\Microsoft\WindowsApps;C:\Users\jar
aco\AppData\Local\hyper\app-3.0.2\resources\bin;C:\Users\jaraco\.local\bin;C:\Users\jaraco\AppData\Roaming\Python\Python39\Scripts;C:\Users\jaraco\.local\bin             
[7324] C:\Users\jaraco\p\importlib_resources\.tox$ 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --python 'C:\Python35\python.exe
' py35-win                                                                                                                                                                
OSError: [WinError 1] Incorrect function: 'C:\\Python35\\python3.dll' -> '\\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35-win\\Script
s\\python3.dll'                                                                                                                                                           
ERROR: invocation failed (exit code 1)                                                                                                                                    
ERROR: InvocationError for command 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --python 'C:\Python35\python.exe' py35-win (exit
ed with code 1)                                                                                                                                                           
py35-win finish: getenv C:\Users\jaraco\p\importlib_resources\.tox\py35-win after 0.53 seconds                                                                            
________________________________________________________________________________ summary ________________________________________________________________________________ 
ERROR:   py35-win: InvocationError for command 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --python 'C:\Python35\python.exe' py
35-win (exited with code 1)                  
@jaraco jaraco added the bug:normal affects many people or has quite an impact label Oct 20, 2020
@jayvdb
Copy link

jayvdb commented Oct 21, 2020

@jaraco , can you try using virtualenv with the UNC path to create the venv, e.g. python -m virtualenv \\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35-win

If it fails, then virtualenv has a bug, but tox could also use a relative path instead of the absolute UNC path. This might be hard to solve in tox v3 because tox uses the deprecated py for paths, but tox v4 will be using pathlib.

If virtualenv works well with UNC paths, tox could be giving it the wrong UNC path, perhaps with wrong escaping.

@gaborbernat
Copy link
Member

Can you modify tox.ini to add VIRTUALENV_* to passenv? Then set VIRTUALENV_VERBOSE=5 env var and rerun it, come back with the trace. Thanks!

@jaraco
Copy link
Author

jaraco commented Oct 21, 2020

Sure. BTW, it's not necessary to modify tox.ini, but just also set TOX_PASSENV:

importlib_resources bugfix/bpo-41490 # $TOX_PASSENV='VIRTUALENV_*'
importlib_resources bugfix/bpo-41490 # $VIRTUALENV_VERBOSE=5importlib_resources bugfix/bpo-41490 # tox -rvv -e py35-win
using tox.ini: C:\Users\jaraco\p\importlib_resources\tox.ini (pid 1664)  removing \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\log
using tox-3.20.1 from c:\users\jaraco\.local\pipx\venvs\tox\lib\site-packages\tox\__init__.py (pid 1664)skipping sdist step
PEP-514 violation in Windows Registry at HKEY_CURRENT_USER/PythonCore/3.9/InstallPath error: missing
PEP-514 violation in Windows Registry at HKEY_LOCAL_MACHINE/PythonCore/2.7/InstallPath error: missing
PEP-514 violation in Windows Registry at HKEY_LOCAL_MACHINE/PythonCore/3.4/InstallPath error: missingPEP-514 violation in Windows Registry at HKEY_LOCAL_MACHINE/PythonCore/3.6/InstallPath error: missing
py35-win uses C:\Python35\python.exe
py35-win start: getenv \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win
py35-win cannot reuse: -r flagpy35-win create: \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win  removing \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win
setting PATH=\\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts;C:\Python35\Scripts\;C:\Python35\;C:\Python39\Scripts\;C:\Python39\;C
:\Program Files\Python39\Scripts\;C:\Program Files\Python39\;C:\Python38\Scripts\;C:\Python38\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32
\WindowsPowerShell\v1.0\;C:\ProgramData\chocolatey\bin;C:\Program Files\dotnet\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Users\jaraco\AppData\Local\Micros
oft\WindowsApps;C:\Users\jaraco\AppData\Local\hyper\app-3.0.2\resources\bin;C:\Users\jaraco\.local\bin;C:\Users\jaraco\AppData\Roaming\Python\Python39\Scripts;C:\Users\ja
raco\.local\bin
[10228] \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox$ 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --p
ython 'C:\Python35\python.exe' py35-win
266 setup logging to NOTSET [DEBUG report:43]
281 find interpreter for spec PythonSpec(path=C:\Python35\python.exe) [INFO builtin:44]
281 Attempting to acquire lock 2423235360368 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\9d5054abfeecd3e752231657b3ba9af93d4764770588e36894323405d4384571.l
ock [DEBUG filelock:270]
281 Lock 2423235360368 acquired on C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\9d5054abfeecd3e752231657b3ba9af93d4764770588e36894323405d4384571.lock [INFO fil
elock:274]
323 got python info of C:\Python35\python.exe from C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\9d5054abfeecd3e752231657b3ba9af93d4764770588e36894323405d438457
1.json [DEBUG via_disk_folder:133]
324 Attempting to release lock 2423235360368 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\9d5054abfeecd3e752231657b3ba9af93d4764770588e36894323405d4384571.l
ock [DEBUG filelock:315]
324 Lock 2423235360368 released on C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\9d5054abfeecd3e752231657b3ba9af93d4764770588e36894323405d4384571.lock [INFO fil
elock:318]
325 proposed PythonInfo(spec=CPython3.5.4.final.0-64, exe=C:\Python35\python.exe, platform=win32, version='3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bi
t (AMD64)]', encoding_fs_io=mbcs-cp1252) [INFO builtin:50]
325 accepted PythonInfo(spec=CPython3.5.4.final.0-64, exe=C:\Python35\python.exe, platform=win32, version='3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bi
t (AMD64)]', encoding_fs_io=mbcs-cp1252) [DEBUG builtin:52]
329 symlink on filesystem does work [DEBUG info:47]
331 filesystem is not case-sensitive [DEBUG info:28]
1133 create virtual environment via CPython3Windows(dest=\\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win, clear=False, global=False) [INFO
 session:52]
1148 create folder \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages [DEBUG _sync:25]
1148 create folder \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts [DEBUG _sync:25]
1148 write \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\pyvenv.cfg [DEBUG pyenv_cfg:34]
1148    home = C:\Python35 [DEBUG pyenv_cfg:38]
1148    implementation = CPython [DEBUG pyenv_cfg:38]
1148    version_info = 3.5.4.final.0 [DEBUG pyenv_cfg:38]
1148    virtualenv = 20.0.35 [DEBUG pyenv_cfg:38]                                                                                                                         
1148    include-system-site-packages = false [DEBUG pyenv_cfg:38]                                                                                                         
1148    base-prefix = C:\Python35 [DEBUG pyenv_cfg:38]                                                                                                                    
1148    base-exec-prefix = C:\Python35 [DEBUG pyenv_cfg:38]                                                                                                               
1148    base-executable = C:\Python35\python.exe [DEBUG pyenv_cfg:38]                                                                                                     
1148 copy C:\Python35\python.exe to \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts\python.exe [DEBUG _sync:52]                   
1181 copy C:\Python35\pythonw.exe to \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts\pythonw.exe [DEBUG _sync:52]                 
1199 symlink C:\Python35\python3.dll to \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts\python3.dll [DEBUG _sync:44]              
1241 OSError: [WinError 1] Incorrect function: 'C:\\Python35\\python3.dll' -> '\\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35-win\\S
cripts\\python3.dll' [ERROR __main__:69]                                                                                                                                  
ERROR: invocation failed (exit code 1)                                                                                                                                    
ERROR: InvocationError for command 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --python 'C:\Python35\python.exe' py35-win (exit
ed with code 1)                                                                                                                                                           
py35-win finish: getenv \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win after 1.53 seconds                                                
________________________________________________________________________________ summary ________________________________________________________________________________ 
ERROR:   py35-win: InvocationError for command 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --python 'C:\Python35\python.exe' py
35-win (exited with code 1)                                                                                 

@jaraco
Copy link
Author

jaraco commented Oct 21, 2020

Indeed, if I try to create a symlink on the local (mounted) filesystem, it fails with the sam eerror:

importlib_resources bugfix/bpo-41490 # os.symlink('C:\\Python35\\python3.dll', 'test.dll')                                                                                
OSError: [WinError 1] Incorrect function: 'C:\\Python35\\python3.dll' -> 'test.dll'                                                                                       

@jaraco
Copy link
Author

jaraco commented Oct 21, 2020

Probably, because this file system is a SMB-mounted AFS file system, it can't store a Windows symlink, which relies on NTFS.

Yet, we see:

329 symlink on filesystem does work [DEBUG info:47]

So it seems that the symlink detection may not be effectively checking that symlinks work on the filesystem where the symlink will be created.

@jaraco
Copy link
Author

jaraco commented Oct 21, 2020

When I am able to create the virtualenv by invoking virtualenv manually, virtualenv never attempts to link the dll:

importlib_resources bugfix/bpo-41490 # ~/.local/pipx/venvs/tox/Scripts/python -m virtualenv -vvvvv .tox/py35-win                                                          
203 setup logging to NOTSET [DEBUG report:43]                                                                                                                             
215 find interpreter for spec PythonSpec(path=C:\Users\jaraco\.local\pipx\venvs\tox\Scripts\python.exe) [INFO builtin:44]                                                 
215 Attempting to acquire lock 2003035829824 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\fe8c250ecc65b9c312d9907a34fb25acb60718286033c0aea00268d1f05cce17.l
ock [DEBUG filelock:270]                                                                                                                                                  
215 Lock 2003035829824 acquired on C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\fe8c250ecc65b9c312d9907a34fb25acb60718286033c0aea00268d1f05cce17.lock [INFO fil
elock:274]                                                                                                                                                                
255 got python info of c:\python39\python.exe from C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\fe8c250ecc65b9c312d9907a34fb25acb60718286033c0aea00268d1f05cce1
7.json [DEBUG via_disk_folder:133]                                                                                                                                        
256 Attempting to release lock 2003035829824 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\fe8c250ecc65b9c312d9907a34fb25acb60718286033c0aea00268d1f05cce17.l
ock [DEBUG filelock:315]                                                                                                                                                  
256 Lock 2003035829824 released on C:\Users\jaraco\AppData\Local\pypa\virtualenv\py_info\1\fe8c250ecc65b9c312d9907a34fb25acb60718286033c0aea00268d1f05cce17.lock [INFO fil
elock:318]                                                                                                                                                                
257 proposed PythonInfo(spec=CPython3.9.0.final.0-64, system=c:\python39\python.exe, exe=C:\Users\jaraco\.local\pipx\venvs\tox\Scripts\python.exe, platform=win32, version
='3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]', encoding_fs_io=utf-8-cp1252) [INFO builtin:50]                                         
257 accepted PythonInfo(spec=CPython3.9.0.final.0-64, system=c:\python39\python.exe, exe=C:\Users\jaraco\.local\pipx\venvs\tox\Scripts\python.exe, platform=win32, version
='3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]', encoding_fs_io=utf-8-cp1252) [DEBUG builtin:52]                                        
261 symlink on filesystem does work [DEBUG info:47]                                                                                                                       
264 filesystem is not case-sensitive [DEBUG info:28]                                                                                                                      
557 create virtual environment via CPython3Windows(dest=\\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win, clear=False, global=False) [INFO 
session:52]                                                                                                                                                               
557 write \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\pyvenv.cfg [DEBUG pyenv_cfg:34]                                                 
557     home = c:\python39 [DEBUG pyenv_cfg:38]                                                                                                                           
557     implementation = CPython [DEBUG pyenv_cfg:38]                                                                                                                     
557     version_info = 3.9.0.final.0 [DEBUG pyenv_cfg:38]                                                                                                                 
557     virtualenv = 20.0.35 [DEBUG pyenv_cfg:38]                                                                                                                         
557     include-system-site-packages = false [DEBUG pyenv_cfg:38]                                                                                                         
557     base-prefix = c:\python39 [DEBUG pyenv_cfg:38]                                                                                                                    
557     base-exec-prefix = c:\python39 [DEBUG pyenv_cfg:38]                                                                                                               
557     base-executable = c:\python39\python.exe [DEBUG pyenv_cfg:38]                                                                                                     
557 remove file \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts\python.exe [DEBUG _sync:38]                                       
572 copy c:\python39\Lib\venv\scripts\nt\python.exe to \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts\python.exe [DEBUG _sync:52]572 remove file \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts\pythonw.exe [DEBUG _sync:38]                                      
588 copy c:\python39\Lib\venv\scripts\nt\pythonw.exe to \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts\pythonw.exe [DEBUG _sync:5
2]                                                                                                                                                                        
683 create virtualenv import hook file \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\_virtualenv.pth [DEBUG api:95]   
690 create \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\_virtualenv.py [DEBUG api:98]                                
702 ============================== target debug ============================== [DEBUG session:54]                                                                         
710 debug via '\\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Scripts\python.exe' 'C:\Users\jaraco\.local\pipx\venvs\tox\lib\site-package
s\virtualenv\create\debug.py' [DEBUG creator:213]                                                                                                                         
709 {                                                                                                                                                                     
  "sys": {                                                                                                                                                                
    "executable": "\\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35-win\\Scripts\\python.exe",                                        
    "_base_executable": "c:\\python39\\python.exe",                                                                                                                       
    "prefix": "\\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35-win",                                                                 
    "base_prefix": "c:\\python39",                                                                                                                                        
    "real_prefix": null,                                                                                                                                                  
    "exec_prefix": "\\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35-win",                                                            
    "base_exec_prefix": "c:\\python39",                                                                                                                                   
    "path": [                                                                                                                                                             
      "c:\\python39\\python39.zip",                                                                                                                                       
      "c:\\python39\\DLLs",                                                                                                                                               
      "c:\\python39\\lib",                                                                                                                                                
      "c:\\python39",                                                                                                                                                     
      "\\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35-win",                                                                         
      "\\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35-win\\lib\\site-packages"                                                      
    ],                                                                                                                                                                    
    "meta_path": [                                                                                                                                                        
      "<class '_virtualenv._Finder'>",                                                                                                                                    
      "<class '_frozen_importlib.BuiltinImporter'>",                                                                                                                      
      "<class '_frozen_importlib.FrozenImporter'>",                                                                                                                       
      "<class '_frozen_importlib_external.PathFinder'>"                                                                                                                   
    ],                                                                                                                                                                    
    "fs_encoding": "utf-8",                                                                                                                                               
    "io_encoding": "cp1252"                                                                                                                                               
  },                                                                                                                                                                      
  "version": "3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]",                                                                            
  "makefile_filename": "c:\\python39\\Lib\\config\\Makefile",                                                                                                             
  "os": "<module 'os' from 'c:\\\\python39\\\\lib\\\\os.py'>",                                                                                                            
  "site": "<module 'site' from 'c:\\\\python39\\\\lib\\\\site.py'>",                                                                                                      
  "datetime": "<module 'datetime' from 'c:\\\\python39\\\\lib\\\\datetime.py'>",                                                                                          
  "math": "<module 'math' (built-in)>",                                                                                                                                   
  "json": "<module 'json' from 'c:\\\\python39\\\\lib\\\\json\\\\__init__.py'>"                                                                                           
} [DEBUG session:55]                                                                                                                                                      
1069 add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=C:\Users\jaraco\AppData\Local\pypa\virtualenv) 
[INFO session:59]                                                                                                                                                         
1126 got embed update of distribution pip from C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\embed\1\pip.json [DEBUG via_disk_folder:133]                       
1156 got embed update of distribution pip from C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\embed\1\pip.json [DEBUG via_disk_folder:133]                       
1159 got embed update of distribution setuptools from C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\embed\1\setuptools.json [DEBUG via_disk_folder:133]         
1163 got embed update of distribution setuptools from C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\embed\1\setuptools.json [DEBUG via_disk_folder:133]         
1160 got embed update of distribution wheel from C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\embed\1\wheel.json [DEBUG via_disk_folder:133]                   
1171 got embed update of distribution wheel from C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\embed\1\wheel.json [DEBUG via_disk_folder:133]                   
1181 install pip from wheel C:\Users\jaraco\.local\pipx\venvs\tox\lib\site-packages\virtualenv\seed\wheels\embed\pip-20.2.3-py2.py3-none-any.whl via CopyPipInstall [DEBUG
 via_app_data:51]                                                                                                                                                         
1191 Attempting to acquire lock 2003042464240 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\pip-20.2.3-py2.py3-none-any.lock [DEBUG fi
lelock:270]                                                                                                                                                               
1191 install setuptools from wheel C:\Users\jaraco\.local\pipx\venvs\tox\lib\site-packages\virtualenv\seed\wheels\embed\setuptools-50.3.1-py3-none-any.whl via CopyPipInst
all [DEBUG via_app_data:51]                                                                                                                                               
1192 Attempting to acquire lock 2003042464288 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any.lock [DEBUG
 filelock:270]                                                                                                                                                            
1194 Lock 2003042464288 acquired on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any.lock [INFO filelock:274]1194 Attempting to release lock 2003042464288 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any.lock [DEBUG
 filelock:315]                                                                                                                                                            
1194 Lock 2003042464288 released on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any.lock [INFO filelock:318]1198 copy C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any\distutils-precedence.pth to \\vmware-host\Shared F
olders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\distutils-precedence.pth [DEBUG _sync:52]                                                     
1192 Lock 2003042464240 acquired on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\pip-20.2.3-py2.py3-none-any.lock [INFO filelock:274]   
1209 Attempting to release lock 2003042464240 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\pip-20.2.3-py2.py3-none-any.lock [DEBUG fi
lelock:315]                                                                                                                                                               
1209 Lock 2003042464240 released on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\pip-20.2.3-py2.py3-none-any.lock [INFO filelock:318]   
1213 copy directory C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\pip-20.2.3-py2.py3-none-any\pip to \\vmware-host\Shared Folders\home\co
de\public\importlib_resources\.tox\py35-win\Lib\site-packages\pip [DEBUG _sync:52]                                                                                        
1226 copy C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any\easy_install.py to \\vmware-host\Shared Folders\ho
me\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\easy_install.py [DEBUG _sync:52]                                                                       
1271 install wheel from wheel C:\Users\jaraco\.local\pipx\venvs\tox\lib\site-packages\virtualenv\seed\wheels\embed\wheel-0.35.1-py2.py3-none-any.whl via CopyPipInstall [D
EBUG via_app_data:51]                                                                                                                                                     
1310 Attempting to acquire lock 2003042464720 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\wheel-0.35.1-py2.py3-none-any.lock [DEBUG 
filelock:270]                                                                                                                                                             
1311 Lock 2003042464720 acquired on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\wheel-0.35.1-py2.py3-none-any.lock [INFO filelock:274] 
1311 Attempting to release lock 2003042464720 on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\wheel-0.35.1-py2.py3-none-any.lock [DEBUG 
filelock:315]                                                                                                                                                             
1311 Lock 2003042464720 released on C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\wheel-0.35.1-py2.py3-none-any.lock [INFO filelock:318] 
1315 copy directory C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\wheel-0.35.1-py2.py3-none-any\wheel to \\vmware-host\Shared Folders\hom
e\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\wheel [DEBUG _sync:52]                                                                                  
1511 copy directory C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any\pkg_resources to \\vmware-host\Shared Fo
lders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\pkg_resources [DEBUG _sync:52]                                                                 
2469 copy directory C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\wheel-0.35.1-py2.py3-none-any\wheel-0.35.1.dist-info to \\vmware-host\S
hared Folders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\wheel-0.35.1.dist-info [DEBUG _sync:52]                                                
3274 copy directory C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any\setuptools to \\vmware-host\Shared Folde
rs\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\setuptools [DEBUG _sync:52]                                                                       
3302 copy C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\wheel-0.35.1-py2.py3-none-any\wheel-0.35.1.virtualenv to \\vmware-host\Shared Fol
ders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\wheel-0.35.1.virtualenv [DEBUG _sync:52]                                                        
3943 generated console scripts wheel.exe wheel3.9.exe wheel3.exe wheel-3.9.exe [DEBUG base:48]                                                                            
13140 copy directory C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any\setuptools-50.3.1.dist-info to \\vmware
-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\setuptools-50.3.1.dist-info [DEBUG _sync:52]                                    
13681 copy C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any\setuptools-50.3.1.virtualenv to \\vmware-host\Sha
red Folders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\setuptools-50.3.1.virtualenv [DEBUG _sync:52]                                            
13862 copy directory C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\setuptools-50.3.1-py3-none-any\_distutils_hack to \\vmware-host\Shared
 Folders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\_distutils_hack [DEBUG _sync:52]                                                            
14249 generated console scripts easy_install-3.9.exe easy_install3.exe easy_install3.9.exe easy_install.exe [DEBUG base:48]                                               
22589 copy directory C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\pip-20.2.3-py2.py3-none-any\pip-20.2.3.dist-info to \\vmware-host\Shar
ed Folders\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\pip-20.2.3.dist-info [DEBUG _sync:52]                                                     
23144 copy C:\Users\jaraco\AppData\Local\pypa\virtualenv\wheel\3.9\image\1\CopyPipInstall\pip-20.2.3-py2.py3-none-any\pip-20.2.3.virtualenv to \\vmware-host\Shared Folder
s\home\code\public\importlib_resources\.tox\py35-win\Lib\site-packages\pip-20.2.3.virtualenv [DEBUG _sync:52]                                                             
23262 generated console scripts pip3.9.exe pip-3.9.exe pip3.exe pip.exe [DEBUG base:48]                                                                                   
23301 add activators for Bash, Batch, Fish, PowerShell, Python, Xonsh [INFO session:64]                                                                                   
23424 write \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win\pyvenv.cfg [DEBUG pyenv_cfg:34]                                               
23511   home = c:\python39 [DEBUG pyenv_cfg:38]                                                                                                                           
23580   implementation = CPython [DEBUG pyenv_cfg:38]                                                                                                                     
23580   version_info = 3.9.0.final.0 [DEBUG pyenv_cfg:38]                                                                                                                 
23580   virtualenv = 20.0.35 [DEBUG pyenv_cfg:38]                                                                                                                         
23580   include-system-site-packages = false [DEBUG pyenv_cfg:38]                                                                                                         
23580   base-prefix = c:\python39 [DEBUG pyenv_cfg:38]                                                                                                                    
23580   base-exec-prefix = c:\python39 [DEBUG pyenv_cfg:38]                                                                                                               
23580   base-executable = c:\python39\python.exe [DEBUG pyenv_cfg:38]                                                                                                     
23580 created virtual environment CPython3.9.0.final.0-64 in 23383ms                                                                                                      
  creator CPython3Windows(dest=\\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35-win, clear=False, global=False)                                
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=C:\Users\jaraco\AppData\Local\pypa\virtualenv)                   
    added seed packages: pip==20.2.3, setuptools==50.3.1, wheel==0.35.1                                                                                                   
  activators BashActivator,BatchActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:17]         

@jaraco
Copy link
Author

jaraco commented Oct 21, 2020

I tried working around the issue by specifying a different directory for tox to create its environments. Since I don't know the difference between a tox work dir and a tox environments working dir, I set both:

importlib_resources bugfix/bpo-41490 # $TOX_WORK_DIR='~/local/tox'                                                                                                        
importlib_resources bugfix/bpo-41490 # $TOX_ENV_DIR='~/local/tox'                                                                                                         
importlib_resources bugfix/bpo-41490 # python -c "import os; print(os.environ['TOX_WORK_DIR'], os.environ['TOX_ENV_DIR'])"                                                
~/local/tox ~/local/tox                                                                                                                                                   
importlib_resources bugfix/bpo-41490 # tox -e py35                                                                                                                        
py35 create: \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35                                                                                  
ERROR: invocation failed (exit code 1), logfile: \\vmware-host\Shared Folders\home\code\public\importlib_resources\.tox\py35\log\py35-0.log                               
=============================================================================== log start =============================================================================== 
OSError: [WinError 1] Incorrect function: 'C:\\Python35\\python3.dll' -> '\\\\vmware-host\\Shared Folders\\home\\code\\public\\importlib_resources\\.tox\\py35\\Scripts\\p
ython3.dll'                                                                                                                                                               
                                                                                                                                                                          
================================================================================ log end ================================================================================ 
ERROR: InvocationError for command 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --python 'C:\Python35\python.exe' py35 (exited w
ith code 1)                                                                                                                                                               
________________________________________________________________________________ summary ________________________________________________________________________________ 
ERROR:   py35: InvocationError for command 'c:\users\jaraco\.local\pipx\venvs\tox\scripts\python.exe' -m virtualenv --no-download --python 'C:\Python35\python.exe' py35 (
exited with code 1)                          

Edit: Oh, as I repeatedly forget, those environment variables are injected, not respected.

@gaborbernat
Copy link
Member

When I am able to create the virtualenv by invoking virtualenv manually, virtualenv never attempts to link the dll

You're not invoking the same command. In the second case you're creating a virtual environment for Python 3.9, not Python 3.5, pass -p 3.5 to have the same. Both should fail then.

@gaborbernat
Copy link
Member

329 symlink on filesystem does work [DEBUG info:47]

So it seems that the symlink detection may not be effectively checking that symlinks work on the filesystem where the symlink will be created.

Indeed, but that's a virtualenv issue, not tox one.

@gaborbernat gaborbernat added the bug:upstream something does not behave as it should, but can't or shouldn't be fixed in tox, but in a dependency label Oct 21, 2020
@jaraco
Copy link
Author

jaraco commented Oct 21, 2020

Good insight, and I agree. After replicating the issue by using -p 3.5, I agree it's an upstream issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug:normal affects many people or has quite an impact bug:upstream something does not behave as it should, but can't or shouldn't be fixed in tox, but in a dependency
Projects
None yet
Development

No branches or pull requests

3 participants