Cannot override virtualenv.ini --copies values on commandline #1784

nils-werner · 2020-04-22T07:51:28Z

Issue

If I set the --copies parameter in ~/.config/virtualenv/virtualenv.ini I cannot override it later with --symlinks on the commandline.

Steps to Reproduce

Create virtualenv.ini file containing

[virtualenv]
copies = True

then run

$ virtualenv default
$ ls -lah default/bin/python*
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:35 default/bin/python
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:35 default/bin/python3
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:35 default/bin/python3.8
$ virtualenv --copies copies
$ ls -lah copies/bin/python*
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:35 copies/bin/python
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:35 copies/bin/python3
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:35 copies/bin/python3.8
$ virtualenv --symlinks symlinks
$ ls -lah symlinks/bin/python*
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:35 symlinks/bin/python
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:35 symlinks/bin/python3
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:35 symlinks/bin/python3.8

As you can see, there are no symlinks in any of the three virtualenvs. I would expect the last virtualenv to have symlinks instead of copies.

Remove virtualenv.ini and retry

$ virtualenv default
$ ls -lah default/bin/python*
lrwxrwxrwx 1 nils nils 15 22. Apr 09:39 default/bin/python -> /usr/bin/python
lrwxrwxrwx 1 nils nils  6 22. Apr 09:39 default/bin/python3 -> python
lrwxrwxrwx 1 nils nils  6 22. Apr 09:39 default/bin/python3.8 -> python
$ virtualenv --copies copies
$ ls -lah copies/bin/python*
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:39 copies/bin/python
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:39 copies/bin/python3
-rwxr-xr-x 1 nils nils 14K 22. Apr 09:39 copies/bin/python3.8
$ virtualenv --symlinks symlinks
$ ls -lah symlinks/bin/python*
lrwxrwxrwx 1 nils nils 15 22. Apr 09:39 symlinks/bin/python -> /usr/bin/python
lrwxrwxrwx 1 nils nils  6 22. Apr 09:39 symlinks/bin/python3 -> python
lrwxrwxrwx 1 nils nils  6 22. Apr 09:39 symlinks/bin/python3.8 -> python

and the expected behaviour shows.

Environment

Provide at least:

OS: Arch Linux Latest
pip list of the host python where virtualenv is installed:

$ pip list
Package             Version    
------------------- -----------
appdirs             1.4.3      
asn1crypto          1.3.0      
attrs               19.3.0     
btrfsutil           1.2.0      
CacheControl        0.12.6     
cachy               0.3.0      
cffi                1.14.0     
chardet             3.0.4      
cleo                0.7.6      
clikit              0.4.2      
colorama            0.4.3      
contextlib2         0.6.0.post1
cryptography        2.8        
distlib             0.3.0      
distro              1.4.0      
docopt              0.6.2      
docutils            0.14       
entrypoints         0.3        
feedparser          5.2.1      
filelock            3.0.12     
html2text           2019.8.11  
html5lib            1.0.1      
idna                2.9        
importlib-metadata  1.5.0      
jeepney             0.4.3      
Jinja2              2.11.2     
jsonschema          3.2.0      
keyring             21.2.0     
lit                 0.10.0.dev0
lockfile            0.12.2     
louis               3.13.0     
MarkupSafe          1.1.1      
meld                3.20.2     
more-itertools      8.2.0      
msgpack             1.0.0      
ordered-set         3.1.1      
packaging           20.3       
pastel              0.2.0      
pep517              0.8.2      
pexpect             4.8.0      
pip                 20.0.2     
pkginfo             1.5.0.1    
ply                 3.11       
poetry              1.0.5      
powerline-gitstatus 1.3.1      
progress            1.5        
ptyprocess          0.6.0      
pwquality           1.4.2      
pycairo             1.19.1     
pycodestyle         2.5.0      
pycparser           2.20       
Pygments            2.6.1      
PyGObject           3.36.0     
pylev               1.3.0      
pyparsing           2.4.7      
pyPEG2              2.15.2     
PyQt5               5.14.2     
PyQt5-sip           12.7.2     
PyQtWebEngine       5.14.0     
pyrsistent          0.16.0     
python-dateutil     2.8.1      
pytoml              0.1.21     
pyxdg               0.26       
PyYAML              5.3.1      
qutebrowser         1.10.2     
requests            2.23.0     
requests-toolbelt   0.9.1      
retrying            1.3.3      
SecretStorage       3.1.2      
setuptools          46.1.3     
shellingham         1.3.1      
six                 1.14.0     
team                1.0        
toml                0.10.0     
tomlkit             0.5.11     
urllib3             1.25.9     
virtualenv          20.0.15    
webencodings        0.5.1      
wheel               0.34.2     
zipp                3.1.0

Output of the virtual environment creation

Make sure to run the creation with -vvv --with-traceback:

$ virtualenv -vvv --with-traceback --symlinks symlinks
23 setup logging to NOTSET [DEBUG report:43]
39 find interpreter for spec PythonSpec(path=/usr/bin/python) [INFO builtin:44]
39 proposed PythonInfo(spec=CPython3.8.2.final.0-64, exe=/usr/bin/python, platform=linux, version='3.8.2 (default, Apr  8 2020, 14:31:25) \n[GCC 9.3.0]', encoding_fs_io=utf-8-utf-8) [INFO builtin:50]
40 accepted PythonInfo(spec=CPython3.8.2.final.0-64, exe=/usr/bin/python, platform=linux, version='3.8.2 (default, Apr  8 2020, 14:31:25) \n[GCC 9.3.0]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:52]
41 filesystem is case-sensitive [DEBUG info:28]
73 create virtual environment via CPython3Posix(dest=/home/nils/Projekte/hydrogen/symlinks, clear=False, global=False) [INFO session:52]
73 write /home/nils/Projekte/hydrogen/symlinks/pyvenv.cfg [DEBUG pyenv_cfg:34]
73 	home = /usr [DEBUG pyenv_cfg:38]
73 	implementation = CPython [DEBUG pyenv_cfg:38]
73 	version_info = 3.8.2.final.0 [DEBUG pyenv_cfg:38]
73 	virtualenv = 20.0.15 [DEBUG pyenv_cfg:38]
73 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
73 	base-prefix = /usr [DEBUG pyenv_cfg:38]
73 	base-exec-prefix = /usr [DEBUG pyenv_cfg:38]
73 	base-executable = /usr/bin/python [DEBUG pyenv_cfg:38]
74 remove file /home/nils/Projekte/hydrogen/symlinks/bin/python [DEBUG _sync:34]
74 copy /usr/bin/python to /home/nils/Projekte/hydrogen/symlinks/bin/python [DEBUG _sync:48]
74 copy /usr/bin/python to /home/nils/Projekte/hydrogen/symlinks/bin/python3 [DEBUG _sync:48]
74 copy /usr/bin/python to /home/nils/Projekte/hydrogen/symlinks/bin/python3.8 [DEBUG _sync:48]
74 create virtualenv import hook file /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/_virtualenv.pth [DEBUG api:76]
75 create /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/_virtualenv.py [DEBUG api:79]
75 ============================== target debug ============================== [DEBUG session:54]
75 debug via /home/nils/Projekte/hydrogen/symlinks/bin/python /usr/lib/python3.8/site-packages/virtualenv/create/debug.py [DEBUG creator:187]
75 {
  "sys": {
    "executable": "/home/nils/Projekte/hydrogen/symlinks/bin/python",
    "_base_executable": "/home/nils/Projekte/hydrogen/symlinks/bin/python",
    "prefix": "/home/nils/Projekte/hydrogen/symlinks",
    "base_prefix": "/usr",
    "real_prefix": null,
    "exec_prefix": "/home/nils/Projekte/hydrogen/symlinks",
    "base_exec_prefix": "/usr",
    "path": [
      "/usr/lib/python38.zip",
      "/usr/lib/python3.8",
      "/usr/lib/python3.8/lib-dynload",
      "/home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.8.2 (default, Apr  8 2020, 14:31:25) \n[GCC 9.3.0]",
  "os": "<module 'os' from '/usr/lib/python3.8/os.py'>",
  "site": "<module 'site' from '/usr/lib/python3.8/site.py'>",
  "datetime": "<module 'datetime' from '/usr/lib/python3.8/datetime.py'>",
  "math": "<module 'math' from '/usr/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so'>",
  "json": "<module 'json' from '/usr/lib/python3.8/json/__init__.py'>"
} [DEBUG session:55]
102 add seed packages via FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/home/nils/.local/share/virtualenv/seed-app-data/v1.0.1) [INFO session:59]
102 Attempting to acquire lock 140676577462016 on /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:270]
102 Lock 140676577462016 acquired on /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:274]
103 get bundled wheel /usr/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/pip-20.0.2-py2.py3-none-any.whl [DEBUG acquire:51]
104 get bundled wheel /usr/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/setuptools-46.1.3-py3-none-any.whl [DEBUG acquire:51]
104 get bundled wheel /usr/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/wheel-0.34.2-py2.py3-none-any.whl [DEBUG acquire:51]
105 install wheel from wheel /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/wheels/wheel-0.34.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
106 install setuptools from wheel /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/wheels/setuptools-46.1.3-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
106 install pip from wheel /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/wheels/pip-20.0.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
107 copy directory /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.dist-info to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/wheel-0.34.2.dist-info [DEBUG _sync:48]
107 copy directory /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.3-py3-none-any/setuptools-46.1.3.dist-info to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/setuptools-46.1.3.dist-info [DEBUG _sync:48]
114 copy /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.virtualenv to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/wheel-0.34.2.virtualenv [DEBUG _sync:48]
114 copy /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.3-py3-none-any/setuptools-46.1.3.virtualenv to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/setuptools-46.1.3.virtualenv [DEBUG _sync:48]
115 copy directory /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/wheel [DEBUG _sync:48]
115 copy /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.3-py3-none-any/easy_install.py to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/easy_install.py [DEBUG _sync:48]
123 copy directory /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.3-py3-none-any/pkg_resources to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/pkg_resources [DEBUG _sync:48]
127 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/wheel to 755 [INFO util:566]
127 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/wheel-3.8 to 755 [INFO util:566]
127 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/wheel3 to 755 [INFO util:566]
127 generated console scripts wheel-3.8 wheel wheel3 [DEBUG base:53]
130 copy directory /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.3-py3-none-any/setuptools to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/setuptools [DEBUG _sync:48]
133 copy directory /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/pip [DEBUG _sync:48]
144 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/easy_install-3.8 to 755 [INFO util:566]
144 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/easy_install to 755 [INFO util:566]
144 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/easy_install3 to 755 [INFO util:566]
144 generated console scripts easy_install easy_install-3.8 easy_install3 [DEBUG base:53]
168 copy /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.virtualenv to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/pip-20.0.2.virtualenv [DEBUG _sync:48]
168 copy directory /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.dist-info to /home/nils/Projekte/hydrogen/symlinks/lib/python3.8/site-packages/pip-20.0.2.dist-info [DEBUG _sync:48]
169 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/pip-3.8 to 755 [INFO util:566]
169 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/pip to 755 [INFO util:566]
169 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/pip3 to 755 [INFO util:566]
170 changing mode of /home/nils/Projekte/hydrogen/symlinks/bin/pip3.8 to 755 [INFO util:566]
170 generated console scripts pip3 pip3.8 pip-3.8 pip [DEBUG base:53]
170 Attempting to release lock 140676577462016 on /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:315]
170 Lock 140676577462016 released on /home/nils/.local/share/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:318]
170 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
172 write /home/nils/Projekte/hydrogen/symlinks/pyvenv.cfg [DEBUG pyenv_cfg:34]
172 	home = /usr [DEBUG pyenv_cfg:38]
172 	implementation = CPython [DEBUG pyenv_cfg:38]
172 	version_info = 3.8.2.final.0 [DEBUG pyenv_cfg:38]
172 	virtualenv = 20.0.15 [DEBUG pyenv_cfg:38]
172 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
172 	base-prefix = /usr [DEBUG pyenv_cfg:38]
172 	base-exec-prefix = /usr [DEBUG pyenv_cfg:38]
172 	base-executable = /usr/bin/python [DEBUG pyenv_cfg:38]
172 created virtual environment CPython3.8.2.final.0-64 in 150ms
  creator CPython3Posix(dest=/home/nils/Projekte/hydrogen/symlinks, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/home/nils/.local/share/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:21]

The text was updated successfully, but these errors were encountered:

gaborbernat · 2020-04-22T09:19:39Z

Thanks for reporting, will look into this; but feel free to put in a PR if you have availability.

gaborbernat · 2020-04-23T10:31:03Z

So the problem here is that in this case both copies and symlinks are set to True, and in such case we favour copy. The options object sadly doesn't know the source of the setting (ini file, env-var or cli flag) so we can't easily tell which to favour, so we picked the option that's more robust... in this case that's favouring copy. Not sure how to fix this.

gaborbernat · 2020-04-23T12:07:19Z

#1785 is an attempt to address this.

gaborbernat · 2020-05-03T14:33:27Z

Hello, this now has been released via https://pypi.org/project/virtualenv/20.0.19/

nils-werner added the bug label Apr 22, 2020

gaborbernat mentioned this issue Apr 23, 2020

Better handling of conflicting --copies and --symlinks #1785

Merged

gaborbernat closed this as completed in #1785 Apr 25, 2020

pypa locked and limited conversation to collaborators Jan 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot override virtualenv.ini --copies values on commandline #1784

Cannot override virtualenv.ini --copies values on commandline #1784

nils-werner commented Apr 22, 2020

gaborbernat commented Apr 22, 2020

gaborbernat commented Apr 23, 2020

gaborbernat commented Apr 23, 2020

gaborbernat commented May 3, 2020

Cannot override virtualenv.ini --copies values on commandline #1784

Cannot override virtualenv.ini --copies values on commandline #1784

Comments

nils-werner commented Apr 22, 2020

gaborbernat commented Apr 22, 2020

gaborbernat commented Apr 23, 2020

gaborbernat commented Apr 23, 2020

gaborbernat commented May 3, 2020