Allow venv to create copies, even when symlinks are supported

[andreacorbellini]

BPO	18807
Nosy	@warsaw, @vsajip
Files	venv-symlink-fix.patch: patch for Lib/venv/init.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/vsajip'
closed_at = <Date 2013-08-24.23:04:36.831>
created_at = <Date 2013-08-22.13:03:46.698>
labels = ['library']
title = 'Allow venv to create copies, even when symlinks are supported'
updated_at = <Date 2014-06-04.17:55:57.490>
user = 'https://bugs.python.org/andreacorbellini'

bugs.python.org fields:

activity = <Date 2014-06-04.17:55:57.490>
actor = 'barry'
assignee = 'vinay.sajip'
closed = True
closed_date = <Date 2013-08-24.23:04:36.831>
closer = 'python-dev'
components = ['Library (Lib)']
creation = <Date 2013-08-22.13:03:46.698>
creator = 'andrea.corbellini'
dependencies = []
files = ['35382']
hgrepos = []
issue_num = 18807
keywords = ['patch']
message_count = 9.0
messages = ['195883', '196104', '219234', '219236', '219241', '219242', '219257', '219258', '219765']
nosy_count = 5.0
nosy_names = ['barry', 'vinay.sajip', 'andrea.corbellini', 'python-dev', 'Dominic.Cerquetti']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue18807'
versions = ['Python 3.4', 'Python 3.5']

[andreacorbellini]

I'd really appreciate if venv could create environments without symlinks.

Working on many Python projects, each one with different requirements, I prefer to keep everything I need in a single virtualenv directory, rather than two (one for the virtualenv and one for the built Python).

So I'd like to have a --copies option that lets me force venv not to create symlinks. I can work on a patch if this issue is accepted.

[python-dev]

New changeset ffb01a6c0960 by Vinay Sajip in branch 'default':
Closes bpo-18807: pyvenv now takes a --copies argument allowing copies instead of symlinks even where symlinks are available and the default.
http://hg.python.org/cpython/rev/ffb01a6c0960

[DominicCerquetti]

Requesting re-open of this issue, using --closes to force no symlinks to be created still results in venv trying to create symlinks.

I'm using Python 3.4 with the following command inside a vagrant Ubuntu 14.04 virtualbox image. The folder is a SMB mount from a windows host, which does not allow symlinks.

Expected behavior: os.symlink() is never called when you run:
python3.4 -m venv --copies

Actual behavior: os.symlink() is still called in a few places such as:
http://hg.python.org/cpython/file/b8e4bb1e1090/Lib/venv/__init__.py
line: 147
line: 215

I have a fix for line 215 that I'm testing now (basically just need to call copier() instead of os.symlink()).

I don't want to mess with line 147 due to it being OSX specific and I have no way to test it. But in theory it should also just be a call to copier()

[vsajip]

While you may be right about line 215, line 147 isn't analogous, because in the latter case it's a symlink to a directory. We don't really want the entire lib directory tree *copied* into lib64 (and in any case, the populating of lib will happen much after venv creatuion, when things are installed into the venv: a copy is of no use here).

[DominicCerquetti]

Preliminary patch for line 215, per earlier description.

While this doesn't appear to break anything and creates both copies and symlinks correctly, I do have these four failing unit tests:

test_multiprocessing_fork
test_multiprocessing_forkserver
test_multiprocessing_main_handling
test_multiprocessing_spawn

I don't have time right now to look into it to see if they're related to my change (at first glance, it looks like not).

[DominicCerquetti]

Ok cool, as you said line 215 then seems to be the only one that needs it.

[python-dev]

New changeset ce1b8b2ddf07 by Vinay Sajip in branch '3.4':
Issue bpo-18807: If copying (no symlinks) specified for a venv, then the python interpreter aliases (python, python3) are now created by copying rather than symlinking.
http://hg.python.org/cpython/rev/ce1b8b2ddf07

New changeset f2adaccc13ab by Vinay Sajip in branch 'default':
Issue bpo-18807: Merged fix from 3.4.
http://hg.python.org/cpython/rev/f2adaccc13ab

[vsajip]

I've made the change - not exactly the same as your patch, which was missing an os.chmod() and doing an unnecessary os.path.join() - but thanks for submitting a patch :-).

However, note that on 64-bit Linux systems (actually any POSIX other than OS X) a symlink lib64 -> lib is still created. Perhaps this could be omitted, but I'm not sure if that would cause problems with pip. I've posted a note on the relevant issue - bpo-21197.

[warsaw]

The os.chmod() will fail if path is a symlink. At the very least it must be guarded by a not os.path.islink() call like above it. I'll add this check to 3.4 and 3.5.