mach_bootstrap.py may find system pip before virtualenv copy

[jbeich]

Likely happens since 1a9263c. Also reported by @Yoric for OS X (MacPorts). To work around one may need to alter search order in PIP_NAMES e.g., prefer pip2.7 before pip-2.7.

Here's how the error looks on FreeBSD:

$ rm -rf ~/.cache/pip
$ ./mach build
Command "/usr/local/bin/python2.7 -c "import setuptools, tokenize;__file__='/tmp/pip-build-NlxkKB/blessings/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-iSyuXi-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-NlxkKB/blessings
Pip failed to execute properly.

Adding a print() before and dropping -q reveals more details:

/usr/local/bin/pip-2.7 install -r /path/to/servo/python/requirements.txt
...
    running install_lib
    error: could not create '/usr/local/lib/python2.7/site-packages/blessings': Perm
ission denied

because virtualenv naming is different from downstream

$ ls -1 /usr/local/bin/pip*
/usr/local/bin/pip
/usr/local/bin/pip-2.7

$ ls -1 python/_virtualenv/bin/pip*
python/_virtualenv/bin/pip
python/_virtualenv/bin/pip2
python/_virtualenv/bin/pip2.7

[jbeich]

Also tracking downstream for FreeBSD.

[frewsxcv]

Relevant issue that prompted that commit: #7784

[zwn]

I believe this issue might be related to #10595. If the 'quote' kicks in for any reason (it is no-op most of the time), the virtualenv fails to activate - meaning the search path for executing binaries is not updated correctly.

[campaul]

Since the virtualenv is already active at this point (assuming another bug hasn't prevented activation), I think it should be safe to replace pip = _get_exec(*PIP_NAMES) with just pip = find_executable("pip"). This solved the issue on my machine, but there may be edge cases I'm not considering.

[jhermann]

For a more robust way and skipping the in-script virtualenv activation (which is tricky at best), you could also do this:

pip_path = os.path.join(virtualenv_path, script_dir, "pip")

(which I think will also work in Windows, but might need checks for .bat/.cmd extensions).

This can then be used for an installation test combined with upgrading, like this:

{pip_path} install -U "pip>8" setuptools wheel # pinning all versions CAN be a good idea, if they're maintained

Please also note that sometimes an old distribute an cause trouble, and maybe should be uninstalled after the upgrade. Adding --log is also a good idea to ease diagnostics of any problems.

[piaoger]

thanks @campaul , I can build servo on my mac successfully by using your solution.

[wafflespeanut]

f83fe9e makes sure that we find pip from virtualenv. Is this still occurring?

[genodeftest]

I ran into this issue before but I think I don't any more. Is there a simple way to test other than running the steps above (which worked fine for me)?

[wafflespeanut]

Try removing python/_virtualenv and run ./mach build -d?

[genodeftest]

Try removing python/_virtualenv and run ./mach build -d?

Works fine for me.

[wafflespeanut]

I'd better close this as fixed. Please ping me if this occurs again.