Creating environment fails with OSError: [Errno 5] Input/output error

[akuzminsky]

To create a a virtual environment tox runs /usr/local/bin/python -m virtualenv --no-download --python /usr/local/bin/python py37. The command fails with an error

# /usr/local/bin/python -m virtualenv --no-download --python /usr/local/bin/python py37
OSError: [Errno 5] Input/output error: '/proxysql-client/py37/bin/python' -> '/proxysql-client/py37/bin/python3'

Apparently it believes a symlink isn't created. Here's an strace output:

stat("/proxysql-client/py37/bin/python", 0x7fff0bafdcf0) = -1 ENOENT (No such file or directory)
getpid()                                = 397
stat("/usr/local/bin/python", {st_mode=S_IFREG|0755, st_size=28368, ...}) = 0
symlink("/usr/local/bin/python", "/proxysql-client/py37/bin/python") = 0
stat("/proxysql-client/py37/bin/python", {st_mode=S_IFREG|0755, st_size=28368, ...}) = 0
chmod("/proxysql-client/py37/bin/python", 0100755) = 0
stat("/proxysql-client/py37/bin/python3", 0x7fff0bafe010) = -1 ENOENT (No such file or directory)
stat("/proxysql-client/py37/bin/python3", 0x7fff0bafdcf0) = -1 ENOENT (No such file or directory)
getpid()                                = 397
link("/proxysql-client/py37/bin/python", "/proxysql-client/py37/bin/python3") = -1 EIO (Input/output error)

It is worth noting tox fails only in a container (from a public image python:3.7) and runs fine on a host MacOS machine.
All virtualenv 20.* versions are affected. Version 16.7.9 works fine.

[akuzminsky]

Workaround in tox.ini

[testenv]
alwayscopy = True

[gaborbernat]

Can you please also post a creation output of virtualenv with -vvv --with-traceback?

[akuzminsky]

# /usr/local/bin/python -m virtualenv --no-download --python /usr/local/bin/python -vvv --with-traceback py37
88 setup logging to NOTSET [DEBUG report:43]
110 find interpreter for spec PythonSpec(path=/usr/local/bin/python) [INFO builtin:44]
110 proposed PythonInfo(spec=CPython3.7.3.final.0-64, exe=/usr/local/bin/python, platform=linux, version='3.7.3 (default, Jun 11 2019, 01:05:09) \n[GCC 6.3.0 20170516]', encoding_fs_io=utf-8-UTF-8) [INFO builtin:50]
110 accepted PythonInfo(spec=CPython3.7.3.final.0-64, exe=/usr/local/bin/python, platform=linux, version='3.7.3 (default, Jun 11 2019, 01:05:09) \n[GCC 6.3.0 20170516]', encoding_fs_io=utf-8-UTF-8) [DEBUG builtin:52]
112 filesystem is case-sensitive [DEBUG info:28]
127 create virtual environment via CPython3Posix(dest=/proxysql-client/py37, clear=False, global=False) [INFO session:24]
127 create folder /proxysql-client/py37/bin [DEBUG _sync:22]
130 create folder /proxysql-client/py37/lib/python3.7/site-packages [DEBUG _sync:22]
133 write /proxysql-client/py37/pyvenv.cfg [DEBUG pyenv_cfg:34]
133     home = /usr/local [DEBUG pyenv_cfg:38]
134     implementation = CPython [DEBUG pyenv_cfg:38]
134     version_info = 3.7.3.final.0 [DEBUG pyenv_cfg:38]
134     virtualenv = 20.0.2 [DEBUG pyenv_cfg:38]
134     include-system-site-packages = false [DEBUG pyenv_cfg:38]
134     base-prefix = /usr/local [DEBUG pyenv_cfg:38]
135     base-exec-prefix = /usr/local [DEBUG pyenv_cfg:38]
135     base-executable = /usr/local/bin/python [DEBUG pyenv_cfg:38]
137 symlink /usr/local/bin/python to /proxysql-client/py37/bin/python [DEBUG _sync:41]
139 hard link /proxysql-client/py37/bin/python to python3 [DEBUG _sync:66]
141 done in 138ms [INFO __main__:24]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.7/site-packages/virtualenv/__main__.py", line 41, in <module>
    run_with_catch()
  File "/usr/local/lib/python3.7/site-packages/virtualenv/__main__.py", line 30, in run_with_catch
    run(args, options)
  File "/usr/local/lib/python3.7/site-packages/virtualenv/__main__.py", line 17, in run
    run_via_cli(args, options)
  File "/usr/local/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 24, in run_via_cli
    session.run()
  File "/usr/local/lib/python3.7/site-packages/virtualenv/session.py", line 18, in run
    self._create()
  File "/usr/local/lib/python3.7/site-packages/virtualenv/session.py", line 25, in _create
    self.creator.run()
  File "/usr/local/lib/python3.7/site-packages/virtualenv/create/creator.py", line 160, in run
    self.create()
  File "/usr/local/lib/python3.7/site-packages/virtualenv/create/via_global_ref/builtin/via_global_self_do.py", line 78, in create
    src.run(self, self.symlinks)
  File "/usr/local/lib/python3.7/site-packages/virtualenv/create/via_global_ref/builtin/ref.py", line 121, in run
    alias_via(dest, link_file)
  File "/usr/local/lib/python3.7/site-packages/virtualenv/util/path/_sync.py", line 67, in link
    os_link(norm(src), norm(dest))
OSError: [Errno 5] Input/output error: '/proxysql-client/py37/bin/python' -> '/proxysql-client/py37/bin/python3'

[gaborbernat]

Eh dockers doesn't like hard links?

[gaborbernat]

Is it docker/for-mac#3553?

[gaborbernat]

Please confirm if you're using docker from Mac and if so what version 🤔 thanks

[akuzminsky]

yes, it's docker on MacOS

$ docker --version
Docker version 19.03.5, build 633a0ea

$ uname -a
Darwin ***.com 18.7.0 Darwin Kernel Version 18.7.0: Sun Dec 1 18:59:03 PST 2019; root:xnu-4903.278.19~1/RELEASE_X86_64 x86_64 i386 MacBookPro15,1 Darwin

[oxfn]

I've also faced this on macOS 10.13.2 and Docker for Mac 2.2.0.0 while preparing reproducible app for another issue on failing tox and coverage in gitlab-runner. In that issue some unwanted symlinks from setuptools package get into coverage report and fail project stats.
virtualenv-issue-1584.zip

[oxfn]

Workaround in tox.ini

[testenv]
alwayscopy = True

Another option is temporarily preinstall 'virtualenv<20' before running tox

[gaborbernat]

While I think this is a docker bug; I think it's reasonable to fallback to symlink here https://github.com/pypa/virtualenv/blob/master/src/virtualenv/create/via_global_ref/builtin/ref.py#L121 if the hard link fails. If anyone can create a PR for it that would be great.

[misl6]

Same issue in Ubuntu 18.04.4 LTS when creating a virtualenv on a FUSE mounted NTFS partition.

[gaborbernat]

Should be resolved via #1596

[gaborbernat]

Hello, a fix for this issue has been released via virtualenv 20.0.4; see https://pypi.org/project/virtualenv/20.0.4/ (https://virtualenv.pypa.io/en/latest/changelog.html#v20-0-4-2020-02-14). Please give a try and report back if your issue has not been addressed; if not, please comment here, and we'll reopen the ticket. We want to apologize for the inconvenience this has caused you and say thanks for having patience while we resolve the unexpected bugs with this new major release.