New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSError 'Not a directory' when creating env on Jython 2.7.0 #326

Closed
pytoxbot opened this Issue Sep 17, 2016 · 12 comments

Comments

5 participants
@pytoxbot

pytoxbot commented Sep 17, 2016

Hi,

I cannot use tox with Jython. I'm using the latest tox 2.3.1, and Jython 2.7.0 on OS X 10.10.5 with Java 1.7.0_79.

I tried to clone this repository and run the test suite using jython. Below is the output from test_venv.py::test_create that shows my issue.

It seems that the subprocess call to virtualenv command that creates the new tox environment is failing because a listdir call in session._initlogpath raises OSError because the envlogdir (where the output from the subprocess is to be logged) does not exist. If I browse the .tox directory in Finder, I can see there is no {envdir}/log subfolder.

Thank you for your support.

Cosimo

(jython-venv) cosimolupo@Cosimo-MBP ~/Documents/Github/tox
$ py.test -v tests/test_venv.py::test_create
============================= test session starts ==============================
platform java1.7.0_79 -- Python 2.7.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- /Users/cosimolupo/Documents/Github/fonttools/jython-venv/bin/jython
cachedir: .cache
tox comes from: '/Users/cosimolupo/Documents/Github/tox/tox/__init__$py.class'
rootdir: /Users/cosimolupo/Documents/Github/tox, inifile: tox.ini
collected 28 items 

tests/test_venv.py::test_create FAILED

=================================== FAILURES ===================================
_________________________________ test_create __________________________________

monkeypatch = <_pytest.monkeypatch.monkeypatch instance at 0x69>
mocksession = <tox._pytestplugin.MockSession instance at 0x6a>
newconfig = <function newconfig at 0x6b>

    def test_create(monkeypatch, mocksession, newconfig):
        config = newconfig([], """
            [testenv:py123]
        """)
        envconfig = config.envconfigs['py123']
        venv = VirtualEnv(envconfig, session=mocksession)
        assert venv.path == envconfig.envdir
        assert not venv.path.check()
        action = mocksession.newaction(venv, "getenv")
>       tox_testenv_create(action=action, venv=venv)

tests/test_venv.py:60: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tox/venv.py:395: in tox_testenv_create
    venv._pcall(args, venv=False, action=action, cwd=basepath)
tox/venv.py:361: in _pcall
    return action.popen(args, cwd=cwd, env=env,
tox/session.py:121: in popen
    fout = self._initlogpath(self.id)
tox/session.py:107: in _initlogpath
    l = logdir.listdir("%s-*" % actionid)
tox/session.py:107: in _initlogpath
    l = logdir.listdir("%s-*" % actionid)
../fonttools/jython-venv/Lib/site-packages/py/_path/local.py:388: in listdir
    names = py.error.checked_call(os.listdir, self.strpath)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <py._error.ErrorMaker object at 0x6c>
func = <java function listdir 0x6d>
args = ('/private/var/folders/jb/rjz76yw92w7144mwqg119jnm0000gn/T/pytest-of-cosimolupo/pytest-3/test_create0/.tox/py123/log',)
kwargs = {}, __tracebackhide__ = False, cls = <class 'py.error.ENOTDIR'>
value = OSError(20, 'Not a directory'), tb = <traceback object at 0x6e>
errno = 20

    def checked_call(self, func, *args, **kwargs):
        """ call a function and raise an errno-exception if applicable. """
        __tracebackhide__ = True
        try:
            return func(*args, **kwargs)
        except self.Error:
            raise
        except (OSError, EnvironmentError):
            cls, value, tb = sys.exc_info()
            if not hasattr(value, 'errno'):
                raise
            __tracebackhide__ = False
            errno = value.errno
            try:
                if not isinstance(value, WindowsError):
                    raise NameError
            except NameError:
                # we are not on Windows, or we got a proper OSError
                cls = self._geterrnoclass(errno)
            else:
                try:
                    cls = self._geterrnoclass(_winerrnomap[errno])
                except KeyError:
                    raise value
>           raise cls("%s%r" % (func.__name__, args))
E           ENOTDIR: [Not a directory]: listdir('/private/var/folders/jb/rjz76yw92w7144mwqg119jnm0000gn/T/pytest-of-cosimolupo/pytest-3/test_create0/.tox/py123/log',)

../fonttools/jython-venv/Lib/site-packages/py/_error.py:84: ENOTDIR
=========================== 1 failed in 2.31 seconds ===========================

peterjc added a commit to peterjc/biopython that referenced this issue Oct 7, 2016

@peterjc

This comment has been minimized.

Contributor

peterjc commented Oct 7, 2016

I just hit this bug or something very similar while trying to use Jython 2.7.0 under TravisCI/Tox

https://travis-ci.org/peterjc/biopython/jobs/165876306

$ tox -c .travis-tox.ini -e $TOXENV --notest
jython-nocov create: /home/travis/build/peterjc/biopython/.tox/jython-nocov
Traceback (most recent call last):
  File "/home/travis/jython/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/home/travis/jython/Lib/site-packages/tox/session.py", line 39, in main
    retcode = Session(config).runcommand()
  File "/home/travis/jython/Lib/site-packages/tox/session.py", line 39, in main
    retcode = Session(config).runcommand()
  File "/home/travis/jython/Lib/site-packages/tox/session.py", line 375, in runcommand
    return self.subcommand_test()
  File "/home/travis/jython/Lib/site-packages/tox/session.py", line 526, in subcommand_test
    if self.setupenv(venv):
  File "/home/travis/jython/Lib/site-packages/tox/session.py", line 434, in setupenv
    status = venv.update(action=action)
  File "/home/travis/jython/Lib/site-packages/tox/session.py", line 434, in setupenv
    status = venv.update(action=action)
  File "/home/travis/jython/Lib/site-packages/tox/venv.py", line 146, in update
    self.hook.tox_testenv_create(action=action, venv=self)
  File "/home/travis/jython/Lib/site-packages/tox/venv.py", line 146, in update
    self.hook.tox_testenv_create(action=action, venv=self)
  File "/home/travis/jython/Lib/site-packages/pluggy.py", line 724, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/home/travis/jython/Lib/site-packages/pluggy.py", line 338, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/travis/jython/Lib/site-packages/pluggy.py", line 332, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: \
  File "/home/travis/jython/Lib/site-packages/pluggy.py", line 596, in execute
    res = hook_impl.function(*args)
  File "/home/travis/jython/Lib/site-packages/tox/venv.py", line 392, in tox_testenv_create
    venv._pcall(args, venv=False, action=action, cwd=basepath)
  File "/home/travis/jython/Lib/site-packages/tox/venv.py", line 361, in _pcall
    return action.popen(args, cwd=cwd, env=env,
  File "/home/travis/jython/Lib/site-packages/tox/session.py", line 121, in popen
    fout = self._initlogpath(self.id)
  File "/home/travis/jython/Lib/site-packages/tox/session.py", line 107, in _initlogpath
    l = logdir.listdir("%s-*" % actionid)
  File "/home/travis/jython/Lib/site-packages/tox/session.py", line 107, in _initlogpath
    l = logdir.listdir("%s-*" % actionid)
  File "/home/travis/jython/Lib/site-packages/py/_path/local.py", line 388, in listdir
    names = py.error.checked_call(os.listdir, self.strpath)
  File "/home/travis/jython/Lib/site-packages/py/_error.py", line 84, in checked_call
    raise cls("%s%r" % (func.__name__, args))
py.error.ENOTDIR: [Not a directory]: listdir('/home/travis/build/peterjc/biopython/.tox/jython-nocov/log',)
The command "tox -c .travis-tox.ini -e $TOXENV --notest" failed and exited with 255 during .
@RonnyPfannschmidt

This comment has been minimized.

Contributor

RonnyPfannschmidt commented Oct 7, 2016

except py.error.ENOENT:

it seems this no lo longer accurate :/ - its not clear to me what python version fixes it

@peterjc

This comment has been minimized.

Contributor

peterjc commented Oct 12, 2016

My earlier comment used tox-2.3.1 virtualenv-15.0.3 on TravisCI, https://travis-ci.org/peterjc/biopython/jobs/165876306

Trying to debug this by pre-creating the directory made no difference, now using tox-2.4.0 virtualenv-15.0.3 on TravisCI, e.g. https://travis-ci.org/peterjc/biopython/jobs/167100293 or https://travis-ci.org/peterjc/biopython/jobs/167103643

@RonnyPfannschmidt

This comment has been minimized.

Contributor

RonnyPfannschmidt commented Oct 12, 2016

@peterjc i believe a extra except clause is needed for jython, would you like to add it? (see the code path i linked)

@peterjc

This comment has been minimized.

Contributor

peterjc commented Oct 12, 2016

I'd want to be able to reproduce this locally first - debugging things on TravisCI is pretty hard, although you can ask them for temporary access to a VM for this kind of thing.

@peterjc

This comment has been minimized.

Contributor

peterjc commented Feb 15, 2017

Persists with Jython 2.7, tox-2.6.0, virtualenv-15.1.0, py-1.4.32 as tested under TravisCI, e.g.

https://travis-ci.org/peterjc/biopython/jobs/201911071

I've looked at the code here and don't understand what an extra except clause would do under Jython. If os.listdir(...) is called on a non-existent folder, then Jython 2.7 returns a reasonable OSError which the code seems to handle already.

$ jython2.7 -c "import os; os.listdir('/does/not/exist')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OSError: [Errno 2] No such file or directory: '/does/not/exist'

@obestwalter obestwalter added this to backlog in Squash all the bugs Mar 10, 2017

@obestwalter obestwalter moved this from triage to ready in Squash all the bugs Mar 10, 2017

LordGaav added a commit to LordGaav/tox that referenced this issue Apr 27, 2017

Also catch ENOTDIR when initializing logdir
There is an implementation difference between CPython and Jython. On
CPython, only ENOENT is raised when the directory does not exist
(see posixmodule.c for instance). On Jython, ENOTDIR is raised when
the directory does not exist, and EACCESS is raised when there is
no read access.

This patch also catches ENOTDIR, but not EACCESS. The code flow
ensures that the logdir is created when it does not exist.

Ref tox-dev#326 .

LordGaav added a commit to LordGaav/tox that referenced this issue Apr 27, 2017

Also catch ENOTDIR when initializing logdir
There is an implementation difference between CPython and Jython. On
CPython, only ENOENT is raised when the directory does not exist
(see posixmodule.c for instance). On Jython, ENOTDIR is raised when
the directory does not exist, and EACCESS is raised when there is
no read access.

This patch also catches ENOTDIR, but not EACCESS. The code flow
ensures that the logdir is created when it does not exist.

Ref tox-dev#326 .

@LordGaav LordGaav referenced this issue Apr 27, 2017

Merged

Also catch ENOTDIR when initializing logdir #512

2 of 3 tasks complete
@LordGaav

This comment has been minimized.

Contributor

LordGaav commented Apr 28, 2017

@RonnyPfannschmidt I have prepared #512 with the fix you proposed. It looks good in my local tests, and Travis is not complaining either.

@obestwalter

This comment has been minimized.

Member

obestwalter commented May 19, 2017

fixed in #512

@obestwalter obestwalter moved this from ready to done in Squash all the bugs May 19, 2017

@peterjc

This comment has been minimized.

Contributor

peterjc commented May 19, 2017

Lovely. I don't know how/when you update the changelog, but this should be of interest to any Jython + tox users.

@obestwalter

This comment has been minimized.

Member

obestwalter commented Jun 2, 2017

@peterjc I think we can cut a new release soonish.

@obestwalter obestwalter added this to done in [released]2.8.x Jul 2, 2017

peterjc added a commit to peterjc/tox that referenced this issue Aug 24, 2017

2.8.0rc1 released on PyPI
Also mention this fixed tox-dev#326 (via pull request tox-dev#512)
@peterjc

This comment has been minimized.

Contributor

peterjc commented Aug 24, 2017

Confirming fixed in tox 2.8.0rc1, which I installed using pip install --pre tox under Jython 2.7.0, e.g.

https://travis-ci.org/peterjc/biopython/jobs/267995703

(The test run timed out, but nevertheless, tox installed and ran under Jython for me)

@obestwalter

This comment has been minimized.

Member

obestwalter commented Aug 24, 2017

Thanks for testing and for your patience!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment