Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[parallel] .tox/log already exists #1137

Closed
JulienPalard opened this issue Jan 14, 2019 · 14 comments
Closed

[parallel] .tox/log already exists #1137

JulienPalard opened this issue Jan 14, 2019 · 14 comments

Comments

@JulienPalard
Copy link

@JulienPalard JulienPalard commented Jan 14, 2019

Related to #641, there is even this actual issue described in it:

Process1 mkdir(logdir)
Process2 mkdir(logdir) which raises py.error.EEXIST exception

just happened to me with tox 3.7.0:

Failed black under process 24272, stdout:

stderr:
Traceback (most recent call last):
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_error.py", line 66, in checked_call
return func(*args, **kwargs)
FileExistsError: [Errno 17] File exists: '/home/mdk/clones/myproj/.tox/log'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/mdk/.pyenv/versions/3.6.5/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/mdk/.pyenv/versions/3.6.5/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/__main__.py", line 4, in <module>
tox.cmdline()
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/session.py", line 47, in cmdline
main(args)
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/session.py", line 54, in main
retcode = build_session(config).runcommand()
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/session.py", line 67, in build_session
return Session(config)
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/tox/session.py", line 394, in	__init__
config.logdir.ensure(dir=1)
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_path/local.py", line 530, in ensure
return p._ensuredirs()
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_path/local.py", line 515, in _ensuredirs
self.mkdir()
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_path/local.py", line 465, in mkdir
py.error.checked_call(os.mkdir, fspath(p))
File "/home/mdk/.venvs/myproj/lib/python3.6/site-packages/py/_error.py", line 86, in checked_call
raise cls("%s%r" % (func.__name__, args))
py.error.EEXIST: [File exists]: mkdir('/home/mdk/clones/myproj/.tox/log',)
@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jan 14, 2019

if you switch to isolated build (which is highly recommend) you might avoid this, in general case we need to hold a lock when writing to these logs but that's tbd

@JulienPalard
Copy link
Author

@JulienPalard JulienPalard commented Jan 15, 2019

By "isolated build" you mean "not using --parallel at all"?

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Jan 15, 2019

@JulienPalard no that's something different, it refers to the build stage, not the execution stage. See: https://tox.readthedocs.io/en/latest/example/package.html?highlight=isolated#setuptools

@JulienPalard
Copy link
Author

@JulienPalard JulienPalard commented Jan 15, 2019

At first glance I don't really get what it's doing. It looks like by default tox is using setuptools, but if I add a pyproject.toml telling tox to use setuptools it works?

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Jan 15, 2019

I mean this is a bug anyway (so marked it accordinlgy now), but activating isolated build together with parallel run should work around it. Can you try like it is described in the docs and let us know if it works? If the docs are unclear there and you are confused, that is also a chance for us to improve them. Thanks!

@JulienPalard
Copy link
Author

@JulienPalard JulienPalard commented Jan 15, 2019

OK for the workaround thing :)

For the moment this is probably fixing the ".tox/log already exists", at least, it didn't happened again after a few tests, but can't say for sure as it was not happening often.

@JulienPalard
Copy link
Author

@JulienPalard JulienPalard commented Jan 15, 2019

What's not clear in the doc for me it's why "isolated_build = True" forces me to use a pyproject.toml. I expected isolated_build = True to isolate the builds but continue to use the default (setuptools) in the absence of pyproject.toml.

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jan 15, 2019

I actually tracked down (and fixed) this yesterday as part of #998. Will open a PR within the next week or so.

@JulienPalard isolated_build = True enforces a new best practice outlined in PEP-517 and PEP-518. All projects should have it going ahead. As tox is all about ensuring your projects build and run always I can see no reason why not to use pyproject.toml, so hence the enforcement. While true the spec outlines that in case of missing pyproject.toml fallback to setuptools, the Python zen does say that explicit is better than implicit.

@JulienPalard
Copy link
Author

@JulienPalard JulienPalard commented Jan 15, 2019

Maybe I'm just a big hater of having so many configuration files but that's a whole other story :] thanks for the great clarification and fast response \o/ \o/ and fixing the issue \o/ and finding a workaround \o/ \o/

Have a good day! :)

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Jan 15, 2019

@gaborbernat

While true the spec outlines that in case of missing pyproject.toml fallback to setuptools, the Python zen does say that explicit is better than implicit.

I tend to agreee, but I think we might need to explain a bit better, what this PEP517/518 shift means and why we enforce it in tox like we do it - openend #1141 for that.

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Jan 15, 2019

Maybe I'm just a big hater of having so many configuration files but that's a whole other story

yes ... but ... this is the standard that will end all other standards!1!!

@adamantike
Copy link
Contributor

@adamantike adamantike commented Feb 12, 2019

@gaborbernat do you know if #1145 had an impact on this issue?

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Feb 12, 2019

Yeah, this should be fixed now. We'll have to do a release, however due to extensive overhaul of the session package some of our plugins may broke, so I want to set up some integration test first 👍to avoid this

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Mar 27, 2019

It turned out that the fix had a bug, #1186 fixes it entirely now.

@tox-dev tox-dev locked and limited conversation to collaborators Jan 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants