Poetry export does not include setuptools even when it is a direct dependency. #1584
Comments
|
Verified that this issue also exists on beta 5 |
|
I think I have a problem that might be related to this issue. I have a service which includes a dependency that pins With the following pyproject.toml If I do poetry install a second time it works without issue. The problem is that on CI it fails every time as CI is always starting fresh. I have poetry 0.12.17 and python 3.7.4. |
|
Can confirm that setuptools as a dependency does not appear to be handled properly. There's a package that depends on setuptools in my lockfile, but there is not associated entry in the lockfile for setuptools itself. Lockfile snippet: |
|
This issue appears to affect |
|
This: fixes my issue. Now the question is: Why are those unsafe and what are the consequences? |
|
It seems there's not too much reasoning behind this (pypa/pip#6459) At the very least, we should introduce an |
|
Also having this issue with 2 packages that require separate versions of setuptools. (pathvalidate and google-auth). Install works the first time, but when I'm assuming this is what's happening:
Once reran, the dependency already exists for google-auth, but it's not the right version, and thus causes the above error. Just a guess, though. |
|
I am considering opening a PR for this. I plan to change the resolving behaviour so that the This will be my first poetry contribution so any advice, e.g. wrt anyone worth running this by before I start work, would be much appreciated. |
|
|
Hello. Packaging setuptools may be needed for some reason poetry may not be aware of, so it is important to provide an option to not ignore it Pinning it inside the virtualenv is also important. pipenv did the mistake of automatically installing the newer version of setuptools on new environment bootstrap, leaving it imposible to pin to a previous, working version in the lockfile (45.0.0 broke something, for a while it was impossible to go back to a version <45). So please allow pinning setuptools and other "special" packages (pip, poetry itself,...). |
|
The reason why they are marked as "unsafe" is because Poetry relies on them internally (well mostly However, Poetry will eventually no longer use them and when that happens this restriction will no longer be needed. Until then, I prefer that we leave it as it is to avoid unexpected side effects. |
|
@sdispater, thanks for getting back to us. Did you see my proposal?
This would seem to address problem of locking a set of packages that are inconsistent due to the unsafe packages being completely ignored at resolve time without affecting the default install/export behaviour. Would you support a PR implementing this? Sorry to push the subject but it's causing us significant pain right now: we're having to add dummy constraints updated manually to workaround this. It's a high maintenance and patchy state of affairs. |
|
Any word on this? Just encountered this same issue; it'd be nice if there were a solution. |
|
Same here. I found
This is not optimal as the pip and setuptools versions are not locked. Any solution to this? Seems to be a critical bug... Issue #1651 seems to be tackling this point, although no activity since February |
|
Same situation here as @ivallesp , Even adding a fake dependency to my project for a more recent setuptools doesn't help, nothing gets installed when I run poetry. |
|
Possibly poetry just should install and manage its setuptools and pip requirements in different/edited/hacked |
|
@earonesty brings up an interesting point. I'd argue that |
Why: The Python buildpack in Cloud.gov uses the requirements.txt file that we are exporting from Poetry. Unfortunately, the sha-256 hashes aren't working because of [some weirdness with Poetry and Setuptools](python-poetry/poetry#1584). How: export the requirements.txt file using `poetry export -f requirements.txt --without-hashes` which fixes the issue. Add this to the CircleCI configuration. Tags: poetry, CircleCI.
Otherwise, the generated py_binary targets depend on non existing dependencies (such as setuptool). Poetry doesn't add several packages in the poetry.lock file: https://github.com/python-poetry/poetry/blob/d2fd581c9a856a5c4e60a25acb95d06d2a963cf2/poetry/puzzle/provider.py#L55 See also python-poetry/poetry#1584 for explanations.
Otherwise, the generated py_binary targets depend on non existing dependencies (such as setuptool). Poetry doesn't add several packages in the poetry.lock file: https://github.com/python-poetry/poetry/blob/d2fd581c9a856a5c4e60a25acb95d06d2a963cf2/poetry/puzzle/provider.py#L55 See also python-poetry/poetry#1584 for explanations.
Otherwise, the generated py_binary targets depend on non existing dependencies (such as setuptool). Poetry doesn't add several packages in the poetry.lock file: https://github.com/python-poetry/poetry/blob/d2fd581c9a856a5c4e60a25acb95d06d2a963cf2/poetry/puzzle/provider.py#L55 See also python-poetry/poetry#1584 for explanations.
Otherwise, the generated py_binary targets depend on non existing dependencies (such as setuptool). Poetry doesn't add several packages in the poetry.lock file: https://github.com/python-poetry/poetry/blob/d2fd581c9a856a5c4e60a25acb95d06d2a963cf2/poetry/puzzle/provider.py#L55 See also python-poetry/poetry#1584 for explanations.
|
@insysion Did you open a PR? This is a serious bug. It causes the installation of older, buggier versions of setuptools like in pypa/setuptools#1963. There's no reason to break such projects. |
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
|
Would My particular use case is to generate a requirements file that I can use without |
|
@bmw There exisits a merge request waiting final review that removes those unsafe packages completely. So there will be no need for such a flag. |
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
|
@bmw @apollo13 jfyi python-poetry/poetry-core#168 (comment) if you add currently unsafe packages to your dependencies, older versions of poetry might not like it all that much. |
|
Also see #3916 (would appreciate some feedback there). |
Generated with poetry export:
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Workaround for a Poetry bug (only fixed in their pre-release which is not yet on PyPI): python-poetry/poetry#1584
|
Hi, are there any updates on this? Just ran into the same issue when I specify https://github.com/pyeve/cerberus/blob/master/setup.py as a dependency edit: I see it's being fixed in an upcoming release #5351 (comment) |
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --with dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
Generated with poetry export:
$ poetry export --without-hashes -f requirements.txt > requirements.txt
$ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.
See: python-poetry/poetry#1584
|
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
-vvvoption).Issue
Using the
pyproject.tomlfile from the Gist and executingpoetry lockI get the following lockfile:setuptools is correctly discovered as dependency of gunicorn (see https://github.com/benoitc/gunicorn/blob/94ab2091173c6037b504f94e56f4e88816d540bf/setup.py#L71-L77 -- it does indeed require it), but the lockfile does not contain any hashes for setuptools.
If I now export to requirements.txt the file looks like this:
Which is not installable with
pip install --require-hashes --force-reinstall -r requirements.txt(note the--require-hasesthere):If I instead add Django via
poetry add Django, thepoetry.lockfile will contain Django's dependencies (pytzandsqlparse):and also export them to the requirements file:
I fear there is some special casing going on for
setuptools, it would be great ifsetuptoolsalso ended up in the lock and requirement files if it is a direct dependency of on of the added packages.The text was updated successfully, but these errors were encountered: