Pipelines broken upgrading 1.2.2 to 1.3+ on Windows

[staticdev]

• Poetry version: 1.3.1
• Python version: 3.10.8 and 3.11.1
• OS version and name: Ubuntu
• pyproject.toml:

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

When upgrading the constraints to poetry==1.3.1 we get on logs:

pipx install --pip-args=--constraint=.github/workflows/constraints.txt poetry
creating virtual environment...
installing poetry...
Fatal error from pip prevented installation. Full pip output in file:
    C:\Program Files (x86)\pipx\logs\cmd_2022-12-18_15.24.01_pip_errors.log

pip seemed to fail to build package:
    html5lib<2.0,>=1.0

Some possibly relevant errors from pip install:
    ERROR: Cannot install poetry==1.3.1 because these package versions have conflicting dependencies.
    ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
Error installing poetry.

Logs can be seen in these repos:

• Bump poetry from 1.2.2 to 1.3.1 in /.github/workflows staticdev/git-portfolio#836
• Bump poetry from 1.2.2 to 1.3.1 in /.github/workflows staticdev/human-readable#493
• Bump poetry from 1.2.2 to 1.3.1 in /.github/workflows staticdev/irpf-investidor#322
• https://github.com/magmax/python-inquirer/actions/runs/3732191428/jobs/6331317578

[dimbleby]

you have requirements files with incompatible requirements, how do you figure this as a poetry bug?

[staticdev]

@dimbleby simple answer to your question.. the only requirement I have is "poetry==1.3.1" =)
and also it works in other OSs (ubuntu and macos).

[dimbleby]

the only requirement I have is "poetry==1.3.1"

You have provided links to diffs showing that you have many more constraints than that.

it works in other OSs

That poetry installation works in lots of places is very bad evidence for poetry installation being broken.

If you were installing poetry via one of the supported methods - https://python-poetry.org/docs/#installation - then that would be potentially interesting.

But it looks as though you are saying "I don't agree with what pip's solver is telling me" and then assigning responsibility for that to poetry.

[staticdev]

@dimbleby I have evidence that even if you consider all the restrictions, poetry is able to resolve dependencies:

[tool.poetry]
name = "sandbox"
version = "0.1.0"
description = ""
authors = ["staticdev"]

[tool.poetry.dependencies]
python = "^3.11"
pip = "==22.3.1"
nox = "==2022.11.21"
nox-poetry = "==1.0.2"
poetry = "==1.3.1"
virtualenv = "==20.17.1"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

poetry lock just works, no error. So the constraints you said have conflicts, not the case. Do you know why would poetry have a conflict resolving dependencies in Linux and MacOS, but not with Windows?

[dimbleby]

Your pipeline is not running poetry, it is using pip / pipx to install poetry.

So if you are confident that poetry's requirements are consistent, then you have a problem with pip or pipx - not with poetry.

[neersighted]

Poetry has a much more sophisticated solver than pip. As you yourself have stated, it is capable of solving your constraints. However, pip is not. There is no bug in Poetry; you are running into limitations of pip or a bug in pip. We simply declare dependencies like any other package.

I would suggest you install Poetry using the recommended methods; if you really must continue to control every single package in the environment, simply maintain an exported requirements.txt based on the pyproject.toml you yourself provided, with poetry export -f requirements.txt. Poetry's solver can solve this and prevent you from using pip's, if you use a fully specified solution.

This is not a Poetry issue, Poetry is not involved here at all, beyond providing a list of required packages.

[staticdev]

@dimbleby @neersighted I am not using pip, in fact I am using pipx as you can see for example here. As far as documentation shows it is an official supported way to install poetry https://python-poetry.org/docs/#installing-with-pipx

Also there is not error on SOLVING anything, all my constraints are all pinned versions:

pip==22.3.1
nox==2022.11.21
nox-poetry==1.0.2
poetry==1.2.2
virtualenv==20.17.1

The error clearly states POETRY has conflicting dependencies and is not able to be installed on Windows.

Since my issue was closed with the argument I am using pip, I am actually not. I am still not convinced it is not a poetry bug. Could you please provide also some evidence from poetry's side it is not?

[dimbleby]

You aren't even running poetry. You agree that poetry's requirements are consistent. What do you think poetry could possibly do better?

[staticdev]

@dimbleby I think you missed the point it is not about using poetry to install anything but installing poetry itself.

It looks to me poetry 1.3.1 is relying on a package html5lib that cannot be build correctly on Windows, maybe the solution is switch dependency or use another restriction on poetry's side if you still want to support it OR go back to the restrictions from 1.2.2 that is working fine.

[dimbleby]

I don't have a windows python installation myself: but poetry is installed thousands of times every day on Windows (including in its own pipelines). It is not relying on a package that cannot be built correctly on windows.

Perhaps your use of constraints is unusual and has exposed a bug in pip.

[staticdev]

I also don't have a Windows installation, the error is occurring exactly in pipelines since many projects I use support Windows and I only use tools and dependencies that also support it. I myself never use Windows.

[neersighted]

We are well aware you are using pipx. However, pipx merely wraps pip, and thus uses pip's solver.

You can see from the file that you linked that you are directly passing arguments to pip, which is wrapped by pipx:

--pip-args=--constraint=.github/workflows/constraints.txt

Injecting additional constraints is not in fact, the suggested/supported usage of pipx.

Furthermore, solving is involved here. You have not provided a fully-specified solution to pip, so it will do the following:

• Download the packages you requested according to the newest version allowed by the constraints you provide.
• Check if all their dependencies (per their constraints) are locally available, and fetch the newest version allowed if not.
• Repeat this recursively until it has every dependency.
• Check for incompatibility and attempt to fetch alternate versions if constraints conflict.

pip solves at install time, without a locking stage like Poetry. Poetry is just another package, and what is happening here is that your constraints are complex enough that pip is failing to come up with a working resolution on every platform.

There are three paths forward here:

• Use the suggested way to install with pipx, and do not inject additional constraints into Poetry's environment.
• Simply pipx install poetry and then note the versions pip solved with, replacing the other versions in your constraints.txt.
• Bail out of using pip's solver by providing a full resolution with the output of poetry export as a requirements/constraints file (if you provide every package, pip will just check compatibility and will not need to solve anything).

Your previous usage happened to work as the intersection of Poetry + your other constraints was solvable by pip. That is no longer true. We, the Poetry project, do not support install methods other than the ones we specify, and the only packages we provide are high-quality Python packages (no different from/special in comparison to anything else on PyPI).

If you try to do something more fancy/opinionated than a 'standard' install method like pip install (e.g. add --constraints to restrict the solution space pip uses), you are responsible for maintaining it.

[staticdev]

@neersighted thanks a lot for the in-depth response and actually you are totally right. I did a test removing constraints and adding pipx install poetry==1.3.1 instead and it is working actually. I will try to isolate the problem so see if I can provide a more detailed issue on pip side. @dimbleby thank you also for your effort.

[neersighted]

If you just want to have Dependabot bump Poetry for you, removing all other versions from the constraints.txt you use should be sufficient:

e.g.

pipx install --pip-args=--contraint=poetry-constraints.txt poetry

poetry-constraints.txt:

poetry==1.3.1

[staticdev]

If you just want to have Dependabot bump Poetry for you, removing all other versions from the constraints.txt you use should be sufficient:

e.g.

pipx install --pip-args=--contraint=poetry-constraints.txt poetry

poetry-constraints.txt:

poetry==1.3.1

Exactly, I also had the same idea.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.