Method for enforcing a minimum version of Poetry

[PetterS]

Poetry 1.1 is not compatible with 1.0, so we need to enforce Poetry version for our development team one way or another.

There is this Stack Overflow question: https://stackoverflow.com/questions/64003868/how-to-enforce-a-minimum-python-poetry-version which currently is unanswered.

Is there a way of specifying a version of Poetry in pyproject.toml that will fail if the executing version of poetry does not match? What effect does [build-system] requires have?

[abn]

At present, poetry does not enforce a version required as in theory it is not necessary per say at the moment as it is the lock file that is not forwards compatible.

root@cc2fc2786035:/foobar# cat poetry.lock | grep lock-version
lock-version = "1.1"
root@cc2fc2786035:/foobar# pip install -q --force poetry==1.0.10
root@cc2fc2786035:/foobar# poetry install
The lock file might not be compatible with the current version of Poetry.
Upgrade Poetry to ensure the lock file is read properly or, alternatively, regenerate the lock file with the `poetry lock` command.
...

That said, the lock version is the closest thing we have at the moment. As you can see above, lock files generated since 1.0.10 if add and verify lock-version metadata in the poetry.lock file. Version 1.1.0 onwards the lock-version is 1.1 (note that this is not tied to poetry version, but to the lock file format).

Lock file generated using poetry>=1.1.0 will raise a warning if used with poetry==1.0.10. For now, in cases where poetry version used to generate the lock file needs to be enforced, you could possible verify this by checking if poetry lock --no-update generates a diff.

As for build-system.requires, this does not have any bearing on how poetry operates as it is relevant only to PEP 517 scenarios. This informaiton is used when a PEP 517 front-end builds the project and needs to know what dependencies to install when doing so. For example, this is used when you do pip install /path/to/poetry/project. And this is now expecte to use poetry-core and not poetry.

[PetterS]

Right, it is the format of the lock file and possibly the virtualenv that is the issue.

But alright, it makes upgrading from 1.0 to 1.1 harder but not impossible.

I have found that if the system version of poetry is 1.0 and 1.1 is added as a dep into the pyproject.toml, the newer version will be used for the project from that point on. That feels a bit nonstandard though.

[abn]

I have found that if the system version of poetry is 1.0 and 1.1 is added as a dep into the pyproject.toml, the newer version will be used for the project from that point on. That feels a bit nonstandard though.

Probably not a good idea if you are adding it as a dependnecy as this will affect your dependency graph.

[PetterS]

Probably not a good idea if you are adding it as a dependnecy as this will affect your dependency graph.

I agree. It would be a dev dependency and for an internal project but I am not looking for creative solutions here. :-)

[kimmoahola]

Would be great to make somehow sure that every team member has recent enough poetry.

In future poetry may add (more) non-backwords compatible features.

Is it possible run bash in https://python-poetry.org/docs/pyproject/#scripts ?

And if there would be some "preinstall" hooks (like in Node's package.json) then it would be possible to create a simple bash script to do the version checking.

Or maybe just add something like

[tool.poetry.checks]
poetry-version="^1.1"

to pyproject.toml

[cpvandehey]

Adding a "plus 1" to this problem. The Node world seems to have solved this problem by specifying node versions with the engineStrict feature.

[wren]

With the addition of dependency groups in poetry v1.2+, this issue again seems to be relevant. Updating pyproject.toml to use the new dependency group format now yields the below error when using poetry v1.1.15, and there's no way to require a minimum poetry version.

  RuntimeError

  The Poetry configuration is invalid:
    - Additional properties are not allowed ('group' was unexpected)

[eric-musliner]

Configuration like this should be enforced and not permitted to run when poetry doesn't meet the minimum required version.

[build-system]
requires = ["poetry-core>=1.2.0"]
build-backend = "poetry.core.masonry.api"

When 1.1 runs against a 1.2 lock file it causes more changes in the diff than are necessary and vice versa.

[neersighted]

The build-system has nothing to do with this issue; Poetry and poetry-core are developed and versioned separately. The "excessive diff" is a result of improvements in Poetry over time:

• Support for locking pip, setuptools, and wheel like any other package (and this sometimes has sharp edges with 1.1 that we largely consider acceptable).
• A more robust lock file format that decreases noise in diffs and is robust against colliding filenames.

If you want to avoid said diffs, you should enforce a consistent version amongst your team. Currently that is a social and not a technical problem; this issue is an ask that we make it a technical one.

[ashirazi2]

@neersighted Sorry, but these dependency managers (and more) version themselves, and warn if you're using an out-of-date version:

• bundler - Gemfile for Ruby
• yarn - package.json for JS

Perhaps you should consider the use-case of a developer who supports more than one project?
I recently experienced a problem with poetry v1.1 overriding and creating a conflicting lockfile on a separate poetry v1.2 project.

[neersighted]

@ashirazi2 I think you misunderstand my comments, and I suggest you read the full discussion before jumping in.

I am referring to the fact that build-system.requires is orthogonal to the ask in this issue; as is the evolution of the lock file format. The fact that poetry-core once had versioning that matched Poetry is a coincidence and doesn't relate to this issue at all.

Currently that is a social and not a technical problem; this issue is an ask that we make it a technical one.

This feature is not rejected, it's still something we want to add but no one has worked on it yet. This is merely a summary of the status quo -- currently this is a social problem, and we want to make it a technical problem.

---

If you need to juggle multiple versions of Poetry, because you are working on a project that has not updated:

1. Please update and open issues for any blockers you find. Some of them may be as-designed/planned, so check for duplicates/release notes; in general we want to make sure everyone is using the latest Poetry as this is a complex project and we're improving it regularly.
2. You can install multiple versions using pipx -- pipx install --suffix @1.1 poetry==1.1.15 for instance will let you keep a copy of Poetry 1.1 handy without conflicting with other copies.

[ashirazi2]

Thanks for the clarification, and the work-around!

[JakeSummers]

I am also running into this issue moving from poetry 1.4.2 to poetry 1.51.

I like @kimmoahola 's proposed solution to this: #3316 (comment)

Full reproduction steps

Poetry version:

$ poetry --version
Poetry (version 1.4.2)

Issue:

$ poetry install

The Poetry configuration is invalid:
  - [source.0] Additional properties are not allowed ('priority' was unexpected)

Due to this part of my pyproject.toml file:

[[tool.poetry.source]]
name = "foo"
url = "https://blablabla"
priority = "explicit"

Updating my poetry version solves this issue:

$ curl -sSL https://install.python-poetry.org | POETRY_VERSION=1.5.1 python3 -

Now poetry install works:

$ poetry install

[valentincalomme]

Add something like

[tool.poetry.checks]
poetry-version="^1.1"

to pyproject.toml

I don't mind the suggestion above. Another suggestion would be to make it part of the configuration. For instance: poetry config poetry-version ^1.7, which if you run with --local would store it in the poetry.toml file.

[travis-cook-sfdc]

If you need to juggle multiple versions of Poetry, because you are working on a project that has not updated:

Please update and open issues for any blockers you find. Some of them may be as-designed/planned, so check for duplicates/release notes; in general we want to make sure everyone is using the latest Poetry as this is a complex project and we're improving it regularly.
You can install multiple versions using pipx -- pipx install --suffix @1.1 poetry==1.1.15 for instance will let you keep a copy of Poetry 1.1 handy without conflicting with other copies.

Agreed that using multiple versions of poetry with pipx is helpful. But, it would still be useful to library developers if they could specific a version constraint for poetry when developing that library.

Both for the reason mentioned above (priority not allowed) as well as the changes with the way that dependencies groups were stored between 1.2 & 1.3, there are minimum allowed versions of poetry based on the way pyproject.toml is structured. It would be nice if that could be enforced in pyproject.toml somewhere.

[mscheifer]

To further clarify why this would be a useful feature, I'd like to give an example.

I tried to set up one of my projects in an IDE. I got this error:

  - 'name' is a required property
  - 'version' is a required property
  - 'description' is a required property
  - 'authors' is a required property
  - Additional properties are not allowed ('package-mode' was unexpected)

It's not that hard to figure out what the problem is by googling "poetry package-mode" and stumbling onto the release notes, but it requires some cognitive load and digging into the project build internals. I knew what the problem was immediately because I wrote the pyproject.toml but a new developer onboarding onto the project might not.

It would be nice if instead the error could have been:

This project requires poetry 1.8. You have 1.4

(at least for versions of poetry going forward)

This kind of problem doesn't come up for all the other development dependencies because they have enforced minimum versions (by poetry of course).