Updating third-party python dependencies using Pants

[sjaq]

I've been evaluating pants and I must say it looks quite amazing!

One thing I can't seem to wrap my head around though is how I can update my third-party dependencies in a python based project. Coming from poetry for all my dependency management I expect there to be some sort of command that helps me update minor versions and even show what dependencies are outdated.

I've tried to play around with the poetry integration, but as far as I can tell the only thing really supported by the poetry_requirements is parsing of a pyproject.toml, I can't depend on Poetry's CLI commands as I would previously, which makes sense as Poetry doesn't understand the full scope of an entire Pants based project.

What is the recommended workflow for checking outdated dependencies and updating them? Am I missing something here or is there really no workflow provided by pants to help with this?

[kaos]

Currently there are no commands/goals in Pants to manage your requirements. *see edit below.

Using a pyproject.toml however allows you to use Poetry for requirements management, and can then use that in your Pants based project.

EDIT Oct 2022:

Just feel like clarifying, that Pants do offer to generate lockfiles from your input requirements, so it is a matter of managing the list of requirements along with any constraints and then regenerating the lockfiles using Pants to get the latest compatible set of dependencies satisfying all constraints given.

[sjaq]

Thanks for the quick reply, that's what I thought. But I'm not really sure how I would use the poetry CLI correctly with pants?

Let me give an example:

• I have a package set up using poetry, that contains a pyproject.toml and a poetry.lock which is built using Pants and contains a BUILD file that uses poetry_requirements().
• In the poetry.lock there's a dependency that is outdated, for instance fastapi is locked by the poetry.lock at version 0.75.0 while the pyproject.toml specifies "^0.75.0" and version 0.75.2 is also available from pypi.
• When I run ./pants generate-lockfiles it generates a lockfile which has fastapi=0.75.2, so it does not consider the lock generated by poetry.

Even if this was not a problem, when you run any poetry CLI command poetry will itself create a separate venv, which does not take into account any other requirements that are present in the entire project's resolves. Which makes the entire poetry CLI unusable right?

In effect this means that versions really need to be pinned to an exact version, otherwise pants might update a minor dependency by itself if I'm not mistaken?

[kaos]

You can ask poetry to export it's lockfile. However I'm not sure if that will work with multiple resolves, if you need that.

I can provide more details tomorrow if that sounds interesting anyway.

[sjaq]

That definitely sounds interesting, this exported lockfile can then be used as a basis for Pants to build it lockfile from I guess right?

[kaos]

Pants can use that exported lockfile (or constraints file, is more accurate) as-is. So there won't actually be a proper lockfile, but pretty close.
This doesn't use the new resolves features, though.

# pants.toml
[python]
# Use poetry export to regenerate this lockfile.
requirement_constraints = "constraints.txt"

Snippet of a Makefile to update the constraints.txt file whenever there are changes to the pyproject.toml file:

constraints.txt: poetry.lock
	poetry export --dev --without-hashes > $@

# Optional, in case that pyproject.toml have been edited rather than changed as a result of running poetry.
poetry.lock: pyproject.toml
	poetry lock --no-update

And then simply use the poetry_requirements() in a BUILD file to pick up the dependencies to use from the pyproject.toml file.