Skip to content

Re-pip-compile all requirements files#1204

Merged
mhucka merged 19 commits intomainfrom
update-requirements
Mar 12, 2026
Merged

Re-pip-compile all requirements files#1204
mhucka merged 19 commits intomainfrom
update-requirements

Conversation

@mhucka
Copy link
Copy Markdown
Contributor

@mhucka mhucka commented Feb 23, 2026

This PR updates some dependencies as well as the scripts used to produce the various environment files, and updates the environment files themselves.

  • dev_tools/requirements/deps/runtime.txt needs to set some constraints on versions of some dependencies because higher versions of those dependencies require Python 3.11, which is incompatible with our goal of maintaining Python 3.10 compatibility (for now)
  • dev_tools/requirements/re-pip-compile-in-docker.sh needed an addition to prevent it including a certain Python package that is incompatible with Python 3.11+
  • dev_tools/requirements/Dockerfile didn't do a pip update -U pip, but should; while at it, I added some options to pip to silence irrelevant warnings so that the logs are less noisy

Note to reviewers: all of the .txt requirements files under dev_tools/requirements/envs/ and dev_tools/requirements/max_compat/ are solely the output of running dev_tools/requirements/re-pip-compile-in-docker.sh.

This is considered best practice and its lack is flagged by the
Dockerfile linter.
Another good practice to follow is to update `pip` itself.
Contourpy is a transitive dependency. Version 1.3.3 needs Python 3.11.
Something is bringing in version 1.3.3 even if I declare this package as
unsafe at `pip-compile` time using the `--unsafe-package` argument. So
far the only solution I've found is to put a version contraint in our
requirements input file.
The package backports.asyncio.runner is a transitive dependency that
should only be installed for Python < 3.11. It doesn't need to be in the
requirements files we produce, and conversely, letting `pip-compile`
include it leads to installation failures with some Python versions. We
can tell `pip-compile` to omit it and this solves the problem.
Copy link
Copy Markdown
Contributor

@pavoljuhas pavoljuhas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should avoid pinning to a specific pip version; this seems to be a side effect of using the --unsafe-package option.

Can we avoid using that option? uv pip compile seems to be able to condition backports.asyncio.runner by Python version.

mhucka added 3 commits March 12, 2026 17:38
What's happening is that for maximum compatibility, we run
run-pip-compile.py in Python 3.10 (see re-pip-compile-in-docker.sh). An
unfortunate behavior of pip-compile is that any use of
`--unsafe-packages` results in pip-compile no longer using its default
list of unsafe packages, and thus, setuptools and pip end up in the
output. So we have to add `--unsafe-package` for them too.
These are the latest files following the addition of more
`--unsafe-package` flags in `run-pip-compile.sh`.
Per Pavol's suggestion, this is a better approach, and it lets us omit
the `--unsafe-package` flag from run-pip-compile.sh.
@mhucka
Copy link
Copy Markdown
Contributor Author

mhucka commented Mar 12, 2026

I think we should avoid pinning to a specific pip version; this seems to be a side effect of using the --unsafe-package option.

Can we avoid using that option? uv pip compile seems to be able to condition backports.asyncio.runner by Python version.

You're right. I thought use the conditional spec in the input to pip-compile would not work, because I thought it was only something for pip, but it turns out it does work.

I removed the unsafe stuff altogether.

mhucka added 2 commits March 12, 2026 17:57
These are the latest files following the addition of more
`--unsafe-package` flags in `run-pip-compile.sh`.
This can be handled in a better way.
@mhucka mhucka requested a review from pavoljuhas March 12, 2026 18:07
@mhucka mhucka added this pull request to the merge queue Mar 12, 2026
Merged via the queue into main with commit 0ac6357 Mar 12, 2026
26 checks passed
@mhucka mhucka deleted the update-requirements branch March 12, 2026 22:09
mhucka added a commit that referenced this pull request Mar 12, 2026
This PR updates some dependencies as well as the scripts used to produce
the various environment files, and updates the environment files
themselves.

* `dev_tools/requirements/deps/runtime.txt` needs to set some
constraints on versions of some dependencies because higher versions of
those dependencies require Python 3.11, which is incompatible with our
goal of maintaining Python 3.10 compatibility (for now)
* `dev_tools/requirements/re-pip-compile-in-docker.sh` needed an
addition to prevent it including a certain Python package that is
incompatible with Python 3.11+
* `dev_tools/requirements/Dockerfile` didn't do a `pip update -U pip`,
but should; while at it, I added some options to pip to silence
irrelevant warnings so that the logs are less noisy

Note to reviewers: all of the `.txt` requirements files under
`dev_tools/requirements/envs/` and `dev_tools/requirements/max_compat/`
are solely the output of running
`dev_tools/requirements/re-pip-compile-in-docker.sh`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants