uv pip compile Does not retain PEP-508 environment markers in output

[CoolCat467]

uv pip compile does not respect PEP-508 Environment Markers (such as ; implementation_name ==).

> uv --version
uv 0.1.11

For example, we have test-requirements.in:

black; implementation_name == "cpython"

and running

> uv pip compile test-requirements.in -o test-requirements.txt

yields black==24.2.0 instead of black==24.2.0 ; implementation_name == "cpython"

[charliermarsh]

I'm surprised that pip-compile retains those markers, but seems like we should too.

[jakkdl]

If removing ; python_version markers depending on the --python-version flag is intended, then maybe the same thing should happen here and you have a flag for specifying platform that makes use of those markers and can be used to generate multiple requirements.txt files depending on version+implementation

[CoolCat467]

This is currently a blocker for python-trio/trio#2958

[charliermarsh]

I'm open to including them but I don't know that it's a "correct" requirement. E.g., in the Trio example, it's not really "safe" to install from that file with a non-CPython implementation, because it could be missing dependencies that are required for PyPy or any other implementation. The locked requirements are only guaranteed to be correct on the same Python platform as that which generated the lock file.

[A5rocks]

I don't think the markers are propagated either, though I haven't checked against pip-compile. We use pip install -r <locked file> so missing dependencies are fine (we don't have hashes because requirement files vary per platform, and since lockfiles are more anti-bitrot than for security).

I think that copying over the marker to the output, even if not necessarily "correct," is a step forward towards platform independent lockfiles. (is there an issue for that? I know there's a section in the readme)

[charliermarsh]

@A5rocks - There's a standardized proposal for multi-platform (but not platform-independent) installer files (to replace requirements.txt for this purpose), we've been participating in the discussion.

[sondr3]

Could markers like platform_system be included at the very least in the beginning? We keep running into issues where developers are updating dependencies on their Windows machines that crash in CI when using uv that works with pip-compile. We've need to add pywin32==306 ; platform_system=='Windows' to our pyproject.toml (using rye) which works with pip-compile but not uv.

[Franky1]

I have this in my requirements.txt file

my-lib@https://github.com/path-to-my-lib-wheel.whl; platform_system=="Windows" and python_version=="3.10"

and uv pip install --upgrade --requirement requirements.txt fails with

error: Couldn't parse requirement in `requirements.txt` at position 180
  Caused by: Missing space before ';', the end of the URL is ambiguous

[charliermarsh]

The grammar is technically ambiguous for some URLs so we require a space before the ; when it follows a URL. pip also accepts this (though does not always require it). So in this case:

my-lib@https://github.com/path-to-my-lib-wheel.whl ; platform_system=="Windows" and python_version=="3.10"

[Franky1]

so we require a space before the ; when it follows a URL

Thanks for the quick reply, that was indeed the solution, tried it and it works now ❤️

[adminy]

Also pip-compile does not support the --trusted-host argument or the --config=pyproject.toml argument.

When you run pip compile it does not put --index-url or --trusted-host in its output, generating the requirements.txt without indication of the source.