Warn that lockfiles are not platform independent

[sondr3]

We just got bit by this at work when trying out rye, one dependency brought in pywin32 three levels down and when ran with rye sync it's included without environment markers in requirements.lock which broke our CI since it runs on Linux. I realize this is an issue with pip-tools (and uv at the moment for that matter) and not with rye, but it may be worth mentioning somewhere.

[mitsuhiko]

Yes, known limitation. It might be worth documenting this for now.

[webknjaz]

I realize this is an issue with pip-tools

If I may, it's not an "issue" with pip-tools but a state of the ecosystem as a whole — cross compilation is something that lacks support in CPython stdlib (there's a PEP attempting to address the lacking bits) and other places.

[sondr3]

Sure, but coming to Python from e.g. Node/C#/Java I don't really care about the state of the ecosystem, I just want tooling that works. Poetry has been great for most of our developers, sans those on Windows who has issues with it. Both the lockfile and the exported requirements.txt that Poetry creates has worked across macOS/Linux/Windows without issues, so solutions exist.

[mitsuhiko]

We all want tooling that works, and that includes lockfiles. I am in discussions with Astral folks regularly on figuring out a good story for this. It just will take a bit longer.

[webknjaz]

so solutions exist.

@sondr3 under a certain restricted set of constraints, yes. But not universally.

Luckily, there's a new discussion hoping to resurrect an effort of standardizing the lockfile format (replacing the rejected one from a few years ago): https://discuss.python.org/t/lock-files-again-but-this-time-w-sdists/46593. This is important as it'd allow for interoperability across tools with shared understanding of how things should work. I haven't read all of it but from what I managed to catch so far, I'm hopeful! I encourage you to read through that discussion and share any use cases that might've been missed there.

[mpizenberg]

While things are discussed to try to come up with a nice solution, could it be possible to generate multiple lock files? For example, my current work setup is using pip-tools, and we generate the following files depending on the platform that runs the compile / sync commands.

# on linux
pip-compile --extra=dev -o requirements/linux-dev.txt
pip-compile --extra=ci -o requirements/linux-ci.txt
# on windows
pip-compile --extra=dev -o requirements/windows-dev.txt

And then depending on the os of the person that uses the project, they either pip-sync the linux or windows requirements.

I haven't found options in rye lock or rye sync that would enable picking the lock file depending on platform. What do you think of providing such temporary workaround?