poetry update removes explicit source from lockfile unexpectedly

[messense]

• Poetry version: 1.7.1
• Python version: 3.10.12
• OS version and name: Ubuntu 22.04
• pyproject.toml:

[tool.poetry]
name = "repro"
version = "0.1.0"
description = ""
authors = []
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"
ruff = "^0.1.15"

[tool.poetry.group.jax.dependencies]
jaxlib = { extras = [
    "cuda12-pip",
], version = "^0.4.21+cuda12.cudnn89", source = "jax-releases" }
jax = { extras = ["cuda12-pip"], version = "^0.4.21" }

[[tool.poetry.source]]
name = "jax-releases"
url = "https://storage.googleapis.com/jax-releases/jax_cuda_releases.html"
priority = "explicit"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

I'm using a in-project virtualenv for this repro, after the initial lock and install, git commit the changes so the worktree is in clean state:

$ git status
On branch main
nothing to commit, working tree clean

now poetry.lock is modified expectedly after executing poetry update ruff:

$ poetry update ruff
Updating dependencies
Resolving dependencies... (1.0s)

No dependencies to install or update

Writing lock file

$ git diff
diff --git a/poetry.lock b/poetry.lock
index ceb3506..8dbcd0f 100644
--- a/poetry.lock
+++ b/poetry.lock
@@ -54,17 +54,8 @@ name = "jaxlib"
 version = "0.4.23+cuda12.cudnn89"
 description = "XLA library for JAX"
 optional = false
-python-versions = ">=3.9"
-files = [
-    {file = "jaxlib-0.4.23+cuda12.cudnn89-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:80ee32afd0e93a42b74027883c20e0ec6b0853da4b9c78969af0a4393d5d25f9"},
-    {file = "jaxlib-0.4.23+cuda12.cudnn89-cp310-cp310-manylinux2014_x86_64.whl", hash = "sha256:8e42000672599e7ec0ea7f551acfcc95dcdd0e22b05a1d1f12f97b56a9fce4a8"},
-    {file = "jaxlib-0.4.23+cuda12.cudnn89-cp311-cp311-manylinux2014_aarch64.whl", hash = "sha256:6d0be32097511a2bf5ae308e6db74c0d12d4c056d8e670ec2e1aae0dfe1e83f5"},
-    {file = "jaxlib-0.4.23+cuda12.cudnn89-cp311-cp311-manylinux2014_x86_64.whl", hash = "sha256:74c51c4671e397c37251e3b73f5c7b08d805d2200714a23352d1e1fbf08d928c"},
-    {file = "jaxlib-0.4.23+cuda12.cudnn89-cp312-cp312-manylinux2014_aarch64.whl", hash = "sha256:d334eafe552eb8cb1eb422dcd5b245edfec0778edfca8fe859dbbbb9b436f169"},
-    {file = "jaxlib-0.4.23+cuda12.cudnn89-cp312-cp312-manylinux2014_x86_64.whl", hash = "sha256:91727a6d4c17f90c9ba983d5f4aa70bfe961768184151af8fb54f4bdf5e8591a"},
-    {file = "jaxlib-0.4.23+cuda12.cudnn89-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:5f7fed1cdb1807413d7c4be5ba80935f93f259ffa2fca21708292368018011b1"},
-    {file = "jaxlib-0.4.23+cuda12.cudnn89-cp39-cp39-manylinux2014_x86_64.whl", hash = "sha256:a2eaf6992c073dd8d5a030193fef5a354909fafc6fd53abb1b952a03d1ef84d2"},
-]
+python-versions = "*"
+files = []

 [package.dependencies]
 ml-dtypes = ">=0.2.0"
@@ -82,15 +73,6 @@ scipy = [
     {version = ">=1.9", markers = "python_version < \"3.12\""},
 ]

-[package.extras]
-cuda11-pip = ["nvidia-cublas-cu11 (>=11.11)", "nvidia-cuda-cupti-cu11 (>=11.8)", "nvidia-cuda-nvcc-cu11 (>=11.8)", "nvidia-cuda-runtime-cu11 (>=11.8)", "nvidia-cudnn-cu11 (>=8.8)", "nvidia-cufft-cu11 (>=10.9)", "nvidia-cusolver-cu11 (>=11.4)", "nvidia-cusparse-cu11 (>=11.7)"]
-cuda12-pip = ["nvidia-cublas-cu12", "nvidia-cuda-cupti-cu12", "nvidia-cuda-nvcc-cu12", "nvidia-cuda-runtime-cu12", "nvidia-cudnn-cu12 (>=8.9)", "nvidia-cufft-cu12", "nvidia-cusolver-cu12", "nvidia-cusparse-cu12"]
-
-[package.source]
-type = "legacy"
-url = "https://storage.googleapis.com/jax-releases/jax_cuda_releases.html"
-reference = "jax-releases"
-
 [[package]]
 name = "ml-dtypes"
 version = "0.3.2"

as you can see the jax-releases explicit source has been removed expectedly. It will be fixed if I run poetry update jaxlib:

$ poetry update jaxlib
Updating dependencies
Resolving dependencies... (0.9s)

No dependencies to install or update

Writing lock file

$  git status
On branch main
nothing to commit, working tree clean

this is pretty annoying, and can lead to #3114.

[dimbleby]

Duplicate #8614 please close

[messense]

Note that I've tried to apply the fix for #8614 but it doesn't work for this.

[messense]

Just tried again with the poetry master branch, the issue persists.

@dimbleby I can close this one if someone re-opens #8614

[dimbleby]

well in that case it is duplicate #8328 instead

edit: probably not actually

[radoering]

#8835 which solved #8614 only handles a fresh lock (without --no-update). There, the DependencyCache makes sure that explicit sources are propagated to transitive dependencies. With a lockfile it's not so easy anymore because dependencies created from locked packages do not have the "explicit source" information so that explicit sources are not searched for the package.

I tried a naive fix in #8948 but that breaks other things (for example if the locked package references a source that does not exist anymore or if an explicit source has been changed in the pyproject.toml). I suppose we have to figure out if a locked package comes from a dependency with an explicit source. However, that's a tough nut to crack for multiple-constraints dependencies with several locked packages with different sources.

[radoering]

I think I have found a robust solution now. Feel free to try #8948. (It's just missing one more test.)

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.