Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dependency Resolution Error Due to Channel Constraints #615

Closed
2 tasks done
flferretti opened this issue Jan 5, 2024 · 4 comments
Closed
2 tasks done

Dependency Resolution Error Due to Channel Constraints #615

flferretti opened this issue Jan 5, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@flferretti
Copy link

flferretti commented Jan 5, 2024 •
edited
Loading

Checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

Ref. ami-iit/jaxsim#63

pixi info:

$ pixi info         
pixi 0.10.0

Platform            : linux-64
Virtual packages    : __unix=0=0
                    : __linux=6.2.0=0
                    : __glibc=2.35=0
                    : __cuda=12.2=0
                    : __archspec=1=x86_64
Cache dir           : /home/flferretti/.cache/rattler/cache
Auth storage        : /home/flferretti/.rattler/rattler_auth_store.json

This results in:

flferretti@IITICUBLAP215:~/jaxsim/examples$ pixi add jaxlib=*=*cuda*
  × could not determine any available versions for jaxlib on linux-64.
  │ Either the package could not be found or version constraints on other
  │ dependencies result in a conflict.
  ╰─▶ Cannot solve the request because of: The following packages are
      incompatible
      |-- jaxlib * *cuda* cannot be installed because there are no viable
      options:
          |-- jaxlib 0.4.18 | 0.4.18 | 0.4.18 | 0.4.18 | 0.4.19 | 0.4.19 |
      0.4.19 | 0.4.19 | 0.4.20 | 0.4.20 | 0.4.20 | 0.4.23 | 0.4.23 | 0.4.23
      would require
              |-- cudatoolkit >=11.8,<12, which cannot be installed because
      there are no viable options:
                  |-- cudatoolkit 11.8.0 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudatoolkit 11.8.0 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudatoolkit 11.8.0 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
          |-- jaxlib 0.4.7 | 0.4.7 | 0.4.7 | 0.4.7 | 0.4.9 | 0.4.9 | 0.4.10
      | 0.4.10 | 0.4.10 | 0.4.10 | 0.4.11 | 0.4.11 | 0.4.12 | 0.4.12 | 0.4.12
      | 0.4.12 | 0.4.12 | 0.4.12 | 0.4.12 | 0.4.12 | 0.4.12 | 0.4.12 | 0.4.12
      | 0.4.14 | 0.4.14 | 0.4.14 | 0.4.14 | 0.4.14 | 0.4.14 | 0.4.14 | 0.4.14
      would require
              |-- cudnn >=8.8.0.121,<9.0a0, which cannot be installed because
      there are no viable options:
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
          |-- jaxlib 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 |
      0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14
      | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14
      | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14
      | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14 | 0.3.14
      | 0.3.14 | 0.3.14 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15
      | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15
      | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15
      | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15
      | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15 | 0.3.15
      | 0.3.15 | 0.3.15 | 0.3.20 | 0.3.20 | 0.3.20 | 0.3.20 | 0.3.20 | 0.3.20
      | 0.3.20 | 0.3.20 | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.22
      | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.22
      | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.22 | 0.3.24 | 0.3.24 | 0.3.24 | 0.3.24
      | 0.3.24 | 0.3.24 | 0.3.24 | 0.3.24 | 0.3.25 | 0.3.25 | 0.3.25 | 0.3.25
      | 0.3.25 | 0.3.25 | 0.3.25 | 0.3.25 | 0.3.25 | 0.3.25 | 0.3.25 | 0.3.25
      | 0.3.25 | 0.3.25 | 0.3.25 | 0.3.25 | 0.4.1 | 0.4.1 | 0.4.1 | 0.4.1 |
      0.4.1 | 0.4.1 | 0.4.1 | 0.4.1 | 0.4.2 | 0.4.2 | 0.4.2 | 0.4.2 | 0.4.2 |
      0.4.2 | 0.4.2 | 0.4.2 | 0.4.3 | 0.4.3 | 0.4.3 | 0.4.3 | 0.4.3 | 0.4.3 |
      0.4.3 | 0.4.4 | 0.4.4 | 0.4.4 | 0.4.6 | 0.4.6 | 0.4.6 | 0.4.6 | 0.4.6 |
      0.4.6 | 0.4.6 | 0.4.6 | 0.4.7 | 0.4.7 | 0.4.7 | 0.4.7 would require
              |-- cudnn >=8.4.1.50,<9.0a0, which cannot be installed because
      there are no viable options:
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.4.1.50 is excluded because due to strict channel
      priority not using this option from: 'https://conda.anaconda.org/conda-
      forge/'
          |-- jaxlib 0.3.10 | 0.3.10 | 0.3.10 | 0.3.10 | 0.3.10 | 0.3.10 |
      0.3.10 | 0.3.10 | 0.3.10 | 0.3.10 | 0.3.10 | 0.3.10 | 0.3.10 | 0.3.10 |
      0.3.10 | 0.3.10 would require
              |-- cudnn >=8.2.1.32,<9.0a0, which cannot be installed because
      there are no viable options:
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.8.0.121 is excluded because due to
      strict channel priority not using this option from: 'https://
      conda.anaconda.org/conda-forge/'
                  |-- cudnn 8.4.1.50 is excluded because due to strict channel
      priority not using this option from: 'https://conda.anaconda.org/conda-
      forge/'
                  |-- cudnn 8.4.0.27 is excluded because due to strict channel
      priority not using this option from: 'https://conda.anaconda.org/conda-
      forge/'
                  |-- cudnn 8.4.0.27 is excluded because due to strict channel
      priority not using this option from: 'https://conda.anaconda.org/conda-
      forge/'
                  |-- cudnn 8.3.2.44 is excluded because due to strict channel
      priority not using this option from: 'https://conda.anaconda.org/conda-
      forge/'
                  |-- cudnn 8.3.2.44 is excluded because due to strict channel
      priority not using this option from: 'https://conda.anaconda.org/conda-
      forge/'
                  |-- cudnn 8.2.1.32 is excluded because due to strict channel
      priority not using this option from: 'https://conda.anaconda.org/conda-
      forge/'
                  |-- cudnn 8.2.1.32 is excluded because due to strict channel
      priority not using this option from: 'https://conda.anaconda.org/conda-
      forge/'

with pixi.toml:

[project]
authors = ["Filippo Luca Ferretti <filippoluca.ferretti@outlook.com>"]
channels = ["nvidia", "conda-forge"]
description = "JAXsim PD controller notebook"
name = "JAXsim examples"
platforms = ["linux-64"]
version = "0.1.0"

[system-requirements]
cuda = "12"
unix = true

[tasks]

[dependencies]
jaxlib = "* *cuda*"

and with:

< channels = ["nvidia", "conda-forge"]
> channels = ["conda-forge", "nvidia"]

results in cudatoolkit being installed from conda-forge.

Issue description

The second TOML does not provide ptxas with cudatoolkit as it's being downloaded from conda-forge instead of nvidia channel, failing with jaxlib with GPU support.

The final version resulted in the following pixi.toml from https://github.com/flferretti/jaxsim/tree/example/notebook:

[project]
authors = ["Filippo Luca Ferretti <filippoluca.ferretti@outlook.com>"]
channels = ["conda-forge", "nvidia"]
description = "JAXsim PD controller notebook"
name = "JAXsim examples"
platforms = ["linux-64"]
version = "0.1.0"

[system-requirements]
cuda = "12"
unix = true

[tasks]
install = "python -m pip install git+https://github.com/ami-iit/jaxsim.git"
notebook = {cmd = "jupyter notebook PD_controller.ipynb", depends_on = ["install"]}

[dependencies]
cuda-nvcc = ">=12.3,<12.4"
ffmpeg = "*"
jax-dataclasses = "*"
jaxlib = "* *cuda*"
jaxlie = "*"
mediapy = "*"
mujoco = "*"
notebook = "*"
pip = "*"
pptree = "*"
rod = "*"
typing-extensions = "*"

and it was obtained following these steps:

  1. Starting with a blank project with only the nvidia channel
  2. pixi add cuda-nvcc
  3. Add conda-forge in channels as: channels = ["conda-forge", "nvidia"]
  4. pixi add "jaxlib=*=*cuda*"
  5. Add the rest of the packages

Expected behavior

I expect pixi to successfully install packages without encountering dependency resolution errors caused by channel constraints.
@flferretti flferretti added the bug Something isn't working label Jan 5, 2024
@ruben-arts
Copy link
Contributor

Hello @flferretti, thanks for the elaborate description.

We have chosen to make the priority always strict in rattler. This is due to a lot of silent issues we have seen where the solver would solve an environment, but it would result in packages that don't work together. If you really want the packages to come from different channels you need to specify the channels per package.

e.g. for you example that would be:

[project]
channels = ["conda-forge", "nvidia"]
#......
[dependencies]  
cuda-nvcc = {channel = "nvidia", version = ">=12.3,<12.4"}
cudnn = "*" #
cudatoolkit = {channel = "nvidia"}
#......

@flferretti
Copy link
Author

Thanks a lot @ruben-arts, I couldn't find anything related in the documentation. I'll give your solution a try

@ruben-arts
Copy link
Contributor

Your welcome! I was just working on that 😄 Checkout : #610

@ruben-arts
Copy link
Contributor

I hope that solved your issue, you can reopen this issue if it didn't!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ruben-arts @flferretti