Skip to content

pixi.lock can silently drift from ci/versions.yml's declared CUDA toolchain version #2770

Description

@juenglin

Description

While relocking cuda_bindings/pixi.lock for an unrelated Cython version bump, we discovered that the default pixi environment (which has an unconstrained cuda-version = "*" per package.host-dependencies) had resolved to CUDA 12.9 packages (cuda-cudart, cuda-nvvm, libcufile, libnvjitlink, etc.) even though ci/versions.yml has declared cuda.build.version: "13.3.0" since 2026-05-26.

The lock was last regenerated on 2026-08-09 (#2532), three months after that version bump, and still locked to 12.9 -- most likely because full CUDA 13.3 conda-forge builds weren't yet available at relock time. Running pixi update cython (completely unrelated to CUDA versions) incidentally triggered a full re-solve of the default solve-group and picked up CUDA 13.3, producing an ~800-line unrelated diff bundled with what should have been a small, targeted lockfile change.

Why this matters

  • ci/tools/check_pixi_cuda_version.py (run in pre-commit) only validates the cuda-version text in pixi.toml against ci/versions.yml; it does not check what versions the corresponding pixi.lock actually resolved. This class of drift is invisible to both CI and pre-commit until someone runs pixi update/pixi lock for an unrelated reason and notices an unexpectedly large diff.
  • Any future targeted lockfile update (e.g. bumping a single test dependency) risks silently smuggling in a large, unrelated CUDA-toolchain bump for the unconstrained default environment. Conversely, a contributor who notices this and tries to avoid the noise by hand-editing the lock risks leaving it stale even longer.

Request

Please design and implement a robust, long-term solution that prevents pixi.lock from drifting away from ci/versions.yml's declared CUDA toolchain version -- or at minimum makes such drift immediately visible instead of silently accumulating. Possible directions (not prescriptive):

  • Extend check_pixi_cuda_version.py (or add a new check) to also parse the relevant pixi.lock entries for the unconstrained default environment and assert they match ci/versions.yml's build.version.
  • Add a scheduled CI job that relocks (or dry-run checks staleness) and fails/opens an issue if the lock has drifted from ci/versions.yml.
  • Reconsider whether the default environment's cuda-version = "*" should instead be pinned to explicitly track ci/versions.yml's build.version, removing the ambiguity at the source.

Where this was found

  • cuda_bindings/pixi.lock and cuda_core/pixi.lock both share the package.host-dependencies: cuda-version = "*" pattern for the unconstrained default environment and are susceptible.
  • cuda_pathfinder/pixi.lock does not have this pattern and was unaffected.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CI/CDCI/CD infrastructuredependenciesPull requests that update a dependency filepackagingAnything related to wheels or Conda packages

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions