Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor for libmamba v2 (WIP) #457

Open
wants to merge 126 commits into
base: main
Choose a base branch
from
Open

Refactor for libmamba v2 (WIP) #457

wants to merge 126 commits into from

Conversation

jaimergp
Copy link
Contributor

@jaimergp jaimergp commented Mar 26, 2024

Description

Adds compatibility for libmamba v2, which enables a few refactors (see below). Supersedes #414.

Progress:

  • Refactor index helpers
  • Refactor repoquery
  • Refactor solver
  • Refactor conflict reports
  • Pass all tests
  • Polish and clean up

Checklist - did you ...

  • Add a file to the news directory (using the template) for the next release's release notes?
  • Add / update necessary tests?
  • Add / update outdated documentation?

Summary of changes

  • Deprecate CONDA_LIBMAMBA_SOLVER_NO_CHANNELS_FROM_INSTALLED. No longer needed with v2. Closes CLS injects channels from installed packages #411
  • Added typing for all methods and functions
  • Added time_recorder decorator for metadata collection and solving loop
  • Removed state.BaseIndexHelper. We just have our index.IndexHelper now
  • The libmamba v1 pool (collection of repos, where a repo represents a loaded repodata.json) is now a database of RepoInfo objects.
  • Do use current_repodata.json if explicitly set in CLI
  • Move to Ruff for pre-commit linting & formatting
  • Logging from libsolv has a HUGE overhead now because it goes from C to C++ to Python to logging to stdout instead of C -> stdout. To enable libsolv verbosity, an extra environment variable CONDA_LIBMAMBA_SOLVER_DEBUG_LIBSOLV is required.
  • allow_uninstall was previously set to false (only true for conda remove), and we set it up for individual solver jobs involving updates or conflicts. With v2, we have individual control over what to "Keep" instead of drop. This required marking important updates as keepers instead. Otherwise they would be uninstalled.
  • Other changes in the test suite discussed in Amend test suite for compatibility with libmamba v2 conda#13784

@conda-bot conda-bot added the cla-signed [bot] added once the contributor has signed the CLA label Mar 26, 2024
@jaimergp
Copy link
Contributor Author

pre-commit.ci autofix

jaimergp added a commit that referenced this pull request Jul 12, 2024
* Port fixed test from #457

* simplify test

* add comment

* Update tests/test_solvers.py

Co-authored-by: Jannis Leidel <jannis@leidel.info>

---------

Co-authored-by: Jannis Leidel <jannis@leidel.info>
@jaimergp
Copy link
Contributor Author

We were debating whether to also add libmamba v1 compatibility in this PR, which would include copying the current code in main to a conda_libmamba_solver.v1 namespace and dispatch automatically from the plugin depending on the detected libmambapy version at runtime.

Sylvain also mentions that there's no need as long as conda allows old conda-libmamba-versions. Worst case, if we needed to release a bugfix, we could do 24.7.x patches til necessary.

That latter option seems simpler than the .v1 namespace, if you ask me, but I'll let this here in case someone has thoughts.

@jaimergp
Copy link
Contributor Author

jaimergp commented Jul 17, 2024

Funny, now that we patched conda-libmamba-solver to prevent libmambapy 2, we can't install it in CI :D This is going to require some hacks or rewriting parts of the CI...

Edit: Built a special conda-libmamba-solver build (off main) but with the libmambapy<2 constraints removed and added it to a label in my channel for now. Thanks to strict priority, this will be chosen before conda-forge's and then a solution will be found.

@conda-bot

This comment was marked as outdated.

@jaimergp jaimergp added the build::review trigger a build for this PR label Jul 19, 2024
@conda-bot conda-bot removed the build::review trigger a build for this PR label Jul 19, 2024
@conda-bot

This comment was marked as outdated.

@jaimergp jaimergp marked this pull request as ready for review July 19, 2024 08:34
@jaimergp
Copy link
Contributor Author

Since the canary workflow has a token issue for the uploads, I built it locally.

conda install -c jaimergp/label/conda-libmamba-solver_dev -c conda-forge/label/mamba_dev -c conda-forge conda-libmamba-solver

@conda-bot
Copy link
Contributor

conda-bot commented Jul 19, 2024

Review build status

The workflow for building and uploading a canary release for conda-libmamba-solver with the label conda-libmamba-solver-pr-457 in channel conda-canary was started by @jaimergp!

Once it's done, use this command to try it out in a new conda environment:

conda install -c conda-canary/label/conda-libmamba-solver-pr-457 conda-libmamba-solver

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed [bot] added once the contributor has signed the CLA
Projects
Status: 🆕 New
2 participants