Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump lockfiles | Force user-installed conda during conda-lock #1555

Merged
merged 4 commits into from
Jul 12, 2023

Conversation

abejgonzalez
Copy link
Contributor

@abejgonzalez abejgonzalez commented Jul 10, 2023

Testing the following theory:

conda-lock under the hood uses conda to solve some extra dependencies that are machine-specific in addition to the pre-computed lockfile dependencies. When it does this it uses a non-optimized version (i.e. without libmamba solver) of conda which is very slow for many dependencies. This PR forces conda-lock to use the conda that the user has installed (by doing which conda and passing that).

This also bumps the lockfiles, makes the default instructions include installing libmamba, and uses the most recent conda version in CI.

Related PRs / Issues:

Type of change:

  • Bug fix
  • New feature
  • Other enhancement

Impact:

  • RTL change
  • Software change (RISC-V software)
  • Build system change
  • Other

Contributor Checklist:

  • Did you set main as the base branch?
  • Is this PR's title suitable for inclusion in the changelog and have you added a changelog:<topic> label?
  • Did you state the type-of-change/impact?
  • Did you delete any extraneous prints/debugging code?
  • Did you mark the PR with a changelog: label?
  • (If applicable) Did you update the conda .conda-lock.yml file if you updated the conda requirements file?
  • (If applicable) Did you add documentation for the feature?
  • (If applicable) Did you add a test demonstrating the PR?
  • (If applicable) Did you mark the PR as Please Backport?

@abejgonzalez
Copy link
Contributor Author

Tested on two machines:

Machine 1 (fresh conda install):

  • With fix = ~1m30s to finish conda setup

Machine 2 (conda already installed and setup):

  • With fix = 1m29s to finish conda setup
  • Without fix = ~15m (14m39s)

So now build-setup.sh should go from O(30m) to O(15m) - really on our machines down to about ~10m

Copy link
Contributor

@jerryz123 jerryz123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@abejgonzalez abejgonzalez changed the title Bump lockfiles | Force user-installed conda during conda-lock [ci skip] Bump lockfiles | Force user-installed conda during conda-lock Jul 11, 2023
@abejgonzalez
Copy link
Contributor Author

@Mergifyio rebase main

@mergify
Copy link

mergify bot commented Jul 12, 2023

rebase main

✅ Branch has been successfully rebased

@abejgonzalez abejgonzalez force-pushed the faster-conda-lockfile-install branch from ba830bb to 1641c8c Compare July 12, 2023 00:21
@abejgonzalez abejgonzalez merged commit 07fc230 into main Jul 12, 2023
53 checks passed
@jerryz123 jerryz123 deleted the faster-conda-lockfile-install branch August 22, 2023 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants