Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: cache comparison key for choosing the next package to resolve #10275

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

radoering
Copy link
Member

@radoering radoering commented Mar 15, 2025

The comparison key for choosing the next dependency to be resolved from the list of unsatisfied dependencies is calculated again and again until the dependency is chosen and removed from the list. Especially for dependencies, which have many dependencies themselves, calculating the comparison key can take some time. Therefore, it makes sense to cache the comparison key of each dependency.

The time for locking #10195 goes down from about 24 seconds to 20 seconds with this PR.

Since the _get_min function was a private function of _choose_next, I had to move it for caching. I also updated the comments. (I did not change the logic.)

Summary by Sourcery

Cache the comparison key used to determine the next package to resolve, improving performance.

Enhancements:

  • Cache the comparison key for each dependency to avoid redundant calculations, especially for dependencies with many sub-dependencies.
  • Move the _get_min function to enable caching of the comparison key.

Copy link

sourcery-ai bot commented Mar 15, 2025

Reviewer's Guide by Sourcery

This pull request optimizes the dependency resolution process by caching the comparison key used to determine the next dependency to resolve. It introduces a Preference class and refactors the logic for calculating the comparison key into a dedicated function, which is then cached using functools.cache. This change improves performance by avoiding redundant calculations, especially for dependencies with many sub-dependencies.

No diagrams generated as the changes look simple and do not need a visual representation.

File-Level Changes

Change Details Files
Introduces a Preference class to encapsulate the criteria for choosing which dependency to solve first.
  • Adds a Preference class with constants DIRECT_ORIGIN, NO_CHOICE, USE_LATEST, LOCKED, and DEFAULT.
  • The Preference class is used to define the priority of dependencies during the resolution process.
src/poetry/mixology/version_solver.py
Caches the comparison key used for choosing the next dependency to resolve.
  • Introduces _get_comp_key to calculate the comparison key for a given dependency.
  • Uses functools.cache to cache the results of _get_comp_key in _get_comp_key_cached.
  • The _choose_next function now uses the cached comparison key _get_comp_key_cached to select the next dependency.
src/poetry/mixology/version_solver.py
Refactors the logic for determining the comparison key of a dependency.
  • The logic previously embedded within _choose_next is extracted into a separate _get_comp_key function.
  • The _get_comp_key function calculates a tuple consisting of preference, number of dependencies with upper bounds, a boolean indicating the presence of dependencies, and the number of available packages.
  • The function now returns a tuple of (preference, -num_deps_upper_bound, not has_deps, -num_packages) to serve as priority for choosing the next package to resolve.
src/poetry/mixology/version_solver.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on
    an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @radoering - I've reviewed your changes - here's some feedback:

Overall Comments:

  • It might be worth exploring if functools.lru_cache would be a better fit than functools.cache here, given that the number of unique dependencies might be quite large.
Here's what I looked at during the review
  • 🟢 General issues: all looks good
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟡 Complexity: 1 issue found
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@radoering radoering requested a review from a team March 15, 2025 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant