Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added CustomPackageOrder #1709

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

isohedronpipeline
Copy link
Contributor

@isohedronpipeline isohedronpipeline commented Apr 6, 2024

This PR carries on the work by @pmolodo (#355) to add a custom package order. This is essentially a generalization of the version_split package order, which provides better functionality for splitting versions within specified ranges, instead of only allowing a single split, as version_split does. This makes the package orderer useful outside of the extremely narrow band of intended uses that the version_split package order was intended to solve.

CustomPackageOrder

This adds support to explicitly specify the way versions should resolve, while still providing flexibility for ranges.
For example consider the following package houdini:

versions = [
    "18.5.499",
    "18.5.625",
    "19.0.350",
    "19.0.720",
    "19.5.632",
    "19.5.679",
    "20.0.413",
    "20.0.528",
]

And consider the following optional plugin plugin that has variants (all latest versions of Houdini):

variants = [
    ["houdini-18.5.625"],
    ["houdini-19.0.720"],
    ["houdini-19.5.679"],
    ["houdini-20.0.528"],
]

But let's consider that the latest version of Houdini is not well tested and we do not know if our pipeline will behave properly with these latest versions. On top of that, consider that we want houdini-19.5 to be the default.
Examples:

  • rez env houdini resolves to houdini-19.5.632 (not latest)
  • rez env houdini plugin resolves to houdini-19.5.679 (19.5 version that is compatible with plugin)
  • rez env houdini-20 resolves to houdini-20.0.413 (not latest 20)
  • rez env houdini plugin resolves to houdini-20.0.528 (20 version that is compatible with plugin)

This is hard to achieve without explicit presets, which makes preference of second order dependencies very difficult to resolve without taking advantage of the graph evaluation of rez. More information here: https://docs.google.com/document/d/1zy3ydSpHWgvatFS--L2PK7AwwOaowr1sdI9B_2eNtFc/edit?usp=sharing

We can now achieve that with the CustomPackageOrder:

package_orderers = [
    {
       "type": "custom",
       "packages": {
           "python": ["19.5.632", "19.5", "20.0.413"],
       }
    }
]

In the above package orderer, we see that we prefer 19.5.632 if no other version constraints are put on it. If 19.5.632 is not suitable (as in the case of plugin not having a requirement match) then we prefer the latest version that fits the VersionRange of 19.5. If that is not suitable, then we prefer 20.0.413 to cause houdini-20 to resolve to a less-than-latest version of Houdini 20.

Lastly, the CustomPackageOrder may take an argument to determine how versions within those version ranges are sorted, e.g. with pypa sorting.

Signed-off-by: Ben Andersen <ben@isohedron.com.au>
@isohedronpipeline isohedronpipeline requested a review from a team as a code owner April 6, 2024 11:30
Copy link

codecov bot commented Apr 6, 2024

Codecov Report

Attention: Patch coverage is 76.19048% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 58.33%. Comparing base (3c75a19) to head (0af2985).
Report is 2 commits behind head on main.

Files Patch % Lines
src/rez/package_order.py 76.19% 9 Missing and 6 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1709      +/-   ##
==========================================
+ Coverage   58.25%   58.33%   +0.08%     
==========================================
  Files         126      126              
  Lines       17157    17220      +63     
  Branches     3504     3520      +16     
==========================================
+ Hits         9995    10046      +51     
- Misses       6496     6503       +7     
- Partials      666      671       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Ben Andersen <ben@isohedron.com.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant