Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-32817: Switch from ButlerURI to ResourcePath #623

Merged
merged 12 commits into from Jan 6, 2022
Merged

Conversation

timj
Copy link
Member

@timj timj commented Dec 30, 2021

Straightforward replacement with a couple of extra clean up commits.

I will see if I can get ButlerURI to give some kind of deprecation warning.

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes

@timj timj requested a review from TallJimbo December 30, 2021 22:50
@codecov
Copy link

codecov bot commented Dec 30, 2021

Codecov Report

Merging #623 (aa58971) into main (df6aacb) will decrease coverage by 0.10%.
The diff coverage is 94.04%.

❗ Current head aa58971 differs from pull request most recent head 2e4d2bc. Consider uploading reports for the commit 2e4d2bc to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##             main     #623      +/-   ##
==========================================
- Coverage   84.18%   84.08%   -0.11%     
==========================================
  Files         237      237              
  Lines       30327    30014     -313     
  Branches     5015     4990      -25     
==========================================
- Hits        25532    25238     -294     
+ Misses       3652     3638      -14     
+ Partials     1143     1138       -5     
Impacted Files Coverage Δ
python/lsst/daf/butler/core/quantum.py 87.30% <ø> (ø)
python/lsst/daf/butler/registries/remote.py 0.00% <0.00%> (ø)
python/lsst/daf/butler/script/queryDatasets.py 86.36% <ø> (ø)
python/lsst/daf/butler/script/retrieveArtifacts.py 85.71% <ø> (ø)
tests/test_uri.py 99.00% <ø> (+1.52%) ⬆️
python/lsst/daf/butler/core/datastore.py 74.00% <50.00%> (ø)
python/lsst/daf/butler/registry/_config.py 80.00% <66.66%> (ø)
python/lsst/daf/butler/core/dimensions/_config.py 83.78% <80.00%> (ø)
python/lsst/daf/butler/core/config.py 91.94% <82.35%> (ø)
python/lsst/daf/butler/datastores/fileDatastore.py 80.11% <96.15%> (+0.02%) ⬆️
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df6aacb...2e4d2bc. Read the comment docs.

Copy link
Member

@TallJimbo TallJimbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every single one of my comments is just a point where we should be using the new ResourcePathExpression instead of something like Union[str, ResourcePath] (to pick up pathlib.Path now, and automatically pick up new types we could coerce in the future).

Most of those are patch commits, but note that I haven't tried to add the ResourcePathExpression imports those will need, and I also haven't tried to fix the many docstrings - the latter just because I don't have solid of a plan for what to change them to.

python/lsst/daf/butler/_butler.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/_butler.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/_butler.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/_butlerConfig.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/_butlerConfig.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/datastores/fileDatastore.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/datastores/fileDatastore.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/registries/remote.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/registries/sql.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/registry/_config.py Outdated Show resolved Hide resolved
This code is tested in ResourcePath unit tests. The remaining
tests demonstrate that the ButlerURI compatibility code
is working.
ButlerURI is still available for now.
numpydoc has no guidance for how to handle a long parameter type
line. Rearrange things slightly to move the type information
to the parameter documentation instead.

See numpy/numpydoc#87
@timj
Copy link
Member Author

timj commented Jan 5, 2022

I can't work out how to attach a deprecation message to ButlerURI. The problem is that ResourcePath only works for ResourcePath or specific subclasses (trying to inject a ButlerURI class into the class hierarchy doesn't work because it triggers the subclass code in ResourcePath). Aliasing ResourcePath to ButlerURI works fine but adding a deprecation message ends up either adding a deprecation message to ResourcePath itself (because they are aliases) or adding a deprecation message to the ButlerURI constructor (so class methods no longer work).

@timj timj merged commit 679b36c into main Jan 6, 2022
@timj timj deleted the tickets/DM-32817 branch January 6, 2022 01:10
@TallJimbo
Copy link
Member

I can't think of a way to add a deprecation message, either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants