Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-35494 Allow execution butler creation to transfer datasets #261

Merged
merged 1 commit into from Jul 14, 2022
Merged

DM-35494 Allow execution butler creation to transfer datasets #261

merged 1 commit into from Jul 14, 2022

Conversation

PaulPrice
Copy link
Contributor

@PaulPrice PaulPrice commented Jul 8, 2022 •
edited

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes

@PaulPrice PaulPrice requested a review from timj July 8, 2022 02:48
@codecov
Copy link

codecov bot commented Jul 8, 2022 •
edited

Codecov Report

Merging #261 (ed36a42) into main (835a13a) will decrease coverage by 0.09%.
The diff coverage is 11.11%.

❗ Current head ed36a42 differs from pull request most recent head 1b9a457. Consider uploading reports for the commit 1b9a457 to get more accurate results

@@            Coverage Diff             @@
##             main     #261      +/-   ##
==========================================
- Coverage   72.32%   72.23%   -0.10%     
==========================================
  Files          60       60              
  Lines        6768     6774       +6     
  Branches     1393     1396       +3     
==========================================
- Hits         4895     4893       -2     
- Misses       1650     1657       +7     
- Partials      223      224       +1
Impacted Files Coverage Δ
python/lsst/pipe/base/executionButlerBuilder.py 17.74% <11.11%> (-0.91%) ⬇️
tests/test_quantumGraph.py 95.13% <0.00%> (-0.70%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 835a13a...1b9a457. Read the comment docs.

@PaulPrice PaulPrice force-pushed the tickets/DM-35494 branch 2 times, most recently from 977e798 to 654c361 Compare July 11, 2022 20:22
timj
timj approved these changes Jul 11, 2022
View reviewed changes
Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This is what I had in mind.

python/lsst/pipe/base/executionButlerBuilder.py Outdated
Location of the execution butler.
dirExists : `bool`
Does the ``outputLocation`` exist, and if so, should it be clobbered?
datastoreRoot : `str`, optional
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we should probably support ResourcePath here as well and then do str(datastoreRoot) below.

@timj timj mentioned this pull request Jul 12, 2022
Merged
2 tasks
@PaulPrice PaulPrice force-pushed the tickets/DM-35494 branch 2 times, most recently from 1428290 to ed36a42 Compare July 14, 2022 18:07
@PaulPrice
executionButler: add option to transfer files
1b9a457 
In some instances, it is helpful/important to transfer files
into the execution butler (e.g., original butler datastore is
on a slow disk). Added an option to buildExecutionButler to
allow transfer of files into a new datastore.
@PaulPrice PaulPrice merged commit 8384841 into main Jul 14, 2022
@PaulPrice PaulPrice deleted the tickets/DM-35494 branch July 14, 2022 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@timj timj timj approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@PaulPrice @timj