Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-40381: Improve mock support and fix bugs in init-input/output handling in QBB #368

Merged
merged 6 commits into from Aug 18, 2023

Conversation

TallJimbo
Copy link
Member

@TallJimbo TallJimbo commented Aug 16, 2023

Depends on lsst/daf_butler#879

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes

@codecov
Copy link

codecov bot commented Aug 16, 2023

Codecov Report

Patch coverage: 23.27% and project coverage change: -0.71% ⚠️

Comparison is base (bb42ac7) 84.11% compared to head (fd98449) 83.40%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #368      +/-   ##
==========================================
- Coverage   84.11%   83.40%   -0.71%     
==========================================
  Files          77       77              
  Lines        9082     9173      +91     
  Branches     1740     1579     -161     
==========================================
+ Hits         7639     7651      +12     
- Misses       1158     1232      +74     
- Partials      285      290       +5     
Files Changed Coverage Δ
...ython/lsst/pipe/base/tests/mocks/_pipeline_task.py 40.77% <9.37%> (-11.52%) ⬇️
python/lsst/pipe/base/graphBuilder.py 60.72% <16.66%> (-0.82%) ⬇️
...ython/lsst/pipe/base/tests/mocks/_storage_class.py 58.47% <47.50%> (-3.23%) ⬇️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

This will give MockDataset more flexibility as to which DatasetRef
attributes can be `None` and free us from using
SerializedDataCoordinate, which is just less convenient than a plane
dict in this context.
Not implementing adjustQuantum was an oversight in the mock
reimplementation of DM-38952.  The other hooks are recent additions
and aren't yet used, but implementing them now saves work on DM-38498.
Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The barren wasteland that is test coverage on this PR really makes me think that at some point we need to add coverage recording for middleware to ci_middleware so we can get some idea of what our coverage really is.

python/lsst/pipe/base/graphBuilder.py Outdated Show resolved Hide resolved
@TallJimbo
Copy link
Member Author

TallJimbo commented Aug 17, 2023

The barren wasteland that is test coverage on this PR really makes me think that at some point we need to add coverage recording for middleware to ci_middleware so we can get some idea of what our coverage really is.

ci_middleware is already recording all of the coverage it's providing when it's run, but it's a pain to use, because you have to run it locally (and be careful about cleaning up coverage outputs from previous runs) and then do coverage combine and coverage report (or set up editor integration) yourself. Since it doesn't run in GitHub Actions, I think we're stuck with Jenkins as our only option for coverage reporting in a friendlier, more automated way.

When all of a task's quanta are skipped (generally because they
already ran successfully) in QG generation, it's cleanest to drop that
task from the QG entirely.  This also includes redefining all of that
task's initOutputs that were pipeline-level initIntermediates as
overall initInputs.
@TallJimbo TallJimbo merged commit 33ad370 into main Aug 18, 2023
12 of 14 checks passed
@TallJimbo TallJimbo deleted the tickets/DM-40381 branch August 18, 2023 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants