Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-38614: more fixes for storage class conversion support #316

Merged
merged 2 commits into from Apr 12, 2023

Conversation

TallJimbo
Copy link
Member

@TallJimbo TallJimbo commented Apr 10, 2023

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes

@codecov
Copy link

codecov bot commented Apr 10, 2023

Codecov Report

Patch coverage: 12.50% and project coverage change: -0.12 ⚠️

Comparison is base (1ecb914) 80.67% compared to head (d76dc2a) 80.56%.

❗ Current head d76dc2a differs from pull request most recent head c4b68be. Consider uploading reports for the commit c4b68be to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #316      +/-   ##
==========================================
- Coverage   80.67%   80.56%   -0.12%     
==========================================
  Files          57       57              
  Lines        6380     6385       +5     
  Branches     1303     1304       +1     
==========================================
- Hits         5147     5144       -3     
- Misses        985      993       +8     
  Partials      248      248              
Impacted Files Coverage Δ
python/lsst/pipe/base/graphBuilder.py 64.68% <0.00%> (-0.63%) ⬇️
python/lsst/pipe/base/executionButlerBuilder.py 15.60% <17.64%> (-0.46%) ⬇️

... and 4 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

If the storage classes are bidirectionally convertible, we just emit
a debug message, since this shouldn't cause any trouble.  If they're
only one-way convertible, it's now an info message rather than a
warning, because it's probably fine and the only problem is that the
code doing the logging is embarrassed that it can't really tell.
Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

existing.storageClass_name,
)
_dict[datasetType] = combined[existing]
elif convertible_to_existing or convertible_from_existing:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code in the graph builder tries to work out whether this is solely an output or solely an input or is an intermediate and so doesn't always require that both directions work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you saying I should drop this message entirely because by the time we get here we've always checked things more thoroughly? I know we've got various checks in various places, but wasn't sure about the flow, so here I just wanted to preserve the original message while making it less scary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I was mainly commenting on the comment saying that we can't tell whether this is an error or not. I'm not sure where in the graph builder this is occurring so it might be that the test that compare outputs to intermediates to inputs happen later. I do remember trying to be careful about only complaining (and failing to build the graph) if you try to use something as an intermediate that won't convert in both directions (it's possible that is buggy of course).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok, all I meant with the comment was that this code right here doesn't have the information it needs to make that determination. Other parts of the graph builder definitely can.

@TallJimbo TallJimbo merged commit 3523509 into main Apr 12, 2023
8 of 10 checks passed
@TallJimbo TallJimbo deleted the tickets/DM-38614 branch April 12, 2023 02:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants