Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-32407: Fix YAML serialization of empty Timespans #590

Merged
merged 2 commits into from Nov 3, 2021

Conversation

andy-slac
Copy link
Contributor

Add special YAML representer and constructor for the _SpecialTimespanBound
class. Extend unit test with a trivial case for an empty Timespan.

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes

@codecov
Copy link

codecov bot commented Nov 2, 2021

Codecov Report

Merging #590 (66f3af7) into master (ca54b62) will decrease coverage by 0.00%.
The diff coverage is 83.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #590      +/-   ##
==========================================
- Coverage   83.52%   83.52%   -0.01%     
==========================================
  Files         241      241              
  Lines       30233    30251      +18     
  Branches     4511     4515       +4     
==========================================
+ Hits        25253    25267      +14     
- Misses       3786     3788       +2     
- Partials     1194     1196       +2     
Impacted Files Coverage Δ
python/lsst/daf/butler/transfers/_yaml.py 88.30% <33.33%> (-1.05%) ⬇️
python/lsst/daf/butler/core/timespan.py 82.80% <88.23%> (+0.34%) ⬆️
tests/test_datastore.py 99.61% <100.00%> (-0.01%) ⬇️
tests/test_simpleButler.py 97.76% <100.00%> (+0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca54b62...66f3af7. Read the comment docs.

Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay. One minor thought on tag naming.



# YAML tag name for _SpecialTimespanBound
_SpecialTimespanBound_yaml_tag = "!lsst.daf.butler._SpecialTimespanBound"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do wonder if we want to have this tag burned into the YAML rather than the notion of EMPTY since the docstring for this class says we should always access it as TimeSpan.EMPTY. Maybe @TallJimbo has thoughts.

Copy link
Member

@TallJimbo TallJimbo Nov 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are allowed to make it whatever string we want, then yes, lsst.daf.butler.Timespan.EMPTY would be preferable.

If it's not difficult to override the YAML serialization to this extent, it'd arguably be even better to map the entire Timespan to a sentinal string value like "EMPTY", since it's really a property of the entire timespan rather than just one bound, but if it's a lot harder it's not worth the effort.

Do we know what the YAML looks like with this solution?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The format before/after this change is on the JIRA here: https://jira.lsstcorp.org/browse/DM-32407?focusedCommentId=340215&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-340215

I was not sure that _SpecialTimespanBound is never going to have any other enum values for it, so I decided to keep it open for possible extensions. I was also thinking about replacing whole empty Timespan with a special YAML key, but this would mean adding more knowledge about Timespan details to YamlRepoExportBackend/YamlRepoImportBackend. Right now these two classes only know that Timespan has begin/end timestamps and it works OK for all cases so far. Making YAML representation more complicated adds more logic to those two classes. Alternatively we can define a special YAML representer for Timespan class itself to encapsulate that logic where it belongs, that adds YAML dependency to Timespan.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a YAML representer to Timespan does feel like the right solution.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'll try something quick today on this ticket.

Add special YAML representer and constructor for the Timespan class.
Extend unit test with a trivial case for an empty Timespan.
Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks for reworking it.

@andy-slac andy-slac merged commit 1393936 into master Nov 3, 2021
@andy-slac andy-slac deleted the tickets/DM-32407 branch November 3, 2021 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants