Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-43925: Add workarounds for pandas bugs when using non-floating-point masked columns. #998

Merged
merged 7 commits into from Apr 18, 2024

Conversation

erykoff
Copy link
Contributor

@erykoff erykoff commented Apr 17, 2024

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes
  • (if changing dimensions.yaml) make a copy of dimensions.yaml in configs/old_dimensions

Copy link

codecov bot commented Apr 17, 2024

Codecov Report

Attention: Patch coverage is 93.75000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 88.97%. Comparing base (1223572) to head (e620aae).

Files Patch % Lines
python/lsst/daf/butler/formatters/parquet.py 77.77% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #998      +/-   ##
==========================================
- Coverage   88.97%   88.97%   -0.01%     
==========================================
  Files         344      344              
  Lines       44199    44226      +27     
  Branches     9100     9106       +6     
==========================================
+ Hits        39327    39351      +24     
- Misses       3556     3557       +1     
- Partials     1316     1318       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@taranu taranu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me; take or leave the comments as you will.

"""
dataframe = arrow_to_pandas(astropy_to_arrow(astropy_table))

if isinstance(index, str):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this raise if it's not a string or None? If not, what about just if index:?

@@ -584,6 +588,23 @@ def testWriteMultiIndexDataFrameReadAsAstropyTable(self):
# This test simply checks that it's readable, but definitely not
# recommended.

@unittest.skipUnless(atable is not None, "Cannot test writing as astropy without astropy.")
def testWriteAstropyTableWithMaskedColsReadAsSingleIndexDataFrame(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is quite the mouthful... though I don't have a better suggestion.

@erykoff erykoff merged commit fd58971 into main Apr 18, 2024
18 checks passed
@erykoff erykoff deleted the tickets/DM-43925 branch April 18, 2024 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants