Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-38689: Update _standardize_muli_index_columns to be faster for dataframe del… #823

Merged
merged 1 commit into from Apr 19, 2023

Conversation

erykoff
Copy link
Contributor

@erykoff erykoff commented Apr 19, 2023

…egates.

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes

@codecov
Copy link

codecov bot commented Apr 19, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01 ⚠️

Comparison is base (220d528) 87.75% compared to head (47f18df) 87.74%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #823      +/-   ##
==========================================
- Coverage   87.75%   87.74%   -0.01%     
==========================================
  Files         268      268              
  Lines       34932    34929       -3     
  Branches     7347     7347              
==========================================
- Hits        30653    30650       -3     
  Misses       3125     3125              
  Partials     1154     1154              
Impacted Files Coverage Δ
python/lsst/daf/butler/delegates/dataframe.py 95.55% <ø> (-0.19%) ⬇️
python/lsst/daf/butler/formatters/parquet.py 93.71% <100.00%> (-0.02%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really follow all of it but it does indeed seem better to use the columns you already knew about.

@erykoff
Copy link
Contributor Author

erykoff commented Apr 19, 2023

All of this code is about dealing with the way you can specify multi-index columns of dataframes as tuples, and how this looks slightly different for arrow tables and dataframes. But I didn't write the original code.
This update here is just making the utility function use a MultiIndex which saves unnecessary overhead when using it in the delegate where we have an explicit dataframe.

@erykoff erykoff merged commit 79aeedf into main Apr 19, 2023
13 checks passed
@erykoff erykoff deleted the tickets/DM-38689 branch April 19, 2023 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants