Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GEMPAK performance updates #3431

Merged
merged 2 commits into from
Mar 14, 2024
Merged

GEMPAK performance updates #3431

merged 2 commits into from
Mar 14, 2024

Conversation

nawendt
Copy link
Contributor

@nawendt nawendt commented Mar 11, 2024

Description Of Changes

Use set instead of list to hold possible elements within the GEMPAK file object. set has better performance when checking for element membership. This change also allowed for modifications to the looping done to unpack elements such that performance would be further increased.

@dopplershift
Copy link
Member

I'm ok with the change. Can you take a look and see if there is a major file type or scenario here that's not covered in our tests? It looks like 30% of the changes here don't get run by the tests.

tests/io/test_gempak.py Dismissed Show dismissed Hide dismissed
tests/io/test_gempak.py Dismissed Show dismissed Hide dismissed
tests/io/test_gempak.py Dismissed Show dismissed Hide dismissed
tests/io/test_gempak.py Dismissed Show dismissed Hide dismissed
tests/io/test_gempak.py Dismissed Show dismissed Hide dismissed
Use `set` instead of `list` to hold possible elements within the
GEMPAK file object. `set` has better performance when checking for
element membership. This change also allowed for modifications to the
looping done to unpack elements such that performace would be further
increased.

Adds climate surface files for testing. This also updates the surface
file type detection that was producing incorrect results. String data
was removed from unmerged sounding unpacking routine as this should not
appear in those file types per GEMPAK documentation.

Add additional tests to improve coverage.
Copy link
Member

@dopplershift dopplershift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work to improve the test coverage!

@dopplershift dopplershift added Type: Enhancement Enhancement to existing functionality Area: IO Pertains to reading data labels Mar 14, 2024
@dopplershift dopplershift merged commit 8f510ae into Unidata:main Mar 14, 2024
38 checks passed
@github-actions github-actions bot added this to the 1.7.0 milestone Mar 14, 2024
@nawendt nawendt deleted the gem-perf branch April 25, 2024 23:34
@dopplershift dopplershift modified the milestones: 1.7.0, 1.6.2 Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: IO Pertains to reading data Type: Enhancement Enhancement to existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants