Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sar-c reader optimization #1580

Merged
merged 19 commits into from Mar 4, 2021
Merged

Conversation

mraspaud
Copy link
Member

@mraspaud mraspaud commented Mar 2, 2021

This PR optimizes the Sar-C reader, among other things by using rioxarray for reading the data.

  • Tests added
  • Fully documented

A simple benchmarking on my laptop reports a reduction of 33% of the memory usage with this PR. The reading is also about 10% faster.

@mraspaud mraspaud self-assigned this Mar 2, 2021
@mraspaud mraspaud added component:readers enhancement code enhancements, features, improvements labels Mar 2, 2021
@codecov
Copy link

codecov bot commented Mar 2, 2021

Codecov Report

Merging #1580 (85f10cf) into master (5504ea4) will increase coverage by 0.25%.
The diff coverage is 96.01%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1580      +/-   ##
==========================================
+ Coverage   92.54%   92.79%   +0.25%     
==========================================
  Files         251      251              
  Lines       36801    37137     +336     
==========================================
+ Hits        34058    34463     +405     
+ Misses       2743     2674      -69     
Flag Coverage Δ
behaviourtests 4.45% <0.00%> (-0.05%) ⬇️
unittests 92.93% <96.01%> (+0.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
satpy/readers/sar_c_safe.py 86.88% <88.58%> (+34.11%) ⬆️
satpy/readers/utils.py 90.27% <93.10%> (+0.09%) ⬆️
satpy/readers/nwcsaf_nc.py 61.27% <94.73%> (+0.86%) ⬆️
satpy/readers/satpy_cf_nc.py 97.56% <95.23%> (-1.04%) ⬇️
satpy/readers/ahi_hsd.py 97.29% <100.00%> (+0.43%) ⬆️
satpy/readers/yaml_reader.py 95.87% <100.00%> (+0.52%) ⬆️
satpy/scene.py 91.46% <100.00%> (-0.19%) ⬇️
satpy/tests/reader_tests/test_ahi_hsd.py 100.00% <100.00%> (ø)
satpy/tests/reader_tests/test_sar_c_safe.py 100.00% <100.00%> (ø)
satpy/tests/reader_tests/test_satpy_cf_nc.py 100.00% <100.00%> (ø)
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5504ea4...b6ee8d0. Read the comment docs.

@ghost
Copy link

ghost commented Mar 2, 2021

DeepCode failed to analyze this pull request

Something went wrong despite trying multiple times, sorry about that.
Please comment this pull request with "Retry DeepCode" to manually retry, or contact us so that a human can look into the issue.

@mraspaud mraspaud marked this pull request as ready for review March 3, 2021 15:48
@mraspaud mraspaud requested a review from djhoese as a code owner March 3, 2021 15:48
Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after skimming

@mraspaud mraspaud added this to the v0.26.0 milestone Mar 3, 2021
@mraspaud mraspaud merged commit 2ae83dc into pytroll:master Mar 4, 2021
@mraspaud mraspaud deleted the feature-sar-optimization branch March 4, 2021 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:readers enhancement code enhancements, features, improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants