Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix eps_l1b reader Delayed usage causing docs failures #2700

Merged
merged 3 commits into from
Dec 18, 2023

Conversation

djhoese
Copy link
Member

@djhoese djhoese commented Dec 18, 2023

This was discussed on slack and first seen failing in #2699. Readthedocs builds were failing with a new version of dask. Sphinx apidocs were trying to analyze if a method was a staticmethod and failing with:

  File "/home/docs/checkouts/readthedocs.org/user_builds/satpy/conda/2699/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 2159, in import_object
    inspect.isstaticmethod(obj, cls=self.parent, name=self.object_name)):
  File "/home/docs/checkouts/readthedocs.org/user_builds/satpy/conda/2699/lib/python3.10/site-packages/sphinx/util/inspect.py", line 214, in isstaticmethod
    if meth:
  File "/home/docs/checkouts/readthedocs.org/user_builds/satpy/conda/2699/lib/python3.10/site-packages/dask/delayed.py", line 645, in __bool__
    raise TypeError("Truth of Delayed objects is not supported")
TypeError: Truth of Delayed objects is not supported

I was able to reproduce it locally with the newest dask and discovered that it was the eps_l1b.py reader. This reader declared a regular class method as a Delayed object. This is generally considered bad practice as it makes the Delayed object non-serializable in distributed environments/workflows. Technically making it a staticmethod should have fixed it but when I tried it locally I couldn't make it work. I instead decided to make it a global function as there was no real reason to tie it to the class. This seems to fix it.

I then checked trollimage which does use delayed staticmethods and it doesn't fail, but that's because the staticmethod isn't made delayed until it is called. For example res = dask.delayed(self._some_static_method)(a, b, c). This type of calling seems fine.

  • Closes #xxxx
  • Tests added
  • Fully documented
  • Add your name to AUTHORS.md if not there already

Delayed objects should always be staticmethods or global functions so they can be easily serialized.
@djhoese djhoese added bug documentation component:readers cleanup Code cleanup but otherwise no change in functionality labels Dec 18, 2023
@djhoese djhoese self-assigned this Dec 18, 2023
Copy link

codecov bot commented Dec 18, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (e638f9d) 95.39% compared to head (9707e70) 95.39%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2700   +/-   ##
=======================================
  Coverage   95.39%   95.39%           
=======================================
  Files         371      371           
  Lines       52690    52690           
=======================================
  Hits        50263    50263           
  Misses       2427     2427           
Flag Coverage Δ
behaviourtests 4.16% <0.00%> (ø)
unittests 96.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@coveralls
Copy link

Pull Request Test Coverage Report for Build 7250373855

  • 5 of 5 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall first build on bugfix-delayed-epsl1b at 95.955%

Totals Coverage Status
Change from base Build 7250054591: 96.0%
Covered Lines: 50389
Relevant Lines: 52513

💛 - Coveralls

Copy link
Member

@mraspaud mraspaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the fix!

@mraspaud mraspaud merged commit 989d9f0 into pytroll:main Dec 18, 2023
18 of 19 checks passed
@djhoese djhoese deleted the bugfix-delayed-epsl1b branch December 18, 2023 17:53
@mraspaud mraspaud changed the title Fix eps_l1b reader Delayed usage causing docs failures Fix eps_l1b reader Delayed usage causing docs failures Feb 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug cleanup Code cleanup but otherwise no change in functionality component:readers documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants