Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix interp function in rayleigh correction to be serializable #64

Merged
merged 1 commit into from
Apr 9, 2019

Conversation

djhoese
Copy link
Member

@djhoese djhoese commented Feb 11, 2019

When using dask in a distributed way, functions passed to map_blocks and delayed can't be internal functions (functions defined inside other functions) because they are not serializable/accessible in a way that dask/multiprocessing can use. This PR refactors the rayleigh correction to not use an internal function. I'm not really sure how to write a test for this.

  • Tests added
  • Tests passed
  • Passes git diff origin/master **/*py | flake8 --diff

@djhoese djhoese added the bug label Feb 11, 2019
@djhoese djhoese self-assigned this Feb 11, 2019
@coveralls
Copy link

Coverage Status

Coverage remained the same at 73.939% when pulling f0eeffc on djhoese:bugfix-serialized-rayleigh into 76627fa on pytroll:master.

Copy link
Collaborator

@adybbroe adybbroe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@djhoese Did you make a "system test" using it with satpy to generate a atm/rayleigh corrected RGB?

@djhoese
Copy link
Member Author

djhoese commented Apr 9, 2019

@adybbroe Yes. I was generating true color images on a cluster. It only moves the function so I'd be surprised if this breaks anything else.

@adybbroe adybbroe merged commit d9e68f8 into pytroll:master Apr 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants