Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add utility functions for generating GeoViews plots #541

Closed
wants to merge 50 commits into from

Conversation

BENR0
Copy link
Collaborator

@BENR0 BENR0 commented Dec 9, 2018

Adds to_geoviews Scene method as show during the Pytroll Developer Week in Darmstadt.
Additionally a new blending function for the MultiScene is added to generate a "timeseries" Scene which in turn can also be used with the geoviews method.

Some documentation is also added. If the data "problem" can be solved Ican also add some Jupyter Notebook examples.

This too can be considered as a work in progress. It's not extensively tested and probably could be improved a lot but maybe is a nice addition to the show method when working in a Notebook.

- Added wishlist to Scene arguments to be able to specify desired
  datasets prior to loading which is important when using multiscene
- Added printing of available dataset ids when no datasets are loaded
  yet
@mraspaud mraspaud added this to In progress in Pytroll Contribution Week at Eumetsat via automation Dec 10, 2018
@djhoese djhoese changed the title Geoviews Add utility functions for generating GeoViews plots Dec 10, 2018
@djhoese
Copy link
Member

djhoese commented Dec 10, 2018

I may need to open a separate issue for the dask display (if I haven't already). I did some hacky features like this in a notebook I have locally but it was extremely slow. It used an HTML table where each table cell was the exact size of the array chunk that it represented. I then made a custom class with a _repr_html_ that also had a numpy-compatible __setitem__ interface so it could be used as a destination with dask.array.store. The idea was that the user would create this custom object with the source data and "display" it. Inside the _repr_html_ it would generate the HTML table and start the computation where __setitem__ was called for each array chunk as it was finished. The __setitem__ would convert the array in to an RGB PNG image and add that image to the HTML table. It was extremely hacky and the conversion to the PNG image was really slow for however I was doing it at the time.

@BENR0
Copy link
Collaborator Author

BENR0 commented Dec 10, 2018

Well that really sounds hacky :-). I did a little research and maybe it is possible using holoviews.streams Pipes but I am not sure. I asked the folks at pyViz but have no answer yet.

@djhoese
Copy link
Member

djhoese commented Dec 10, 2018

Yeah I talked to the ipywidgets folks about this about 6 months ago. They had different packages that I could try out (ipyleaflet, etc). I am also the maintainer of the vispy project which has a notebook widget for doing webgl displays but this may be too complicated. @mraspaud and I brought this up again on slack last week after ipywidgets (I think) announced a new streaming API for streaming video. I'm not sure what would need to go in to that to make image data (a numpy array) look like a stream.

In the end it would be cool if we could have two types of displays:

  1. a single image-like rectangle that gets filled in as chunks get computed
  2. a map-like widget (ipyleaflet, geoviews, etc) with coastlines and pan-zoom capabilities that had the individual chunks appear as computed. Same as 1, but on a map.

satpy/scene.py Outdated Show resolved Hide resolved
satpy/scene.py Show resolved Hide resolved
satpy/scene.py Outdated Show resolved Hide resolved
satpy/scene.py Outdated Show resolved Hide resolved
satpy/scene.py Show resolved Hide resolved
satpy/scene.py Show resolved Hide resolved
satpy/scene.py Show resolved Hide resolved
satpy/utils.py Outdated Show resolved Hide resolved
satpy/utils.py Outdated Show resolved Hide resolved
satpy/utils.py Outdated Show resolved Hide resolved
satpy/scene.py Outdated Show resolved Hide resolved
from satpy.node import DependencyTree
from satpy.readers import DatasetDict, load_readers
from satpy.resample import (resample_dataset,
prepare_resampler, get_area_def)
from satpy.writers import load_writer
from pyresample.geometry import AreaDefinition, BaseDefinition
from pyresample.utils import proj4_str_to_dict
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F401 'pyresample.utils.proj4_str_to_dict' imported but unused

satpy/scene.py Outdated Show resolved Hide resolved
"""
try:
import geoviews as gv
from cartopy import crs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F401 'cartopy.crs' imported but unused

@@ -261,3 +261,4 @@ def atmospheric_path_length_correction(data, cos_zen, limit=88.):
corr = corr.where(cos_zen > limit).fillna(corr_lim)

return data * corr

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

W391 blank line at end of file

@djhoese
Copy link
Member

djhoese commented Dec 18, 2018

@BENR0 How goes the git hell with this one?

@BENR0
Copy link
Collaborator Author

BENR0 commented Dec 19, 2018

@djhoese unfortunately currently I am busy with preparations for a paper presentation so I don't find enough time to take care of the PRs.

I already fixed the superfluous imports which stickler came up with this is due to the change from handling the crs manually vs using the to_cartopy_crs method. The wishlist issue is a left over from the git hell ;-) which is also fixed. Just hasen't been pushed yet.

I guess there are still some "lose ends" which in the long run should be improved so this functionality fits in better (geoviews as well as the to_xarray_dataset method) but as you said as long as its only used with geoviews this should be fine for now.

So the things which are left to take care of from my point of view are:

  • check documentation again if any changes are needed due to the latest changes
  • prepare Jupyter notebook with examples (Probably I could just modify/extend the one I showed in Darmstadt). It would be nice to have some data somewhere online I could use for the notebook. You said you maybe have some data which could be used?

Apart from that maybe you have some more Ideas/ wishes for changes. In that case please let me know.

@djhoese
Copy link
Member

djhoese commented Jan 28, 2019

Closing since #567 replaces it.

@djhoese djhoese closed this Jan 28, 2019
Pytroll Contribution Week at Eumetsat automation moved this from In progress to Done Jan 28, 2019
@BENR0 BENR0 deleted the geoviews branch June 11, 2019 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:scene enhancement code enhancements, features, improvements PCW Pytroll Contributors' Week
Development

Successfully merging this pull request may close these issues.

None yet

4 participants