Reimplement Path, Contours and Polygons plots #1991

philippjfr · 2017-10-10T02:56:15Z

Over the past few months I've slowly been laying the groundwork to support more powerful Path, Contours and Polygons elements and in this PR I am also finally exposing this new functionality in plotting classes. This PR finishes off some final issues with the internally facing API and will reimplements the plotting classes based on this new API.

New features:

Path, Contours and Polygons can now have any number of value dimensions associated with them which are revealed while hovering and which can be colored by.
Each geometry in a Contours or Polygons element can now have a different value associated with it, so you no longer have to create huge NdOverlays with one element per geometry.
Path now allows supplying varying value dimensions which are drawn as paths with continuously varying color.

Here is a demo of the new (and old) features.

This PR will address the following issues:

and will allow me to finish off my geopandas interface in geoviews:

Added GeoPandas data interface geoviews#88

jbednar · 2017-10-10T12:42:48Z

Fabulous! Was the county name meant to be visible in the hover?

philippjfr · 2017-10-10T12:44:36Z

Fabulous! Was the county name meant to be visible in the hover?

Yes, not quite done yet :-)

philippjfr · 2017-10-10T12:51:45Z

Actually nevermind, it does work, I just hadn't declared County as a dimension.

philippjfr · 2017-10-19T15:46:19Z

@jlstevens This is ready for review once you have time.

philippjfr · 2017-10-20T15:31:07Z

Additional user API is not part of this PR btw, the two main things I think we should still add are:

Allow groupby by value dimensions on Contours/Polygon (but not Path)
Allow selecting subpaths by index and value dimensions

jlstevens · 2017-10-20T16:02:22Z

It looks like you've found a few more things to improve in this PR. I'll review it once you think it is ready again...

philippjfr · 2017-10-20T16:41:04Z

It looks like you've found a few more things to improve in this PR. I'll review it once you think it is ready again...

It's ready, I just hadn't pushed stuff I'd done before.

jlstevens · 2017-10-21T20:01:09Z

doc/Tutorials/Introduction.ipynb

-    "green05 = levels.Overlay.Green[0.5]\n",
-    "green05 + green05.Channel + green05.Channel.Green.sample(y=0.0)"
+    "green05 = levels.Contours.Green\n",
+    "green05 + chans.Channel.Green + chans.Channel.Green.sample(y=0.0)"


Doesn't this have implications for backwards compatibility?

Guess it has some, it's no longer overlaid by default. I could have made it be overlaid again but I generally don't want to.

jlstevens · 2017-10-21T20:02:02Z

examples/gallery/demos/bokeh/texas_choropleth_example.ipynb

-    "                for name, xs, ys, rate in zip(county_names, county_xs, county_ys, county_rates)}\n",
-    "\n",
-    "choropleth = hv.NdOverlay(county_polys, kdims=['County'])"
+    "choropleth = hv.Polygons(counties, ['lons', 'lats'], [('detailed name', 'County'), 'Unemployment'])"


I assume the old code still works and the new approach is just more succinct/flexible/efficient.

jlstevens · 2017-10-21T20:03:48Z

examples/reference/elements/bokeh/Polygons.ipynb

-    "Polygons with values can be used to build heatmaps with arbitrary shapes."
+    "In order to efficiently represent the scalar values associated with each path the dictionary format is preferable since it can store the scalar values without expanding them into a whole column. Additionally it allows passing multiple columns as a single array by specifying the dimension names as a tuple.\n",
+    "\n",
+    "In this example we will createa list of random polygons each with an associated ``level`` value. Polygons will default to using the first value dimension as the ``color_index`` but for clarity we will define the ``color_index`` explicitly:"


createa -> create a

jlstevens · 2017-10-21T20:06:05Z

examples/user_guide/Exporting_and_Archiving.ipynb

@@ -155,7 +155,7 @@
   "source": [
    "%%opts Contours (linewidth=1.3) Image (cmap=\"gray\")\n",
    "cs = contours(penguins[:,:,'R'], levels=[0.10,0.80])\n",
-    "cs"
+    "penguins[:, :, 'R'] * cs"


The operation no longer just adds contours on top of the input and return it? That's fine as long as there is a switch to restore the old behavior. I also suspect we may have made this change a while back.

I see you changed the default - see comment below.

We would at least need to document this change of behavior in the changelog/release notes.

Right, may start requiring committers adding backwards compatibility changes to the CHANGELOG when they are made so we don't forget about them. Will do that here.

jlstevens · 2017-10-21T20:07:34Z

holoviews/core/data/interface.py

    gridded = False

+    # Denotes whether the interface expects multiple ragged arrays


Does it have to be arrays? Isn't it ragged 'data' (i.e multiple elements of different lengths)?

jlstevens · 2017-10-21T20:09:50Z

holoviews/operation/element.py

@@ -415,22 +415,17 @@ class contours(Operation):
    filled = param.Boolean(default=False, doc="""
        Whether to generate filled contours""")

-    overlaid = param.Boolean(default=True, doc="""
+    overlaid = param.Boolean(default=False, doc="""


Ok, this answers my earlier question.

jlstevens · 2017-10-21T20:11:04Z

holoviews/plotting/bokeh/path.py

+
+    color_index = param.ClassSelector(default=None, class_=(util.basestring, int),
+                                      allow_None=True, doc="""
+      Index of the dimension from which the color will the drawn""")


Would be nice to get the op proposal in soon...

jlstevens · 2017-10-21T20:12:26Z

tests/testoperation.py

+        contour = Contours([[(-0.5,  0.416667, 0.5), (-0.25, 0.5, 0.5)],
+                             [(0.25, 0.5, 0.5), (0.5, 0.45, 0.5)]],
+                            vdims=img.vdims)
+        print(contour.range(2))


Stray print.

Thanks I'll delete it.

jlstevens · 2017-10-21T20:23:00Z

holoviews/core/data/multipath.py

@@ -20,7 +20,9 @@ class MultiInterface(Interface):

    datatype = 'multitabular'

-    subtypes = ['dataframe', 'dictionary', 'array', 'dask']
+    subtypes = ['dictionary', 'dataframe', 'array', 'dask']


I think it makes sense to try the more general dictionary (i.e standard python literals) format first. Might be other implications I haven't figured out yet. Then again, MultiInterface is pretty new so it probably doesn't matter wrt backwards compatibility.

Right, the point here is that it doesn't accidentally expand your scalar values into a dataframe column for instance.

Makes sense.

jlstevens · 2017-10-21T20:24:23Z

Looks good. My main comments are:

I am assuming old code still works and that many of the updates to the notebooks just show the new, cleaner way of doing things.
We need to be clear about any changes to behavior, e.g the way the contour function no longer overlays thing.
The change in how you use attribute access in the introduction notebook seems like the most worrying change I noticed...
Reading notebook diffs is tricky - I can't tell if you only updated the element notebooks or added material to showcase all the new functionality. Github's notebook rendering seems off (a problem with the HTML maybe?):

If you haven't added material it would be good to see more examples (e.g of the split method).

Anyway, none of this is critical and once a few of my comments above are addressed (typos, stray print and other minor things), I'm happy to merge. The main other thing I would like is more examples for the docs.

philippjfr · 2017-10-22T11:27:33Z

I am assuming old code still works and that many of the updates to the notebooks just show the new, cleaner way of doing things.

Correct.

We need to be clear about any changes to behavior, e.g the way the contour function no longer overlays thing.

Yes, contours needs a note about backward compatibility.

The change in how you use attribute access in the introduction notebook seems like the most worrying change I noticed...

This is a result of two things 1) it's no longer overlaid by default and b) contours now returns a single Contours element not an NdOverlay of them so this falls under "noting contours backward compatibility changes".

Reading notebook diffs is tricky - I can't tell if you only updated the element notebooks or added material to showcase all the new functionality. Github's notebook rendering seems off (a problem with the HTML maybe?):

Not sure what you're referring to, looks fine to me. Anyway, I completely overhauled the element reference notebooks adding more material.

If you haven't added material it would be good to see more examples (e.g of the split method).

I'm working on a "Working with Geometries" user guide which will cover stuff like this. Will see how far I get with that, but I can add it in a later PR.

jlstevens · 2017-10-22T12:29:15Z

Ok great!

Should contours get a compatibility flag in config? If we find ourselves with a number of small backwards incompatibility issues, we could group them under a new config flag.

That doesn't need to be decided right away and I'll merge once you get rid of that stray print and fix that typo.

jlstevens · 2017-10-22T13:04:08Z

The tests have now passed except for on transient error (build restarted and still running).

Merging.

philippjfr added tag: API tag: backend: bokeh tag: component: data type: feature A major new feature tag: backend: mpl tag: component: plotting status: WIP labels Oct 10, 2017

philippjfr added 20 commits October 18, 2017 00:14

Added scalar validation to interfaces and Contours

1321c4b

Fixed MultiInterface dimension_values expand

8e7e682

Improved functionality of Path.split

68d3605

Reimplemented bokeh PathPlots based on new interface

25c1e22

Rewrote contours operation using new interface

d1c9c01

Small fixes to bokeh path plots

10b6dc8

Fixed Contours levels deprecation warning

8000134

Rewrote matplotlib path plots

a5e7411

Small fixes for mpl path plots

9adf459

Fixes for levels in contours operation

8db534a

Optimized datashader path splitting

17ea673

Added support for passing stacked arrays to DictInterface

115ac00

Overhauled docs with recent path improvements

ea5d64e

Small fixes for Path constructor

d0e2f2d

Fixes for contours operation

a69a622

Fixed Path colorbars

4d9f5ee

Improved contours operation

2cc0e42

Fixes for dictionary interface

8cdb15b

Fixed Path comparisons

c7493af

Small fixes for contours operation

712d04b

philippjfr added 2 commits October 19, 2017 19:29

Add class attribute to denote multi interfaces

1711099

Ensure splitting MultiInterface does not expand dictionary data

1b2602d

Improved Path.split when not using multi-interface

0a73811

Added static_datasource optimization for bokeh paths

bf217c5

jlstevens reviewed Oct 21, 2017

View reviewed changes

Addressed various review comments

6e9ec29

jlstevens merged commit 73038d7 into master Oct 22, 2017

This was referenced Oct 22, 2017

Improving Path/Curve for easy setting of color gradient along line #877

Closed

Add support for holes to Polygons #2017

Closed

philippjfr deleted the new_path_plots branch October 28, 2017 16:22

philippjfr added this to the v1.9 milestone Nov 2, 2017

pyup-bot mentioned this pull request Nov 3, 2017

Update holoviews to 1.9.0 zenotech/zCFDSuperBuild#104

Closed

pyup-bot mentioned this pull request Nov 13, 2017

Update holoviews to 1.9.1 zenotech/zCFDSuperBuild#120

Closed

pyup-bot mentioned this pull request Dec 12, 2017

Update holoviews to 1.9.2 zenotech/zCFDSuperBuild#139

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reimplement Path, Contours and Polygons plots #1991

Reimplement Path, Contours and Polygons plots #1991

philippjfr commented Oct 10, 2017 •

edited

Loading

jbednar commented Oct 10, 2017

philippjfr commented Oct 10, 2017

philippjfr commented Oct 10, 2017

philippjfr commented Oct 19, 2017

philippjfr commented Oct 20, 2017

jlstevens commented Oct 20, 2017

philippjfr commented Oct 20, 2017 •

edited

Loading

jlstevens Oct 21, 2017

philippjfr Oct 22, 2017

jlstevens Oct 21, 2017

philippjfr Oct 22, 2017

jlstevens Oct 21, 2017

philippjfr Oct 22, 2017

jlstevens Oct 21, 2017

jlstevens Oct 21, 2017

jlstevens Oct 21, 2017

philippjfr Oct 22, 2017

jlstevens Oct 21, 2017

philippjfr Oct 22, 2017

jlstevens Oct 21, 2017

jlstevens Oct 21, 2017

philippjfr Oct 22, 2017

jlstevens Oct 21, 2017 •

edited

Loading

philippjfr Oct 22, 2017

jlstevens Oct 21, 2017

philippjfr Oct 22, 2017

jlstevens Oct 22, 2017

jlstevens commented Oct 21, 2017

philippjfr commented Oct 22, 2017 •

edited

Loading

jlstevens commented Oct 22, 2017

jlstevens commented Oct 22, 2017

		gridded = False

		# Denotes whether the interface expects multiple ragged arrays

Reimplement Path, Contours and Polygons plots #1991

Reimplement Path, Contours and Polygons plots #1991

Conversation

philippjfr commented Oct 10, 2017 • edited Loading

jbednar commented Oct 10, 2017

philippjfr commented Oct 10, 2017

philippjfr commented Oct 10, 2017

philippjfr commented Oct 19, 2017

philippjfr commented Oct 20, 2017

jlstevens commented Oct 20, 2017

philippjfr commented Oct 20, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jlstevens Oct 21, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jlstevens commented Oct 21, 2017

philippjfr commented Oct 22, 2017 • edited Loading

jlstevens commented Oct 22, 2017

jlstevens commented Oct 22, 2017

philippjfr commented Oct 10, 2017 •

edited

Loading

philippjfr commented Oct 20, 2017 •

edited

Loading

jlstevens Oct 21, 2017 •

edited

Loading

philippjfr commented Oct 22, 2017 •

edited

Loading