Added DeckGL.Polygon Layer w/ JS controls #4227

hughhhh · 2018-01-17T21:53:03Z

Added new layer with advanced JS controls

mistercrunch

You need a png thumbnail here as well here

mistercrunch · 2018-01-17T22:05:50Z

superset/assets/javascripts/explore/stores/controls.jsx

+    type: 'SelectControl',
+    label: t('Polygon Column'),
+    validators: [v.nonEmpty],
+    description: t('Select the polygon column'),


Looks like we're assuming JSON formatted string here based on the code bellow. We should be clear about that here. The column the user points to should contain JSON array(N) of array(2).

I wonder if we ever use polyline for polygons, seems like it should work for both. We may want to support both as we do for deck_path (optional). That's only immediately useful if we have that input format internally.

mistercrunch · 2018-01-17T22:08:18Z

superset/assets/javascripts/explore/stores/visTypes.js

+        label: t('Query'),
+        expanded: true,
+        controlSetRows: [
+          ['polygon', 'row_limit'],


We may want to support the reverse control here. Seems like so far I've seen both lat/long long/lat in different places.

…ckgl.polygon

As they are external resources.

closes #4222 Related to: #4016

PR here #3518 missed a line of code while merging conflicts with time_pivot viz

* Improve geoJSON * Addressing comments * lint

hughhhh · 2018-01-18T00:24:53Z

superset/viz.py

+        return d[self.form_data.get('line_column')]
+
+    def query_obj(self):
+        d = super(DeckPolygon, self).query_obj()


@mistercrunch not sure if you wanted me to move this functionality into the baseClass? I think this is fine since we are still leveraging the line_column controls. Let me know what you think.

In theory we could have some intermediate (or mixin class) in between the base class that would be common to deck_path and deck_polygon, but in general the python philosophy is to keep inheritance schemes simple when possible. I'm fine leaving it as is as it's little code duplication as it is.

Now that I think about it, could we just have DeckPolygon derive DeckPath? Aren't they pretty much identical?

mistercrunch · 2018-01-18T16:29:49Z

Man that screenshot looks amazing.

mistercrunch · 2018-01-18T16:42:31Z

superset/viz.py

+        'polyline': polyline.decode,
+    }
+
+    def get_position(self, d):


My recent PR got rid of get_position, you may need to rebase and remove this method.

mistercrunch · 2018-01-18T16:46:44Z

superset/viz.py

+
+    def get_properties(self, d):
+        fd = self.form_data
+        deser = self.deser_map[fd.get('line_type')]


We could save one hash lookup per row (get_properties gets called for every row in the df) by setting self.deser = self.deser_map[fd.get('line_type')] in __init__. (I missed out on this simple optimization in DeckPath...)

mistercrunch · 2018-01-18T16:49:07Z

superset/viz.py

+        if fd.get('reverse_long_lat'):
+            polygon = (polygon[1], polygon[0])
+        return {
+            'polygon': polygon,


Since this is pretty much the only difference between DeckPath and DeckPolyline, I'd suggest have DeckPolyline derive DeckPath, and renaming this key to something generic like points. If we wanted to have a different key on each context, you could do it through a class (static) attribute that would be different for each class.

mistercrunch · 2018-01-18T16:52:52Z

It took me a moment to realize how similar path and polygon are (a polygon is pretty much just a closed path). Now that we know they are pretty much identical from a backend standpoint, we shouldn't have to duplicate any code as one can derive the other.

williaster · 2018-01-18T18:53:11Z

@hughhhh can you give a screenshot of the new UI controls? 😍

mistercrunch · 2018-01-18T20:50:10Z

@williaster

* Working polygon layer for deckGL * add js controls * add thumbnail * better description * refactor to leverage line_column controls * templates: open code and documentation on a new tab (apache#4217) As they are external resources. * Fix tutorial doesn't match the current interface apache#4138 (apache#4215) * [bugfix] markup and iframe viz raise 'Empty query' (apache#4225) closes apache#4222 Related to: apache#4016 * [bugfix] time_pivot entry got missing in merge conflict (apache#4221) PR here apache#3518 missed a line of code while merging conflicts with time_pivot viz * Improve deck.gl GeoJSON visualization (apache#4220) * Improve geoJSON * Addressing comments * lint * refactor to leverage line_column controls * refactor to use DeckPathViz * oops

williaster · 2018-01-18T22:52:04Z

hmm, does this not directly tie database entries (user-entered JS) to the current ES version/version of babel we are using, as well as the implementation of a given component (e.g., now we are tied to the contract of d.object d.object.props etc)?

that seems like a nightmare for backwards compatibility?

mistercrunch · 2018-01-20T01:01:07Z

I'm assuming JS won't evolve in a way that isn't backward compatible here. The supported language is what the "vm" module specs out. That code is sandboxely evaled at runtime, it's not babelized. Here's the construct we're tied to: https://nodejs.org/api/vm.html#vm_script_runinnewcontext_sandbox_options

The contract should not change here, at least not in incremental release of deck.gl and/or Superset.

I've been meaning to add an environment feature flag for this if you think it's necessary to allow folks to turn it off, and possibly a RBAC perm so that only some users are allowed to author JS. Let me know what you think.

Note that we haven't pushed a release since this has been merged and wanted to bring this up at our next committer meeting.

mistercrunch · 2018-01-20T01:04:07Z

Quick about the fact that this JS construct is extremely powerful in that it allows us to tackle the very long tail of visualization properties and attributes that [power] users may want to set or bind to the data in specific ways. It would takes dozens of new controls to expose a fraction of what is possible through JS.

williaster · 2018-01-22T18:21:28Z

thanks for all the additional context etc., I just didn't want to merge something that might have big API/migration implications lightly. I agree that this provides a much more flexible system for data munging and visual attribute mapping. the sandboxed vm seems reasonable.

let's discuss further at the next committer meeting 👍

* Working polygon layer for deckGL * add js controls * add thumbnail * better description * refactor to leverage line_column controls * templates: open code and documentation on a new tab (apache#4217) As they are external resources. * Fix tutorial doesn't match the current interface apache#4138 (apache#4215) * [bugfix] markup and iframe viz raise 'Empty query' (apache#4225) closes apache#4222 Related to: apache#4016 * [bugfix] time_pivot entry got missing in merge conflict (apache#4221) PR here apache#3518 missed a line of code while merging conflicts with time_pivot viz * Improve deck.gl GeoJSON visualization (apache#4220) * Improve geoJSON * Addressing comments * lint * refactor to leverage line_column controls * refactor to use DeckPathViz * oops

alldoami · 2018-07-26T21:31:32Z

@mistercrunch how do I change one of the props in the functions of DeckGL? For example, I want to change the color prop in grid.jsx within Superset. Is that possible?

* Working polygon layer for deckGL * add js controls * add thumbnail * better description * refactor to leverage line_column controls * templates: open code and documentation on a new tab (apache#4217) As they are external resources. * Fix tutorial doesn't match the current interface apache#4138 (apache#4215) * [bugfix] markup and iframe viz raise 'Empty query' (apache#4225) closes apache#4222 Related to: apache#4016 * [bugfix] time_pivot entry got missing in merge conflict (apache#4221) PR here apache#3518 missed a line of code while merging conflicts with time_pivot viz * Improve deck.gl GeoJSON visualization (apache#4220) * Improve geoJSON * Addressing comments * lint * refactor to leverage line_column controls * refactor to use DeckPathViz * oops

hughhhh added 3 commits January 16, 2018 19:16

Working polygon layer for deckGL

a2304b6

add js controls

9e94150

add thumbnail

1ec204a

mistercrunch requested changes Jan 17, 2018

View reviewed changes

hughhhh and others added 9 commits January 17, 2018 15:33

better description

4e99c37

Merge branch 'master' of github.com:apache/incubator-superset into de…

835a13e

…ckgl.polygon

refactor to leverage line_column controls

e549c9a

templates: open code and documentation on a new tab (#4217)

2a7ae47

As they are external resources.

Fix tutorial doesn't match the current interface #4138 (#4215)

b42f06e

[bugfix] markup and iframe viz raise 'Empty query' (#4225)

f75dc51

closes #4222 Related to: #4016

[bugfix] time_pivot entry got missing in merge conflict (#4221)

828014c

PR here #3518 missed a line of code while merging conflicts with time_pivot viz

Improve deck.gl GeoJSON visualization (#4220)

65e6e60

* Improve geoJSON * Addressing comments * lint

refactor to leverage line_column controls

cd33bee

hughhhh commented Jan 18, 2018

View reviewed changes

mistercrunch reviewed Jan 18, 2018

View reviewed changes

hughhhh added 3 commits January 18, 2018 11:29

refactor to use DeckPathViz

7e8726b

fixed conflicts

3173cdc

oops

d49bcbd

mistercrunch approved these changes Jan 18, 2018

View reviewed changes

mistercrunch merged commit 5079b2a into apache:master Jan 18, 2018

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.23.0 labels Feb 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added DeckGL.Polygon Layer w/ JS controls #4227

Added DeckGL.Polygon Layer w/ JS controls #4227

hughhhh commented Jan 17, 2018 •

edited

mistercrunch left a comment

mistercrunch Jan 17, 2018

mistercrunch Jan 17, 2018

mistercrunch Jan 17, 2018

hughhhh Jan 18, 2018

mistercrunch Jan 18, 2018 •

edited

mistercrunch Jan 18, 2018

mistercrunch commented Jan 18, 2018

mistercrunch Jan 18, 2018

mistercrunch Jan 18, 2018

mistercrunch Jan 18, 2018

mistercrunch commented Jan 18, 2018

williaster commented Jan 18, 2018

mistercrunch commented Jan 18, 2018

williaster commented Jan 18, 2018

mistercrunch commented Jan 20, 2018

mistercrunch commented Jan 20, 2018

williaster commented Jan 22, 2018

alldoami commented Jul 26, 2018

Added DeckGL.Polygon Layer w/ JS controls #4227

Added DeckGL.Polygon Layer w/ JS controls #4227

Conversation

hughhhh commented Jan 17, 2018 • edited

mistercrunch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mistercrunch Jan 18, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mistercrunch commented Jan 18, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mistercrunch commented Jan 18, 2018

williaster commented Jan 18, 2018

mistercrunch commented Jan 18, 2018

williaster commented Jan 18, 2018

mistercrunch commented Jan 20, 2018

mistercrunch commented Jan 20, 2018

williaster commented Jan 22, 2018

alldoami commented Jul 26, 2018

hughhhh commented Jan 17, 2018 •

edited

mistercrunch Jan 18, 2018 •

edited