Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support vector tile source for additional viz types #120

Merged
merged 22 commits into from Oct 21, 2018

Conversation

akacarlyann
Copy link
Collaborator

@akacarlyann akacarlyann commented Aug 9, 2018

Adds support for vector tile source for MapViz, CircleViz, GraduatedCircleViz and HeatmapViz via the VectorMixin class. Adds support for allowing JSON/GeoJSON join-data to be passed as data argument as a filename, URL, or dictionary. Closes #71.

…o a VectorMixin class; fixes linestring bug with interpolation of color for certain color lookups with match-type
…tility and logic to viz.py to facilitate loading data from JSON object, list of Python dicts, GeoJSON filename
@akacarlyann
Copy link
Collaborator Author

@ryanbaumann Still working on tests and docs for this PR, but it's pretty close :)

@akacarlyann
Copy link
Collaborator Author

@ryanbaumann Can you take a look and see if the tests I've included are sufficient? Still need to update docs, but I should be able to get this done before the weekend :)

@ryanbaumann
Copy link
Contributor

@akacarlyann looks like the external vector tile support for circle, graduatedcircle, and heatmap layers is all based on using the data-join technique. One use case we want to also support is to style data based on properties already in the vector tile itself - is that possible with your new branch here?

For example, if all we wanted to do was replace the geojson sources in this example https://nbviewer.jupyter.org/github/mapbox/mapboxgl-jupyter/blob/master/examples/notebooks/point-viz-types-example.ipynb with vector tile sources. No data-join required from a local data frame, just sourcing the data from an external vector tile source.

@akacarlyann
Copy link
Collaborator Author

Sure, that was working in one of the intermediate steps. I'll make sure it's still working with the idea be that data can be passed in as [] or None. I think the heatmap example still works with an empty list for data maybe...

@akacarlyann
Copy link
Collaborator Author

Hey @ryanbaumann , I've added a flag for disabling vector data join. I'm also working on making sure join data is supported when passed as JSON/GeoJSON, a dictionary, or a filepath, so feel free to take a look.

@@ -29,7 +29,7 @@ Parameter | Description | Example
data | GeoJSON Feature Collection or JSON Join-Data | 'points.geojson'
vector_url | optional property to define vector data source (supported for basic MapViz, CircleViz, GraduatedCircleViz, HeatmapViz, ChoroplethViz, LinestringViz) | 'mapbox://mapbox.mapbox-terrain-v2'
vector_layer_name | property to define target layer of vector source | 'contour'
vector_join_property | property to aid in determining color for styling vector layer | 'ele'
vector_join_property | property of features in vector tile data to use as link to joined json data | 'ele'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Name of the property in the reference vector tile source to join to the JSON` data passed to this function

@ryanbaumann
Copy link
Contributor

Going to try testing this now on various viz types @akacarlyann

@akacarlyann
Copy link
Collaborator Author

@ryanbaumann Have you had a chance to take a look at this? Hoping to spend some time on the project later this week :)

@ryanbaumann
Copy link
Contributor

Awesome @akacarlyann. Unfortunately no, I haven't had much time to check this out. Thank you for the reminder - I'll do it ASAP.

@ryanbaumann
Copy link
Contributor

Largely this looks good @akacarlyann and will be super powerful for adding vector data sources.

One question - could eliminate the need for the disable_data_join=True flag when using a vector tile source? It should be the default behavior that most users will want if they are not making a Choropleth map.

@akacarlyann
Copy link
Collaborator Author

@ryanbaumann Can you explain a little more of what you mean? Under the current vector visualization paradigm, the user has to pass a json data object and the vector tile URL and layer name. But if a user wants to style the layer only using the data within the vector tile source, they must still provide json data for the layer filter applied to the vector data. disable_data_join=True just prevents the empty data param from filtering out all the features in the vector source.

I can refactor this but it will take a little more logic to smartly determine when data should be ignored. Alternately, join data could be added in a separate argument?

@ryanbaumann
Copy link
Contributor

But if a user wants to style the layer only using the data within the vector tile source, they must still provide json data for the layer filter applied to the vector data. disable_data_join=True just prevents the empty data param from filtering out all the features in the vector source.

Understood @akacarlyann. I think this is a reasonable assumption to make then, as long as we document it well, which you have.

Checking out the branch for merge now with the latest changes merged into master.

@akacarlyann
Copy link
Collaborator Author

@ryanbaumann looks like one check is failing now due to updates on master in the last few weeks. I'll look into fixing that, but otherwise, how is this PR looking?

@ryanbaumann
Copy link
Contributor

@akacarlyann looking good with a master update - thank you! Ill approve once passing.

@akacarlyann
Copy link
Collaborator Author

@ryanbaumann ready to go!

@ryanbaumann ryanbaumann merged commit 0125251 into mapbox:master Oct 21, 2018
@ryanbaumann
Copy link
Contributor

ryanbaumann commented Oct 21, 2018

Woho! Thanks for bearing with me on the time to merge - your PR here opens up a huge number of use cases for developers visualizing massive tiled data sources in Jupyter and Python that's too big to fit into the browser memory. Thanks, @akacarlyann!

akacarlyann added a commit to akacarlyann/mapboxgl-jupyter that referenced this pull request Nov 11, 2018
* Update text-size to use expression based on viz.label_size property

* Add new label properties to choropleth viz templates

* Move label properties to base class (leverage inheritance to reduce repeated arguments)

* Extract vector_color_map and numeric map (for height or line_width) to a VectorMixin class; fixes linestring bug with interpolation of color for certain color lookups with match-type

* Extend CircleViz with VectorMixin (start GraduatedCircleViz, HeatmapViz, ClusteredCircleViz)

* Extend vector data loading to base map and CircleMap -- datadriven styling for radius needs work

* Refine color mapping in VectorMixin and update templates, viz.py

* Update CircleViz template files with Jinja inheritance, establish {% block circle %} tag

* Update Vector layer example for CircleViz; add geojson_file_to_dict utility and logic to viz.py to facilitate loading data from JSON object, list of Python dicts, GeoJSON filename

* Refine function for parsing GeoJSON and JSON input (esp for vector visualizations) and add SourceDataError

* Add support for GraduatedCircleViz template to use vector source data layer

* Add support for HeatmapViz to use vector data source

* Add and refine tests; update utility name for geojson_to_dict etc.

* Change FileNotFoundError to IOError for Python2.7 support

* Enable data from vector layers to be used for data-driven style (without using data-join technique)

* Update docs for VectorMixin class, vector properties and label properties inherited from MapViz parent class

* Update geojson_to_dict to geojson_to_dict_list and add docs; organize MapBox.create_html()
ryanbaumann pushed a commit that referenced this pull request Mar 19, 2019
* Add  function

* Add test notebook for variable radius legend

* Fix link markup in readme (#124)

* Update README.rst (#123)

Add import so that example runs

* upgrade to gl js v49 (#125)

* upgrade to v0.9.0

* Support vector tile source for additional viz types (#120)

* Update text-size to use expression based on viz.label_size property

* Add new label properties to choropleth viz templates

* Move label properties to base class (leverage inheritance to reduce repeated arguments)

* Extract vector_color_map and numeric map (for height or line_width) to a VectorMixin class; fixes linestring bug with interpolation of color for certain color lookups with match-type

* Extend CircleViz with VectorMixin (start GraduatedCircleViz, HeatmapViz, ClusteredCircleViz)

* Extend vector data loading to base map and CircleMap -- datadriven styling for radius needs work

* Refine color mapping in VectorMixin and update templates, viz.py

* Update CircleViz template files with Jinja inheritance, establish {% block circle %} tag

* Update Vector layer example for CircleViz; add geojson_file_to_dict utility and logic to viz.py to facilitate loading data from JSON object, list of Python dicts, GeoJSON filename

* Refine function for parsing GeoJSON and JSON input (esp for vector visualizations) and add SourceDataError

* Add support for GraduatedCircleViz template to use vector source data layer

* Add support for HeatmapViz to use vector data source

* Add and refine tests; update utility name for geojson_to_dict etc.

* Change FileNotFoundError to IOError for Python2.7 support

* Enable data from vector layers to be used for data-driven style (without using data-join technique)

* Update docs for VectorMixin class, vector properties and label properties inherited from MapViz parent class

* Update geojson_to_dict to geojson_to_dict_list and add docs; organize MapBox.create_html()

* Fix bug with df_to_geojson and non-sequential indices (#132)

* Support variable radius legend with legend_function='radius' setting for GraduatedCircleViz

* Update docs

* Move LegendError to  method `create_html`

* Bugfix for LegendError test

* Refactor legend placement scripts and add updateAttribMargin and updateLegendMargin functions in main.html to automatically place secondary legends
@akacarlyann akacarlyann deleted the vector-mixin branch December 2, 2019 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants