[loader] Vector tiles support

[vpicavet]

iTowns should support vector tiles at some point.

Vector tiles are an efficient vector format for rendering of 2D data. Vector tile services start to be available, and server-side components do exist.

• https://www.mapbox.com/vector-tiles/specification/
• https://www.mapbox.com/vector-tiles/
• https://en.wikipedia.org/wiki/Vector_tiles
• http://wiki.openstreetmap.org/wiki/Vector_tiles#Ongoing_projects.2C_people

Implementing vector tiles support suppose having Clamping available first ( #181 ).

We could use the following services for testing purposes : 

• Klokan : https://maps.klokantech.com/
• MapBox
• Mapzen

[zarov]

I'm currently working on integrating vector tiles, and I identified a few step ahead:

• load a pbf/mvt file and parse it
• support pbf/mvt loading for TMS, WMS, WFS and WMTS
• display the result as an image (similar to RasterProvider)
• display the result as a Mesh

First I'm concentrating on solving one way, and then the others (that should be smooth).

Below are a list of PR related to this work (will be updated):

• feat(core): compute TMS coordinates from north to south #639 feat(core): compute TMS coordinates from north to south

[zarov]

Let's lay a solid basis, with the aim to easily iterate later on this subject (for example when expanding style possibilities). Here's a more detailed point of view I got.

Tile loading

Vector tiles are accessible through TMS, WMTS, WMS and WFS. Currently, no provider can fetch pbf files. A method needs to be added for this, that could be used like OGCWebServiceHelper.getColorTextureByUrl for example.

Question: do we add all the methods related to vector tiles in a new file, like VectorTile_Provider.js ? I don't think it's a good idea to create a new single provider, as it would be a pain to add support for all protocols in it. I was thinking more of adding a VectorTileHelper.js, featuring some public methods:

• getVectorTileRawByUrl: would return a "raw" vector tile, not the loaded pbf but a Mapbox VectorTile.
• getVectorTileGeoJSONByUrl: would return a vector tile as a GeoJSON (needed for next methods, so why not making it available)
• getVectorTileTextureByUrl: if the layer.type is color, would return a texture, using the Feature2Texture class.
• getVectorTileMeshByUrl: if the layer.type is geometry, would return a mesh, using the Feature2Mesh class.

Those methods could be used in each Provider, using the supportedFormats array: a new entry with application/x-protobuf;type=mapbox-vector needs to specify the calling of a method above.

Tile styling

The style argument in Feature2Texture.createTextureFromFeature could be improved:

• by allowing it to be a function that specify the style given properties in the feature.
• by allowing to load a complex file like this one.

The same reasoning can be applied to Feature2Mesh, but more limited.

Let's discuss it further !

[peppsac]

We should start rethinking other Providers / Drivers as well. For instance WMTS_Provider does different things for elevation and color textures while it's completely unrelated to the WMTS protocol.

An option is to reorganize providers: ImageryProvider, ElevationProvider, VectorProvider for instance. They would then use some protocol aware code (WMTSProtocolHandler ? TMSProtocoHandler?) that would simply build the correct url and fetch the result.

And maybe VectorProvider isn't even useful, since it'll provide textures or meshes. You could declare an imagery layer using the xyz protocol and a vector-tiles data source.

Then ImageryProvider would do:

return tmsProtocolHandler.fetch(params).then(tile => VectorTile.toImageryTexture(tile) 

For a layer using a raster data source ImageryProvider would do:

return tmsProtocolHandler.fetch(params).then(tile => Raster.toImageryTexture(tile) 
return tmsProtocolHandler.fetch(params).then(tile => Raster.toElevationTexture(tile) 

WDYT?

[mbredif]

Please allow me to jump in, as this discussion is very much related to recent experience with students having a really hard time to customize itowns with new datasources, new data formats or new portraying styles (without even talking about interaction) and facing monolithic providers that they have a hard time to customize.

We definitely need this discussion and clarification of the roles of providers/protocolhandlers, loaders/drivers and feature_transformer/stylizers. Current providers are basically responsible for orchestrating the whole transformation from datasources/servers to Three.js objects, so it is very cumbersome to mix and match different protocols, formats and styles.

This discussion has multiple facets, my proposition, definitely up to discussion, would be that :

• The responsibility of providers should be restricted only to protocol access, ie handing over datablobs (=serialized datasets, file content, file chunks) from data requests, possibly taking into account filtering (eg, by extent, attribute...). Providers may also be responsible for cataloguing (discovery of available datasets, getcap...) and format negotiation.
• The loaders/drivers should be gathered in a single place (they are currently scattered into the Providers, ThreeExtended and Scheduler ) and should expose a more standardized interface.
• Having a registry of loaders, could help to decouple the loaders from the providers. Ideally decoupling could enable the the coding of each provider and loader as independant npm packages (ie depending only on a core itowns package). That would help customization a lot !
• The responsibility of loaders could be to transform a datablob into a set of Three.js objects. These objects may however not be directly renderable : maybe it is only a float32array of positions from a 3dtiles/pnts or potree tile, a texture from an image file, the geometry of a mesh...
• Then Stylizers could be responsible for assembling three.js renderable objects (geometry and material) from the features and style parameters (eg SLD/SE for 2D features). The current use of features as a pivot dataset before rendering as textures or meshes definitely goes in the right direction and should be generalized. Ideally the pivot format for features should however be as close as possible to three.js/webgl to save any conversion roundtrip if the data format is suitable for direct upload to the gpu. The goal here would be to be able to work on point cloud stylisation (eye dome lighting, projective texturing, splatting...) without caring on from where the points came from (potree, 3dtiles, geojson, drag and dropped PLY file...)
• Optionally, transformers could massage/filter/augment/whatever features into new features before stylisation. 2D polygon to 3D mesh extrusion is a good example.

[autra]

I quite agree with you @mbredif on the role of providers. For the rest, no opinion yet :-)

could enable the the coding of each provider and loader as independant npm packages

Note that this is already possible with providers, at least. Also, if you import iTowns with a module bundler, you can already customize a lot of things (but I do agree that this process still requires too much inner knowledge of iTowns :-/ ).

[peppsac]

@mbredif I agree - it's more or less what I wrote in my comment, but yours is way clearer :)

[zarov]

Solved in multiple PRs