[SDW Best Practices Update]: Discuss Tiling in the BP

[situx]

Add something on Tile concept (noting the work in OGC to standardize the conceptual model and the tiles API), perhaps a BP - needs further discussion in the group. We only touched on this.

[PeterParslow]

ISO/TC 211's "tiling people" (well, the convenor of the imagery/EO group) could provide something but not for a few months (i.e. beyond April 2023) - perhaps an update after this next release?

[situx]

I would appreciate getting a text on this. We could surely fit that in a minor update after an official release, so highly appreciated if the people could provide something beyond April 2023

[PeterParslow]

Natural Resources Canada / Graham Wilkes (convenor of ISO/TC 211 WG6 which covers quite a lot of tiling) are working on a 'best practice on geospatial tiling' document with the intention that it is available (licenced) to W3C to include in the SDW BP.

I expect it to cover conceptual tiling (2D, 3D, cartesian & projected); the use of tiles for analysis & delivery; vector & raster tiling; OGC API Tiles; and related considerations around CRSs, accessibility, the relation to coverages.

[PeterParslow]

Canada's proposed report on best practices in tiling would contain:

• Conceptual tiling of 2D, 3D, nD space for geocentric, projected and geographic coordinate reference systems
• The OGC Two-Dimensional Tile Matrix Sets and TileSet metadata Standard
• Tiling gridded coverage data, maps and imagery
• Use of TileMatrixSets in OGC WMTS and OGC GeoPackage
• Tiling vector features and point clouds
• Accessing data of interest efficiently through the OGC API family of standards (Tiles, Maps, Coverages, 3D GeoVolumes)
• Tiling 3D data (bounding volume hierarchies, glTF, 3D Tiles and i3s)
• Relationship between 2D TileMatrixSets and axis-aligned Discrete Global Grid Systems
• Tiles as a mechanism to facilitate parallel and on-demand processing
• Accessibility considerations

If there's anything else / different we would like to see included, then we should let them know before the end of April 2023, when work will start in earnest.

It sounds as if it will provide a sizeable section to attach in some way to the SDW BP.

@situx @Iza82

[chris-little]

@PeterParslow @situx @Iza82 The OGC EDR API SWG started to explore access to data tiles (Cube is a query type) through the OGC API-EDR standard and the use of CoverageJSON, which also supports an intrinsic tile structure. Collaboration with OGC API-Maps, Coverages, 3D Geovolumes, etc is being pursued.
So it would be good if API-EDR was added to the list in Bullet 6, and perhaps adding the CoverageJSON container format somewhere?

[rob-metalinkage]

Conceptually tiling is independent of encoding format, but formats may have canonical support for tiling. The BP should be format agnostic, not using a closed list, but have examples and references to known implementations.

[PeterParslow]

@chris-little : I am happy to suggest OGC API EDR is added at "the OGC API family of standards (Tiles, Maps, Coverages, 3D GeoVolumes)"

I'm not sure what the W3C BP should say about the OGC / community differences between CoverageJSON (as per API EDR) and CIS JSON (as per API Coverages / the OGC Coverages SWG). From conversations overheard (as it were) it doesn't seem that there is an OGC best practice on encoding coverages in JSON. That could perhaps be said in the W3C BP? It isn't the only place where OGC is happy to have two "competing" / overlapping standards.

[chris-little]

@PeterParslow I think that the key differences between the CoverageJSON and CIS JSON standard encodings are in the use cases. CoverageJSON is specifically for small, relatively lightweight, generic clients, such as a browser or Leaflet, and is agnostic about its server. CIS JSON seems to require more GIS specific clients and servers.
I think that is probably worth mentioning in the W3C BP, but not sure how much re-writing that would entail.

[PeterParslow]

An update: the paper that Natural Resources Canada sponsored is now ready for reuse. It was written largely by Jérôme St-Louis (@jerstlouis), with input from Peter Rushforth (@prushforth) and Graham Wilkes (@wilkesg)
I'll let Graham explain how they would like this taken forward. We envisage "mining" the document for information to include in the best practice whilst having the document itself as an annex or external reference.

@situx / @lvdbrink : I think you'll need to invite Jérôme and Graham to the group

[situx]

@PeterParslow, thanks for the update. I believe I do not have sufficient rights to invite anyone, so maybe @lvdbrink can do it. From my point of view, I would think it is good to have it as an external reference. At first thought, putting the paper as a whole into the annex might be too much. Then again, I have not read it yet, so maybe it will change my mind if I do. Is it already accessible somewhere?

[PeterParslow]

I was hoping Graham would upload it (my tagging of him & the others appears to have worked).

Here it is.
Spatial Data on the Web Best Practices - Tiling geospatial data - Final version.pdf

[wilkesg]

@PeterParslow My apologies for the tardiness on posting! Yes, thank you Peter for sharing. We encourage use of this document for the tiling best practices as it was designed to be used for this. Noting that we request any text please be attributed to Natural Resources Canada. Jerome St-Louis was the lead author on putting together this text, with support from Aliyan Haq and Peter Rushforth of Natural Resources Canada. We share it under our open data licence, https://open.canada.ca/en/open-government-licence-canada Cheers, Graham

[wilkesg]

Some or all of you may know Jerome, but a link to his linkedin profile if you like to connect, https://www.linkedin.com/in/jeromestlouis/?originalSubdomain=ca

[situx]

This looks quite comprehensive. Looking through the document, it may very well become its own chapter in the Best Practice document. You are right; some parts could be left out or shortened (maybe the history part), but it would fit very well as its own chapter in the current document.
What is missing, I think, are the actual Practices that are derived from this document that we always used to list.
These need to be created with numbers and short descriptions, and then we could incorporate much of the text.

@wilkesg @jerstlouis @PeterParslow did I understand you correctly in that you would like to issue a pull request to include that text into the Best Practices, or would that be something we would need to work on as editors?

[PeterParslow]

I think that first we should agree (using this thread?) what to extract as the "best practices", then see how the document might need to be restructured to be a section in the SDWBP - both those before we could create a pull request.

[jerstlouis]

@situx @PeterParslow About the 2DTMS history part, despite the title, the intent of this section really was to discuss the long-standing use of 2DTMS in GeoPackages (for storage / offline exchange) and WMTS, both of which are still widely used today, and at the same time to attribute where these 2DTMSs come from -- the 2DTMS standard didn't just appear out of nowhere in 2019. Leaving out the section completely would miss out a lot of important information, so perhaps we could rewrite it so that it looks less like a history (starting with the title, we can go back to the original proposed "Use of TileMatrixSets in OGC WMTS and OGC GeoPackage"), if that is problematic.

In the document I tried to highlight some of the Best Practices with a bold "Best Practice". Use of any of the OGC / ISO standards reference is a "Best Practice" in any context where they are applicable, but where this is not already identified as such, clarifying that context might require a bit more text.

As I mentioned by e-mail to @wilkesg and @PeterParslow, I am happy to keep working with you all to better adapt the content to the SDWBP purpose.

[situx]

From my point of view, we would need to explain briefly what tiling is and where it came from both 2D and 3D. That would be one section for me. Next, we would need to create sections that explain the aspects of the work that we would need to summarize as best practice statements, e.g. "Provide 2D Tiling Services for sharing map data using XXX Standards" and then categorize these in the categories present in the Best Practice document. These would likely be Access, Reuse, Interoperability and possibly Discovery. I think that could be a way forward.
In the end, we have a section with an introduction and each aspect of tiling framed into a best practice statement. This section could be added to the document.

[wilkesg]

From the document, I took some time to try and answer these questions.
Why we tile data? Tiling in itself is one big best practice for the web. Tiling data geospatially is essential to manage and deliver large datasets on the web. Tiling involves dividing geographical space into smaller, manageable ‘tiles’ or subsets, which can contain raster images, vector data, point clouds, or 3D models. These tiles enable various benefits, such as optimizing data storage and retrieval, supporting parallel processing for data analytics, and enhancing web map accessibility by enabling alternative rendering styles. Tiling provides a versatile and scalable solution for handling geospatial information effectively in web applications.
Where did it come from? Tiling data for web mapping and geospatial standards was initiated and developed by organizations and communities working on geographic information systems (GIS) and web mapping. It evolved as a best practice in the geospatial industry rather than having a single individual or organization credited with its inception.
Here are the specific considerations for tiling summarized:

1. Use of Generic Tile Containers: Adopting tile container formats like COG (Cloud Optimized GeoTIFF) and PMTiles, which support HTTP range requests for efficient access to individual tiles, is recommended for serving various types of geospatial data.
2. Efficient Vector Tile Encoding: Emphasizes the efficiency of vector tile encoding, which quantizes geometry coordinates to a grid at an appropriate resolution, making it suitable for visualization while also achieving good compression.
3. Tiling Point Clouds: Tiling point cloud data is advised to handle large, dense point cloud datasets by setting an upper limit on points per tile and using clustering or filtering algorithms for lower resolution tiles.
4. Use of Efficient Formats: Recommends using efficient formats like LAS/LAZ for point cloud data and Cloud Optimized Point Cloud (COPC) based on LAZ for HTTP range access.
5. Tiling 3D Data: Suggests tiling large-scale 3D environments using standards like OGC 3D Tiles and I3S to efficiently exchange and visualize multi-resolution 3D scenes.
6. Accessing Data via OGC API: Promotes the use of OGC API standards for accessing geospatial data as REST Web APIs, enabling consistent and modular access mechanisms for different types of data.
7. Raw Data Tiles for Accessibility: Proposes using raw data tiles (e.g., vector tiles and gridded coverage tiles) to enhance web map accessibility, as they allow for alternative rendering styles, larger fonts, and improved accessibility devices for users with disabilities.
8. Parallel and On-Demand Processing: Highlights the role of tiling in facilitating parallel and on-demand processing for large geospatial datasets, aiding memory management and distributed processing.
9. Interoperability with DGGS: Discusses the use of Discrete Global Grid Systems (DGGS) and their integration with tiling practices to support spatial queries efficiently.
10. Data Partitioning and Optimization: Recognizes the practice of partitioning geospatial data into tiles as an effective means to optimize data storage, retrieval, and processing.

[lvdbrink]

Wow, this is quite a list! Reading this, my first thought is: wouldn't this be better as its own best practice document?

Are there specific reasons/arguments for including this in the SDWBP? Should it be in a joint W3C/OGC publication or is it allright if it's just OGC?

Noting that the Spatial data on the Web working group may not exist in a few months time...

[situx]

I would say we should at least have a section about Tiling in the SDWBP, since tiling is essential for hosting geodata. To what extent the information contained in the linked paper should be incorporated, that is up for discussion here I suppose.

[lvdbrink]

My personal feeling is that we should have a section on Tiling but keep it limited to roughly the same size as the other individual best practices in the document. Otherwise, the document would start leaning significantly in the direction of tiling / data visualization, whereas the intent is (or at least was, when we wrote the first version) to have a focus on data sharing - on the data itself, not on 'pictures' of maps. I'm not trying to diminish the importance of this topic, but it should not change the focus of the document as a whole IMHO.

[PeterParslow]

Agree regarding the relative size of sections.

But, Linda, remember tiling is for data, not just for maps.

[cportele]

remember tiling is for data, not just for maps.

For feature data: While it is true that in principle tiling can be used to spatially organize feature data, the content of the tiles in most tilesets (that I am aware of) in the common formats Mapbox Vector Tiles, 3D Tiles or i3s is optimized for client-side visualization at the level of the tile in the tiling scheme, not for analytics. The "Best Practice" should focus on how tiling is used in practice, not how it could be used.

[chris-little]

The "Best Practice" should focus on how tiling is used in practice, not how it could be used.

Though the "Best Practice" may not be the majority practice.

[jerstlouis]

In my view, the main goal and advantage of OGC API - Tiles over WMTS, and the reason we developed the Tileset Metadata in 2DTMS 2.0, is the ability to share data tiles for purposes other than visualization and for client-side visualization / styling / queries (no need for server round-trip "GetMapInfo").

This applies to any type of data, whether that is coverage tile, vector tiles, 3D mesh tiles, or point cloud tiles.

Even with the commonly distributed Mapbox Vector Tiles tilesets, at least the most detailed tiled zoom level should be suitable for purposes other than visualization. The MVT spec states its purpose as:

This document specifies a space-efficient encoding format for tiled geographic vector data. It is designed to be used in browsers or server-side applications for fast rendering or lookups of feature data.

[PeterParslow]

A thought: if "we" want to publish the whole paper somewhere and then reference it from the SDWBP, perhaps with an extra "best practice on tiling" (some detail, reference out to the document for more).... then we could host it at ISO/TC 211. As mentioned earlier, I suspect it could also be hosted at OGC.