[FEATURE] parallel tile generation and road network connectivity

[bdon]

Hi, thanks for creating this - it could be a good match to other map tooling I'm developing including a tile archive format and canvas renderer.

One thing I noticed with whipping up a quick example is that the basemap result for motorway network has gaps in it - see attached screenshot. The root cause for the gaps is that OSM ways that connect two other high-importance ways - like highway=motorway, assigned to appear at a lower zoom level like z9 might actually be tagged at a lower level like highway=primary and not be included in z9. This is because OSM tags are based on physical features and not cartographic importance.

The approach I usually take to solve this is to generalize roads via a new tag class outside of the OSM tagging conventions based on connectivity to other ways. This would break the parallel design described in ARCHITECTURE.md because feature generation at the early stage could not proceed independently per-feature. I also don't think it could fit into the advanced joining/grouping in README#Limitations because the late-stage, pre-write processing of features at the tile level would not have access to those missing ways.

Is this mode of processing something you would consider in scope for Planetiler, if it necessitates changing the tile generation strategy? There may be ways to accommodate it into the existing design if candidate ways were saved out similar to boundary relations but the quantity of candidates would be very large.

[msbarry]

Thanks for reaching out @bdon! Could you elaborate on exactly how that connectivity check would work?

The line merging algorithm is pretty simple now, it just groups linestrings in each tile with the exact same tags and runs them through the JTS LineMerger. The main issues I've seen are:

• short bridge sections interrupt the segment and prevent the merge
• roads that split into 2 lanes (or appear to split because the lane endpoints collapse to the same pixel when rounded) confuse the line merger because it stops merging lines at 3-way merges

I spot checked a few of those gaps, and they they mostly seem to be due to those 2 issues - is there a specific spot where you see the motorway/primary/motorway issue causing a gap?

A few ideas I had to improve line merging:

• support grouping by a subset of tags, and aggregating the remaining ones so short bridge segments don't cause these gaps
• prevent the line merging routine from combining ways that go in separate directions so divided highways don't generate 3-way intersections (probably need this anyway since line merging might be breaking the oneway tag right now).
• make the line merging routine "gap-tolerant" so that it can combine lines with the same tags with endpoints a few pixels apart instead of requiring exactly matching endpoints

On a side note, I was wondering if it might make sense to have a native pmtiles output option for planetiler? How bad is it to convert a 100gb mbtiles file to pmtiles?

[bdon]

Essentially the grouping by a subset of tags would get most of the way there - I mentioned primary > motorway but I think this happens more often for motorway_link that connects motorway at flying/cloverleaf junctions. The side effect of doing grouping by the tag sets is your motorways will become "hairy", since off-ramps are also tagged as motorway_link, so the connectivity check is something like:

1. if both ends of the motorway_link connect to a motorway and nothing else, promote it to a motorway
2. if one end of the motorway_link connects to a motorway but the other end connects only to x, demote it to x

the python utility in the pmtiles package should be workable for converting mbtiles to pmtiles, but a native one shouldn't be hard to reproduce in Java, the logic is pretty short: https://github.com/protomaps/PMTiles/blob/master/python/pmtiles/writer.py

[msbarry]

Ok thanks for the explanation. I think the first step here is to implement a custom line merge routine to be able to customize any of this stuff since the JTS one is a bit of a black box.

I'll try doing that then add the ability to:

• Filter what's being grouped
• group by a subset of tags and specify how to combine the remaining ones
• optionally prevent line merging from changing the direction of linestrings

Let's see how that looks and if we still need to can try adding smarter support for 3-way merges or "fuzzy" matching on endpoints.

That would also open up the door to that custom connectivity check at tile-write time. I don't think it would need a different overall architecture, just maybe pass through more ways during osm processing then either upgrade them or throw them out at tile write time. Alternatively, it could keep track of what highway classes terminate at certain node ids during osm pass1, then use that to upgrade certain ways during pass 2.

[bdon]

This is somewhat implemented in protomaps/basemaps#19 - I don't think we need to restructure planetiler core to accomplish this in a good-enough way. But for now this is possible consuming the tile post-process APIs. Closing issue!