Use globally unique object ID-s

[jaakla]

It looks like the schema page does not specify how are are object ID-s set in the vector tiles. They seem to be sequential for every tile (1,2,3...), right? Now when there are several copies of same objects (e.g. POIs) due to buffers, then the same object in different tile will have different ID.

This seems to cause labelling issues with some renderers, for example in Mapnik or CARTO SDK. At least the latter expects that vector tile has global ID numbering at least within a layer. This could be easily achieved by copying a "natural" global UID, e.g. osm_id value from OSM database to object IDs, as some other vector tile providers do. This way renderer knows that object with id 12345 in one tile is same as object with same id 12345 is the neighbouring tile.

[sfkeller]

Keeping the same id is not only for renderers, but even more for clients/readers (like my Vector Tiles Reader https://github.com/geometalab/Vector-Tiles-Reader-QGIS-Plugin) which have to merge objects cut apart by the tiling process, and to deduplicate objects from overlapping buffers.

[davvo]

I would also like to see this happen, since we are drawing buildings with different colors based on its ID, and would like to get the same color between different tiles and also between different zoom levels.

[tomchadwin]

I've just run into this issue with Leaflet.VectorGrid. I can't label features without a unique id field - as @jaakla says, using osm_id is a strong candidate.

[klokan]

OpenMapTiles v3.7 is going to have OSM derived object ID for point features in layers:

• place
• poi
• water_name

The IDs are going to be very probably internal - available for decoders of PBFs and from JavaScript while reading, but not exposed as attributes to the styles.

In the future (v3.8?) we may add globally unique IDs into other layers. On polygons and lines it is more complicated. Ideally we should choose always the same IDs as in https://osmnames.org/ (for relations, multisegment lines, etc. generalized forms) - this may mean cleaning of OSMNames data too: http://osmnames.org/download/.

[jirik]

This was partially implemented in #378 as @klokan described.

This seems to cause labelling issues with some renderers, for example in Mapnik or CARTO SDK. At least the latter expects that vector tile has global ID numbering at least within a layer.

@jaakla Any feedback about if it helped you solve rendering issues with Mapnik or CARTO SDK or not is highly welcome. From the first tests I have a feeling that it did not help in case of Mapnik. The 3.7 release will be deployed to free.tilehosting.com till the end of this week, so you can try it then.

[jaakla]

The CARTO SDK rendering issue seems to be resolved with this, thanks! It was pretty minor still, so probably outsiders did not even notice it.

[jaakla]

Just for info - missing ID issue was specifically visible in this CARTO SDK issue: CartoDB/mobile-sdk#176 , as the server will be updated now to OMT 3.7, then should disappear from live.

[amirmasud]

The IDs are going to be very probably internal - available for decoders of PBFs and from JavaScript while reading, but not exposed as attributes to the styles.

@klokan Is there any way to get osm id of a place/poi/water_name via mapbox gl ?
I tried queryRenderedFeatures and querySourceFeatures methods but none of them had osm_id in the result features.

[amatissart]

We have a similar use case where we'd like to set a consistent osm id in our tiles.

Due to how imposm mappings work, the current osm_id is impractical as it may contain different osm types depending on the table, and even all 3 node/way/relation types with potential conflicts when using "type_mappings" (eg in the aerodrome_label layer).

It has been considered in the past to set use_single_id_space option to workaround this issue.
Wouldn't this approach also help to build osm_id_hash more directly ?

[amatissart]

Here is an example of how we plan to use use_single_id_space to build both original osm_id (e.g osm:relation:123456) and osm_id_hash.

CREATE OR REPLACE FUNCTION osm_hash_from_imposm(imposm_id bigint)
RETURNS bigint AS $$
    SELECT CASE
        WHEN imposm_id < -1e17 THEN (-imposm_id-1e17) * 10 + 4 -- Relation
        WHEN imposm_id < 0 THEN  (-imposm_id) * 10 + 1 -- Way
        ELSE imposm_id * 10 -- Node
    END::bigint;
$$ LANGUAGE SQL IMMUTABLE;

CREATE OR REPLACE FUNCTION global_id_from_imposm(imposm_id bigint)
RETURNS TEXT AS $$
    SELECT CONCAT(
        'osm:',
        CASE WHEN imposm_id < -1e17 THEN CONCAT('relation:', -imposm_id-1e17)
             WHEN imposm_id < 0 THEN CONCAT('way:', -imposm_id)
             ELSE CONCAT('node:', imposm_id)
        END
    );
$$ LANGUAGE SQL IMMUTABLE;

For some context see https://github.com/QwantResearch/openmaptiles/pull/9/files#diff-11a4b6bc390567301a75fa8562893b9c

Maybe that is a bit out of scope of this issue, but that's something we are implementing, and would find useful upstream.

[nyurik]

PostGIS 3.0+ ST_AsMVT supports feature_id_name as its 5th parameter, so now sqltomvt uses them too.