Preserve OSM IDs

[dabreegster]

Is it possible to plumb through the original OSM way ID in combined_network.gpkg? This would help give some hints for map-matching. It'd still be an imperfect match, because

1. We need to use exactly the same dump of OSM data. Maybe that'd mean synchronizing with CycleStreets, if they use Geofabrik or something else public?
2. NPW imports a different set of the OSM ways. I think it's a superset of what CycleStreets considers using for routing, though, so it'd be OK to try matching.

[Robinlovelace]

Yes, it's a good idea but feels like a Phase 4 job as it involves a deep refactor of the way we're generating the different layers.

[mvl22]

We need to use exactly the same dump of OSM data. Maybe that'd mean synchronizing with CycleStreets, if they use Geofabrik or something else public?

We can work to a specific Geofabrik datestamp if necessary, as the project has a dedicated build setup.

In practice OSM data doesn't change all that match, but yes some small fraction of a percent may be out.

As a general principle, it is generally helpful to maintain OSM IDs through a pipeline whenever possible.

NPW imports a different set of the OSM ways. I think it's a superset of what CycleStreets considers using for routing, though, so it'd be OK to try matching.

We use everything relevant to cycling, but of course things like motorways get dismissed quickly from the processing. Not sure what difference you were mainly referring to.

[dabreegster]

We use everything relevant to cycling, but of course things like motorways get dismissed quickly from the processing. Not sure what difference you were mainly referring to.

https://github.com/nptscot/npw/blob/8766e143feb759b283b938eb65f7ea703d9ee84f/backend/src/existing.rs#L28
When None is returned here, NPW discards it, otherwise it's part of the editable network. You're right, there shouldn't be differences here. If there were cases where either of us merges two OSM ways into one at a degree-2 node, there could be some messiness, but shouldn't be relevant here.

For more context, the map matching here is used from combined_network.gpkg to determine the high/medium/low cycling flow layers, which is one of the key metrics. The quality of the map-matching isn't great right now, but I'll try to make improvements without requiring an OSM ID to help match.

[mvl22]

If there were cases where either of us merges two OSM ways into one at a degree-2 node

That should logically only be done where the OSM tags exactly match between the two Ways, or where any difference is genuinely completely irrelevant to the effect on cycling.

[Robinlovelace]

Many-to-many relationships can be represented, e.g.

npt_id	osm_id
1	9,6
2	5
3	5
4	2

question for all: should we only include 1 OSM id (the main one that overlaps with the linestring) or many?

[dabreegster]

What's the "main overlapping linestring"? Aren't you getting back a list of routing steps at the OSM way level, and doing something with them? What are the NPT IDs?

[Robinlovelace]

NPT ID is what I wrote for want of a better term for the route networks we're using:

• Mostly-OSM-level ways that sometimes are shorter and sometimes longer than OSM ways, shown here: https://nptscot.github.io/#/rnet/#13.76/55.93906/-3.19774
• The simplified network, which contains linestrings that are mostly longer and will generally have multiple OSM ids, shown here: https://nptscot.github.io/#/rnet-simplified/#13.76/55.93906/-3.19774
• The coherent network which is mostly the same as the simplified network: https://nptscot.github.io/#/coherentnetwork/#13.76/55.93906/-3.19774

May have been better to say "Row number", right? In any case interested to hear with your thoughts on segments that contain multiples OSM ids, we could say how many meters of each runs parallel to each linestring using the weighting functionality in the anime crate. My guess is that this would be the most useful representation:

row_number	osm_ids
1	9,6

etc.

No immediate plans to do this as I say but will be good to get feedback on what the resulting osm_ids field will look like.