Errors on unusual relation roles

[jparta]

I'm running the following query:

[out:json][timeout:25];
(
  node(60.11,24.55,60.24,24.85);
  way(60.11,24.55,60.24,24.85);
  relation(60.11,24.55,60.24,24.85);
);
out body;
\>;
out skel qt;

And getting these errors:

Errors

Ref for multiline relation not found in index {'ref': 15513851, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685382, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685382, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685382, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 10002448, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685383, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 399906, 'role': 'destination', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 399906, 'role': 'destination', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 15513851, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 448858, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 448858, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 2050744, 'role': '', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 2474203, 'role': '', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685382, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685382, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685383, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 10002448, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685382, 'role': 'platform', 'type': 'relation'} Failed to convert relation to shape: {'id': 9376828, 'members': [{'ref': 26991652, 'role': '', 'type': 'way'}, {'ref': 26991649, 'role': '', 'type': 'way'}, {'ref': 8321534, 'role': '', 'type': 'relation'}, {'ref': 8321533, 'role': '', 'type': 'relation'}], 'tags': {'name': '510 Herttoniemi ' '(M)–Pasila–Meilahti–Tapiola(M)–Westendinasema', 'network': 'HSL', 'network:wikidata': 'Q473211', 'public_transport:version': '2', 'ref': '510', 'route_master': 'bus', 'type': 'route_master'}, 'type': 'relation'} Ref for multiline relation not found in index {'ref': 15513851, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685382, 'role': 'platform', 'type': 'relation'} Ref for multiline relation not found in index {'ref': 11685382, 'role': 'platform', 'type': 'relation'} Failed to convert relation to shape: {'id': 12842503, 'members': [{'ref': 13321691, 'role': 'outline', 'type': 'relation'}, {'ref': 875521179, 'role': 'part', 'type': 'way'}, {'ref': 992408323, 'role': 'part', 'type': 'way'}, {'ref': 992408326, 'role': 'part', 'type': 'way'}, {'ref': 992408327, 'role': 'part', 'type': 'way'}, {'ref': 992408317, 'role': 'part', 'type': 'way'}, {'ref': 992408322, 'role': 'part', 'type': 'way'}, {'ref': 992408318, 'role': 'part', 'type': 'way'}], 'tags': {'type': 'building'}, 'type': 'relation'} Failed to convert relation to shape: {'id': 14921282, 'members': [{'ref': 537248097, 'role': 'outline', 'type': 'way'}, {'ref': 14921281, 'role': 'part', 'type': 'relation'}, {'ref': 1116239370, 'role': 'part', 'type': 'way'}], 'tags': {'type': 'building'}, 'type': 'relation'} Failed to convert relation to shape: {'id': 14921283, 'members': [{'ref': 537248096, 'role': 'outline', 'type': 'way'}, {'ref': 14921280, 'role': 'part', 'type': 'relation'}, {'ref': 1116239371, 'role': 'part', 'type': 'way'}], 'tags': {'type': 'building'}, 'type': 'relation'} Failed to convert relation to shape: {'id': 14921284, 'members': [{'ref': 630120572, 'role': 'outline', 'type': 'way'}, {'ref': 14921279, 'role': 'part', 'type': 'relation'}, {'ref': 1116239369, 'role': 'part', 'type': 'way'}], 'tags': {'type': 'building'}, 'type': 'relation'}

It seems the roles causing problems are
'platform'
'destination'
'' (empty)
'outline'
'part'

Is this a known issue, or a "won't fix"?

Python 3.11.3
osm2geojson 0.2.4

[rapkin]

Hi, it would be better if you could find a smaller example. That way, I will be able to test with some data. Also, it would be perfect if you could provide some feedback on how you think this should work. This library is under development, and I do not rule out that some nuances of the OSM standard may not be taken into account.

[jparta]

Here is a JSON object where the keys are small bounding boxes which contain features causing errors, and the values are (roles, types) tuples of the features where errors were encountered.

https://gist.github.com/jparta/086261c5a9dfa9a262edea8e02d83c31

Here's a function to parse the keys:

def bbox_string_to_tuple(bbox_string: str):
    as_list = bbox_string.strip("()").replace(" ", "").split(",")
    return tuple(map(float, as_list))

Query template:

query_template = """
[out:json][timeout:25];
(
  node({bbox});
  way({bbox});
  relation({bbox});
);
out body;
\>;
out skel qt;
"""

[jparta]

I don't quite understand the error, because I don't know what causes a feature to not be found in the refs_index. What I expect it to do is to parse the features without errors, or give a more helpful error explaining why the feature cannot be parsed.

[cbeddow]

Here is some code I made that corrected this on the Overpass response, then you can send data back to osm2geojson.json2geojson(). Not perfect, but a start:

        elements_relations_without_members = [feature for feature in data['elements'] if 'members' not in feature and feature['type'] == 'relation']
        elements_relations_with_all_members = [feature for feature in data['elements'] if feature['type'] == 'relation' and 'members' in feature and all(
            ref in data['elements'] for ref in feature['members']
        )]
        elements_with_members_missing = [feature for feature in data['elements'] if feature['type'] == 'relation' and 'members' in feature and not all(
            ref in data['elements'] for ref in feature['members']
        )]
        fixed_elements = []
        for el in elements_with_members_missing:
            refs = [mem['ref'] for mem in el['members']]
            members = [obj for obj in el['members'] if obj['ref'] in [ent['id'] for ent in data['elements']]]
            el['members'] = members                       
            fixed_elements.append(el)
        recombined_elements = elements_not_relations + elements_relations_without_members + elements_relations_with_all_members + fixed_elements
        data['elements'] = recombined_elements```

[rapkin]

@cbeddow I see that your code has some elements which are already implemented in the current logic of the osm2geojson package, but maybe some other unique cases are handled by this code. I'll try to compare this logic with what we have in this package and maybe improve it. But, unfortunately, I don't have a fair amount of free time soon. So if you want to make a PR with your changes, I'll appreciate that. In any case - thank you!