JOSM validation: merge duplicate geometries

[roelderickx]

Story

When converting a shapefile to osm it happens that the JOSM validator reports several warnings or errors. Some common examples:

• Error: Duplicated ways. These are ways of which all nodes are the same and which also have the same tags. Both duplicate ways may exist in the opposite direction (for JOSM this is only valid if either no oneway tag or if oneway=no is specified) or when they are circular they may have a different start point.
• Warning: Ways with same position. These are duplicate ways, but with different tags

As mentioned in JOSM issue 10761, ogr2osm is responsible to avoid these errors and warnings. Just like duplicate nodes are already merged, duplicate ways and duplicate relations should be merged as well.

There is however an issue regarding tags. Currently when two duplicate nodes are merged the tags of the first node prevail while the tags of the second node are simply omitted. It would be better to keep the tags of the second node when the first node does not contain any tags. The same logic is valid for duplicate ways and duplicate relations.

Some use-cases require nodes, ways or relations not to be merged. To avoid merging nodes there is a translation method get_unique_node_identifier, which can be redefined to return an integer counter or a combination of coordinates and tags. Comparable method(s) must be available to disable or limit merging ways or relations.

Modifications

• Merge ways
• Merge relations
• Add support for multipoint (these translate as multiple nodes, which are already merged)
• Add support for multilinestring (these translate as multiple ways, which are already merged)
• Add support for multipolygon (these translate as a single relation, special case)
• Add support for collections
• Merge tags

Test cases

• Duplicate nodes
• Duplicate ways
• Reverse duplicate ways
• Duplicate relations
• Multipoint support
• Multilinestring support
• Multipolygon support
• Collection support
• Duplicate multipoint
• Duplicate multilinestring
• Duplicate multipolygon
• Duplicate collections
• Merge tags

[roelderickx]

Duplicate nodes, ways and relations

When trying to add a geometry the first thing to do is check if a duplicate geometry already exists.

• For nodes this means the coordinate of the node to be added is the same as the coordinate of an already added node
• For ways this means all nodes of the way to be added are the same as all nodes of an already added way. Additionally, the order of the nodes in both ways should be either the same or reversed. In case of circular ways the first node may be different.
• For relations this means all members of the relation to be added are the same and have the same role as all members of an already added relation.

As a result a number of potentially duplicate geometries may be found. The next step is to match the tags of all of these geometries with the tags of the geometry to be added. If the tags can be merged then a duplicate is succesfully found, if the tags of none of the potentially duplicate geometries can be merged a new geometry has to be added to the output file. To create valid osm files the tags of duplicate geometries should always be merged.

Note that this logic overlaps with the translation method get_unique_node_identifier. Therefore, this translation method will be deprecated in favour of a new translation method merge_tags. This new method will be called for all duplicate geometries, whether it be nodes, ways or relations. The method should either confirm both geometries are duplicates and return the merged tags or None if both geometries are to be considered different.

A branch josmvalidation has been added where the above logic is implemented.

[roelderickx]

Multipoint, multilinestring and multipolygon

Both multilinestrings and multipolygons are already supported.

• A multilinestring is a set of linestrings, which can be translated to multiple ways.
• A multipolygon is a set of polygons will be translated to a single relation containing all polygons (each translated as an outer way and optionally a number of inner ways) in stead of multiple relations each containing one polygon.

Support for multipoints has been added, they will be translated to multiple nodes.

Collections

Shapefile collections contain one or more points, linestrings, relations or a mix of all those types. There are presumably little to no shapefiles containing geometry collections since they will crash ogr2osm and no issue has been reported before.

Current functionality

A top-level relation is created containing the nodes, ways and relations generated from the children of the collection, each with role member. Meanwhile the problem causing the crash is addressed and the role of ways has been modified to outer for the first way and inner for consecutive ways. The JOSM validator is now slightly more positive about it but this behaviour is only correct under limited circumstances.

Proposed functionality

The points and linestrings in the collection can be added to the osm file as nodes and ways, but analog to the functionality of multipoints and multilinestrings they should not be added to a parent relation. The remaining polygons can then be regarded as a multipolygon.

[roelderickx]

All modifications regarding merging of nodes, ways and relations including their tags are implemented and merged into master. I am in the process of integration testing, everything looks fine and performance is ok as well. If no new issues arise this will become version 1.1.0.