Graph building hangs on 'Intersecting unconnected areas' on latest Master

[clinct]

Been trying to build the Dutch graph with the latest Master today, it hung for more then 40 minutes on "intersecting unconnected areas", while normally a graph build takes around 25 minutes on the same machine.

After reading through the last commits and checking-out different commits, the issue seems to be introduced with commit 9856f2f: "Fix #1562 - handle intersecting unconnected P+R."

I'm using the pbf from: http://download.geofabrik.de/europe/netherlands-latest.osm.pbf

Is this expected to take so long? Should I just keep it running? Or does it point to something going wrong?

[abyrd]

It is not expected to be slow, this is probably just unexpected inefficiency. Some of these geometric algorithms (including the visibility graph algorithm) can break down with certain inputs. This code was added recently and seems to need some revision. With tomorrow's holiday, @laurentg is out of the office but we can probably improve it later in the week.

[laurentg]

It's not expected to be so slow, but I'm a bit afraid of the performance for large graph as the performance may not be O(n). To minimize memory usage we only store P+R areas in the spatial index, and then query for all ways the P+R areas intersecting them, thus one spatial index query per edge. Spatial index query may not be O(1) depending on the spatial index implementation.

If it's a problem you can safely disable L335 in OSMDatabase processUnconnectedAreas(); waiting for a proper fix/workaround.

[laurentg]

See #1617 on performance: my fears are overstated.

[abyrd]

This change is included in bugfix tag 0.12.1. @mattwigway reports that a New York graph build has been stuck overnight.

[laurentg]

Which extract is @mattwigway using, and which branch? I re-tested here with the whole of new-york state (http://download.geofabrik.de/north-america/us/new-york-latest.osm.pbf from a few weeks ago) with the fix on my branch and it builds fine.

[abyrd]

He is using tag 0.12.1. I assume that the OSM data he's using is similar to that NY extract... but it might have come out of Vanilla Extract. I hope we're not feeding OTP invalid input due to a bug or lacking feature in Vex. @mattwigway can you confirm?

[laurentg]

Just to make sure I re-tested with the latest version from download.geofabrik.de (from a few minutes ago) and it builds fine. For the record intersecting takes 4.6 seconds on my laptop (total build time is 5.2 minutes). I tested on branch "unconnected-P+R", but everything in it is in master (https://github.com/laurentg/OpenTripPlanner-plannerstack/compare/unconnected-P%2BR)

[abyrd]

@mattwigway has traced this to a specific way in Hackettstown, New Jersey: http://www.openstreetmap.org/way/305082958

So it should be reproducible with a New Jersey OSM extract.

The road already shares a node with its park and ride lot, but both the road and the lot exist in two perfectly overlaid copies. Therefore each copy of the road intersects its twin's copy of the park and ride lot without sharing a node, and the intersection perfectly coincides with an existing node. Presumably a zero-length split occurs at the end of a line segment, and this continues recursively.

While this is a pretty crazy situation, it seems that the same problem could arise with less unusual inputs. A road would just have to pass precisely over a node in a parking lot, or the road could have a node in exactly the same location as one in the parking lot without sharing it.

Matt is implementing an epsilon-distance check which will reuse an existing node if the split node is too close to the end of a line segment.

[mattwigway]

It doesn't need an NJ OSM extract, a tiny slice will do: http://api.openstreetmap.org/api/0.6/map?bbox=-74.80558,40.84353,-74.80136,40.84541