Mode "TRANSIT" sometimes leads to crash of detailed_itineraries function if Transit is not available for a specific departure time

[SRN1973]

This is an addition to my issue #144

I encounterd a really strange behaviour in the detailed_itineraries function, that seems to be somehow connected with the parameters passed for transportation mode and departure_datetime. For some points the function crashes with an error message if the mode parameter contains "TRANSIT". However when changing the departure_datetime e.g. by adding 2400 seconds then the calculation is performed as expected...The same is true when removing "TRANSIT" from the mode parameter (what is not an option of course).

One of the strange things is that the verbose output indicates that r5 deliverd a result for the path - request that resulted in the crash...

---

OPERATING SYSTEM:
Ubuntu 20.04.1 LTS
RAM: 1 TB
120 cernels

INPUT FILES:
germany-latest-osm.pbf (https://download.geofabrik.de/europe/germany-latest.osm.pbf) with droped relations as recommended here: #141
-GTFS files (whole public transport for Germany):
https://download.gtfs.de/germany/fv_free/latest.zip
https://download.gtfs.de/germany/nv_free/latest.zip
https://download.gtfs.de/germany/rv_free/latest.zip

REPRODUCIBLE CODE:
The error can be reproduced by running following code for a network.dat constructed from the files above.
(@ mvpsaraiva I have sent you a private email to your uk email address with a download link to my network.dat in case you are interested in a reproducible example)

#This departure time will cause the error
departure_datetime <- as.POSIXct("20-10-2020 09:00:00",format = "%d-%m-%Y %H:%M:%S",tz = "Europe/Berlin")

#This departure time results in a result as expected
departure_datetime <- as.POSIXct("20-10-2020 09:40:00",format = "%d-%m-%Y %H:%M:%S",tz = "Europe/Berlin")

r5r_core <- setup_r5(data_path = data_path, version = "6.0.1", verbose=FALSE)

detailed_itineraries(r5r_core = r5r_core,
origins = data.frame(id=c("1"), lat=(52.63280), lon=c(7.272448)),
destinations = data.frame(id=c("1"), lat=(52.69058), lon=c(7.285789)),
mode = c( "WALK", TRANSIT)
mode_egress = "WALK",
departure_datetime = departure_datetime,
max_walk_dist = Inf,
shortest_path = TRUE,
verbose = TRUE,
drop_geometry = TRUE,
n_threads = 40
)

ERROR MESSAGE IF THE FUNCTION CRASHES:

...
esource limiting issue and paths may be incorrect.
13:55:10.204 [main] DEBUG com.conveyal.r5.streets.StreetRouter - Using distance limit of 1000 meters
13:55:10.205 [main] DEBUG com.conveyal.r5.streets.StreetRouter - Routing took 1 msec
13:55:10.205 [main] WARN com.conveyal.r5.streets.StreetRouter - Setting a distance limit when distance is not the dominance function, this is a resource limiting issue and paths may be incorrect.
13:55:10.205 [main] DEBUG com.conveyal.r5.streets.StreetRouter - Using distance limit of 1000 meters
13:55:10.205 [main] DEBUG com.conveyal.r5.streets.StreetRouter - Routing took 0 msec
13:55:10.205 [main] INFO c.c.r.p.builder.PointToPointQuery - Returned 10 options
13:55:10.205 [main] INFO c.c.r.p.builder.PointToPointQuery - Took 53 ms
Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
java.lang.NullPointerException
In addition: Warning messages:
1: In assert_points_input(origins, "origins") :
'origins$id' forcefully cast to character.
2: In assert_points_input(destinations, "destinations") :
'destinations$id' forcefully cast to character.

[SRN1973]

In Addition to my last comment. When experimenting a little bit further in order to isolate the reasons for the strange behaviour I ralized that the error occurs in connection with the setting of the "walk_speed"-parameter (not for all to-from combinations but for single combinations).

I assume that this parameter has to be in km/h and then gets converted in m/s by the strange 5/18 multiplier). That is, if I insert a walking speed in the interval >= 2 to <=5.4 I receive the error explained in my last comment. Lower or higher walking speed values lead to a result. (Dependent on the max_walk_dist respectively max_trip_duration -parameters (e.g. higher or lower values) for these walking speed values I receive either a NULL result or a route result.)

To be concise: following walking speeds in meters per second do lead to the error described:
1.4, 1.3, 1.2, 1.1, 1.0, 0.9, 0.8, 0.7, 0.3, 0.1
Following walking speeds in meters per second work:
0.2, 0.4, 0.5, 1.5 and above

For me this is problematic as especially the range of 1.3 m/s to 1.45 m/s is the normal walking speed of people according to several studies...

(@mvpsaraiva with the data I provided you you should be able to reproduce the bahaviour by setting following parameters)

departure_datetime <- as.POSIXct("20-10-2020 09:00:00",format = "%d-%m-%Y %H:%M:%S",tz = "Europe/Berlin")
mode <- c("WALK", "TRANSIT")

detailed_itineraries(r5r_core = r5r_core,
origins = data.frame(id=c(1), lat=(52.63280), lon=c(7.272448)), #vonPlace,
destinations = data.frame(id=c(1), lat=(52.69058), lon=c(7.285789)), #)zuPlace
walk_speed= 4.6 , # (~1.3 m/s)
mode = mode,
mode_egress = "WALK",
departure_datetime = departure_datetime,
max_walk_dist = 600000,
shortest_path = TRUE,
verbose = TRUE,
drop_geometry = TRUE,
max_trip_duration = 600000,
n_threads = 1
)

[mvpsaraiva]

Hi @SRN1973. Sorry about the late response, but this was a tricky bug to catch.

Both problems are the same: the departure time and walking speed actually triggered the same bug. Some of your gtfs feeds have a shapes.txt file, while others don't, which caused r5r to think they all had shapes. Although you're not using the itineraries' geometries in your results (by using drop_geometry = TRUE), r5r used that geometry to calculate travel distances. In short, r5r was trying to calculate distances over a NULL shape, hence the java.lang.NullPointerException.

Depending on the departure time and walking speed, R5 found a different shortest path. In your reprex, if the chosen path included the route 10524 from the nv gtfs, which has no shape, the program crashed.

If you use the current development version of r5r, installed from GitHub, this issue should be solved. I'll keep testing here, but let me know if it works now.

Best,
Marcus

[SRN1973]

Thank you for fixing this bug...however feeding all my from-to destinations to the detailed_itineraries function I still encounter a quite similar error when trying to request a connection for following points (with the same network and routing parameters etc. I provided you last time [see above]...)

From Point:
id : 8521
lat: 52.40011
lon: 12.12148

To Point:
id: 2799
lat: 52.40698
lon : 12.17273

The error message is not very helpful to me. The problem seems to be the " java.lang.ArrayIndexOutOfBoundsException" error not the additional warning:

Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
java.lang.ArrayIndexOutOfBoundsException
In addition: Warning messages:
1: In assert_points_input(origins, "origins") :
'origins$id' forcefully cast to character.
2: In assert_points_input(destinations, "destinations") :
'destinations$id' forcefully cast to character.

However, I suspect the problematic point is the from point as it lies more than 500 m away from every street etc. in the middle of a meadow...Is there some kind limit in r5 or r5r that prevents that points with a certain distance ggreater than x get connected with the road network?

The problem with this error is, that, when feeding a bigger "chunk" of from to connections to the detailed_itineraries function nothing is returned in the end if it contains such an errorenous request... although the function seems to correctly calculate an output for the other from to requests in the function...So I think it would be helpful to add a mechanism that keeps the correct results and only omitts the problematic ones instead of returning a totally empty result object...
...maybe with a second result object that registers the problematic requests for further inspection later on...

p.s. Now the development function's output is much more detailed. However, is there a way to supress the console output of the routing progress? This would be helpful.

regards,
Stefan

[mvpsaraiva]

Hi @SRN1973. I think we fixed this issue, and have discussed a lot via email. Thanks for your feedback.
I'm closing this issue now, but feel free to open it again (or a new one) if you have further questions.