Error with buildning routable transport network

[manneolfv]

Hello,

I am trying to build a network in r5r. I've used this guide https://ipeagit.github.io/r5r/articles/r5r.html. I've checked if I have the correct Java version
rJava::.jcall("java.lang.System", "S", "getProperty", "java.version") [1] "11.0.18"
In my data_path I have one osm (Sweden) in pbf format. I also have a some gtfs feeds and two csv files with points.

> list.files(data_path) [1] "blekinge.zip" "halland.zip" "jlt.zip" "klt.zip" "krono.zip" [6] "sj.zip" "skane.zip" "sweden-latest.osm.pbf" "sweden-latest.osm.pbf.mapdb" "sweden-latest.osm.pbf.mapdb.p" [11] "test_res.csv" "test_res2.csv"

When I use the following code

r5r_core <- setup_r5(data_path = data_path)

I get this error message:
"Error in setup_r5(data_path = data_path) : java.lang.NullPointerException"

I've downloaded my map from geofabrik. Is it something with my GTFS feeds? I have feeds from several regions and need to include them all.

Thanks in advance

Regards

[rafapereirabr]

Hi @manneolfv ,

Please run the r5r::r5r_sitrep() function to generate a situation report, and paste the output below. This will help us find eventual bugs.

Also , try running this code below. This is a reproducible example with a sample data form r5r. If you don't have any problems running this example, it means everything is alright with the installation of r5r.

options(java.parameters = "-Xmx2G")

library(r5r)

# build transport network
data_path <- system.file("extdata/poa", package = "r5r")
r5r_core <- setup_r5(data_path)

# load origin/destination points
points <- read.csv(file.path(data_path, "poa_points_of_interest.csv"))

departure_datetime <- as.POSIXct(
  "13-05-2019 14:00:00",
  format = "%d-%m-%Y %H:%M:%S"
)

ttm <- travel_time_matrix(
  r5r_core,
  origins = points,
  destinations = points,
  mode = c("WALK", "TRANSIT"),
  departure_datetime = departure_datetime,
  max_trip_duration = 60
)

head(ttm)

[dhersz]

I'd also try running setup_r5() with verbose output (r5r_core <- setup_r5(data_path = data_path, verbose = TRUE)) to see if it gives any more hints of what the problem might be.

[manneolfv]

Hi again,

Thanks for the help. I realized that I did not have the latest updates for all of the packages. But now when I've updated I have another problem buildning the network (this error comes before the last one so there is a possibility that I still would receive the same one as before as well):

"Error in setup_r5(data_path = data_path) :
java.lang.IllegalArgumentException: Geographic extent of street layer (1045411 km2) exceeds limit of 975000 km2."

I have a pbf file with all parts of Sweden. I am only interested in the southern parts so if it is possible to reduce the size that would be a solution. Another one maybe could be to change the limit - is that possible?

[manneolfv]

And here is the output from r5r::r5r_sitrep()

$r5r_package_version
[1] ‘1.0.0’

$r5_jar_version
[1] "6.8"

$java_version
[1] "11.0.18"

$set_memory
[1] "-Xmx2G"

$session_info
R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=Swedish_Sweden.utf8 LC_CTYPE=Swedish_Sweden.utf8 LC_MONETARY=Swedish_Sweden.utf8 LC_NUMERIC=C LC_TIME=Swedish_Sweden.utf8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] data.table_1.14.6 sf_1.0-9 r5r_1.0.0 devtools_2.4.5 usethis_2.1.6 ggplot2_3.4.1 tidyverse_2.0.0 osmextract_0.4.1 writexl_1.4.2

loaded via a namespace (and not attached):
[1] Rcpp_1.0.9 prettyunits_1.1.1 class_7.3-20 ps_1.7.2 digest_0.6.31 utf8_1.2.2 mime_0.12 R6_2.5.1 backports_1.4.1
[10] e1071_1.7-12 pillar_1.8.1 rlang_1.0.6 rstudioapi_0.14 miniUI_0.1.1.1 urlchecker_1.0.1 callr_3.7.3 checkmate_2.1.0 stringr_1.5.0
[19] htmlwidgets_1.6.1 munsell_0.5.0 proxy_0.4-27 shiny_1.7.4 compiler_4.2.2 httpuv_1.6.9 pkgconfig_2.0.3 pkgbuild_1.4.0 htmltools_0.5.4
[28] tidyselect_1.2.0 tibble_3.1.8 fansi_1.0.3 crayon_1.5.2 dplyr_1.0.10 withr_2.5.0 later_1.3.0 grid_4.2.2 xtable_1.8-4
[37] gtable_0.3.1 lifecycle_1.0.3 DBI_1.1.3 magrittr_2.0.3 units_0.8-1 scales_1.2.1 KernSmooth_2.23-20 cli_3.6.0 stringi_1.7.12
[46] cachem_1.0.7 fs_1.6.1 promises_1.2.0.1 remotes_2.4.2 ellipsis_0.3.2 generics_0.1.3 vctrs_0.5.1 tools_4.2.2 glue_1.6.2
[55] purrr_1.0.1 processx_3.8.0 pkgload_1.3.2 fastmap_1.1.1 colorspace_2.1-0 sessioninfo_1.2.2 classInt_0.4-8 memoise_2.0.1 rJava_1.0-6
[64] profvis_0.3.7

[rafapereirabr]

Hi @manneolfv . Did you manage to run the reproducible example I suggest above?

Regarding the geographic extent limit, unfortunately, this is a limit hardcoded upstream in R5 so we cannot change it. The solution here would be to crop the OSM to your area of interest

[mvpsaraiva]

I have a pbf file with all parts of Sweden. I am only interested in the southern parts so if it is possible to reduce the size that would be a solution. Another one maybe could be to change the limit - is that possible?

Hi @manneolfv. As @rafapereirabr said, we cannot change the work area size limit. But you can crop the .pbf file to cover only the areas you're interested in. You can use this guide.

[mvpsaraiva]

This seems to be a bug upstream in R5: issue 196. It was fixed in R5 6.9, so we need to update r5r as well.

[rafapereirabr]

Thanks, Marcus. I've updated r5r to use R5's latest v6.9. The new r5r v1.0.1 is on its way to cran.

[manneolfv]

Hi @manneolfv . Did you manage to run the reproducible example I suggest above?

Regarding the geographic extent limit, unfortunately, this is a limit hardcoded upstream in R5 so we cannot change it. The solution here would be to crop the OSM to your area of interest

Hi, yes I the reproducible example worked fine for me.

I will try to crop the OSM and when I've done that I will update here if it works fine afterwards.

[rafapereirabr]

Nice. Thanks. Please use the new r5r v1.0.1 from CRAN

[manneolfv]

Hello again,

I've updated the packages and now I am using r5r v1.0.1. I've also downloaded a new pbf file from Protomaps with the area I am interested in. I still receive the following error
Error in setup_r5(data_path = data_path, verbose = TRUE) : java.lang.NullPointerException

And the output from r5r_sitrep()
$r5r_package_version
[1] ‘1.0.1’

$r5_jar_version
[1] "6.9"

$java_version
[1] "11.0.18"

$set_memory
[1] "-Xmx2G"

$session_info
R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=Swedish_Sweden.utf8 LC_CTYPE=Swedish_Sweden.utf8 LC_MONETARY=Swedish_Sweden.utf8 LC_NUMERIC=C LC_TIME=Swedish_Sweden.utf8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] data.table_1.14.6 sf_1.0-9 r5r_1.0.1 devtools_2.4.5 usethis_2.1.6 lubridate_1.9.2 forcats_1.0.0 stringr_1.5.0 dplyr_1.1.0 purrr_1.0.1
[11] readr_2.1.4 tidyr_1.3.0 tibble_3.1.8 ggplot2_3.4.1 tidyverse_2.0.0 osmextract_0.4.1 rJava_1.0-6

loaded via a namespace (and not attached):
[1] Rcpp_1.0.10 prettyunits_1.1.1 ps_1.7.2 class_7.3-21 digest_0.6.31 utf8_1.2.3 mime_0.12 R6_2.5.1 backports_1.4.1
[10] e1071_1.7-13 pillar_1.8.1 rlang_1.0.6 rstudioapi_0.14 miniUI_0.1.1.1 urlchecker_1.0.1 callr_3.7.3 checkmate_2.1.0 htmlwidgets_1.6.1
[19] munsell_0.5.0 proxy_0.4-27 shiny_1.7.4 compiler_4.2.2 httpuv_1.6.9 pkgconfig_2.0.3 pkgbuild_1.4.0 htmltools_0.5.4 tidyselect_1.2.0
[28] fansi_1.0.4 crayon_1.5.2 tzdb_0.3.0 withr_2.5.0 later_1.3.0 grid_4.2.2 xtable_1.8-4 gtable_0.3.1 lifecycle_1.0.3
[37] DBI_1.1.3 magrittr_2.0.3 units_0.8-1 scales_1.2.1 KernSmooth_2.23-20 cli_3.6.0 stringi_1.7.12 cachem_1.0.7 remotes_2.4.2
[46] fs_1.6.1 promises_1.2.0.1 ellipsis_0.3.2 generics_0.1.3 vctrs_0.5.2 tools_4.2.2 glue_1.6.2 hms_1.1.2 pkgload_1.3.2
[55] processx_3.8.0 fastmap_1.1.1 timechange_0.2.0 colorspace_2.1-0 sessioninfo_1.2.2 classInt_0.4-9 memoise_2.0.1 profvis_0.3.7
`

[mvpsaraiva]

Hi! Can you paste here the full output of setup_r5()? Also, can you share the .pbf file?

[manneolfv]

Hi,

The output is very long, like a thousands rows since it starts the setup but after a while crashes with the error. But I saved the first rows from the output and then I saved the last part as well, here it is:
output_setupr5r.txt

The pbf file is too big to upload here (254 MB). Do you know where I could share it to you?

[mvpsaraiva]

That's interesting, because the error seems to be happening when loading the GTFS, after the .pbf is loaded. So it maybe a different t bug than that solved by R5 6.9.

First, try removing the gtfs files from the folder and run setup_r5 only on the street network. If it works, than there might be something wrong with the gtfs. If it doesn't work, then send us the .pbf via some google drive or Dropbox link.

[manneolfv]

Hi,

Thanks for the help, the pbf looks fine since it worked for me when I removed the gtfs files from the folder. It also works for me when I am calculating travel times with car, but the problem with transit mode remains. Which gives me a hint that it is the GTFS feed that causes the problem. It worked for me building a network when I updated my GTFS feed. The former feed I edited becauseI needed to remove routes coded as "taxi".

Now I have a similar error I've encountered before: Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, : java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Taxi route_type code not supported: 1501

I've removed rows that included this route type before, and then it did not work building the network. Do you know how I can recode these route types as bus?

[stupidpupil]

Are you able to share both the original and your edited GTFS zips?

[manneolfv]

Hi,

yes, thank you. I've uploaded the feeds to google drive since it is too big to share here.

original gtfs - https://drive.google.com/file/d/1iJJIRYEDmS6nbFftDqN59k9veiez7iWS/view?usp=sharing

adjusted gtfs - https://drive.google.com/file/d/1RQB2gQ1mhQLOX-XtYka_fD2MgXQCRKy6/view?usp=sharing

[stupidpupil]

At least the 'adjusted' halland and blekinge GTFS zips seem to be invalid - they have entries in trips.txt without corresponding entries in routes.txt . I'd guess that R5 wouldn't like that.

(There are some issues with the attributions.txt files as well, but I'd guess that R5 wouldn't really care so much about these.)

[rafapereirabr]

I would suggest you use gtfstools::validate_gtfs() to have a full check on your GTFS data. This function can take a few minutes for large GTFS feeds. Code example below:

library(gtfstools)

local_gtfs_path <- "./users/desktop/gtfs.zip"

gtfs <- gtfstools::read_gtfs(local_gtfs_path)

# download validator
gtfstools::download_validator(path = '.')

# validate gtfs
gtfstools::validate_gtfs(gtfs = local_gtfs_path, 
                         validator_path  = 'gtfs-validator-v4.0.0.jar',
                         output_path = './')

See the outpu report in html.

[manneolfv]

Hi again,

I've managed to build the network with the GTFS files now after coding the routes 1501 to 700 (from taxi to bus). Previously I got the following error message: Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, : java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Taxi route_type code not supported: 1501

But now when I am trying to calculate the travel times with the GTFS feeds it only uses walk as transport mode. And therefore doesn't even calculate the travel times between two citites that are in different regions. The only textfiles I've edited is those named "routes" and I've just changed 1501 to 700 for the routes coded as taxi. Could something be wrong now with the GTFS after I've edited? I've never worked with GTFS feeds before and therefore I am not sure how to recode.

I've tried with multiple dates and it doesn't seem to be the problem with this issue.

[rafapereirabr]

Hi @route_type. According to the GTFS specification, bus routes should have a route_type coded with value 3, and not 700. Please refer to the GTFS documentation.

Also, please make sure you departure date is covered by the calendar of your GTFS data.

We recommend you use the gtfstools package to manipulate GTFS data in R.

[rafapereirabr]

closing this issue due to inactivity. Happy to reponen it if the issue persists