read_sf GeoPackage crashes R #1606

alazarolop · 2021-02-19T19:50:53Z

Hi,

I've just installed sf from GitHub (0.9-8) on macOS Big Sur (11.2.1).
There is no error when building the package, but when I try to read a GeoPackage, R crashes.

r$> library(sf)                                                                                                                                                               
Linking to GEOS 3.9.1, GDAL 3.2.1, PROJ 7.2.1

r$> world <- sf::read_sf("gadm36_levels.gpkg", layer = "level0")                                          

 *** caught segfault ***
address 0x100000000, cause 'memory not mapped'

Traceback:
 1: CPL_read_ogr(dsn, layer, query, as.character(options), quiet,     type, fid_column_name, drivers, wkt_filter, promote_to_multi,     int64_as_string, dsn_exists, dsn_isdb)
 2: st_read.character(..., quiet = quiet, stringsAsFactors = stringsAsFactors,     as_tibble = as_tibble)
 3: st_read(..., quiet = quiet, stringsAsFactors = stringsAsFactors,     as_tibble = as_tibble)
 4: sf::read_sf("gadm36_levels.gpkg",     layer = "level0")

I don't get any clue from the Traceback in order to fix it.
I'm able to read the GeoPackage with ogrinfo tough, so I guess it's not a problem with GDAL installation.

a@MBP gadm36_levels_gpkg % ogrinfo gadm36_levels.gpkg
INFO: Open of `gadm36_levels.gpkg'
      using driver `GPKG' successful.
1: level0 (Multi Polygon)
2: level1 (Multi Polygon)
3: level2 (Multi Polygon)
4: level3 (Multi Polygon)
5: level4 (Multi Polygon)
6: level5 (Multi Polygon)
a@MBP gadm36_levels_gpkg %

Moreover, I'm able to read a SHP file with sf.

r$> read_sf("pgsql-ogr-fdw/data/pt_two.shp")                                                                                                 
Simple feature collection with 2 features and 4 fields
geometry type:  POINT
dimension:      XY
bbox:           xmin: -0.581035 ymin: 0.2443327 xmax: 0.098329 ymax: 0.7032013
geographic CRS: WGS 84
# A tibble: 2 x 5
  name    age height birthdate               geometry
  <chr> <int>  <dbl> <date>               <POINT [°]>
1 Peter    45   5.6  1965-04-12  (0.098329 0.7032013)
2 Paul     33   5.84 1971-03-25 (-0.581035 0.2443327)

Could it be a problem with my sf installation or perhaps a bug with GeoPackage in sf?
I was able to read the file before upgrading to latest version of GDAL, GEOS, Proj and sf.

Thank you in advance.

The text was updated successfully, but these errors were encountered:

edzer · 2021-02-19T21:17:32Z

is this a binary or a source install?

alazarolop · 2021-02-19T21:42:26Z

From source.

I've also rebuilt from source rgdal and rgeos with no luck.

edzer · 2021-02-19T21:46:03Z

Have you tried binary installs?

alazarolop · 2021-02-20T08:27:25Z

No, I built R from source to get OpenBLAS support and so AFAIK I can't install binaries, am I right?

Is there a way I could extract some debug information more than the Traceback?

PS: I've also installed everything from scratch in a new R installation, but I get the same outcome.

edzer · 2021-02-20T11:18:18Z

Is there a way I could extract some debug information more than the Traceback?

Yes, but I'm not a mac user so can't help you with that.

I guess you're using dynamic linking; a common source of problems is that you have more than one version of GDAL, GEOS and or PROJ installed on your system.

alazarolop · 2021-02-20T11:43:17Z

I'll check it out and I'll also recompile them from source.

I'll close the ticket in the meantime.
Thank you a lot for the support Edzer.

edzer · 2021-02-22T11:09:01Z

Solved?

alazarolop · 2021-02-22T11:40:59Z

Not really. I've rebuilt from source GDAL and R packages rgeos, rgdal, sf; but I'm getting the same issue.
I closed cause I said I would in the meantime and I can't also give more information on the topic.

alazarolop · 2021-02-22T12:02:02Z

I've just tried to rebuilt R from source and install the packages from scratch with no luck neither.

edzer · 2021-02-22T12:22:12Z

Did you make 100% sure there is one and only one of the GEOS, GDAL and PROJ libraries on your system?

alazarolop · 2021-02-22T12:24:53Z

I apologise for closing and reopening the issue.

I get the same issue using rgdal approach, either trying to metadata or the data:

rgdal::ogrInfo("data/project_sites.gpkg")

 *** caught segfault ***
address 0x100000000, cause 'memory not mapped'

Traceback:
 1: ogrListLayers(dsn = dsn)
 2: rgdal::ogrInfo("data/project_sites.gpkg")

readOGR("data/project_sites.gpkg")

 *** caught segfault ***
address 0x100000000, cause 'memory not mapped'

Traceback:
 1: ogrListLayers(dsn = dsn)
 2: readOGR("data/project_sites.gpkg")

So I guess it's exceed sf and can be related to GDAL. It's working in the command line tough:

ogrinfo data/project_sites.gpkg
INFO: Open of `data/project_sites.gpkg'
      using driver `GPKG' successful.
1: sites (Point)
2: sites_sum (Point)

My session info:

R version 4.0.4 (2021-02-15)
Platform: x86_64-apple-darwin20.3.0 (64-bit)
Running under: macOS Big Sur 11.2.1

Matrix products: default
BLAS/LAPACK: /usr/local/Cellar/openblas/0.3.13/lib/libopenblasp-r0.3.13.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_4.0.4

alazarolop · 2021-02-22T12:26:16Z

Did you make 100% sure there is one and only one of the GEOS, GDAL and PROJ libraries on your system?

Yes. I only install them using Homebrew and there is only one version for each. I don't have any installation from a third repository.

edzer · 2021-02-22T12:47:18Z

Did you compile the command line tool ogrinfo from source too?

alazarolop · 2021-02-22T12:49:47Z

I first used a binary downloaded from Homebrew and then I compiled it from source, in both cases I could read the GeoPackage without issues.

edzer · 2021-02-27T13:56:11Z

Since this happens both with rgdal and sf, it seems like a problem with your GDAL setup. I suggest to close here.

alazarolop · 2021-02-28T11:36:42Z

Thank you for the support.

alazarolop · 2021-02-28T19:00:23Z

@edzer I solved it with a workaround: creating a connection as a SQLite file and then reading it.
For future references:

x <- dbConnect(RSQLite::SQLite(), "data/project_sites.gpkg") %>%
  sf::st_read(layer = "sites", as_tibble = TRUE)

pat-s · 2021-03-07T08:44:36Z

Facing the same issue

macOS 11.2.1
R 4.0.4
sf 0.9-7

library(sf) 
st_read(system.file("gpkg/nc.gpkg", package="sf"), quiet = TRUE)                      
Linking to GEOS 3.9.1, GDAL 3.2.1, PROJ 7.2.1

 *** caught segfault ***
address 0x100000000, cause 'memory not mapped'

Traceback:
 1: CPL_read_ogr(dsn, layer, query, as.character(options), quiet,     type, fid_column_name, drivers, wkt_filter, promote_to_multi,     int64_as_string, dsn_exists, dsn_isdb)
 2: st_read.character(system.file("gpkg/nc.gpkg", package = "sf"),     quiet = TRUE)
 3: st_read(system.file("gpkg/nc.gpkg", package = "sf"), quiet = TRUE)

rsbivand · 2021-03-07T19:30:23Z

Please consider running R -d gdb and reporting the output.

alazarolop · 2021-03-07T20:13:02Z

Hi @rsbivand , this is my output:

a@MBP ~ % R -d gdb
/usr/local/bin/R: line 298: exec: gdb: not found

rsbivand · 2021-03-08T08:49:37Z

Try (untried) R -d lldb.

alazarolop · 2021-03-08T10:29:56Z

Try (untried) R -d lldb.

a@MBP ~ % R -d lldb
(lldb) target create "/usr/local/Cellar/r/4.0.4/lib/R/bin/exec/R"
Current executable set to '/usr/local/Cellar/r/4.0.4/lib/R/bin/exec/R' (x86_64).
(lldb)

Then, I could type text but not R commands.

edzer · 2021-03-08T10:34:31Z

(if this works like gdb, type r to run the R process, then the commands that lead to the crash)

alazarolop · 2021-03-08T10:44:35Z

Thanks for the clue @edzer . Now, it works like a charm.

> library(sf)
Linking to GEOS 3.9.1, GDAL 3.2.1, PROJ 7.2.1
> st_read(system.file("gpkg/nc.gpkg", package="sf"), quiet = TRUE)
Simple feature collection with 100 features and 14 fields
Geometry type: MULTIPOLYGON
Dimension:     XY
Bounding box:  xmin: -84.32385 ymin: 33.88199 xmax: -75.45698 ymax: 36.58965
Geodetic CRS:  NAD27

> sessionInfo()
R version 4.0.4 (2021-02-15)
Platform: x86_64-apple-darwin20.3.0 (64-bit)
Running under: macOS Big Sur 11.2.2

Matrix products: default
BLAS/LAPACK: /usr/local/Cellar/openblas/0.3.13/lib/libopenblasp-r0.3.13.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/POSIX/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] sf_0.9-8

loaded via a namespace (and not attached):
 [1] compiler_4.0.4     magrittr_2.0.1     class_7.3-18       tools_4.0.4
 [5] DBI_1.1.1          units_0.7-0        Rcpp_1.0.6         KernSmooth_2.23-18
 [9] grid_4.0.4         e1071_1.7-4        classInt_0.4-3

If I run it like usual without the flag R, I get the error. How could I add this fixed into my R workflow / loading process.

edzer · 2021-03-08T10:49:24Z

Oh, that is nasty! Could you try running the script with valgrind, i.e. within R -d valgrind ?

alazarolop · 2021-03-08T10:55:15Z

Ok! Unfortunately, I get the same output than with R -d gdb:

a@MBP ~ % R -d valgrind
/usr/local/bin/R: line 298: exec: valgrind: not found

I'm not sure if I needed to add something at the end to run it or similar:

R -d valgrind -e 'sf::st_read(system.file("gpkg/nc.gpkg", package="sf"), quiet = TRUE)'

But it outputs the same error.

edzer · 2021-03-08T11:05:59Z

You'd need to install valgrind.

alazarolop · 2021-03-08T11:12:50Z

It will take a bit of time, cause it's not available in Homebrew for macOS. I'll let you know as soon as I get it.

alazarolop · 2021-03-08T11:19:00Z

Unfortunately, it's not available for macOS 11.x

checking for a supported OS... ok (darwin20.3.0)
checking for the kernel version... unsupported (20.3.0)
configure: error: Valgrind works on Darwin 10.x, 11.x, 12.x, 13.x, 14.x, 15.x, 16.x and 17.x (Mac OS X 10.6/7/8/9/10/11 and macOS 10.12/13)

Is there another alternative that we could try?

ManuelSpinola · 2021-03-18T16:34:11Z

I am having the same problem.

Reading geopackage files crushes R

I am using macOS Big Sur 11.2.3

library(sf)
Linking to GEOS 3.9.0, GDAL 3.2.1, PROJ 7.2.1

sessionInfo()
R version 4.0.4 (2021-02-15)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur 10.16

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] sp_1.4-5

loaded via a namespace (and not attached):
[1] pillar_1.5.1 compiler_4.0.4 tools_4.0.4 downlit_0.2.1
[5] digest_0.6.27 lattice_0.20-41 viridisLite_0.3.0 jsonlite_1.7.2
[9] evaluate_0.14 lifecycle_1.0.0 tibble_3.1.0 gtable_0.3.0
[13] pkgconfig_2.0.3 rlang_0.4.10 cli_2.3.1 DBI_1.1.1
[17] distill_1.2 yaml_2.2.1 xfun_0.22 dplyr_1.0.5
[21] httr_1.4.2 knitr_1.31 generics_0.1.0 vctrs_0.3.6
[25] htmlwidgets_1.5.3 grid_4.0.4 tidyselect_1.1.0 glue_1.4.2
[29] data.table_1.14.0 R6_2.5.0 fansi_0.4.2 plotly_4.9.3
[33] rmarkdown_2.7 tidyr_1.1.3 ggplot2_3.3.3 purrr_0.3.4
[37] magrittr_2.0.1 scales_1.1.1 ellipsis_0.3.1 htmltools_0.5.1.1
[41] assertthat_0.2.1 colorspace_2.0-0 utf8_1.2.1 tinytex_0.30
[45] lazyeval_0.2.2 munsell_0.5.0 crayon_1.4.1

rsbivand · 2021-03-31T16:27:28Z

Same question on https://stat.ethz.ch/pipermail/r-sig-geo/2021-March/028632.html. I relied there:

Please refer to the associated sf issue on github: #1606 for completeness. Please confirm that you are not using the MacOS sf CRAN binary package, but have installed GDAL, PROJ and GEOS yourself. Please also confirm that you see the same error with every GPKG file.

rsbivand · 2021-03-31T16:51:02Z

Further, with R 4.0.4 on Rosetta but with CRAN binary packages, there is no problem. It is most likely that the problem is caused by user-installed upstream software components, and sf or rgdal installed from source, possibly with libsqlite3 also installed from source. Please report back if any other drivers fail - RSQLite does not use local binaries, but bundles the software library itself.

alazarolop · 2021-04-03T10:44:19Z

GPKG read and write work again flawlessly with the latest commits of sf (370f934) and the latest release of GDAL (3.2.2_1) in Homebrew. Thank you a lot for your support!

rsbivand · 2021-04-03T13:27:20Z

OK, for your installation, the GDAL 3.2.1 binary was broken but 3.2.2 is not, or your source GDAL 3.2.1 was broken for reasons unknown, but 3.2.2 is not. These systems are deterministic, so it would have helped (others, later on) to establish and reproduce what was broken. However, good to hear that 3.2.2 appears to work.

ManuelSpinola · 2021-04-03T13:54:47Z

Thank you very much Roger and Alberto. It works after updating GDAL 3.2.2 Manuel El sáb, 3 abr 2021 a las 7:27, Roger Bivand ***@***.***>) escribió:

…

OK, for your installation, the GDAL 3.2.1 binary was broken but 3.2.2 is not, or your source GDAL 3.2.1 was broken for reasons unknown, but 3.2.2 is not. These systems are deterministic, so it would have helped (others, later on) to establish and reproduce what was broken. However, good to hear that 3.2.2 appears to work. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1606 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFI3FB6MKTSPKQENYS4QKMTTG4JUHANCNFSM4X45562A> .

-- *Manuel Spínola, Ph.D.* Instituto Internacional en Conservación y Manejo de Vida Silvestre Universidad Nacional Apartado 1350-3000 Heredia COSTA RICA ***@***.*** ***@***.***> ***@***.*** Teléfono: (506) 8706 - 4662 Personal website: Lobito de río <https://sites.google.com/site/lobitoderio/> Institutional website: ICOMVIS <http://www.icomvis.una.ac.cr/>

alazarolop · 2021-04-03T18:39:59Z

@rsbivand I wouldn't say GDAL was broken at that point, since it was working properly and it was reading GPKG files in version 3.2.1. Many things have been upgraded since then, including GDAL version (not the way it was built in Homebrew) but also the sf R package (I haven't checked all the commits since the issue was opened). That's way I couldn't determine the reason, although I agree with you that it would have been interested to know it. Thank you for your support.

JimShady · 2022-10-27T07:42:43Z

I'm sporadically getting this issue (caught segfault, memory not mapped) on Ubuntu, with GDAL 3.0.2. Seems to normally happen with larger geopackages.

It seems the solution is to update GDAL to > 3.2.2 ? Or is it more complex than that?

Thanks

edzer · 2022-10-27T08:05:07Z

It seems the solution is to update GDAL to > 3.2.2 ? Or is it more complex than that?

reinstall sf after updating GDAL.

JimShady · 2022-10-27T08:14:17Z

It seems the solution is to update GDAL to > 3.2.2 ? Or is it more complex than that?

reinstall sf after updating GDAL.

Cheers

alazarolop closed this as completed Feb 22, 2021

alazarolop reopened this Feb 22, 2021

alazarolop closed this as completed Feb 28, 2021

alazarolop reopened this Mar 7, 2021

alazarolop closed this as completed Apr 3, 2021

read_sf GeoPackage crashes R #1606

read_sf GeoPackage crashes R #1606

Comments

alazarolop commented Feb 19, 2021

edzer commented Feb 19, 2021

alazarolop commented Feb 19, 2021

edzer commented Feb 19, 2021

alazarolop commented Feb 20, 2021 • edited Loading

edzer commented Feb 20, 2021

alazarolop commented Feb 20, 2021

edzer commented Feb 22, 2021

alazarolop commented Feb 22, 2021

alazarolop commented Feb 22, 2021

edzer commented Feb 22, 2021

alazarolop commented Feb 22, 2021 • edited Loading

alazarolop commented Feb 22, 2021

edzer commented Feb 22, 2021

alazarolop commented Feb 22, 2021

edzer commented Feb 27, 2021

alazarolop commented Feb 28, 2021

alazarolop commented Feb 28, 2021

pat-s commented Mar 7, 2021

rsbivand commented Mar 7, 2021

alazarolop commented Mar 7, 2021

rsbivand commented Mar 8, 2021

alazarolop commented Mar 8, 2021

edzer commented Mar 8, 2021

alazarolop commented Mar 8, 2021 • edited Loading

edzer commented Mar 8, 2021

alazarolop commented Mar 8, 2021 • edited Loading

edzer commented Mar 8, 2021

alazarolop commented Mar 8, 2021

alazarolop commented Mar 8, 2021 • edited Loading

ManuelSpinola commented Mar 18, 2021

rsbivand commented Mar 31, 2021

rsbivand commented Mar 31, 2021

alazarolop commented Apr 3, 2021 • edited Loading

rsbivand commented Apr 3, 2021

ManuelSpinola commented Apr 3, 2021 via email

alazarolop commented Apr 3, 2021

JimShady commented Oct 27, 2022

edzer commented Oct 27, 2022

JimShady commented Oct 27, 2022

alazarolop commented Feb 20, 2021 •

edited

Loading

alazarolop commented Feb 22, 2021 •

edited

Loading

alazarolop commented Mar 8, 2021 •

edited

Loading

alazarolop commented Mar 8, 2021 •

edited

Loading

alazarolop commented Mar 8, 2021 •

edited

Loading

alazarolop commented Apr 3, 2021 •

edited

Loading