Submission: osmextract

[agila5]

Submitting Author: Andrea Gilardi (@agila5)
Repository: https://github.com/ITSLeeds/osmextract
Version submitted: 0.1.0
Editor: @annakrystalli
Reviewer 1: @potterzot
Reviewer 2: @salvafern
Archive: TBD
Version accepted: TBD

---

• Paste the full DESCRIPTION file inside a code block below:

Package: osmextract
Type: Package
Title: Download and Read OpenStreetMap Data Extracts
Version: 0.1.0
Authors@R: c(
    person("Andrea", "Gilardi", email = "a.gilardi5@campus.unimib.it", role=c("aut", "cre"),
      comment = c(ORCID = "0000-0002-9424-7439")),
    person("Robin", "Lovelace", role = c("aut"),
      comment = c(ORCID = "0000-0001-5679-6536")),
    person("Barry", "Rowlingson", role=c("ctb"),
      comment = c(ORCID = "0000-0002-8586-6625"))
    )
Description: This package is used to download, convert and read Open Street
    Map data extracts downloaded from several providers. 
License: GPL-3
Encoding: UTF-8
LazyData: true
RoxygenNote: 7.1.1
Roxygen: list(markdown = TRUE)
URL: https://github.com/itsleeds/osmextract
BugReports: https://github.com/itsleeds/osmextract/issues
Depends: R (>= 3.5.0)
Imports: 
    sf (>= 0.8.1),
    utils,
    tools
Suggests: 
    testthat,
    knitr,
    rmarkdown,
    covr
VignetteBuilder: knitr

Scope

• Please indicate which category or categories from our package fit policies this package falls under: (Please check an appropriate box below. If you are unsure, we suggest you make a pre-submission inquiry.):

  • data retrieval
  • data extraction
  • data munging
  • data deposition
  • workflow automation
  • version control
  • citation management and bibliometrics
  • scientific software wrappers
  • field and lab reproducibility tools
  • database software bindings
  • geospatial data
  • text analysis

• Explain how and why the package falls under these categories (briefly, 1-2 sentences):

osmextract is for accessing and downloading OpenStreetMap data from a variety (current 2) of providers. OpenStreetMap has numerous scientific applications, including in transport, ecological analysis of human impacts on the environment, spatial statistics and land use change and the package provides a way of extracting the unstructure compressed .pbf files into modern geographic data structures (sf data frames).

• Who is the target audience and what are scientific applications of this package?

The target audience is researchers, practitioners and citizens who want to use OpenStreetMap data for scientific research, especially people who require large datasets (from city to continental scale) for their work.

• Are there other R packages that accomplish the same thing? If so, how does yours differ or meet our criteria for best-in-category?

osmdata provides an R interface to the Overpass API, which is ideal for downloading small OSM datasets. However, the API is rate limited, making it hard to download large datasets.

• (If applicable) Does your package comply with our guidance around Ethics, Data Privacy and Human Subjects Research?

We believe the package complies with ethical guidelines in the Internet Research: Ethical Guidelines 3.0 document.
It makes it easier for researchers to access and make use of data that is already in the public domain, under the conditions of the adhering to the conditions of the OdBL.

• If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

Technical checks

Confirm each of the following by checking the box.

• I have read the guide for authors and rOpenSci packaging guide.

This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions, created with roxygen2.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, including reporting of test coverage using services such as Travis CI, Coveralls and/or CodeCov.

Publication options

• Do you intend for this package to go on CRAN?
• Do you intend for this package to go on Bioconductor?
• Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Options

• The package has an obvious research application according to JOSS's definition.
  • The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
  • The package is deposited in a long-term repository with the DOI:
  • (Do not submit your package separately to JOSS)

• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

MEE Options

• The package is novel and will be of interest to the broad readership of the journal.
• The manuscript describing the package is no longer than 3000 words.
• You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
• (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
• (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
• (Please do not submit your package separately to Methods in Ecology and Evolution)

Code of conduct

• I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

[annakrystalli]

Hello @agila5 and many thanks for your submission! Apologies for the slow response but many of us have been on holiday.

I'll be your editor for this submission and will begin initial editor's checks now. Will be back in touch shortly! 👍

[annakrystalli]

Editor checks:

• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

Editor comments

Hello again @agila5 ! 👋

I've now completed an initial pass of editor's checks. There's a few minor points but most importantly there is n error when building one of the vignettes (see below) and this is causing problems for me to fully complete the checks.

If you could have a look and let me know when it's sorted, I can have another look. 👍

Problem with building vignettes

I'm hitting a problem when building the vignettes from source locally, with same issue building from GitHub also:

This is also causing problems when running devtools::check().

pkg_dir <- "/Users/Anna/Documents/workflows/rOpenSci/editorials/osmextract"
devtools::install(pkg_dir, dependencies = T, build_vignettes = T)
#>      checking for file ‘/Users/Anna/Documents/workflows/rOpenSci/editorials/osmextract/DESCRIPTION’ ...  ✓  checking for file ‘/Users/Anna/Documents/workflows/rOpenSci/editorials/osmextract/DESCRIPTION’
#>   ─  preparing ‘osmextract’:
#>      checking DESCRIPTION meta-information ...  ✓  checking DESCRIPTION meta-information
#>   ─  installing the package to build vignettes
#>      creating vignettes ...  E  creating vignettes (7.7s)
#>    --- re-building ‘osmextract.Rmd’ using rmarkdown
#>    trying URL 'https://github.com/ITSLeeds/osmextract/raw/master/inst/its-example.osm.pbf'
#>    Content type 'application/octet-stream' length 40792 bytes (39 KB)
#>    ==================================================
#>    downloaded 39 KB
#>    
#>    Quitting from lines 566-573 (osmextract.Rmd) 
#>    Error: processing vignette 'osmextract.Rmd' failed with diagnostics:
#>    NA value(s) in bounding box. Trying to plot empty geometries?
#>    --- failed re-building ‘osmextract.Rmd’
#>    
#>    --- re-building ‘providers.Rmd’ using rmarkdown
#>    --- finished re-building ‘providers.Rmd’
#>    
#>    SUMMARY: processing the following file failed:
#>      ‘osmextract.Rmd’
#>    
#>    Error: Vignette re-building failed.
#>    Execution halted
#> 
#> Error in (function (command = NULL, args = character(), error_on_status = TRUE, : System command 'R' failed, exit status: 1, stdout + stderr:
#> E> * checking for file ‘/Users/Anna/Documents/workflows/rOpenSci/editorials/osmextract/DESCRIPTION’ ... OK
#> E> * preparing ‘osmextract’:
#> E> * checking DESCRIPTION meta-information ... OK
#> E> * installing the package to build vignettes
#> E> * creating vignettes ... ERROR
#> E> --- re-building ‘osmextract.Rmd’ using rmarkdown
#> E> trying URL 'https://github.com/ITSLeeds/osmextract/raw/master/inst/its-example.osm.pbf'
#> E> Content type 'application/octet-stream' length 40792 bytes (39 KB)
#> E> ==================================================
#> E> downloaded 39 KB
#> E> 
#> E> Quitting from lines 566-573 (osmextract.Rmd) 
#> E> Error: processing vignette 'osmextract.Rmd' failed with diagnostics:
#> E> NA value(s) in bounding box. Trying to plot empty geometries?
#> E> --- failed re-building ‘osmextract.Rmd’
#> E> 
#> E> --- re-building ‘providers.Rmd’ using rmarkdown
#> E> --- finished re-building ‘providers.Rmd’
#> E> 
#> E> SUMMARY: processing the following file failed:
#> E>   ‘osmextract.Rmd’
#> E> 
#> E> Error: Vignette re-building failed.
#> E> Execution halted

^{Created on 2020-09-29 by the reprex package (v0.3.0)}

Warning during testing

devtools::test() throws an error in:

test-read.R:3: error: oe_read: simplest examples work
#> cannot open URL ''  

and bunch of warnings

pkg_dir <- "/Users/Anna/Documents/workflows/rOpenSci/editorials/osmextract"
devtools::test(pkg_dir)
#> Loading osmextract
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright.
#> Any product made from OpenStreetMap must cite OSM as the data source.
#> Geofabrik data are taken from https://download.geofabrik.de/
#> For usage details of bbbike data see https://download.bbbike.org/osm/
#> OpenStreetMap_fr data are taken from http://download.openstreetmap.fr/
#> Testing osmextract
#> ✓ |  OK F W S | Context
#> ⠏ |   0       | download⠙ |   1   1   | download✓ |   6   1   | download [0.6 s]
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> test-download.R:3: warning: oe_download: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> ⠏ |   0       | find⠼ |   1   4   | find✓ |   3   4   | find [0.6 s]
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> test-find.R:2: warning: oe_find: simplest example works
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-find.R:6: warning: oe_find: simplest example works
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-find.R:6: warning: oe_find: simplest example works
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-find.R:6: warning: oe_find: simplest example works
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> ⠏ |   0       | get✓ |   1   1   | get
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> test-get.R:2: warning: oe_get: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> ⠏ |   0       | match⠏ |   4   6   | match⠸ |   8   16   | match⠴ |  14   22   | match✓ |  26   28   | match [0.4 s]
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> test-match.R:2: warning: oe_match: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:2: warning: oe_match: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:3: warning: oe_match: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:3: warning: oe_match: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:14: warning: oe_match: sfc_POINT objects
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:14: warning: oe_match: sfc_POINT objects
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:18: warning: oe_match: sfc_POINT objects
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:18: warning: oe_match: sfc_POINT objects
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:23: warning: oe_match: sfc_POINT objects
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:24: warning: oe_match: sfc_POINT objects
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:38: warning: oe_match: sfc_POINT objects
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:38: warning: oe_match: sfc_POINT objects
#> st_centroid does not give correct centroids for longitude/latitude data
#> 
#> test-match.R:43: warning: oe_match: sfc_POINT objects
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:43: warning: oe_match: sfc_POINT objects
#> st_centroid does not give correct centroids for longitude/latitude data
#> 
#> test-match.R:43: warning: oe_match: sfc_POINT objects
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:43: warning: oe_match: sfc_POINT objects
#> st_centroid does not give correct centroids for longitude/latitude data
#> 
#> test-match.R:50: warning: oe_match: numeric input
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:50: warning: oe_match: numeric input
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:55: warning: oe_match: different providers, match_by or max_string dist args
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:56: warning: oe_match: different providers, match_by or max_string dist args
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:56: warning: oe_match: different providers, match_by or max_string dist args
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:58: warning: oe_match: different providers, match_by or max_string dist args
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:59: warning: oe_match: different providers, match_by or max_string dist args
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:59: warning: oe_match: different providers, match_by or max_string dist args
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:60: warning: oe_match: different providers, match_by or max_string dist args
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:84: warning: oe_check_pattern: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:85: warning: oe_check_pattern: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-match.R:86: warning: oe_check_pattern: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> ⠏ |   0       | providers✓ |   1   6   | providers
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> test-providers.R:2: warning: oe_providers works correctly
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-providers.R:2: warning: oe_providers works correctly
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-providers.R:2: warning: oe_providers works correctly
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-providers.R:2: warning: oe_providers works correctly
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-providers.R:2: warning: oe_providers works correctly
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-providers.R:2: warning: oe_providers works correctly
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> ⠏ |   0       | readx |   0 1 1 1 | read
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> test-read.R:3: warning: oe_read: simplest examples work
#> URL '': status was 'URL using bad/illegal format or missing URL'
#> 
#> test-read.R:3: error: oe_read: simplest examples work
#> cannot open URL ''
#> Backtrace:
#>  1. osmextract::oe_read(f) tests/testthat/test-read.R:3:2
#>  2. osmextract::oe_download(...) R/read.R:84:4
#>  3. utils::download.file(...) R/download.R:131:4
#> 
#> test-read.R:23: skip: or_read: simplest example with a URL works
#> Reason: empty test
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> ⠏ |   0       | updateThe .gpkg files are going to be removed.
#> ⠹ |   1   2   | update✓ |   1   2   | update [0.4 s]
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> test-update.R:4: warning: oe_update(): simplest example works
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-update.R:10: warning: oe_update(): simplest example works
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> ⠸ |   1   3   | update⠹ |   3       | vectortranslate✓ |   7   2   | vectortranslate [0.2 s]
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> test-vectortranslate.R:42: warning: oe_get_keys: simplest examples work
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> 
#> test-vectortranslate.R:49: warning: oe_get_keys: returns error with wrong inputs
#> st_crs<- : replacing crs does not reproject data; use st_transform for that
#> ─────────────────────────────────────────────────────────────────────────────────────────────────────
#> 
#> ══ Results ══════════════════════════════════════════════════════════════════════════════════════════
#> Duration: 2.3 s
#> 
#> OK:       45
#> Failed:   1
#> Warnings: 46
#> Skipped:  1

^{Created on 2020-09-29 by the reprex package (v0.3.0)}

goodpractice output

There's a few minor issues flagged by goodpractice::gp()

pkg_dir <- "/Users/Anna/Documents/workflows/rOpenSci/editorials/osmextract"
goodpractice::gp(pkg_dir)
#> Preparing: covr
#> Warning in MYPREPS[[prep]](state, quiet = quiet): Prep step for test coverage
#> failed.
#> Preparing: cyclocomp
#> Warning in MYPREPS[[prep]](state, quiet = quiet): Prep step for cyclomatic
#> complexity failed.
#> Preparing: description
#> Preparing: lintr
#> Preparing: namespace
#> Preparing: rcmdcheck
#> Warning in MYPREPS[[prep]](state, quiet = quiet): Prep step for rcmdcheck
#> failed.
#> ── GP osmextract ───────────────────────────────────────────────────────────────
#> 
#> It is good practice to
#> 
#>   ✖ use '<-' for assignment instead of '='. '<-' is the standard, and R
#>     users and developers are used it and it is easier to read your code
#>     for them if you use '<-'.
#> 
#>     R/download.R:61:13
#>     R/download.R:83:14
#>     R/download.R:88:13
#>     R/download.R:110:14
#>     R/download.R:118:16
#>     ... and 119 more lines
#> 
#>   ✖ avoid long code lines, it is bad for readability. Also, many people
#>     prefer editor windows that are about 80 characters wide. Try make
#>     your lines shorter than 80 characters
#> 
#>     R/download.R:40:1
#>     R/find.R:23:1
#>     R/find.R:57:1
#>     R/get.R:9:1
#>     R/get.R:72:1
#>     ... and 17 more lines
#> 
#> ────────────────────────────────────────────────────────────────────────────────

^{Created on 2020-09-29 by the reprex package (v0.3.0)}

[agila5]

Hi @annakrystalli. First of all, thank you very much for testing our R package and helping us make it better.

I've now completed an initial pass of editor's checks. There's a few minor points but most importantly there is n error when building one of the vignettes (see below) and this is causing problems for me to fully complete the checks.

I tried to reproduce the errors using my laptop (running Windows 10) and a server (running Ubuntu 16.04), but I couldn't replicate it. Could you please share the output of sessionInfo() and sf::sf_extSoftVersion()? @Robinlovelace suggested it may be a problem related to an old version of GDAL. Moreover, we recently changed the osmextract.Rmd vignette, could you please try running the checks using the most recent version of the package?

I think that the error in devtools::test has the same root cause as the error when installing the package.

and bunch of warnings

The warning messages are caused by a version of GDAL that does not support WKT, while the provider's data in osmextract was created using WKT. The problem is documented in the sf issue tracker. I added a note in the README:

https://github.com/ITSLeeds/osmextract/blob/1655ab4186c9a1402584234c3a9e808659940044/README.Rmd#L178-L187

and a warning message in .onLoad():

https://github.com/ITSLeeds/osmextract/blob/1655ab4186c9a1402584234c3a9e808659940044/R/zzz.R#L12-L25

These warnings will tend to fade as OSs update and should disapper when installing newer version of PROJ and GDAL.

Sorry for these problems, we'll try to fix them as soon as possible.

[mpadge]

These warnings will tend to fade as OSs update and should disapper when installing newer version of PROJ and GDAL.

For what it's worth, the package installs and tests clean on my (linux) system with absolute latest versions of everything (well, okay, not yet R4.0.3 😏):

R.Version()$version.string
#> [1] "R version 4.0.2 (2020-06-22)"
system2 ("gdalinfo", "--version", stdout = TRUE)
#> [1] "GDAL 3.0.4, released 2020/01/28"
system2 ("proj", "--version", stdout = TRUE)
#> [1] "Rel. 6.3.2, May 1st, 2020"                                            

^{Created on 2020-10-01 by the reprex package (v0.3.0)}

(Tests just give the couple of warnings about st_centroid applied to lon-lat data.)

[annakrystalli]

Thanks for the prompt response @agila5 !

The message is useful as is the note in the README. However I would move that to the installation section and be explicit about the minimum versions of the GDAL and PROJ libraries required for full functionality.

Now onto the fix I had a few issues with (so anticipate others might too!). Apologies for not sharing platform details earlier! Indeed I am on macos, here's full platform info:

sessioninfo::platform_info()
#>  setting  value                       
#>  version  R version 3.6.2 (2019-12-12)
#>  os       macOS Catalina 10.15.6      
#>  system   x86_64, darwin15.6.0        
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_GB.UTF-8                 
#>  ctype    en_GB.UTF-8                 
#>  tz       Europe/London               
#>  date     2020-10-04

I've now updated GDAL to 3.1.2 using homebrew which also updated PROJ4 library to 7.1.1

==> Upgrading 1 outdated package:
gdal 2.4.4_2 -> 3.1.2_1

==> Installing gdal dependency: proj
==> Pouring proj-7.1.1.catalina.bottle.tar.gz

Just a note that I kept getting an error even after reinstalling sf from CRAN. It was still linking to an earlier version of proj.

> library(sf)
Linking to GEOS 3.7.2, GDAL 2.4.2, PROJ 5.2.0

FYI, I solved it by:

• Set environment variable PKG_CONFIG_PATH to, in my case, /usr/local/lib/pkgconfig/, the path to to the pkgconfig/ directory containing a proj.pc file with details of the newest version, as suggested in this issue.
• Reinstalled sf from GitHub with remotes::install_github("r-spatial/sf") (reinstalling from CRAN didn't solve the linking problem)

Which now works!

library("sf")
#> Linking to GEOS 3.8.1, GDAL 3.1.2, PROJ 7.1.1

But! I'm still having issues, see below:

tests

Loading osmextract
Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright.
Any product made from OpenStreetMap must cite OSM as the data source.
Geofabrik data are taken from https://download.geofabrik.de/
For usage details of bbbike data see https://download.bbbike.org/osm/
OpenStreetMap_fr data are taken from http://download.openstreetmap.fr/
Testing osmextract
✓ |  OK F W S | Context
⠏ |   0       | downloadtrying URL 'https://github.com/ITSLeeds/osmextract/raw/master/inst/its-example.osm.pbf'
Content type 'application/octet-stream' length 40792 bytes (39 KB)
==================================================
downloaded 39 KB

✓ |   6       | download [1.0 s]
✓ |   3       | find [0.6 s]
✓ |   1       | get
✓ |  26   3   | match [0.7 s]
───────────────────────────────────────────────────────────────────────────────
test-match.R:38: warning: oe_match: sfc_POINT objects
st_centroid does not give correct centroids for longitude/latitude data

test-match.R:43: warning: oe_match: sfc_POINT objects
st_centroid does not give correct centroids for longitude/latitude data

test-match.R:43: warning: oe_match: sfc_POINT objects
st_centroid does not give correct centroids for longitude/latitude data
───────────────────────────────────────────────────────────────────────────────
✓ |   1       | providers
x |   1 1   1 | read
───────────────────────────────────────────────────────────────────────────────
test-read.R:3: error: oe_read: simplest examples work
The input file_path does not correspond to any existing file and it doesn't look like a URL.
Backtrace:
 1. osmextract::oe_read(f) tests/testthat/test-read.R:3:2

test-read.R:23: skip: or_read: simplest example with a URL works
Reason: empty test
───────────────────────────────────────────────────────────────────────────────
⠏ |   0       | updateThe .gpkg files are going to be removed.
✓ |   1       | update [0.6 s]
✓ |   7       | vectortranslate [0.3 s]

══ Results ════════════════════════════════════════════════════════════════════
Duration: 3.4 s

OK:       46
Failed:   1
Warnings: 3
Skipped:  1

checks

Also a couple more related errors in examples being thrown up by devtools::check()

E  checking examples (2.8s)
   Running examples in ‘osmextract-Ex.R’ failed
   The error most likely occurred in:
   
   > base::assign(".ptime", proc.time(), pos = "CheckExEnv")
   > ### Name: oe_read
   > ### Title: Read a '.pbf' or '.gpkg' object from file or URL
   > ### Aliases: oe_read
   > 
   > ### ** Examples
   > 
   > # Read an existing .pbf file
   > my_pbf = system.file("its-example.osm.pbf", package = "osmextract")
   > oe_read(my_pbf, quiet = FALSE)
   Error: The input file_path does not correspond to any existing file and it doesn't look like a URL.
   Execution halted
✓  checking for unstated dependencies in ‘tests’ ...
─  checking tests ...
E  Running ‘testthat.R’ (4s)
   Running the tests in ‘tests/testthat.R’ failed.
   Last 13 lines of output:
     ==================================================
     downloaded 39 KB
     
     ── 1. Error: oe_read: simplest examples work (@test-read.R
     The input file_path does not correspond to any existing file and it doesn't look like a URL.
     Backtrace:
      1. osmextract::oe_read(f)
     
     The .gpkg files are going to be removed.
     ══ testthat results  ═════════════════════════════════════
     [ OK: 46 | SKIPPED: 1 | WARNINGS: 3 | FAILED: 1 ]
     1. Error: oe_read: simplest examples work (@test-read.R#3) 
     
     Error: testthat unit tests failed
     Execution halted
✓  checking for unstated dependencies in vignettes ...
✓  checking package vignettes in ‘inst/doc’ ...
✓  checking re-building of vignette outputs (10.6s)
✓  checking for detritus in the temp directory
   
   See
     ‘/Users/Anna/Documents/workflows/rOpenSci/editorials/osmextract.Rcheck/00check.log’
   for details.
   
── R CMD check results ─────────────────────────────────── osmextract 0.1.0 ────
Duration: 46.5s

> checking examples ... ERROR
  Running examples in ‘osmextract-Ex.R’ failed
  The error most likely occurred in:
  
  > base::assign(".ptime", proc.time(), pos = "CheckExEnv")
  > ### Name: oe_read
  > ### Title: Read a '.pbf' or '.gpkg' object from file or URL
  > ### Aliases: oe_read
  > 
  > ### ** Examples
  > 
  > # Read an existing .pbf file
  > my_pbf = system.file("its-example.osm.pbf", package = "osmextract")
  > oe_read(my_pbf, quiet = FALSE)
  Error: The input file_path does not correspond to any existing file and it doesn't look like a URL.
  Execution halted

goodpractice

A few more minor issues thrown up by GP

── GP osmextract ──────────────────────────────────────────────────────────────

It is good practice to

  ✖ use '<-' for assignment instead of '='. '<-' is the
    standard, and R users and developers are used it and it is easier
    to read your code for them if you use '<-'.

    R/download.R:61:13
    R/download.R:83:14
    R/download.R:88:13
    R/download.R:110:14
    R/download.R:119:16
    ... and 122 more lines

  ✖ avoid long code lines, it is bad for readability. Also,
    many people prefer editor windows that are about 80 characters
    wide. Try make your lines shorter than 80 characters

    R/download.R:40:1
    R/find.R:23:1
    R/find.R:57:1
    R/get.R:9:1
    R/get.R:73:1
    ... and 22 more lines

  ✖ fix this R CMD check NOTE: Note: found 263 marked UTF-8
    strings

spellcheck

One valid (from what I can tell) spelling error thrown up by devtools::spell_check()

Antartica             geofabrik_zones.Rd:18

Coverage

I still can't get a coverage breakdown because I'm getting the same error when running covr::package_coverage()

Error: Failure in `/private/var/folders/8p/87cqdx2s34vfvcgh04l6z72w0000gn/T/RtmppGl6ac/R_LIBS61956a128da7/osmextract/osmextract-tests/testthat.Rout.fail` .osm.pbf' Content type 'application/octet-stream' length 40792 bytes (39 KB) ================================================== downloaded 39 KB ── 1. Error: oe_read: simplest examples work (@test-read.R#3) ──────────────── The input file_path does not correspond to any existing file and it doesn't look like a URL. Backtrace: 1. osmextract::oe_read(f) The .gpkg files are going to be removed. ══ testthat results ══════════════════════════════════════════════════════════ [ OK: 46 | SKIPPED: 1 | WARNINGS: 3 | FAILED: 1 ] 1. Error: oe_read: simplest examples work (@test-read.R#3) Error: testthat unit tests failed Execution halted

Any ideas? It seems @mpadge didn't hit this issue 🤔

[agila5]

Hi @annakrystalli, and thank you very much again for your feedback.

The message is useful as is the note in the README. However I would move that to the installation section and be explicit about the minimum versions of the GDAL and PROJ libraries required for full functionality.

I just moved the warning message in the installation section, thanks!

goodpractice
A few more minor issues thrown up by GP

We decided to adopt = as the assignment operator, so the first set of warning messages is somewhat "expected". I recently changed some of the "long lines of code" and, at the moment, the remaining ones should represent only long URL(s). I'm not sure about the UTF8 strings, but I think that's not a big deal.

spellcheck
One valid (from what I can tell) spelling error thrown up by devtools::spell_check()

Fixed!

Now onto the fix I had a few issues with (so anticipate others might too!). Apologies for not sharing platform details earlier! Indeed I am on macos, here's full platform info:

First of all, thanks for your useful feedback , I will add a link to the README pointing to this discussion as soon as we fix these problems. Unfortunately, I have very little experience with macOS + sf, so I have no idea what's going on 😢 The only thing that I may suggest is running (I read the following command here):

install.packages("sf", type = "mac.binary")
remotes::install_github("ITSLeeds/osmextract")

The first line of code should overwrite the previous version of sf, and install a "new" binary version with its own GEOS/GDAL/PROJ. The main drawback is that this "new" version of sf is not linked to the external GEOS/GDAL/PROJ (and this is the same behaviour that occurs on Windows, I think. @Robinlovelace please correct me here). I checked this approach with a friend of mine, and we installed the osmextract package without any problem.

Another question: do you face any problem when you install another R package that depends on sf?

EDIT: More info on the error in devtools::test(). I think that the error in oe_read() is caused by an erroneous installation of osmextract. The first two lines of that test are:

https://github.com/ITSLeeds/osmextract/blob/4fafa757fe0c7f701bd1b4833113346843ed6b1a/tests/testthat/test-read.R#L1-L2

Then, the object f is passed to oe_read(), which returns an error since (for some reason that I don't understand), f does not link any existing file and it's not a URL.

[annakrystalli]

Hi @agila5 ! And thanks for your response!

BTW I did manage to install! I just had to follow these steps to get everything to work properly:

FYI, I solved it by:

• Set environment variable PKG_CONFIG_PATH to, in my case, /usr/local/lib/pkgconfig/, the path to to the pkgconfig/ directory containing a proj.pc file with details of the newest version, as suggested in this issue.
• Reinstalled sf from GitHub with remotes::install_github("r-spatial/sf") (reinstalling from CRAN didn't solve the linking problem)
  Which now works!

But do you think the test failure and broken example still are an sf installation issue? The main issue in tests and checks seems to be:

  > my_pbf = system.file("its-example.osm.pbf", package = "osmextract")
  > oe_read(my_pbf, quiet = FALSE)
  Error: The input file_path does not correspond to any existing file and it doesn't look like a URL.

[agila5]

But do you think the test failure and broken example still are an sf installation issue?

I think that the error is caused by some problems related to the installation of osmextract. The function oe_read is used to read-in a .pbf or .gpkg file, that can be specified using a file-path or a URL. It should fail only in two cases: 1) the input data points to a non-existing file or 2) the input data points to a non-existing URL.

The object my_pbf should represent the path to the its-example.osm.pbf file, which is one of the files included in the package (see https://github.com/ITSLeeds/osmextract/tree/master/inst). For example, I see:

list.files("C:/Users/Utente/Documents/R/win-library/3.6/osmextract/")
#>  [1] "data"                "DESCRIPTION"         "help"               
#>  [4] "html"                "INDEX"               "its-example.osm.pbf"
#>  [7] "Meta"                "NAMESPACE"           "osmconf.ini"        
#> [10] "R"

Could you compare that output with the analogous command on macOS? That example is clearly failing, and (I think) that should imply that there were some problems during the installation of the package. Could you try reinstalling it again?

I can succesfully replicate the same behaviour only when I run devtools::test after deleting the "its-example.osm.pbf" file (which is clearly a patological example):

# remove "old" osmextract
remove.packages("osmextract")
#> Removing package from 'C:/Users/Utente/Documents/R/win-library/3.6'
#> (as 'lib' is unspecified)

# check that it's removed
library("osmextract")
#> Error in library("osmextract"): there is no package called 'osmextract'

# download master branch
osmextract_master <- tempfile(fileext = ".zip")
download.file(
  url = "https://github.com/ITSLeeds/osmextract/archive/master.zip", 
  destfile = osmextract_master
)

# unzip 
unzip(osmextract_master, exdir = tempdir())
list.files(tempdir())
#> [1] "file2a2c23283a3e"     "file2a2c3f246c27"     "file2a2c54c36595.dll"
#> [4] "file2a2c7f3d50ae.zip" "osmextract-master"

# delete "its-example.osm.pbf" from osmextract-master
file.remove(file.path(tempdir(), "osmextract-master", "inst", "its-example.osm.pbf"))
#> [1] TRUE

# run tests
devtools::test(file.path(tempdir(), "osmextract-master"))
#> Loading osmextract
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright.
#> Any product made from OpenStreetMap must cite OSM as the data source.
#> Geofabrik data are taken from https://download.geofabrik.de/
#> For usage details of bbbike data see https://download.bbbike.org/osm/
#> OpenStreetMap_fr data are taken from http://download.openstreetmap.fr/
#> Testing osmextract
#> v |  OK F W S | Context
#> / |   0       | download- |   1       | downloadv |   6       | download [0.6 s]
#> / |   0       | find\ |   2       | findv |   3       | find [0.3 s]
#> / |   0       | getv |   1       | get
#> / |   0       | match/ |   4       | match\ |   6       | match- |  10   3   | matchv |  26   3   | match [0.5 s]
#> ----------------------------------------------------------------------------------------------------------------------------
#> test-match.R:38: warning: oe_match: sfc_POINT objects
#> st_centroid does not give correct centroids for longitude/latitude data
#> 
#> test-match.R:43: warning: oe_match: sfc_POINT objects
#> st_centroid does not give correct centroids for longitude/latitude data
#> 
#> test-match.R:43: warning: oe_match: sfc_POINT objects
#> st_centroid does not give correct centroids for longitude/latitude data
#> ----------------------------------------------------------------------------------------------------------------------------
#> / |   0       | providersv |   1       | providers
#> / |   0       | readx |   1 1   1 | read
#> ----------------------------------------------------------------------------------------------------------------------------
#> test-read.R:3: error: oe_read: simplest examples work
#> The input file_path does not correspond to any existing file and it doesn't look like a URL.
#> Backtrace:
#>  1. osmextract::oe_read(f) C:\Users\Utente\AppData\Local\Temp\Rtmp82J8ZK\osmextract-master/tests/testthat/test-read.R:3:2
#> 
#> test-read.R:23: skip: or_read: simplest example with a URL works
#> Reason: empty test
#> ----------------------------------------------------------------------------------------------------------------------------
#> / |   0       | updateThe .gpkg files are going to be removed.
#> v |   1       | update
#> / |   0       | vectortranslate- |   1       | vectortranslate| |   3       | vectortranslatev |   7       | vectortranslate [0.4 s]
#> 
#> == Results =================================================================================================================
#> Duration: 2.1 s
#> 
#> OK:       46
#> Failed:   1
#> Warnings: 3
#> Skipped:  1

^{Created on 2020-10-07 by the reprex package (v0.3.0)}

Sorry for all installation problems 😢