New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wateRinfo package #255

Open
stijnvanhoey opened this Issue Oct 2, 2018 · 6 comments

Comments

Projects
None yet
4 participants
@stijnvanhoey

stijnvanhoey commented Oct 2, 2018

Summary

  • What does this package do? (explain in 50 words or less):

wateRinfo facilitates the access to a variety of environmental water-related data about Flanders (Belgium) available on waterinfo.be, a website managed by the Flemish Environmental Agency.

  • Paste the full DESCRIPTION file inside a code block below:
Package: wateRinfo
Title: Download Time Series Data from Waterinfo.be
Version: 0.2.0
Description: The wateRinfo package provides an R interface to the data available
    at waterinfo.be, the data portal provided by the Flemish Environmental
    Agency. The package provides R functions to check the available stations for 
    a given variable, check the available variable for a given station and 
    download time series using the database identifiers as well as station 
    names.
Authors@R: c(
    person("Stijn", "Van Hoey", role = c("aut", "cre"), email = "stijn.vanhoey@inbo.be", comment = c(ORCID = "0000-0001-6413-3185")),
    person("Willem", "Maetens", role = "ctb", email = "w.maetens@vmm.be"),
    person("Peter", "Desmet", role = "ctb", email = "peter.desmet@inbo.be", comment = c(ORCID = "0000-0002-8442-8025"))
    )
License: MIT + file LICENSE
URL: https://github.com/inbo/wateRinfo, https://inbo.github.io/wateRinfo
BugReports: https://github.com/inbo/wateRinfo/issues
Depends:
    R (>= 2.10)
Imports:
    dplyr,
    httr,
    jsonlite,
    openssl,
    lubridate (>= 1.6.0),
    rlang,
    utils
Suggests:
    covr,
    ggplot2,
    knitr,
    rmarkdown,
    testthat
LazyData: true
Encoding: UTF-8
VignetteBuilder: knitr
RoxygenNote: 6.1.0.9000
  • URL for the package (the development repository, not a stylized html page):

https://github.com/inbo/wateRinfo

  • Please indicate which category or categories from our package fit policies this package falls under *and why(? (e.g., data retrieval, reproducibility. If you are unsure, we suggest you make a pre-submission inquiry.):

data retrieval: because the package provides R access to environmental data (water level and tidal information, discharge data, water quality variables, meteorological variables...) provided by the Flemish Environmental Agency. Without the package, people need to download data manually using the web interface or using the available API calls. The web interface, although evolving, does not support downloads for a list of stations and variables leading to lots of clicks to download data. The API documentation is limited and only available in Dutch. The package overcomes these limitations, supporting downloads of multiple variables and stations more efficiently.

  •   Who is the target audience and what are scientific applications of this package?  

Anyone interested in using the water-related data from waterinfo.be, for example hydrologists and ecologists. Scientific applications are ranging from environmental prediction and risk assessment modelling studies (e.g hydrological flood models) to ecological studies requiring environmental data.

To our knowledge, there is no other R package to retrieve data from waterinfo.be. Although some loose scripts have been circulating among researchers from different institutes, this package aims to support a collaborative/community effort.

  •   If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

Requirements

Confirm each of the following by checking the box. This package:

  • does not violate the Terms of Service of any service it interacts with.
  • has a CRAN and OSI accepted license.
  • contains a README with instructions for installing the development version.
  • includes documentation with examples for all functions.
  • contains a vignette with examples of its essential functions and uses.
  • has a test suite.
  • has continuous integration, including reporting of test coverage, using services such as Travis CI, Coveralls and/or CodeCov.
  • I agree to abide by ROpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

Publication options

  • Do you intend for this package to go on CRAN?
  • Do you wish to automatically submit to the Journal of Open Source Software? If so:
    • The package has an obvious research application according to JOSS's definition.
    • The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
    • The package is deposited in a long-term repository with the DOI:
    • (Do not submit your package separately to JOSS)
  • Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:
    • The package is novel and will be of interest to the broad readership of the journal.
    • The manuscript describing the package is no longer than 3000 words.
    • You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
    • (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
    • (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
    • (Please do not submit your package separately to Methods in Ecology and Evolution)

Detail

  • Does R CMD check (or devtools::check()) succeed? Paste and describe any errors or warnings:

  • Does the package conform to rOpenSci packaging guidelines? Please describe any exceptions:

    • NEWS file - there is one now, but only started with version 0.1.1 and using the ropensci structure since 0.2.0
    • Package name: Unfortunately, the name contains capital letters (wateRinfo), but this is how the package is already in use for almost a year
    • The main download function names are setup as get_** which does not use the object_verb() recommendation. As these are the main functions, we would keep the naming as such.
    • We did not use the @family tag as this provides full control on the order in between different sections. Considering the limited set of functionalities, the management of the pkgdown yml file is still feasible.
    • We do run the documentation website locally before deployment.
    • The current set of untested functionalities consists of interactions (mainly exceptions) with the waterinfo.be API which are hard to replicate.
  • If this is a resubmission following rejection, please explain the change in circumstances:

  • If possible, please provide recommendations of reviewers - those with experience with similar packages and/or likely users of your package - and their GitHub user names:

@sckott

This comment has been minimized.

Show comment
Hide comment
@sckott

sckott Oct 2, 2018

Member

thanks for your submission @stijnvanhoey - editors are discussing and we'll get back to you soon

Member

sckott commented Oct 2, 2018

thanks for your submission @stijnvanhoey - editors are discussing and we'll get back to you soon

@sckott

This comment has been minimized.

Show comment
Hide comment
@sckott

sckott Oct 3, 2018

Member

@stijnvanhoey et al. Do other regions of Belgium have water data available? If so, can the pkg be generalized to use those other data?

Member

sckott commented Oct 3, 2018

@stijnvanhoey et al. Do other regions of Belgium have water data available? If so, can the pkg be generalized to use those other data?

@stijnvanhoey

This comment has been minimized.

Show comment
Hide comment
@stijnvanhoey

stijnvanhoey Oct 3, 2018

Good point. I'll try to provide an overview of the information I'm aware of. Notice that we have for water-related issues different governmental levels: national level and regional levels, i.e. Flanders, Walloon and Brussels). Rivers are monitored on regional (and subregional) level:

  • The main data source for meteorological data (rainfall, evapotranspiration,...) on the national level is the Royal Meteorological Institute of Belgium, but they do not provide open data for the moment.
  • For the Walloon region, there is no data on the open data platform, but there data on discharge on their aqualim website. The latter presents data from the last 30 days online and a form to download data as an excel-sheet when providing contact details etc. For water quality, there are two websites: aquapol and aquaphyc providing water quality data for a number of stations. I do not have a direct idea how these could be incorporated (no REST or something alike services)?
  • For Brussels, the is no water data available on the open data platform. Their main river is Zenne, see the map. I do know of a project belini with a monitoring component, but the data seems not available as open data or a webservice.
  • The Flemish environmental agency (VMM) also provides data about water quality, but they told us their water quality system is currently in redevelopment (with other endpoints), so rather something to take into scope when the new system will be in place.
  • There is a database/webportal on soil and groundwater in Flanders, called DOV, which is, among other subsoil data (e.g. geological data), the aggregator for groundwater time series. There is a current initiative to increase the number of data contributors to DOV and a Python package for data access to DOV is currently in development as well, https://github.com/dov-vlaanderen/pydov. As the groundwater level data is thematically (and technically) linked to the other data sources of DOV (geological interpretations, boreholes,...) we consider it out of scope for the wateRinfo package.

Notice that we opt for GBIF to publish species occurrences related to water, e.g. fish, so this data is covered by the rgbif package.

As such, to my knowledge, there is no direct other source of data to incorporate. I'm adding @WillemMaetens (VMM), @pietercolpaert (open knowledge), @Sachagobeyn, @bartpannemans, @pjhaest to the discussion, maybe they know about other initiatives?

stijnvanhoey commented Oct 3, 2018

Good point. I'll try to provide an overview of the information I'm aware of. Notice that we have for water-related issues different governmental levels: national level and regional levels, i.e. Flanders, Walloon and Brussels). Rivers are monitored on regional (and subregional) level:

  • The main data source for meteorological data (rainfall, evapotranspiration,...) on the national level is the Royal Meteorological Institute of Belgium, but they do not provide open data for the moment.
  • For the Walloon region, there is no data on the open data platform, but there data on discharge on their aqualim website. The latter presents data from the last 30 days online and a form to download data as an excel-sheet when providing contact details etc. For water quality, there are two websites: aquapol and aquaphyc providing water quality data for a number of stations. I do not have a direct idea how these could be incorporated (no REST or something alike services)?
  • For Brussels, the is no water data available on the open data platform. Their main river is Zenne, see the map. I do know of a project belini with a monitoring component, but the data seems not available as open data or a webservice.
  • The Flemish environmental agency (VMM) also provides data about water quality, but they told us their water quality system is currently in redevelopment (with other endpoints), so rather something to take into scope when the new system will be in place.
  • There is a database/webportal on soil and groundwater in Flanders, called DOV, which is, among other subsoil data (e.g. geological data), the aggregator for groundwater time series. There is a current initiative to increase the number of data contributors to DOV and a Python package for data access to DOV is currently in development as well, https://github.com/dov-vlaanderen/pydov. As the groundwater level data is thematically (and technically) linked to the other data sources of DOV (geological interpretations, boreholes,...) we consider it out of scope for the wateRinfo package.

Notice that we opt for GBIF to publish species occurrences related to water, e.g. fish, so this data is covered by the rgbif package.

As such, to my knowledge, there is no direct other source of data to incorporate. I'm adding @WillemMaetens (VMM), @pietercolpaert (open knowledge), @Sachagobeyn, @bartpannemans, @pjhaest to the discussion, maybe they know about other initiatives?

@Sachagobeyn

This comment has been minimized.

Show comment
Hide comment
@Sachagobeyn

Sachagobeyn Oct 4, 2018

Dear all,

With respect to water quality data hosted by the VMM (Flanders): as I am currently aware, they are not available open source. They can be requested via the VMM website, however from my own experience, requests are often declined. An alternative for this type of information is the European Union Open Data Portal, in which ecological water quality, and a number of physico-chemical water quality variables are reported. For the latter, it should noted that data are only available for a few years. In addition, the datasets are not 'callable' as they are in waterinfo (zips hosted on the website).

http://cdr.eionet.europa.eu/be
https://www.eea.europa.eu/data-and-maps

Sachagobeyn commented Oct 4, 2018

Dear all,

With respect to water quality data hosted by the VMM (Flanders): as I am currently aware, they are not available open source. They can be requested via the VMM website, however from my own experience, requests are often declined. An alternative for this type of information is the European Union Open Data Portal, in which ecological water quality, and a number of physico-chemical water quality variables are reported. For the latter, it should noted that data are only available for a few years. In addition, the datasets are not 'callable' as they are in waterinfo (zips hosted on the website).

http://cdr.eionet.europa.eu/be
https://www.eea.europa.eu/data-and-maps

@sckott

This comment has been minimized.

Show comment
Hide comment
@sckott

sckott Oct 5, 2018

Member

Thanks very much @stijnvanhoey and @Sachagobeyn for the thorough overview.

It sounds like there are no other data sources at the moment, so we can move forward with this submission.

assigning an editor now

Member

sckott commented Oct 5, 2018

Thanks very much @stijnvanhoey and @Sachagobeyn for the thorough overview.

It sounds like there are no other data sources at the moment, so we can move forward with this submission.

assigning an editor now

@karthik

This comment has been minimized.

Show comment
Hide comment
@karthik

karthik Oct 6, 2018

Member

👋 @stijnvanhoey. I’ll be editing your submission. Stay tuned for more instructions from me shortly.

Member

karthik commented Oct 6, 2018

👋 @stijnvanhoey. I’ll be editing your submission. Stay tuned for more instructions from me shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment