/
webapi_vignette.Rmd
159 lines (124 loc) · 6.5 KB
/
webapi_vignette.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
---
title: "WebAPI functionality"
author: "Koen Hufkens"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{WebAPI functionality}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
```{r setup, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
opts <- options(keyring_warn_for_env_fallback = FALSE)
# load the library
library(ncdf4)
library(terra)
library(maps)
library(ecmwfr)
# grab demo nc files
ncfile <- list.files(system.file(package="ecmwfr"),"*.nc", recursive = TRUE, full.names = TRUE)
ncfile <- ncfile[grepl("webapi.nc", ncfile)]
```
# ecmwfr
Programmatic interface to the ['ECMWF' web API services](https://confluence.ecmwf.int/display/WEBAPI/ECMWF+Web+API+Home). Allows for easy downloads of ECMWF [public data](https://confluence.ecmwf.int/display/WEBAPI/Access+ECMWF+Public+Datasets).
## Setup
Before starting save your provided ECMWF key to your local keychain. In this example I use my R development account with the key hidden using Xs. The package does not allow you to use your key inline in scripts to limit security issues when sharing scripts on github or otherwise.
```{r eval = FALSE}
# set a key to the keychain
wf_set_key(user = "khrdev@outlook.com",
key = "XXXXXXXXXXXXXXXXXXXXXX",
service = "webapi")
# you can retrieve the key using
wf_get_key(user = "khrdev@outlook.com")
# the output should be the key you provided
# in this case represented by the fake X string.
# "XXXXXXXXXXXXXXXXXXXXXX"
```
Alternatively, you can call the `wf_set_key()` function with only the service argument. The function will then redirect you to the correct login page where you can find your login credentials, which can be copied and pasted into the command prompt.
```{r eval = FALSE}
# set a key to the keychain
wf_set_key(service = "webapi")
```
Before you can download any data you have to make sure to accept the terms and conditions here:
[https://apps.ecmwf.int/datasets/licences/general/](https://apps.ecmwf.int/datasets/licences/general/).
## ECMWF Data Requests (Public Data Sets)
To download data use the `wf_request()` function, together with your email and a request string syntax [as documented](https://confluence.ecmwf.int/display/WEBAPI/Brief+request+syntax#Briefrequestsyntax-Syntax). Instead of `json` formatting the function uses a simple `R` list for all the arguments.
The conversion from a MARS or python based query to the list format can be automated if you use the RStudio based Addin. By selecting and using Addin -> Mars to list (or 'Python to list') you dynamically convert queries copied from either ECMWF or CDS based services.
```{r eval = FALSE, message=FALSE, warning=FALSE}
# this is an example of a request
request <- list(stream = "oper",
levtype = "sfc",
param = "167.128",
dataset = "interim",
step = "0",
grid = "0.75/0.75",
time = "00",
date = "2014-07-01/to/2014-07-02",
type = "an",
class = "ei",
area = "73.5/-27/33/45",
format = "netcdf",
target = "tmp.nc")
# an example download using fw_request()
# using the above request list()
ncfile <- wf_request(user = "khrdev@outlook.com",
request = request,
transfer = TRUE,
path = tempdir(),
verbose = FALSE)
```
The download operation might take a while. A progress indicator will keep you informed on the status of your request. The download will expire after a default time out of one hour. You will be informed of where to track your job request on [the ECMWF website](https://apps.ecmwf.int/webmars/joblist/), and provided with a download url which can be used with the `wt_transfer()` funtion to download the data on a later moment. Keep in mind that you can only have 3 active downloads and 20 job items queued!
If the download is successful you can visualize one layer in the retrieved stack of temperatures at 2 meters using the plotting routine below.
```{r fig.width = 7, fig.height = 7}
# Open NetCDF file and plot the data
# (trap read error on mac - if gdal netcdf support is missing)
r <- try(terra::rast(ncfile))
if(!inherits(r, "try-error")) {
terra::plot(r[[1]], main = "2 meter temperature (Kelvin)")
maps::map("world", add = TRUE)
}
```
## ECMWF mars Requests
The `ecmwfr` package ([`wf_request()`](../references/wf_request.html)) also allows to submit _mars_ requests. Note that this requires a member state login or a commercial user account!
The `ecmwfr` syntax for _mars_ requests is identical to the ones to the public ECMWF API except that one has to specify `dataset = "mars"`. The example below downloads the 2 meter temperature forecast (parameter `167`) from the 00 UTC ECMWF HRES run January 22, 2019, only 12 ahead forecast (`step`). A NetCDF file with a regular `0.125 x 0.125` degree horizontal resolution for a custom area in northern Europe will be downloaded (`area`). **Note** that the `grid` specification _is required_ if `format = "netcdf"` as grib to NetCDF conversion is only able for regular longitude-latitude grids.
```{r mars example, eval = FALSE}
# this is an example of a request
request <- list("dataset" = "mars",
"class" = "od",
"date" = "2019-01-22",
"expver" = "1",
"levtype" = "sfc",
"param" = "167.128",
"step" = "0",
"stream" = "oper",
"time" = "12",
"type" = "fc",
"grid" = "0.125/0.125",
"area" = "50/0/30/15",
"format" = "netcdf",
"target" = "mars-data.nc")
# Submit mars request and wait for netcdf file
wf_request(user = "khrdev@outlook.com",
transfer = TRUE,
path = "~",
request = request,
verbose = FALSE)
```
## Listing Ancillary data
You can list ancillary data such as user information, datasets and services using the following calls.
```{r eval = FALSE}
# user info
user_info <- wf_user_info(user = "khrdev@outlook.com")
head(user_info)
# services
services <- wf_services(user = "khrdev@outlook.com")
head(services)
# datasets
datasets <- wf_datasets(user = "khrdev@outlook.com")
head(datasets)
```