-
Notifications
You must be signed in to change notification settings - Fork 12
/
entitylist_download.Rd
205 lines (166 loc) · 7.26 KB
/
entitylist_download.Rd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/entitylist_download.R
\name{entitylist_download}
\alias{entitylist_download}
\title{Download an Entity List as CSV.}
\usage{
entitylist_download(
pid = get_default_pid(),
did = "",
url = get_default_url(),
un = get_default_un(),
pw = get_default_pw(),
local_dir = here::here(),
filter = NULL,
etag = NULL,
overwrite = TRUE,
retries = get_retries(),
odkc_version = get_default_odkc_version(),
orders = c("YmdHMS", "YmdHMSz", "Ymd HMS", "Ymd HMSz", "Ymd", "ymd"),
tz = get_default_tz(),
verbose = get_ru_verbose()
)
}
\arguments{
\item{pid}{The numeric ID of the project, e.g.: 2.
Default: \code{\link{get_default_pid}}.
Set default \code{pid} through \code{ru_setup(pid="...")}.
See \code{vignette("Setup", package = "ruODK")}.}
\item{did}{(chr) The name of the Entity List, internally called Dataset.
The function will error if this parameter is not given.
Default: "".}
\item{url}{The ODK Central base URL without trailing slash.
Default: \code{\link{get_default_url}}.
Set default \code{url} through \code{ru_setup(url="...")}.
See \code{vignette("Setup", package = "ruODK")}.}
\item{un}{The ODK Central username (an email address).
Default: \code{\link{get_default_un}}.
Set default \code{un} through \code{ru_setup(un="...")}.
See \code{vignette("Setup", package = "ruODK")}.}
\item{pw}{The ODK Central password.
Default: \code{\link{get_default_pw}}.
Set default \code{pw} through \code{ru_setup(pw="...")}.
See \code{vignette("Setup", package = "ruODK")}.}
\item{local_dir}{The local folder to save the downloaded files to,
default: \code{here::here}.
If the folder does not exist it will be created.}
\item{filter}{(str) A valid filter string.
Default: NULL (no filtering, all Entities returned).}
\item{etag}{(str) The etag value from a previous call to
\code{entitylist_download()}. The value must be stripped of the \verb{W/\\"} and \verb{\\"},
which is the format of the etag returned by \code{entitylist_download()}.
If provided, only new entities will be returned.
If the same \code{local_dir} is chosen and \code{overwrite} is set to \code{TRUE},
the downloaded CSV will also be overwritten, losing the previously
downloaded Entities.
Default: NULL (no filtering, all Entities returned).}
\item{overwrite}{Whether to overwrite previously downloaded file,
default: FALSE}
\item{retries}{The number of attempts to retrieve a web resource.
This parameter is given to \code{\link[httr]{RETRY}(times = retries)}.
Default: 3.}
\item{odkc_version}{The ODK Central version as a semantic version string
(year.minor.patch), e.g. "2023.5.1". The version is shown on ODK Central's
version page \verb{/version.txt}. Discard the "v".
\code{ruODK} uses this parameter to adjust for breaking changes in ODK Central.
Default: \code{\link{get_default_odkc_version}} or "2023.5.1" if unset.
Set default \code{get_default_odkc_version} through
\code{ru_setup(odkc_version="2023.5.1")}.
See \code{vignette("Setup", package = "ruODK")}.}
\item{orders}{(vector of character) Orders of datetime elements for
lubridate.
Default:
\code{c("YmdHMS", "YmdHMSz", "Ymd HMS", "Ymd HMSz", "Ymd", "ymd")}.}
\item{tz}{A timezone to convert dates and times to.
Read \code{vignette("setup", package = "ruODK")} to learn how \code{ruODK}'s
timezone can be set globally or per function.}
\item{verbose}{Whether to display debug messages or not.
Read \code{vignette("setup", package = "ruODK")} to learn how \code{ruODK}'s
verbosity can be set globally or per function.}
}
\value{
A list of four items:
\itemize{
\item entities (tbl_df) The Entity List as tibble
\item http_status (int) The HTTP status code of the response.
200 if OK, 304 if a given etag finds no new entities created.
\item etag (str) The ETag to use in subsequent calls to \code{entitylist_download()}
\item downloaded_to (fs_path) The path to the downloaded CSV file
\item downloaded_on (POSIXct) The time of download in the local timezone
}
}
\description{
\ifelse{html}{\href{https://lifecycle.r-lib.org/articles/stages.html#maturing}{\figure{lifecycle-maturing.svg}{options: alt='[Maturing]'}}}{\strong{[Maturing]}}
}
\details{
\subsection{CSV file}{
The downloaded CSV file is named after the entity list name.
The download location defaults to the current workdir, but can be modified
to a different folder path which will be created if it doesn't exist.
Entity Lists can be used as Attachments in other Forms, but they can also be
downloaded directly as a CSV file.
The CSV format closely matches the OData Dataset (Entity List) Service
format, with columns for system properties such as \verb{__id} (the Entity UUID),
\verb{__createdAt}, \verb{__creatorName}, etc., the Entity Label, and the
Dataset (Entity List) or Entity Properties themselves.
If any Property for an given Entity is blank (e.g. it was not captured by
that Form or was left blank), that field of the CSV is blank.
}
\subsection{Filter}{
The ODK Central \verb{$filter} query string parameter can be used to filter on
system-level properties, similar to how filtering in the OData Dataset
(Entity List) Service works.
Of the \href{https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#_Toc31358948}{OData filter specs }
ODK Central implements a \href{https://docs.getodk.org/central-api-odata-endpoints/#data-document}{growing set of features }.
\code{ruODK} provides the parameter \code{filter} (str) which, if set, will be passed
on to the ODK Central endpoint as is.
}
\subsection{Resuming downloads through ETag}{
The ODK Central endpoint supports the \href{https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag}{\code{ETag} header }, which can
be used to avoid downloading the same content more than once.
When an API consumer calls this endpoint, the endpoint returns a value in
the \code{ETag} header.
If you pass that value in the \href{https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-None-Match}{\code{If-None-Match} header }
of a subsequent request,
then if the Entity List has not been changed since the previous request,
you will receive 304 Not Modified response; otherwise you'll get the new
data.
\code{ruODK} provides the parameter \code{etag} which can be set from the output of
a previous call to \code{entitylist_download()}. \code{ruODK} strips the \verb{W/\\"} and
\verb{\\"} from the returned etag and expects the stripped etag as parameter.
}
}
\examples{
\dontrun{
# See vignette("setup") for setup and authentication options
# ruODK::ru_setup(svc = "....svc", un = "me@email.com", pw = "...")
ds <- entitylist_list(pid = get_default_pid())
ds1 <- entitylist_download(pid = get_default_pid(), did = ds$name[1])
# ds1$entities
# ds1$etag
# ds1$downloaded_to
# ds1$downloaded_on
ds2 <- entitylist_download(
pid = get_default_pid(),
did = ds$name[1],
etag = ds1$etag
)
# ds2$http_status == 304
newest_entity_date <- as.Date(max(ds1$entities$`__createdAt`))
ds3 <- entitylist_download(
pid = get_default_pid(),
did = ds$name[1],
filter = glue::glue("__createdAt le {newest_entity_date}")
)
}
}
\seealso{
\url{https://docs.getodk.org/central-api-dataset-management/#datasets}
Other entity-management:
\code{\link{entity_detail}()},
\code{\link{entity_list}()},
\code{\link{entity_update}()},
\code{\link{entitylist_detail}()},
\code{\link{entitylist_list}()}
}
\concept{entity-management}