Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,13 @@ Imports:
Suggests:
astgrepr,
covr,
knitr,
rmarkdown,
stringi,
testthat (>= 3.0.0),
tibble,
tibblify
VignetteBuilder: knitr
Remotes:
wranglezone/tibblify
Config/testthat/edition: 3
Expand Down
188 changes: 188 additions & 0 deletions vignettes/nectar.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
---
title: "nectar"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{nectar}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```

```{r setup}
library(nectar)
```

nectar is a framework for building R packages that wrap web APIs. It provides a set of opinionated functions that handle the most common patterns in API wrappers: authentication, request preparation, pagination, response parsing, and tidying. This vignette demonstrates how to use nectar's core functions together, using the [Crossref Unified Resource API](https://api.crossref.org/swagger-ui/index.html) as a real-world example.

## Preparing a request with `req_prepare()`

The main entry point in nectar is `req_prepare()`. It wraps `httr2::request()` and a collection of `req_*` functions into a single, composable call. You provide the base URL and any options you need, such as authentication, pagination, and response tidying, and `req_prepare()` stores those settings directly on the request object so that downstream functions can use them automatically.

Here we prepare a request to the Crossref `/works` endpoint. We ask for ten results per page (`rows = 10`), select only the "publisher" and "DOI" fields, tell `{httr2}` to concatenate the `select` parameter with commas (`.multi`), and set the `cursor` parameter to `"*"` to trigger cursor-based pagination:


## Authentication with `req_auth_api_key()`

Many APIs accept an optional key (or, as in Crossref's case, an email address) to identify your application and gain access to a higher rate limit. nectar provides `req_auth_api_key()` for this purpose. You can pass it through `req_prepare()` via the `auth_fn` and `auth_args` arguments:

```{r auth, eval = FALSE}
req <- req_prepare(
"https://api.crossref.org/works",
query = list(
rows = 10, cursor = "*", select = c("publisher", "DOI"), .multi = "comma"
),
auth_fn = req_auth_api_key,
auth_args = list("mailto", api_key = "your@email.com", location = "query")
)
Comment thread
jonthegeek marked this conversation as resolved.
```

If you need to remove the key (for example, to fall back to anonymous access), pass `api_key = NULL`:

```{r auth-remove, eval = FALSE}
req <- req_auth_api_key(
req, # From above, with "your@email.com" attached as the key.
parameter_name = "mailto",
api_key = NULL,
location = "query"
)
```

## Response tidying with `resp_tidy_json()`

The Crossref API returns JSON. nectar's `resp_tidy_json()` function parses the JSON response body and converts the result to a tibble using `tibblify::tibblify()`. You can extract a nested path from the response at the same time with the `subset_path` argument.

For Crossref, the actual work items live at `message$items` in the response body. You can attach `resp_tidy_json()` to the request via the `tidy_fn` argument in `req_prepare()`, so that every response is automatically tidied when you call `resp_parse()` later:

```{r tidy, eval = FALSE}
req <- req_prepare(
"https://api.crossref.org/works",
query = list(
rows = 10, cursor = "*", select = c("publisher", "DOI"), .multi = "comma"
),
tidy_fn = resp_tidy_json,
tidy_args = list(subset_path = c("message", "items"))
)
```

## Pagination with `iterate_with_json_cursor()`

Crossref uses cursor-based pagination: each response contains a `message$next-cursor` field that should be sent as the `cursor` query parameter in the next request. nectar's `iterate_with_json_cursor()` generates the iterator function for this pattern, given the parameter name and the path to the cursor value in the response body:

```{r paginate, eval = FALSE}
iterate_xref <- iterate_with_json_cursor(
param_name = "cursor",
next_cursor_path = c("message", "next-cursor")
)
```

You can attach this iterator to the request via the `pagination_fn` argument in `req_prepare()`:

```{r prepare-full, eval = FALSE}
req <- req_prepare(
"https://api.crossref.org/works",
query = list(
rows = 10, cursor = "*", select = c("publisher", "DOI"), .multi = "comma"
),
tidy_fn = resp_tidy_json,
tidy_args = list(subset_path = c("message", "items")),
pagination_fn = iterate_xref
)
```

## Performing the request with `req_perform_opinionated()`

`req_perform_opinionated()` performs the request. If a pagination function is attached to the request (as above), it automatically uses `httr2::req_perform_iterative()` to fetch multiple pages; otherwise it falls back to a single `httr2::req_perform()` call. It also applies a retry policy to handle transient errors.

The `max_reqs` argument controls how many pages to fetch (default: `2`). During development, keep it small. Set it to `Inf` once you are confident the request works:

```{r perform, eval = FALSE}
resps <- req_perform_opinionated(req, max_reqs = 2)
```

The result is always a list of `httr2_response` objects with additional class `nectar_responses`, so downstream handling is consistent regardless of whether one or many pages were fetched.

## Parsing the response with `resp_parse()`

`resp_parse()` converts the raw responses into a usable R object. Because the request was prepared with `tidy_fn = resp_tidy_json`, `resp_parse()` will find that function automatically and apply it to each response, then combine the results:

```{r parse, eval = FALSE}
result <- resp_parse(resps)
result
```

## Putting it all together

The three core functions compose naturally into a pipeline. Here is the full workflow in one expression:

```{r pipeline, eval = FALSE}
result <- req_prepare(
"https://api.crossref.org/works",
query = list(
rows = 10, cursor = "*", select = c("publisher", "DOI"), .multi = "comma"
),
tidy_fn = resp_tidy_json,
tidy_args = list(subset_path = c("message", "items")),
pagination_fn = iterate_with_json_cursor(
param_name = "cursor",
next_cursor_path = c("message", "next-cursor")
)
) |>
req_perform_opinionated(max_reqs = 3) |>
resp_parse()

result
Comment thread
jonthegeek marked this conversation as resolved.

#> # A tibble: 30 × 2
#> publisher DOI
#> <chr> <chr>
#> 1 Springer Fachmedien Wiesbaden 10.1007/978-3-658-17671-6_18-1
#> 2 Springer International Publishing 10.1007/978-3-031-23161-2_300726
#> 3 Elsevier 10.1016/b978-0-08-102696-0.00020-8
#> 4 Springer International Publishing 10.1007/978-3-031-28170-9_6
#> 5 Springer Nature Singapore 10.1007/978-981-16-8679-5_306
#> 6 Springer Singapore 10.1007/978-981-15-1636-8_42
#> 7 Springer Berlin Heidelberg 10.1007/978-3-642-33832-8_41
#> 8 Springer Nature Switzerland 10.1007/978-3-031-72371-1_11
#> 9 Springer International Publishing 10.1007/978-3-319-18938-3
#> 10 Springer International Publishing 10.1007/978-3-319-19932-0_5
#> # i 20 more rows
#> # i Use `print(n = ...)` to see more rows
```

The resulting tibble contains the DOIs from the first two pages of Crossref works. Once you are ready to fetch all pages, replace `max_reqs = 2` with `max_reqs = Inf`.

## Building an API package with nectar

In practice, you would wrap these calls inside exported functions in your own package. For example, a `crossref` package might provide:

```{r package-example, eval = FALSE}
# R/works.R (inside a hypothetical "crossref" package)
works <- function(
rows = 20,
select = NULL,
mailto = NULL
) {
req_prepare(
"https://api.crossref.org/works",
query = list(rows = rows, cursor = "*", select = select),
auth_fn = req_auth_api_key,
auth_args = list("mailto", api_key = mailto, location = "query"),
tidy_fn = resp_tidy_json,
tidy_args = list(subset_path = c("message", "items")),
pagination_fn = iterate_with_json_cursor(
param_name = "cursor",
next_cursor_path = c("message", "next-cursor")
)
) |>
req_perform_opinionated() |>
resp_parse()
}
```

Users of the package never need to know about cursors, retry logic, or JSON parsing; nectar handles it all.
Loading