-
Notifications
You must be signed in to change notification settings - Fork 0
Add basic usage vignette #78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
81e833b
Initial plan
Copilot e25ea3b
Add basic usage vignette with Crossref API example
Copilot dea647e
Update _pkgdown.yml
jonthegeek 20e6374
Remove em dashes from vignette prose
Copilot 6b2c5c1
Apply suggestions from code review
jonthegeek File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,188 @@ | ||
| --- | ||
| title: "nectar" | ||
| output: rmarkdown::html_vignette | ||
| vignette: > | ||
| %\VignetteIndexEntry{nectar} | ||
| %\VignetteEngine{knitr::rmarkdown} | ||
| %\VignetteEncoding{UTF-8} | ||
| --- | ||
|
|
||
| ```{r, include = FALSE} | ||
| knitr::opts_chunk$set( | ||
| collapse = TRUE, | ||
| comment = "#>" | ||
| ) | ||
| ``` | ||
|
|
||
| ```{r setup} | ||
| library(nectar) | ||
| ``` | ||
|
|
||
| nectar is a framework for building R packages that wrap web APIs. It provides a set of opinionated functions that handle the most common patterns in API wrappers: authentication, request preparation, pagination, response parsing, and tidying. This vignette demonstrates how to use nectar's core functions together, using the [Crossref Unified Resource API](https://api.crossref.org/swagger-ui/index.html) as a real-world example. | ||
|
|
||
| ## Preparing a request with `req_prepare()` | ||
|
|
||
| The main entry point in nectar is `req_prepare()`. It wraps `httr2::request()` and a collection of `req_*` functions into a single, composable call. You provide the base URL and any options you need, such as authentication, pagination, and response tidying, and `req_prepare()` stores those settings directly on the request object so that downstream functions can use them automatically. | ||
|
|
||
| Here we prepare a request to the Crossref `/works` endpoint. We ask for ten results per page (`rows = 10`), select only the "publisher" and "DOI" fields, tell `{httr2}` to concatenate the `select` parameter with commas (`.multi`), and set the `cursor` parameter to `"*"` to trigger cursor-based pagination: | ||
|
|
||
|
|
||
| ## Authentication with `req_auth_api_key()` | ||
|
|
||
| Many APIs accept an optional key (or, as in Crossref's case, an email address) to identify your application and gain access to a higher rate limit. nectar provides `req_auth_api_key()` for this purpose. You can pass it through `req_prepare()` via the `auth_fn` and `auth_args` arguments: | ||
|
|
||
| ```{r auth, eval = FALSE} | ||
| req <- req_prepare( | ||
| "https://api.crossref.org/works", | ||
| query = list( | ||
| rows = 10, cursor = "*", select = c("publisher", "DOI"), .multi = "comma" | ||
| ), | ||
| auth_fn = req_auth_api_key, | ||
| auth_args = list("mailto", api_key = "your@email.com", location = "query") | ||
| ) | ||
| ``` | ||
|
|
||
| If you need to remove the key (for example, to fall back to anonymous access), pass `api_key = NULL`: | ||
|
|
||
| ```{r auth-remove, eval = FALSE} | ||
| req <- req_auth_api_key( | ||
| req, # From above, with "your@email.com" attached as the key. | ||
| parameter_name = "mailto", | ||
| api_key = NULL, | ||
| location = "query" | ||
| ) | ||
| ``` | ||
|
|
||
| ## Response tidying with `resp_tidy_json()` | ||
|
|
||
| The Crossref API returns JSON. nectar's `resp_tidy_json()` function parses the JSON response body and converts the result to a tibble using `tibblify::tibblify()`. You can extract a nested path from the response at the same time with the `subset_path` argument. | ||
|
|
||
| For Crossref, the actual work items live at `message$items` in the response body. You can attach `resp_tidy_json()` to the request via the `tidy_fn` argument in `req_prepare()`, so that every response is automatically tidied when you call `resp_parse()` later: | ||
|
|
||
| ```{r tidy, eval = FALSE} | ||
| req <- req_prepare( | ||
| "https://api.crossref.org/works", | ||
| query = list( | ||
| rows = 10, cursor = "*", select = c("publisher", "DOI"), .multi = "comma" | ||
| ), | ||
| tidy_fn = resp_tidy_json, | ||
| tidy_args = list(subset_path = c("message", "items")) | ||
| ) | ||
| ``` | ||
|
|
||
| ## Pagination with `iterate_with_json_cursor()` | ||
|
|
||
| Crossref uses cursor-based pagination: each response contains a `message$next-cursor` field that should be sent as the `cursor` query parameter in the next request. nectar's `iterate_with_json_cursor()` generates the iterator function for this pattern, given the parameter name and the path to the cursor value in the response body: | ||
|
|
||
| ```{r paginate, eval = FALSE} | ||
| iterate_xref <- iterate_with_json_cursor( | ||
| param_name = "cursor", | ||
| next_cursor_path = c("message", "next-cursor") | ||
| ) | ||
| ``` | ||
|
|
||
| You can attach this iterator to the request via the `pagination_fn` argument in `req_prepare()`: | ||
|
|
||
| ```{r prepare-full, eval = FALSE} | ||
| req <- req_prepare( | ||
| "https://api.crossref.org/works", | ||
| query = list( | ||
| rows = 10, cursor = "*", select = c("publisher", "DOI"), .multi = "comma" | ||
| ), | ||
| tidy_fn = resp_tidy_json, | ||
| tidy_args = list(subset_path = c("message", "items")), | ||
| pagination_fn = iterate_xref | ||
| ) | ||
| ``` | ||
|
|
||
| ## Performing the request with `req_perform_opinionated()` | ||
|
|
||
| `req_perform_opinionated()` performs the request. If a pagination function is attached to the request (as above), it automatically uses `httr2::req_perform_iterative()` to fetch multiple pages; otherwise it falls back to a single `httr2::req_perform()` call. It also applies a retry policy to handle transient errors. | ||
|
|
||
| The `max_reqs` argument controls how many pages to fetch (default: `2`). During development, keep it small. Set it to `Inf` once you are confident the request works: | ||
|
|
||
| ```{r perform, eval = FALSE} | ||
| resps <- req_perform_opinionated(req, max_reqs = 2) | ||
| ``` | ||
|
|
||
| The result is always a list of `httr2_response` objects with additional class `nectar_responses`, so downstream handling is consistent regardless of whether one or many pages were fetched. | ||
|
|
||
| ## Parsing the response with `resp_parse()` | ||
|
|
||
| `resp_parse()` converts the raw responses into a usable R object. Because the request was prepared with `tidy_fn = resp_tidy_json`, `resp_parse()` will find that function automatically and apply it to each response, then combine the results: | ||
|
|
||
| ```{r parse, eval = FALSE} | ||
| result <- resp_parse(resps) | ||
| result | ||
| ``` | ||
|
|
||
| ## Putting it all together | ||
|
|
||
| The three core functions compose naturally into a pipeline. Here is the full workflow in one expression: | ||
|
|
||
| ```{r pipeline, eval = FALSE} | ||
| result <- req_prepare( | ||
| "https://api.crossref.org/works", | ||
| query = list( | ||
| rows = 10, cursor = "*", select = c("publisher", "DOI"), .multi = "comma" | ||
| ), | ||
| tidy_fn = resp_tidy_json, | ||
| tidy_args = list(subset_path = c("message", "items")), | ||
| pagination_fn = iterate_with_json_cursor( | ||
| param_name = "cursor", | ||
| next_cursor_path = c("message", "next-cursor") | ||
| ) | ||
| ) |> | ||
| req_perform_opinionated(max_reqs = 3) |> | ||
| resp_parse() | ||
|
|
||
| result | ||
|
jonthegeek marked this conversation as resolved.
|
||
|
|
||
| #> # A tibble: 30 × 2 | ||
| #> publisher DOI | ||
| #> <chr> <chr> | ||
| #> 1 Springer Fachmedien Wiesbaden 10.1007/978-3-658-17671-6_18-1 | ||
| #> 2 Springer International Publishing 10.1007/978-3-031-23161-2_300726 | ||
| #> 3 Elsevier 10.1016/b978-0-08-102696-0.00020-8 | ||
| #> 4 Springer International Publishing 10.1007/978-3-031-28170-9_6 | ||
| #> 5 Springer Nature Singapore 10.1007/978-981-16-8679-5_306 | ||
| #> 6 Springer Singapore 10.1007/978-981-15-1636-8_42 | ||
| #> 7 Springer Berlin Heidelberg 10.1007/978-3-642-33832-8_41 | ||
| #> 8 Springer Nature Switzerland 10.1007/978-3-031-72371-1_11 | ||
| #> 9 Springer International Publishing 10.1007/978-3-319-18938-3 | ||
| #> 10 Springer International Publishing 10.1007/978-3-319-19932-0_5 | ||
| #> # i 20 more rows | ||
| #> # i Use `print(n = ...)` to see more rows | ||
| ``` | ||
|
|
||
| The resulting tibble contains the DOIs from the first two pages of Crossref works. Once you are ready to fetch all pages, replace `max_reqs = 2` with `max_reqs = Inf`. | ||
|
|
||
| ## Building an API package with nectar | ||
|
|
||
| In practice, you would wrap these calls inside exported functions in your own package. For example, a `crossref` package might provide: | ||
|
|
||
| ```{r package-example, eval = FALSE} | ||
| # R/works.R (inside a hypothetical "crossref" package) | ||
| works <- function( | ||
| rows = 20, | ||
| select = NULL, | ||
| mailto = NULL | ||
| ) { | ||
| req_prepare( | ||
| "https://api.crossref.org/works", | ||
| query = list(rows = rows, cursor = "*", select = select), | ||
| auth_fn = req_auth_api_key, | ||
| auth_args = list("mailto", api_key = mailto, location = "query"), | ||
| tidy_fn = resp_tidy_json, | ||
| tidy_args = list(subset_path = c("message", "items")), | ||
| pagination_fn = iterate_with_json_cursor( | ||
| param_name = "cursor", | ||
| next_cursor_path = c("message", "next-cursor") | ||
| ) | ||
| ) |> | ||
| req_perform_opinionated() |> | ||
| resp_parse() | ||
| } | ||
| ``` | ||
|
|
||
| Users of the package never need to know about cursors, retry logic, or JSON parsing; nectar handles it all. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.