Skip to content

Commit

Permalink
Snapshot documentation improvements (#1682)
Browse files Browse the repository at this point in the history
Fixes #1548. Fixes #1629. Fixes #1649.
  • Loading branch information
hadley committed Sep 22, 2022
1 parent 0630319 commit 1c9947d
Show file tree
Hide file tree
Showing 8 changed files with 173 additions and 96 deletions.
2 changes: 1 addition & 1 deletion R/expect-output.R
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#' Does code print output to the console?
#'
#' Test for output produced by `print()` or `cat()`. This is best used for
#' very simple output; for more complex cases use [verify_output()].
#' very simple output; for more complex cases use [expect_snapshot()].
#'
#' @export
#' @family expectations
Expand Down
58 changes: 35 additions & 23 deletions R/snapshot.R
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,9 @@
#' (e.g. this is a useful error message). Learn more in
#' `vignette("snapshotting")`.
#'
#' * `expect_snapshot()` captures all messages, warnings, errors, and
#' output from code.
#' * `expect_snapshot_output()` captures just output printed to the console.
#' * `expect_snapshot_error()` captures an error message and
#' optionally checks its class.
#' * `expect_snapshot_warning()` captures a warning message and
#' optionally checks its class.
#' * `expect_snapshot_value()` captures the return value.
#'
#' (These functions supersede [verify_output()], [expect_known_output()],
#' [expect_known_value()], and [expect_known_hash()].)
#' `expect_snapshot()` runs code as if you had executed it at the console, and
#' records the results, including output, messages, warnings, and errors.
#' If you just want to compare the result, try [expect_snapshot_value()].
#'
#' @section Workflow:
#' The first time that you run a snapshot expectation it will run `x`,
Expand Down Expand Up @@ -50,11 +42,9 @@
#' @param error Do you expect the code to throw an error? The expectation
#' will fail (even on CRAN) if an unexpected error is thrown or the
#' expected error is not thrown.
#' @param variant `r lifecycle::badge("experimental")`
#'
#' If not-`NULL`, results will be saved in `_snaps/{variant}/{test.md}`,
#' so `variant` must be a single string of alphanumeric characters suitable
#' for use as a directory name.
#' @param variant If non-`NULL`, results will be saved in
#' `_snaps/{variant}/{test.md}`, so `variant` must be a single string
#' suitable for use as a directory name.
#'
#' You can use variants to deal with cases where the snapshot output varies
#' and you want to capture and test the variations. Common use cases include
Expand Down Expand Up @@ -206,8 +196,24 @@ snap_header <- function(state, header) {
}
}

#' Snapshot helpers
#'
#' @description
#' `r lifecycle::badge("questioning")`
#'
#' These snapshotting functions are questioning because they were developed
#' before [expect_snapshot()] and we're not sure that they still have a
#' role to play.
#'
#' * `expect_snapshot_output()` captures just output printed to the console.
#' * `expect_snapshot_error()` captures an error message and
#' optionally checks its class.
#' * `expect_snapshot_warning()` captures a warning message and
#' optionally checks its class.
#'
#' @inheritParams expect_snapshot
#' @keywords internal
#' @export
#' @rdname expect_snapshot
expect_snapshot_output <- function(x, cran = FALSE, variant = NULL) {
edition_require(3, "expect_snapshot_output()")
variant <- check_variant(variant)
Expand All @@ -228,7 +234,7 @@ expect_snapshot_output <- function(x, cran = FALSE, variant = NULL) {
#' always fail (even on CRAN) if an error of this class isn't seen
#' when executing `x`.
#' @export
#' @rdname expect_snapshot
#' @rdname expect_snapshot_output
expect_snapshot_error <- function(x, class = "error", cran = FALSE, variant = NULL) {
edition_require(3, "expect_snapshot_error()")
expect_snapshot_condition(
Expand All @@ -240,7 +246,7 @@ expect_snapshot_error <- function(x, class = "error", cran = FALSE, variant = NU
}

#' @export
#' @rdname expect_snapshot
#' @rdname expect_snapshot_output
expect_snapshot_warning <- function(x, class = "warning", cran = FALSE, variant = NULL) {
edition_require(3, "expect_snapshot_warning()")
expect_snapshot_condition(
Expand Down Expand Up @@ -273,6 +279,12 @@ expect_snapshot_condition <- function(base_class, x, class, cran = FALSE, varian
)
}

#' Snapshot testing for values
#'
#' Captures the result of function, flexibly serializing it into a text
#' representation that's stored in a snapshot file. See [expect_snapshot()]
#' for more details on snapshot testing.
#'
#' @param style Serialization style to use:
#' * `json` uses [jsonlite::fromJSON()] and [jsonlite::toJSON()]. This
#' produces the simplest output but only works for relatively simple
Expand All @@ -284,11 +296,11 @@ expect_snapshot_condition <- function(base_class, x, class, cran = FALSE, varian
#' * `serialize()` produces a binary serialization of the object using
#' [serialize()]. This is all but guaranteed to work for any R object,
#' but produces a completely opaque serialization.
#' @param ... For `expect_snapshot_value()` only, passed on to
#' [waldo::compare()] so you can control the details of the comparison.
#' @export
#' @param ... Passed on to [waldo::compare()] so you can control the details of
#' the comparison.
#' @inheritParams expect_snapshot
#' @inheritParams compare
#' @rdname expect_snapshot
#' @export
expect_snapshot_value <- function(x,
style = c("json", "json2", "deparse", "serialize"),
cran = FALSE,
Expand Down
4 changes: 3 additions & 1 deletion _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,9 @@ reference:

- title: Snapshot testing
contents:
- starts_with("expect_snapshot_")
- expect_snapshot
- expect_snapshot_value
- expect_snapshot_file
- starts_with("snapshot_")

- title: Test helpers
Expand Down
2 changes: 1 addition & 1 deletion man/expect_output.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

69 changes: 6 additions & 63 deletions man/expect_snapshot.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

52 changes: 52 additions & 0 deletions man/expect_snapshot_output.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

62 changes: 62 additions & 0 deletions man/expect_snapshot_value.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 13 additions & 7 deletions vignettes/snapshotting.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -162,9 +162,12 @@ Since they can't automatically find the reference output, they instead just prin
## Other types of output

So far we've focussed on snapshot tests for output printed to the console.
But `expect_snapshot()` also captures messages, errors, and warnings.
But `expect_snapshot()` also captures messages, errors, and warnings[^1].
The following function generates a some output, a message, and a warning:

[^1]: We no longer recommend `expect_snapshot_output()`, `expect_snapshot_warning()`, or `expect_snapshot_error()`.
Just use `expect_snapshot()`.

```{r}
f <- function() {
print("Hello")
Expand Down Expand Up @@ -210,15 +213,18 @@ test_that("you can't add weird thngs", {
})
```

## Other types of snapshot
## Snapshotting values

`expect_snapshot()` is the most used snapshot function because it records everything: the code you run, printed output, messages, warnings, and errors.
But sometimes you just want to capture the output or errors in which you might want to use `expect_snapshot_output()` or `expect_snapshot_error()`.
This can happen when the associated code is lengthy or doesn't add much in terms of capturing the test's intent.
`expect_snapshot_error()` predates `expect_snapshot()` and, if history had unfolded differently, `expect_snapshot(error = TRUE, ...)` probably would have made `expect_snapshot_error()` unnecessary.
If you care about the return value rather than any side-effects, you may might to use `expect_snapshot_value()` instead.
It offers a number of serialisation approaches that provide a tradeoff between accuracy and human readability.

Or rather than caring about side-effects, you may want to check that the value of an R object stays the same.
In this case, you can use `expect_snapshot_value()` which offers a number of serialisation approaches that provide a tradeoff between accuracy and human readability.
```{r}
test_that("can snapshot a simple list", {
x <- list(a = list(1, 5, 10), b = list("elephant", "banana"))
expect_snapshot_value(x)
})
```

## Whole file snapshotting

Expand Down

0 comments on commit 1c9947d

Please sign in to comment.