doctest.Rmd

---
title: "doctest"
output: 
  html_document:
    toc: true
    toc_depth: 3
    toc_float: true
    theme: flatly
vignette: >
  %\VignetteIndexEntry{doctest}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

# Introduction to doctest

```{r, child="../man/rmd/intro.Rmd"}
```

```{r, child="../man/rmd/example.Rmd"}
```

```{r, child="../man/rmd/usage.Rmd"}
```

## Adding doctests to your package

Here's a simple workflow to start using doctest:

1. Alter your package DESCRIPTION as above.

2. In your roxygen documentation, replace `@examples` by `@doctest`.

3. In the package directory run `roxygen2::roxygenize()` 
or `devtools::document()` to create documentation. You should see Rd files created as normal in the `man/` directory, including `\examples` sections.

4. Add `@expect` tags to your `@doctest` sections.

5. Run `roxygenize()` again. You will now see new files created in 
the `tests/testthat` directory, with the name `test-doctest-<topic name>.R`.

6. Run `devtools::test()` and check that your tests pass. 

At present, you can't use doctest from the RStudio keyboard shortcut
`Ctrl + Shift + D`, because this always uses the standard roxygen2 roclets. 
However, you can bind the RStudio addin "Devtools: document a package" to a
keyboard shortcut. This will use the roclets from your package DESCRIPTION 
file.

You don't need to add doctest as a dependency to your package. Just like
roxygen2 itself, you can use it to create help files and tests without it 
being installed for users. However, you may wish to add it in `Suggests:`, 
to help other developers working on the package:

```r
usethis::use_package("doctest", type = "Suggests")
```


## Tags

The doctest package adds these tags to roxygen:


### `@doctest`

Use `@doctest` instead of `@examples`: 

```r
#' @doctest
#' 
#' # ... examples for your function
```

The content of `@doctest` will be used in the .Rd "examples" section, and in a testthat test.

You can have more than one `@doctest` section. Each section creates one test
like `test_that("Test name", {...})`. You can name the doctest, or leave it blank
for a default name. All the sections will be merged into a single .Rd example.


```r
#' @doctest Positive numbers
#' x <- 1
#' @expect equal(x)
#' abs(x)
#'
#' @doctest Negative numbers
#' x <- -1
#' @expect equal(-x)
#' abs(x)
```


### `@expect`

`@expect` writes a testthat expectation.

```r
#' @expect equal(4)
#' 2 + 2

```

You can use any `expect_*` function from `testthat`. Omit the `expect_` at the
start of the call.

The expression on the next line will be substituted as the first argument
into the `expect` call:

```r
expect_equal(2 + 2, 4)
```

Use a dot `.` to substitute in different places:

```r
#' @expect equal(., rev(.))
#' c("T", "E", "N", "E", "T")
```

This becomes:

```r
expect_equal(c("T", "E", "N", "E", "T"), rev(c("T", "E", "N", "E", "T")))
```

### `@expectRaw`

`@expectRaw` writes an expectation, without substituting the next expression:

```r
#' x <- 2 + 2
#' @expectRaw equal(x, 4)
```


### `@snap`

`@snap` is shorthand for `@expect snapshot()`. This creates a 
[snapshot test](https://testthat.r-lib.org/articles/snapshotting.html),
which is useful for checking that complex examples haven't changed.


### `@testRaw`

`@testRaw` adds an arbitrary line of code to your test:

```r
#' @testRaw skip_on_cran("Takes too long to run")
#' 
#' @expect equal(6765)
#' fib(20)
```

Tests are only written if they contain at least one `@expect` or 
`@expectRaw` tag, so use those tags to create expectations, not `@testRaw`.


### `@omit` and `@resume`

While `@testRaw` includes a line of code in the test but not the example, 
`@omit` does the opposite: it includes all following code in the example
but not the test. You can use `@resume` to restart including lines without 
creating a new expectation.

```r
#' myfunc(1)
#' 
#' @omit
#' # No need to test plotting
#' plot(1:10, my_func(1:10))
#' 
#' @resume
#' x <- NA
#' @expect warning()
#' myfunc(x)
```

If you are using `@testRaw` and `@omit` a lot, it is probably a good idea
to separate out the test and the example. You can do this by renaming the
`test-doctest-` file, and removing the "Generated by doctest" line within it.
Then change your `@doctest` tag back to `@examples`.


### `@doctestExample`

`@doctestExample filename.R` includes the R code in filename.R as an example.
It is a drop-in replacement for roxygen2's `@example`. The R code isn't
checked for doctest tags and isn't included in any tests.


## Caveats

* Don't use `@doctest` and `@examples` in the same topic. That won't work. 

* Doctest currently ignores `\dontrun` and `\donttest` macros. Potentially,
  that could lead to dangerous code being included in tests. To
  avoid this, use the `@omit` tag.

* Each `@doctest` section should include a complete
  self-contained example, that would work inside a `test_that` expression.
  Don't rely on variables from a previous `@doctest`.

* You can include expectations within e.g. `if` blocks or `for` loops. 
  Don't forget that each roxygen tag must be indented with a single space:

``` r
#' # Right:
#' if (TRUE) {
#' @expect equals(4)
#'   2+2
#' }
#' # Wrong:
#' if (TRUE) {
#'   @expect equals(4)
#'   2+2
#' }
```

## Writing good doctests

Tests and documentation are similar, but not identical. Tests need to cover
difficult corner cases. Examples need to convey the basics to the user. 
I like the following advice:

> ... write the best possible documentation, and [R] makes sure the code
> samples in your documentation actually compile and run [and do what they
> are supposed to do]

*Programming Rust*, Blandy, Orendorff and Tindall, 2021

In particular, use doctest as an *addition* to manually created tests, not a 
*substitute* for them. Use doctest to make sure your examples do what they expect,
and for simple tests of basic functionality. If it's hard to specify what to test
for, consider using `@snap` to capture output:

```r
#' @snap
summary(model)
```

For more complex test cases,  write a test file manually.

To see an example of using the doctest package in "production", check out
`vignette("conversion")`.

## Related packages

The [roxytest](https://mikldk.github.io/roxytest/) and
[roxut](https://github.com/bryanhanson/roxut) packages both allow you to write
tests in roxygen blocks. Doctest is slightly different because it combines tests
with examples. The [exampletestr](https://github.com/rorynolan/exampletestr/)
package uses roxygen examples to generate a test skeleton which you can fill in
yourself.