The goal of {testdown}
is to generate a bookdown report of
{testthat}
results
You can install the dev version of {testdown}
- from r-universe
# Enable universe(s) by thinkr-open
options(repos = c(
thinkropen = 'https://thinkr-open.r-universe.dev',
CRAN = 'https://cloud.r-project.org'))
# Install some packages
install.packages('testdown')
- from GitHub with:
remotes::install_github("ThinkR-open/testdown")
This package has two exported functions:
This function turns the {testthat}
results into a {bookdown}
report.
It takes:
- A
project_name
, which is the name you want to give to the project. The default isNULL
, which will then be converted tobasename(here::here())
. - An
author
, if you want your report to be have an author name. Default is NULL, and then the report won’t have any name on it. pkg
, the path to the package that will be documented. Default ishere::here()
environment
, a name for the testing environment. Default ishere::here()
book_path
, the folder where you want the{bookdown}
report to be created. Default is"tests/testdown"
.with_help
Should the help appendix be added? Default isTRUE
.open
Should the report be opened once compiled? Default isinteractive()
.
This function will compile the report for an example package contained
inside {testdown}
.
You can add a description
to your tests using the roxygen tag
@description
. Note that this test must be on the line just before
the expectation.
Here is an example:
test_that("hello_world() works", {
#' @description Testing that hello_world("colin") returns "Hello world, my name is colin!"
expect_equal(hello_world("colin"), "Hello world, my name is colin!")
#' @description Testing that hello_world("colin") returns a character vector
expect_is(hello_world("colin"), "character")
#' @description Testing that hello_world("colin") has World in it
expect_match(hello_world("colin"), "World")
#' @description Testing that hello_world("colin") has Hello in it
expect_match(hello_world("colin"), "Hello")
})
There are three scopes when it comes to tests:
-
“Test files”, which are the files included inside the
test/testthat/
folder, and which contain a testing infrastructure. Each file contains one ore more test(s). -
“Tests”, which start with
test_that("",
. Each test contains one or more expectation(s). -
“Expectations”, which (usually) start with
expect_
(note thatskip_
functions are also considered as expectations). An expectation only contains one statement. If you put a roxygen tag@description
just before your expectation, it will be displayed in the testing results (otherwise the ‘Description’ column will be blank).
To sum up, Test files contains one or more test(s), which contain one or more expectation(s).
-
✅ Success: Passed (the expectation is met).
-
❌ Failure: Failed (the expectation is not met).
-
⚠️ Warning: The expectation returned a warning. -
❌ Error (test stopped): the expectation has returned an error, and the current test was stopped (i.e further expectations in the current test will not be launched).
-
🔄 Skip: This expectation has validated a “skip” expectation for the current test.
-
🔄 Was Skipped: The expectation has been skipped and is reported as a ‘skipped expectation’, due to a
skip_if
or an error previously in the test. Note that when an expectation is skipped, the Description, and Test time are not retrieved.
The index offers a summary of:
-
When the tests were run
-
The package (Title, Version, and Description of the package)
-
The project
- Name of the project
- Environment the tests were run into (aka the directory)
- Details about the test results
-
Results overview
-
The testing infrastructure
- R version
- Operating system
- Locale
- Package versions
-
Glossary of terms
This page offers a summary of the results, grouped by test.
Details of the table:
-
File: Name of the file that contains the test. This column is clickable and will open the corresponding page.
-
Test: Label given to the test.
-
Expectations: Number of expectations contained in the test.
-
Result: If the test contains one or more error(s), failed expectation(s), or warning(s) ❌, otherwise ✅.
-
Time spent: Time spent on this particular test.
Each test file has its own chapter, detailing the results of the expectations.
The summary at the top gives an overview of the results for the given test file.
Then, details are included about the tested file:
-
Test: Label of the test (i.e. the text in
test_that("label of the test", ...)
) -
Description: The description of the test, as detailed in the
#' @description
roxygen tag used just before the expectation. If no description was set, it will be blank. -
Expectation: Raw name of the expectation, i.e the R code used.
-
Result: Success / Failure / Warning / Errored / Skip / Was skipped (see “Expectations status” at the top of this page).
-
Location: Name of the file containing the test, with the line number where the expectation is located (for example,
test-hello-world.R#9
means the expectation started at line 9, in thetest-hello-world.R
file) -
Test_time: When was the test run.
Aggregation of expectations that returned a failure/error, warning, or were skipped.
-
Location: Name of the file containing the test, with the line number where the expectation is located (for example,
test-hello-world.R#9
means the expectation started at line 9, in thetest-hello-world.R
file) -
Test: Label of the test (i.e. the text in
test_that("label of the test", ...)
) -
Description: The description of the test, as detailed in the
#' @description
roxygen tag used just before the expectation. If no description was set, it will be blank. -
Expectation: Raw name of the expectation, i.e the R code used.
-
Message: Message output by R when the expectation failed/warned/skip.
For reliable results, all expectations should start by expect
, notably
if you need to count the skipped tests. In other words, the skipped
expectations count relies on counting all the functions starting with
expect
, so naming custom expectation differently will prevent this
count from being correct.
{testdown}
report relies on {testthat}
and the results are based on
these outputs.
Here are known example of results that will be discarded by {testthat}
and hence will make {testdown}
behave in a weird way.
- Using
{withr}
test_that("Files exist", {
with_dir(
"new/dir",
{
#' @description Checking that style.css is created
expect_file_exists("tests/testdown/style.css")
#' @description Checking that _bookdown.yml is created
expect_file_exists("tests/testdown/_bookdown.yml")
#' @description Checking that _output.yml is created
expect_file_exists("tests/testdown/_output.yml")
}
)
})
As testthat doesn’t count the expectations from with_dir
, this will
make the {testdown}
result weird. Always add the expectations at the
top level of your test_that.
- Loops
Same goes with for loops:
for (i in names(df)){
#' @description Checking the names of the output are correct
expect_true(
i %in% c(
"context",
"test",
"expectation",
"description",
"location",
"test_time",
"result",
"file",
"message"
)
)
}
If ever you use this format, {testthat}
won’t catch the tests, so they
won’t be reported.
- HTML expectations
As testdown render the text straight in html, if your expecatation contains html, it will break the rendering. For example, the following test will break the rendering:
expect_match(
tag,
"<h2>this</h2>"
)
The development of this package has been sponsored by:
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.