expectation() function now expects an expectation type (one of "success", "failure", "error", "skip", "warning") as first argument. If you're creating your own expectations, you'll need to use
expect() instead (#437).
The expectation system got a thorough overhaul (#217). This primarily makes it easier to add new expectations in the future, but also included a thorough review of the documentation, ensuring that related expectations are documented together, and have evocative names.
One useful change is that most expectations invisibly return the input
object. This makes it possible to chain together expectations with magrittr:
factor("a") %>% expect_type("integer") %>% expect_s3_class("factor") %>% expect_length(1)
(And to make this style even easier, testthat now re-exports the pipe, #412).
The exception to this rule are the expectations that evaluate (i.e.
for messages, warnings, errors, output etc), which invisibly return
NULL. These functions are now more consistent: using
NA will cause a failure if there is a errors/warnings/mesages/output (i.e. they're not missing), and will
NULL fail if there aren't any errors/warnings/mesages/output. This previously didn't work for
expect_output() (#323), and the error messages were confusing with
expect_error(..., NA) (#342, @nealrichardson + @krlmlr, #317).
Another change is that
expect_output() now requires you to explicitly print the output if you want to test a print method:
expect_output("a", "a") will fail,
expect_output(print("a"), "a") will succeed.
There are six new expectations:
expect_type()checks the type of the object (#316),
expect_s3_class()tests that an object is S3 with given class,
expect_s4_class()tests that an object is S4 with given class (#373).
I recommend using these more specific expectations instead of the
expect_length()checks that an object has expected length.
expect_failure()are new expectations designed
specifically for testing other expectations (#368).
A number of older features have been deprecated:
expect_less_than()have been deprecated. Please
takes_less_than()has been deprecated.
not()has been deprecated. Please use the explicit individual forms
expect_warning(.., NA)and so on.
Expectations are conditions
Now all expectations are also conditions, and R's condition system is used to signal failures and successes (#360, @krlmlr). All known conditions (currently, "error", "warning", "message", "failure", and "success") are converted to expectations using the new
as.expectation(). This allows third-party test packages (such as
assertive) to seamlessly establish
testthat compatibility by issuing custom error conditions (e.g.,
structure(list(message = "Error message"), class = c("customError", "error", "condition"))) and then implementing
assertthat package contains an example.
The reporters system class has been considerably refactored to make existing reporters simpler and to make it easier to write new reporters. There are two main changes:
- Reporters classes are now R6 classes instead of Reference Classes.
- Each callbacks receive the full context:
add_results()is passed context and test as well as the expectation.
test_end()both get the context and test.
context_end()get the context.
- Warnings are now captured and reported in most reporters.
- The reporter output goes to the original standard output and is not affected by
- The default summary reporter lists all warnings (#310), and all skipped
tests (@krlmlr, #343). New option
the number of reports printed by the summary reporter. The default is 15
MinimalReportercorrect labels errors with E and failures with F (#311).
FailReporterto stop in case of failures or errors after all tests
- New functions
capture_warnings()selectively capture function output. These are
to allow other types out output to percolate up (#410).
try_again()allows you to retry code multiple times until it succeeds
test_package()now attach testthat so
all testing functions are available.
source_test_helpers()gets a useful default path: the testthat tests
directory. It defaults to the
test_env()to be consistent with the
other source functions (#415).
test_file()now loads helpers in the test directory before running
the tests (#350).
test_path()makes it possible to create paths to files in
that work interactively and when called from tests (#345).
setup_test_dir()has been removed. If you used it previously, instead use
source_file()exports the function testthat uses to load files from disk.
logicalthat indicates if all tests were successful
find_reporter()(and also all high-level testing functions) support a vector
of reporters. For more than one reporter, a
with_reporter()is used internally and gains new argument
start_end_reporter = TRUE(@krlmlr, 355).
set_reporter()returns old reporter invisibly (#358, @krlmlr).
- Comparing integers to non-numbers doesn't raise errors anymore, and falls
back to string comparison if objects have different lengths. Complex numbers
are compared using the same routine (#309, @krlmlr).
compare.chacter()recieved another overhaul. This
should improve behaviour of edge cases, and provides a strong foundation for
further work. Added
compare.POSIXt()for better reporting of datetime
detailed output of differences (#319, @krlmlr).
- Added Catch v1.2.1 for unit testing of C++ code.
?use_catch()for more details. (@kevinushey)