Skip to content

Commit

Permalink
Apply a linter to the documentation
Browse files Browse the repository at this point in the history
The same way we have errors for our compiled code, it is nice to also apply the same techniques to the documentation. As such,
I ran markdownlint (https://github.com/DavidAnson/markdownlint) to the documentation files. Most of the suggestions were trivial but
actually, make the file structure a bit more reasonable to maintain. In addition to this, there were a couple of typos that I fixed while going through the documentation.

In general, I think this makes the "dev" part of the documentation a bit more consistent and nicer.
  • Loading branch information
Thorius committed Oct 30, 2018
1 parent de54478 commit 98096c1
Show file tree
Hide file tree
Showing 20 changed files with 347 additions and 205 deletions.
22 changes: 13 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
RapidCheck [![Build Status](https://travis-ci.org/emil-e/rapidcheck.svg?branch=master)](https://travis-ci.org/emil-e/rapidcheck) [![Build status](https://ci.appveyor.com/api/projects/status/8hms56ghn27agpcj/branch/master?svg=true)](https://ci.appveyor.com/project/emil-e/rapidcheck/branch/master)
==========
# RapidCheck [![Build Status](https://travis-ci.org/emil-e/rapidcheck.svg?branch=master)](https://travis-ci.org/emil-e/rapidcheck) [![Build status](https://ci.appveyor.com/api/projects/status/8hms56ghn27agpcj/branch/master?svg=true)](https://ci.appveyor.com/project/emil-e/rapidcheck/branch/master)

RapidCheck is a C++ framework for property based testing inspired by QuickCheck and other similar frameworks. In property based testing, you state facts about your code that given certain precondition should always be true. RapidCheck then generates random test data to try and find a case for which the property doesn't hold. If such a case is found, RapidCheck tries to find the smallest case (for some definition of smallest) for which the property is still false and then displays this as a counterexample. For example, if the input is an integer, RapidCheck tries to find the smallest integer for which the property is false.

Sounds interesting? Why don't you read the **[User Guide](doc/user_guide.md)** to learn more!

## Why RapidCheck? ##
## Why RapidCheck?

There are existing implementations of property based testing but the ones that I have found are either (in my humble opinion) a bit clunky or are missing essential features such as test case shrinking.

Let's throw together a list of features:
Expand All @@ -16,8 +17,9 @@ Let's throw together a list of features:
- Stateful based on commands in the vein of Erlang QuickCheck
- Integration with popular testing frameworks such as Boost Test, Google Test and Google Mock

## Prerequisites and installation ##
RapidCheck makes extensive use of C++11 and thus requires a compliant compiler. RapidCheck continuous integration builds using Clang 3.5, GCC 4.9 and Visual Studio 2015 so any later versions should also work.
## Prerequisites and installation

RapidCheck makes extensive use of C++11 and thus requires a compliant compiler. RapidCheck continuous integration builds using Clang 3.5, GCC 4.9 and Visual Studio 2015 so any later versions should also work.

RapidCheck uses CMake and is built like any other CMake project. If your own project uses CMake you can simply have RapidCheck as a subdirectory and add the following to your `CMakeLists.txt`:

Expand All @@ -26,7 +28,8 @@ RapidCheck uses CMake and is built like any other CMake project. If your own pro

This will give you both linking and include directories.

## Quick introduction ##
## Quick introduction

A common first example is testing a reversal function. For such a function, double reversal should always result in the original list. In this example we will use the standard C++ `std::reverse` function:

```C++
Expand Down Expand Up @@ -54,7 +57,7 @@ The property above also forms part of a specification of the reversal function:

If we were to run this, RapidCheck would (hopefully) output the following:

```
```text
Using configuration: seed=9928307433081493900
- double reversal yields the original value
Expand All @@ -63,7 +66,7 @@ OK, passed 100 tests

Here, RapidCheck tells us that it ran 100 test cases and all of them passed. It also tells us the configuration that was used, in particular the random seed. If there was a bug in the implementation of `std::reverse` we could get the following output instead:

```
```text
Falsifiable after 12 tests and 10 shrinks
std::tuple<std::vector<int>>:
Expand All @@ -80,5 +83,6 @@ Here RapidCheck tells us that it found a case for which the property does not ho

Can you guess what the bug is? The fact that there are exactly 10 items should give a clue. In this case, the bug is that the implementation sets the first element to `0` when `l0.size() >= 10`. This is also the reason for the initial `0`, the problem doesn't manifest when all elements are zero. How did this bug happen? Who knows!

## Thanks ##
## Thanks

Big thanks to my employer, Spotify, for making it possible for me to spend work time improving RapidCheck.
4 changes: 2 additions & 2 deletions doc/Gen.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
`Gen<T>`
=======
# `Gen<T>`

_This section is incomplete._
4 changes: 2 additions & 2 deletions doc/Seq.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
`Seq<T>`
=======
# `Seq<T>`

_This section is incomplete._

`Seq<T>` implements a lazy sequence (often called a "stream") of values of type `T`. It has a super simple interface for retrieving these values consisting only of the method `Maybe<T> next()`. As can be seen, it returns a `Maybe<T>` which is similar to `boost::optional`, Haskell's `Maybe` or other similar types that either contain a value or be empty. `next` successively returns the values in the sequence until the sequence is exhausted after which it will return an empty `Maybe` to signal that there are no more values.
Expand Down
4 changes: 2 additions & 2 deletions doc/Shrinkable.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
`Shrinkable<T>`
==============
# `Shrinkable<T>`

_This section is incomplete._

`Shrinkable<T>` is a fundamental template class in RapidCheck. An instance of `Shrinkable<T>` represents some value of type `T` and provides a way to access all the ways in which this value can be shrunk. It has value semantics which means that it can be copied and passed around just like any other value. It has two member functions:
Expand Down
38 changes: 24 additions & 14 deletions doc/assertions.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Assertions
==========
# Assertions

There are multiple ways to fail a test case:

- Returning `false`
Expand All @@ -10,7 +10,8 @@ Each is equally valid but using RapidCheck's assertions are a convenient way to

**NOTE:** The assertion macros are implemented using exceptions. The particular exceptions are an implementation detail but you should ensure your properties are exception safe or be prepared to leak resources.

## Capturing ##
## Capturing

In a lot of cases, RapidCheck can capture the expansion of the expression used in the assertion macros in a way inspired by the [Catch.hpp](https://github.com/philsquared/Catch) framework. This means you can write your assertions in a style that is natural to C++:

```C++
Expand All @@ -21,7 +22,7 @@ RC_ASSERT(foo == bar);
If used in a property, RapidCheck might print something similar:
```
```text
main.cpp:24:
foo == bar
Expand All @@ -32,32 +33,41 @@ Expands to:
However, in some cases, this capturing might fail to include the information that you want.

## Reference

The selection of assertion macros is currently rather slim. Suggestions on how to improve this is welcome.

### `RC_ASSERT(expression)` ###
### `RC_ASSERT(expression)`

Fails the test case if `expression` evaluates to `false`.

### `RC_ASSERT_FALSE(expression)` ###
### `RC_ASSERT_FALSE(expression)`

Fails the test case if `expression` evaluates to `true`. Use this instead of `RC_ASSERT(!(...))` since RapidCheck cannot capture the expression in that case.

### `RC_ASSERT_THROWS(expression)` ###
### `RC_ASSERT_THROWS(expression)`

Fails the test case if `expression` does not throw an exception.

### `RC_ASSERT_THROWS_AS(expression, ExceptionType)` ###
### `RC_ASSERT_THROWS_AS(expression, ExceptionType)`

Fails the test case if `expression` does not throw an exception that matches `ExceptionType`.

### `RC_FAIL(msg)` ###
### `RC_FAIL(msg)`

Unconditionally fails the test case with `msg` as message.

### `RC_SUCCEED_IF(expression)` ###
### `RC_SUCCEED_IF(expression)`

Causes the test case to immediately succeed if `expression` evaluates to `true`.

### `RC_SUCCEED(msg)` ###
### `RC_SUCCEED(msg)`

Unconditionally makes the test case succeed with `msg` as the message.

### `RC_PRE(expression)` ###
### `RC_PRE(expression)`

Discards the test case if `expression` evaluates to `false`.

### `RC_DISCARD(msg)` ###
Unconditionally discards the test case with `msg` as the message.
### `RC_DISCARD(msg)`

Unconditionally discards the test case with `msg` as the message.
16 changes: 10 additions & 6 deletions doc/boost.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,24 @@
Boost type support
==================
# Boost type support

RapidCheck comes with support for (currently, a very limited set of) common Boost library types. This support is available through the `extras/boost` module. You can either directly add the `extras/boost/include` directory to your include path or link against the `rapidcheck_boost` target in your `CMakeLists.txt`. You can then simply `#include <rapidcheck/boost.h>`.

The Boost support is currently very limited so if you feel that you're missing some very essential type, please file an issue. Even better, submit a pull request complete with tests!

## Arbitrary ##
## Arbitrary

The following types currently have `Arbitrary` specializations:

- `boost::optional<T>`

## Generators reference ##
## Generators reference

All the Boost generators are located in the `rc::gen::boost` namespace.

#### `Gen<boost::optional<T>> optional(Gen<T> gen)` ####
### `Gen<boost::optional<T>> optional(Gen<T> gen)`

Equivalent to `gen::maybe` but for `boost::optional<T>` instead.

```
```C++
// Example:
const auto optionalSmallInt = *gen::boost::optional(gen::inRange(0, 100));
```
21 changes: 13 additions & 8 deletions doc/boost_test.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
Boost Test integration
=======================
# Boost Test integration

RapidCheck comes with support for integrating with Boost Test and allows you to easily create properties that are also Boost Test test cases.

# Usage #
## Usage

This support is available through the `extras/boost_test` module. You can either directly add the `extras/boost_test/include` directory to your include path or link against the `rapidcheck_boost_test` target in your `CMakeLists.txt`. You can then simply `#include <rapidcheck/boost_test.h>`. Note that `rapidcheck/boost_test.h` needs to be included after any Boost Test headers.

## Reference ##
### `RC_BOOST_PROP(TestName, (args...))` ###
## Reference

### `RC_BOOST_PROP(TestName, (args...))`

Defines a RapidCheck property as a Boost Test test. This macro essentially works the same as the Boost `BOOST_AUTO_TEST_CASE` macro but it includes an extra parenthesized list of arguments that will be generated by RapidCheck, just like when using `rc::check`:

```C++
Expand All @@ -21,7 +24,7 @@ RC_BOOST_PROP(MyTestCase,
The parenthesis around the argument list are required because of how the preprocessor works and can not be omitted. This also means that if you don't want any arguments generated, you need to include an empty set of parenthesis:
```C++
// If you don't have any arguments, you have to have an empty paren
// If you don't have any arguments, you have to have empty parentheses
RC_BOOST_PROP(MyTestCase, inRangeValueIsInRange, ()) {
const auto range = *rc::gen::arbitrary<std::pair<int, int>>();
const auto x = *rc::gen::inRange(range.first, range.second);
Expand All @@ -30,8 +33,10 @@ RC_BOOST_PROP(MyTestCase, inRangeValueIsInRange, ()) {
}
```

### `RC_BOOST_FIXTURE_PROP(Name, Fixture, (args...))` ###
### `RC_BOOST_FIXTURE_PROP(Name, Fixture, (args...))`

Analogous to the Boost `BOOST_FIXTURE_TEST_CASE` macro, defines a RapidCheck property as a Boost Test fixture based test. `Fixture` names the test fixture class. The fixture will be reinstantiated for every test case that is run. Otherwise, this macro works the same as `RC_BOOST_PROP`.

# Assertions #
## Assertions

RapidCheck will treat any exception as a property failure so you should be able to use any assertion mechanism that signals failures as exceptions. However, Boost Test assertions are not implemented using exceptions which means that you should avoid them and use RapidCheck assertions such as `RC_ASSERT` in your RapidCheck properties instead.
9 changes: 5 additions & 4 deletions doc/configuration.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
Configuration
=============
# Configuration

RapidCheck has several parameters that affect testing. These can be configured by setting the `RC_PARAMS` environment variable to a configuration string. The configuration string as the format `<key1>=<value1> <key2>=<value2> ...`. Values containing spaces can be quoted and backslash works as an escape character, just as you would expect.

When RapidCheck is initialized, it prints the configuration that is used excluding parameters that are set to their default value. For example, when running a test without changing the defaults, the following could be printed:

```
```text
Using configuration: seed=8482903616249937941
```

This allows you to reproduce the exact test in case of a failure.

## Reference ##
## Reference

The following settings are provided:

- `seed` - The global random seed used. This is a 64-bit integer. If not set, a random one is chosen using the system random device.
Expand Down
29 changes: 17 additions & 12 deletions doc/debugging.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,45 @@
Debugging failures
==================
# Debugging failures

In a lot of cases, the cause of a property failure is evident in the counterexample given by RapidCheck. However, if this is not the case, RapidCheck provides some tools to let you observe the failure in a debugger.

## Setting breakpoints ##
## Setting breakpoints

When RapidCheck finds a failing test case, it shrinks the test case by recursively trying smaller versions of the original test case. It repeats this process until it cannot find a smaller test case which still fails the property. Since this minimal test case is probably the one that you want to debug, RapidCheck makes it easy for you by calling the `rc::beforeMinimalTestCase` function before the minimal test case is run one last time. By setting a breakpoint on this function, you can pause execution to set up your debugging environment (breakpoints, watchpoints etc.) for the final run.

Please note that this is called for every failing property so you may want to use the facilities of a test framework (i.e. Boost UTF or Google Test) to filter out the specific property you want to debug.

## Reproducing a failure ##
## Reproducing a failure

RapidCheck randomizes its seed each time the test program is run. This means that if you simply rerun a test program after a failure you might not see the bug again. Fortunately, RapidCheck gives you some control over this randomness so that the exact same failure can be reproduced.

### Reusing seeds ###
### Reusing seeds

RapidCheck prints the exact configuration (including the seed) that it uses when the program starts:

```
```text
Using configuration: seed=12776003016957408636
```

If we rerun the program with the same configuration we will get an identical run:

```
```text
RC_PARAMS="seed=12776003016957408636" ./my_test
```

### Reproduce mode ###
### Reproduce mode

If the bug occurred after large number of test cases, waiting for test cases that are known to be successful to run before the failing test case is run can be time consuming. For example, if the bug occurred after 2000 tests and 200 shrinks, it is not interesting to run the 1999 (successful) test cases before that and it is not really necessary to try all the shrunk versions of the failure that were tried but could not reproduce the bug.

Fortunately, RapidCheck will print a string that encodes the necessary information to avoid this to the console if there are failures:

```
```text
Some of your RapidCheck properties had failures. To reproduce these, run with:
RC_PARAMS="reproduce=BMjUhBXakNEalN2aFhXYtBHbl9CZpZXaklmbnJUeUVmbNF2alNXQsxmT11mYlJ3cFFXdhxGZj9A1PXTXZQ2YPQ9z10VGkN2DU_cNdlBZj9A1PXTXZAIgCAAEPAAAAMgADQA"
```

Simply run your test again with this configuration to reproduce the failure:

```
```text
RC_PARAMS="reproduce=BMjUhBXakNEalN2aFhXYtBHbl9CZpZXaklmbnJUeUVmbNF2alNXQsxmT11mYlJ3cFFXdhxGZj9A1PXTXZQ2YPQ9z10VGkN2DU_cNdlBZj9A1PXTXZAIgCAAEPAAAAMgADQA" ./my_test
```

Expand All @@ -45,7 +49,8 @@ While RapidCheck will try to perform as little unnecessary work as possible in r

*Note:* This feature currently does not work with the Catch.hpp integration.

## Logging ##
## Logging

Since the property function may be called many times, using classic debugging techniques such as printing to `stdout` or `stderr` can prove confusing. The `RC_LOG` macro provides a better way to log information during the execution of a test case. Information logged this way will only be printed on failure after the minimal test case has been found.

This macro can be use in two ways. If given no arguments, it works like an `std::ostream`:
Expand All @@ -62,7 +67,7 @@ RC_LOG("It's all broken!");
As stated above, none of this will have any effect on the success or failure of the test case. The information will only be printed when the test case fails otherwise. For example:
```
```text
Falsifiable after 38 tests and 2 shrinks
std::tuple<User>:
Expand Down
4 changes: 2 additions & 2 deletions doc/displaying.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Displaying values
=================
# Displaying values

RapidCheck often needs to display values as strings, such as when printing assertion messages or counterexamples. To do this, RapidCheck calls the `rc::show(const T &, std::ostream &)` template function. This template takes a value to display and a stream to output the string representation to. Given a value `v` and an output stream `os`, calling this template will do one of the following

- If there is a valid overload for a call to `showValue(v, os)`, calls this overload.
Expand Down
Loading

0 comments on commit 98096c1

Please sign in to comment.