Skip to content

Commit

Permalink
Merge branch 'develop'
Browse files Browse the repository at this point in the history
* develop:
  replace do/while by while to avoid shrinking unshrinkable values
  add links to sets
  update the documentation
  display the variable name before displaying the associated value
  remove outdated note
  mark Set interface as internal
  add test to fail on purpose to check the output of property list
  add stateful testing example
  fix associating the wrong values when rethrowing the previous failure
  add possibility to do stateful testing
  allow to filter sequences
  make sure a sequence is never shrunk with fewer elements than the specified lower bound
  always return the specified number of elements
  fix sequence type declarations
  add sequence set
  • Loading branch information
Baptouuuu committed Apr 25, 2020
2 parents eb43e12 + d22a8a7 commit 119b375
Show file tree
Hide file tree
Showing 27 changed files with 1,231 additions and 62 deletions.
146 changes: 113 additions & 33 deletions README.md
Expand Up @@ -12,7 +12,22 @@ When I run tests I need some data to assert the validity of my code, the first a

The goal of this library is to help build higher order sets to facilitate the understanding of tests.

**Note**: the library only generates primitives types, any user defined type set must be declared in its dedicated package.
`BlackBox` comes with the `Set`s of primitives:
- [`Integers`](src/Set/Integers.php) -> `int`
- [`RealNumbers`](src/Set/RealNumbers.php) -> `float`
- [`Strings`](src/Set/Strings.php) -> `string`
- [`UnsafeStrings`](src/Set/UnsafeStrings.php) -> `string` (found [here](https://github.com/minimaxir/big-list-of-naughty-strings))
- [`Regex`](src/Set/Regex.php) -> `string`

User defined elements `Set`s can be defined with:
- [`Elements`](src/Set/Elements.php)
- [`FromGenerator`](src/Set/FromGenerator.php)

Higher order `Set`s allows you create structures:
- [`Decorate`](src/Set/Decorate.php) -> map a type `A` to a type `B`, ie an `int` to an object `Age`
- [`Composite`](src/Set/Composite.php) -> map many types to another unique type, ie `string $firstname` and `string $lastname` to an object `User($firstname, $lastname)`
- [`Sequence`](src/Set/Sequence.php) -> create an `array` containing multiple elements of the same type
- [`Either`](src/Set/Either.php) -> will generate either a type `A` or `B`, ie to create nullable `int`s via `Either(Integers::any(), Elements::of(null))`

## Installation

Expand All @@ -28,69 +43,134 @@ use Innmind\BlackBox\{
PHPUnit\BlackBox,
};

final class User
final class Counter
{
private $firstName;
private $lastName;
private int $current;

public function __construct(string $firstName, string $lastName)
public function __construct(int $initial = 0)
{
$this->firstName = $firstName;
$this->lastName = $lastName;
$this->current = $initial;
}

public function firstName(): string
public function up(): void
{
return $this->firstName;
if ($this->current === 100) {
return;
}

++$this->current;
}

public function greet(): string
public function down(): void
{
return "Hi, I'm {$this->firstName}";
if ($this->current === 0) {
return;
}

--$this->current;
}
}

class UserSet
{
public static function any(): Set
public function current(): int
{
return Set\Composite::immutable(
function($firstName, $lastName): User {
return new User($firstName, $lastName);
},
Set\Strings::any(), // firstNames
Set\Strings::any() // lastNames
);
return $this->current;
}
}

class UserTest extends \PHPUnit\Framework\TestCase
class CounterTest extends \PHPUnit\Framework\TestCase
{
use BlackBox;

public function testUserGreetsWithHisFirstName()
public function testCounterValueIsAlwaysHigherAfterCoutningUp()
{
$this
->forAll(UserSet::any())
->then(function(User $user) {
$this->assertSame(
"Hi, I'm {$user->firstName()}",
$user->greet()
);
->forAll(
Set\Integers::between(0, 100), // counter bounds
)
->then(function(int $initial) {
$counter = new Counter($initial);
$counter->up();

$this->assertGreaterThan($initial, $counter->value());
});
}
}
```

This really simple example show how the test class is focused on the behaviour and not about the construction of the test data.

**Note**: here the `User` class is not mutable, but in your application it's likely that such class (meaning an entity) would be mutable, in such case you MUST use `Composite::mutable()` otherwise a mutated object would bleed between the iterations of the test.

By default the library supports the shrinking of data to help you find the smallest possible set of values that makes your test fail. To help you ease the debugging of the code you can use the printer class `Innmind\BlackBox\PHPUnit\ResultPrinterV8` that will print the set of generated data that made your test fail.

![](printer.png)

**Important**: shrinking use recursion to find the smallest value thus generating deep call stacks, so you may need to disable the xdebug `Maximum function nesting level` option.
### Stateful testing

When we write tests we tend to focus on evaluating the behaviour when doing one action (like in our counter example above). This technique help us cover most of our code, but when we deal we stateful systems (such as a counter, an entity or a daemon) it becomes harder to make sure all succession of mutations will always result in a coherent new state.

Once again Property Based Testing can help us improve the coverage of behaviours. Instead of describing the initial test to the framework and manually do one action, we describe to the framework all the properties that our system must hold and the framework will try to find a succession of actions that will break our properties.

If we reuse the counter example from above, the property would be written like this:

```php
use Innmind\BlackBox\Property;
use PHPUnit\Framework\Assert;

final class CountingUpAlwaysEndInAHigherCount implements Property
{
public function name(): string
{
return 'Counting up always end in a higher count';
}

public function applicableTo(object $counter): bool
{
return $counter->current() < 100; // since upper bound is 100
}

public function ensureHelBy(object $counter): object
{
$initial = $counter->current();
$counter->up();

Assert::assertGreaterThan($initial, $counter->current());

return $counter;
}
}
```

We could also write the inverse property `CountingDownAlwaysEndInALowerCount`, the test in phpunit would then look like this:

```php
class CounterTest extends \PHPUnit\Framework\TestCase
{
use BlackBox;

public function testProperties()
{
$this
->forAll(
Set\Properties::of(
new CountingUpAlwaysEndInAHigherCount,
new CountingDownAlwaysEndInALowerCount,
),
Set\Integers::between(0, 100), // counter bounds
)
->then(function($scenario, $initial) {
$scenario->ensureHelBy(new Counter($initial));
});
}
}
```

**Note**: you should declare the properties as the first set of `forAll` to make sure it is shrunk first.

The above example would generate multiple scenarii of counting up and down (it tries to apply up to 100 properties per scenario). In the case it found a failing scenario, it would be displayed as follow in your terminal:

![](state_error.png)

**Note**: this counter example is used as the test process of this framework, all properties to prove the behaviour of the counter can be found in the [`fixtures/`](fixtures/) folder.

**Note 2**: this example was taken from an article by [Johannes Link](https://twitter.com/johanneslink) on [Model-based Testing](https://johanneslink.net/model-based-testing/).

## Configuration

Expand Down
3 changes: 2 additions & 1 deletion composer.json
Expand Up @@ -28,7 +28,8 @@
},
"autoload-dev": {
"psr-4": {
"Tests\\Innmind\\BlackBox\\": "tests/"
"Tests\\Innmind\\BlackBox\\": "tests/",
"Fixtures\\Innmind\\BlackBox\\": "fixtures/"
}
},
"require-dev": {
Expand Down
46 changes: 46 additions & 0 deletions fixtures/Counter.php
@@ -0,0 +1,46 @@
<?php
declare(strict_types = 1);

namespace Fixtures\Innmind\BlackBox;

final class Counter
{
private int $value;
private bool $correct = true;

public function __construct(int $initial = 0)
{
$this->value = $initial;
}

public static function failOnPurpose(): self
{
$self = new self;
$self->correct = false;

return $self;
}

public function down(): void
{
if ($this->correct && $this->value === 0) {
return;
}

--$this->value;
}

public function up(): void
{
if ($this->correct && $this->value === 100) {
return;
}

++$this->value;
}

public function current(): int
{
return $this->value;
}
}
30 changes: 30 additions & 0 deletions fixtures/DownAndUpIsAnIdentityFunction.php
@@ -0,0 +1,30 @@
<?php
declare(strict_types = 1);

namespace Fixtures\Innmind\BlackBox;

use Innmind\BlackBox\Property;
use PHPUnit\Framework\Assert;

final class DownAndUpIsAnIdentityFunction implements Property
{
public function name(): string
{
return 'Down and up return to the initial value';
}

public function applicableTo(object $counter): bool
{
return $counter->current() > 0;
}

public function ensureHeldBy(object $counter): object
{
$initial = $counter->current();
$counter->down();
$counter->up();
Assert::assertSame($initial, $counter->current());

return $counter;
}
}
29 changes: 29 additions & 0 deletions fixtures/DownChangeState.php
@@ -0,0 +1,29 @@
<?php
declare(strict_types = 1);

namespace Fixtures\Innmind\BlackBox;

use Innmind\BlackBox\Property;
use PHPUnit\Framework\Assert;

final class DownChangeState implements Property
{
public function name(): string
{
return 'Down change state';
}

public function applicableTo(object $counter): bool
{
return $counter->current() > 0;
}

public function ensureHeldBy(object $counter): object
{
$initial = $counter->current();
$counter->down();
Assert::assertLessThan($initial, $counter->current());

return $counter;
}
}
28 changes: 28 additions & 0 deletions fixtures/LowerBoundAtZero.php
@@ -0,0 +1,28 @@
<?php
declare(strict_types = 1);

namespace Fixtures\Innmind\BlackBox;

use Innmind\BlackBox\Property;
use PHPUnit\Framework\Assert;

final class LowerBoundAtZero implements Property
{
public function name(): string
{
return 'Counter can not go lower than 0';
}

public function applicableTo(object $counter): bool
{
return $counter->current() < 2;
}

public function ensureHeldBy(object $counter): object
{
$counter->down();
Assert::assertSame(0, $counter->current());

return $counter;
}
}
30 changes: 30 additions & 0 deletions fixtures/UpAndDownIsAnIdentityFunction.php
@@ -0,0 +1,30 @@
<?php
declare(strict_types = 1);

namespace Fixtures\Innmind\BlackBox;

use Innmind\BlackBox\Property;
use PHPUnit\Framework\Assert;

final class UpAndDownIsAnIdentityFunction implements Property
{
public function name(): string
{
return 'Up and down return to the initial value';
}

public function applicableTo(object $counter): bool
{
return $counter->current() < 99;
}

public function ensureHeldBy(object $counter): object
{
$initial = $counter->current();
$counter->up();
$counter->down();
Assert::assertSame($initial, $counter->current());

return $counter;
}
}
29 changes: 29 additions & 0 deletions fixtures/UpChangeState.php
@@ -0,0 +1,29 @@
<?php
declare(strict_types = 1);

namespace Fixtures\Innmind\BlackBox;

use Innmind\BlackBox\Property;
use PHPUnit\Framework\Assert;

final class UpChangeState implements Property
{
public function name(): string
{
return 'Up change state';
}

public function applicableTo(object $counter): bool
{
return $counter->current() < 100;
}

public function ensureHeldBy(object $counter): object
{
$initial = $counter->current();
$counter->up();
Assert::assertGreaterThan($initial, $counter->current());

return $counter;
}
}

0 comments on commit 119b375

Please sign in to comment.