Skip to content

Commit

Permalink
Additional tests to prep for first release.
Browse files Browse the repository at this point in the history
  • Loading branch information
daylerees committed Feb 6, 2016
1 parent db6d892 commit ab9aea4
Show file tree
Hide file tree
Showing 9 changed files with 204 additions and 132 deletions.
8 changes: 7 additions & 1 deletion CHANGELOG.md
@@ -1,3 +1,9 @@
# Changelog

- Will be maintained after alpha releases are out of the way. :)
## Next version [???]

## v1.0.0 [06-02-2016]

- First version released.
- 100% test coverage.
- Full documentation.
99 changes: 35 additions & 64 deletions CONTRIBUTING.md
@@ -1,66 +1,37 @@
# Contribution Guide

This guide will be updated with more thorough content, but for now, the best thing you could do to help the project is to create `Journal` packages.

Journals are used to export the information from experiments to data stores for later inspection. They implement the `Scientist\Journals\Journal` interface:

```php
<?php

namespace Scientist\Journals;

use Scientist\Report;
use Scientist\Experiment;

/**
* Class Journal
*
* Journals allow the scientist to record experiment results in a
* variety of different ways.
*
* @package \Scientist
*/
interface Journal
{
/**
* Dispatch a report to storage.
*
* @param \Scientist\Experiment $experiment
* @param \Scientist\Report $report
*
* @return mixed
*/
public function report(Experiment $experiment, Report $report);
}

```

When you implement the `report()` method, the registered Journal will receive an instance of the experiment, and the report from the executed experiment. You can interrogate these instances for all kinds of useful information, and ship that information to a data storage platform.

Here's some useful methods:

```php
$report->getName(); // Get the experiment name. (string)
$report->getControl(); // Get the Scientist\Result instance for the control. (Scientist\Result)
$report->getTrial('name'); // Get the result instance for a trial by name. (Scientist\Result)
$report->getTrials(); // Get an assoc array of trial result instances (array)
```

And on the result instance:

```php
$result->getValue(); // Get the resulting value of the trial/control callback. (mixed)
$result->isMatch(); // Did the result match the control? Use on trial results. (boolean)
$result->getStartTime(); // Float callable start microtime. (float)
$result->getEndTime(); // Float callable end microtime. (float)
$result->getTime(); // Get the execution microtime. (float)
$result->getStartMemory(); // Get memory usage before calling. (integer)
$result->getEndMemory(); // Get memory usage after calling. (integer)
$result->getMemory(); // Get different in memory usage. (integer)
```

If you create a new journal (and a composer package, format scientist-xxx-journal) then please be sure to raise an issue and let me know! I'll update the docs to mentions it.

Thanks for all your support!

- Dayle.
The best thing a contributor can do for the project right now, is to submit supporting resources. Here's some great ideas for you.

## Create Bridge Packages

Bridges allow Scientist to be used seamlessly with other frameworks and libraries. Learn more about Bridges [in the documentation](https://scientist.readme.io/docs/bridges).

Please name your package `vendorname/scientist-<platform>-bridge`.

## Create Journals

Journals allow the Scientist to export his experiment data to data storage platforms for latest inspection.

Please name your package `vendorname/scientist-<platform>-journal`.

See more information [in the documentation](https://scientist.readme.io/docs/journals-1).

## Create Matchers

Matchers allow for complex data types to be matched using Scientist.

Please name your package `vendorname/scientist-<platform>-matcher`.

See more information [in the documentation](https://scientist.readme.io/docs/matchers-1).

## Create Interfaces

Interfaces allow for unique ways to view your experiment data.

Please name your package `vendorname/scientist-<platform>-interface`.

See more information [in the documentation](https://scientist.readme.io/docs/interfaces).

---

Thanks for supporting the project!
6 changes: 3 additions & 3 deletions src/Experiment.php
Expand Up @@ -257,14 +257,14 @@ public function run()
}

/**
* Execute the experiment and return a result.
* Execute the experiment and return a report.
*
* @return \Scientist\Report
*/
public function result()
public function report()
{
$this->params = func_get_args();

return $this->laboratory->getResult($this);
return $this->laboratory->getReport($this);
}
}
2 changes: 1 addition & 1 deletion src/Intern.php
Expand Up @@ -74,7 +74,7 @@ protected function runTrials(Experiment $experiment)
*/
protected function determineMatches(Matcher $matcher, Result $control, array $trials = [])
{
foreach ($trials as $name => $trial) {
foreach ($trials as $trial) {
if ($matcher->match($control->getValue(), $trial->getValue())) {
$trial->setMatch(true);
}
Expand Down
18 changes: 9 additions & 9 deletions src/Laboratory.php
Expand Up @@ -81,8 +81,8 @@ public function experiment($name)
public function runExperiment(Experiment $experiment)
{
if ($experiment->shouldRun()) {
$result = $this->getResult($experiment);
return $result->getControl()->getValue();
$report = $this->getReport($experiment);
return $report->getControl()->getValue();
}

return call_user_func_array(
Expand All @@ -98,26 +98,26 @@ public function runExperiment(Experiment $experiment)
*
* @return \Scientist\Report
*/
public function getResult(Experiment $experiment)
public function getReport(Experiment $experiment)
{
$result = (new Intern)->run($experiment);
$this->reportToJournals($experiment, $result);
$report = (new Intern)->run($experiment);
$this->reportToJournals($experiment, $report);

return $result;
return $report;
}

/**
* Report experiment result to registered journals.
*
* @param \Scientist\Experiment $experiment
* @param \Scientist\Report $result
* @param \Scientist\Report $report
*
* @return void
*/
protected function reportToJournals(Experiment $experiment, Report $result)
protected function reportToJournals(Experiment $experiment, Report $report)
{
foreach ($this->journals as $journal) {
$journal->report($experiment, $result);
$journal->report($experiment, $report);
}
}
}
54 changes: 0 additions & 54 deletions tests/ExampleTest.php

This file was deleted.

59 changes: 59 additions & 0 deletions tests/InternTest.php
@@ -0,0 +1,59 @@
<?php

use Scientist\Intern;
use Scientist\Report;
use Scientist\Experiment;

class InternTest extends PHPUnit_Framework_TestCase
{
public function test_that_intern_can_be_created()
{
$i = new Intern;
$this->assertInstanceOf(Intern::class, $i);
}

public function test_that_intern_can_run_an_experiment()
{
$i = new Intern;
$e = new Experiment('test experiment');
$e->control(function () { return 'foo'; });
$v = $i->run($e);
$this->assertInstanceOf(Report::class, $v);
$this->assertEquals('foo', $v->getControl()->getValue());
}

public function test_that_intern_can_match_control_and_trial()
{
$i = new Intern;
$e = new Experiment('test experiment');
$e->control(function () { return 'foo'; });
$e->trial('bar', function () { return 'foo'; });
$v = $i->run($e);
$this->assertInstanceOf(Report::class, $v);
$this->assertTrue($v->getTrial('bar')->isMatch());
}

public function test_that_intern_can_mismatch_control_and_trial()
{
$i = new Intern;
$e = new Experiment('test experiment');
$e->control(function () { return 'foo'; });
$e->trial('bar', function () { return 'bar'; });
$v = $i->run($e);
$this->assertInstanceOf(Report::class, $v);
$this->assertFalse($v->getTrial('bar')->isMatch());
}

public function test_that_intern_can_match_and_mismatch_control_and_trial()
{
$i = new Intern;
$e = new Experiment('test experiment');
$e->control(function () { return 'foo'; });
$e->trial('bar', function () { return 'foo'; });
$e->trial('baz', function () { return 'baz'; });
$v = $i->run($e);
$this->assertInstanceOf(Report::class, $v);
$this->assertTrue($v->getTrial('bar')->isMatch());
$this->assertFalse($v->getTrial('baz')->isMatch());
}
}
1 change: 1 addition & 0 deletions tests/Journals/JournalTest.php
Expand Up @@ -10,6 +10,7 @@ class JournalTest extends PHPUnit_Framework_TestCase
public function test_that_journals_can_be_created()
{
$s = new StandardJournal;
$this->assertInstanceOf(StandardJournal::class, $s);
}

public function test_that_a_journal_can_be_added_to_a_laboratory()
Expand Down
89 changes: 89 additions & 0 deletions tests/LaboratoryTest.php
@@ -0,0 +1,89 @@
<?php

use Scientist\Report;
use Scientist\Laboratory;

class LaboratoryTest extends PHPUnit_Framework_TestCase
{
public function test_laboratory_can_be_created()
{
$l = new Laboratory;
$this->assertInstanceOf(Laboratory::class, $l);
}

public function test_laboratory_can_run_experiment_with_no_journals()
{
$v = (new Laboratory)
->experiment('test experiment')
->control(function () { return 'foo'; })
->trial('trial name', function () { return 'foo'; })
->run();

$this->assertEquals('foo', $v);
}

public function test_laboratory_can_fetch_report_for_experiment_with_no_journals()
{
$r = (new Laboratory)
->experiment('test experiment')
->control(function () { return 'foo'; })
->trial('trial', function () { return 'bar'; })
->report();

$this->assertInstanceOf(Report::class, $r);
$this->assertEquals('foo', $r->getControl()->getValue());
$this->assertEquals('bar', $r->getTrial('trial')->getValue());
$this->assertInternalType('float', $r->getControl()->getStartTime());
$this->assertInternalType('float', $r->getControl()->getEndTime());
$this->assertInternalType('float', $r->getControl()->getTime());
$this->assertInternalType('float', $r->getTrial('trial')->getStartTime());
$this->assertInternalType('float', $r->getTrial('trial')->getEndTime());
$this->assertInternalType('float', $r->getTrial('trial')->getTime());
$this->assertInternalType('integer', $r->getControl()->getStartMemory());
$this->assertInternalType('integer', $r->getControl()->getEndMemory());
$this->assertInternalType('integer', $r->getControl()->getMemory());
$this->assertInternalType('integer', $r->getTrial('trial')->getStartMemory());
$this->assertInternalType('integer', $r->getTrial('trial')->getEndMemory());
$this->assertInternalType('integer', $r->getTrial('trial')->getMemory());
$this->assertInternalType('null', $r->getControl()->getException());
$this->assertInternalType('null', $r->getTrial('trial')->getException());
$this->assertFalse($r->getTrial('trial')->isMatch());
}

public function test_that_exceptions_are_thrown_within_control()
{
$this->setExpectedException(Exception::class);

$v = (new Laboratory)
->experiment('test experiment')
->control(function () { throw new Exception; })
->trial('trial', function () { return 'foo'; })
->run();
}

public function test_that_exceptions_are_swallowed_within_the_trial()
{
$r = (new Laboratory)
->experiment('test experiment')
->control(function () { return 'foo'; })
->trial('trial', function () { throw new Exception; })
->report();

$this->assertInstanceOf(Report::class, $r);
$this->assertInternalType('null', $r->getControl()->getException());
$this->assertInstanceOf(Exception::class, $r->getTrial('trial')->getException());
}

public function test_that_control_and_trials_receive_parameters()
{
$r = (new Laboratory)
->experiment('test experiment')
->control(function ($one, $two) { return $one; })
->trial('trial', function ($one, $two) { return $two; })
->report('Panda', 'Giraffe');

$this->assertInstanceOf(Report::class, $r);
$this->assertEquals('Panda', $r->getControl()->getValue());
$this->assertEquals('Giraffe', $r->getTrial('trial')->getValue());
}
}

0 comments on commit ab9aea4

Please sign in to comment.