New testing library #344

thekid · 2023-01-05T20:38:00Z

Scope of Change

This RFC suggests to create a new testing library for the XP Framework. It will only work with baseless, single-instance test classes.

Rationale

Drop support for extends TestCase, supporting of which costs lots of code
Drop support for seldomly used features such as test timeouts, exotic Values variations or external provider methods
Simplify test actions, these should have not use exceptions for flow control
Integrate Assert::that() extended assertions to simplify commonly used test code

Functionality

In a nutshell, tests reside inside a class and are annotated with the Test attribute. Except for the changed namespace (see below) and the changed default output, this is basically identical to the current usage of the unittest library.

use test\{Assert, Test};

class CalculatorTest {

  #[Test]
  public function addition() {
    Assert::equals(2, (new Calculator())->add(1, 1));
  }
}

To run this test, use the test subcommand:

$ xp test CalculatorTest.class.php
> [PASS] CalculatorTest
  ✓ addition

Tests:       1 succeeded, 0 skipped, 0 failed
Memory used: 1556.36 kB (1610.49 kB peak)
Time taken:  0.001 seconds

Naming

The library will be called test, because it can not only run unittests. The top-level package used follows this, meaning the Assert class' fully qualified name is test.Assert.

Assert DSL

The following shorthand methods exist on the Assert class:

equals(mixed $expected, mixed $actual) - check two values are equal. Uses the util.Objects::equal() method internally, which allows overwriting object comparison.
notEquals(mixed $expected, mixed $actual) - opposite of above
true(mixed $actual) - check a given value is equal to the true boolean
false(mixed $actual) - check a given value is equal to the false boolean
null(mixed $actual) - check a given value is null
instance(string|lang.Type $expected, mixed $actual) - check a given value is an instance of the given type.

Extended assertions

The Assert::that() method starts an assertion chain:

Fluent assertions

Each of the following may be chained to Assert::that():

is(unittest.assert.Condition $condition) - Asserts a given condition matches
isNot(unittest.assert.Condition $condition) - Asserts a given condition does not match
isEqualTo(var $compare) - Asserts the value is equal to a given comparison
isNotEqualTo(var $compare) - Asserts the value is not equal to a given comparison
isNull() - Asserts the value is null
isTrue() - Asserts the value is true
isFalse() - Asserts the value is false
isInstanceOf(string|lang.Type $type) - Asserts the value is of a given type

Transformation

Transforming the value before comparison can make it easier to create the value to compare against. This can be achieved by chaining map() to Assert::that():

$records= $db->open('select ...');
$expected= ['one', 'two'];

// Before
$actual= [];
foreach ($records as $record) {
  $actual[]= $r['name'];
}
Assert::equals($expected, $actual);

// After
Assert::that($records)->mappedBy(fn($r) => $r['name'])->isEqualTo($expected);

Values and providers

#[Values] is a basic provider implementation. It can either return a static list of values as follows:

use test\{Assert, Test, Values};

class CalculatorTest {

  #[Test, Values([[0, 0], [1, 1], [-1, 1]])]
  public function addition($a, $b) {
    Assert::equals($a + $b, (new Calculator())->add($a, $b));
  }
}

...or invoke a method, as seen in this example:

use test\{Assert, Test, Values};

class CalculatorTest {

  private function operands(): iterable {
    yield [0, 0];
    yield [1, 1];
    yield [-1, 1];
  }

  #[Test, Values(from: 'operands')]
  public function addition($a, $b) {
    Assert::equals($a + $b, (new Calculator())->add($a, $b));
  }
}

Provider implementation example

use test\Provider;
use lang\reflection\Type;

class StartServer implements Provider {
  private $connection;

  /** Starts a new server */
  public function __construct(string $bind, ?int $port= null) {
    $port??= rand(1024, 65535);
    $this->connection= "Socket({$bind}:{$port})"; // TODO: Actual implementation ;)
  }

  /**
   * Returns values
   *
   * @param  Type $type
   * @param  ?object $instance
   * @return iterable
   */
  public function values($type, $instance= null) {
    return [$this->connection];
  }
}

Provider values are passed as method argument, just like #[Values], the previous only implementation.

use test\{Assert, Test};

class ServerTest {

  #[Test, StartServer('0.0.0.0', 8080)]
  public function connect($connection) {
    Assert::equals('Socket(0.0.0.0:8080)', $connection);
  }
}

Provider values are passed to the constructor and the connection can be used for all test cases. Note: The provider's values() method is invoked with $instance= null!

use test\{Assert, Test};

#[StartServer('0.0.0.0', 8080)]
class ServerTest {

  public function __construct(private $connection) { }

  #[Test]
  public function connect() {
    Assert::equals('Socket(0.0.0.0:8080)', $this->connection);
  }
}

Test prerequisites

Prerequisites can exist on a test class or a test method. Unlike in the old library, they do not require the Action annotation, but stand alone.

use test\{Assert, Test};
use test\verify\{Runtime, Condition};

#[Condition('class_exists(Calculator::class)')]
class CalculatorTest {

  #[Test, Runtime(php: '^8.1')]
  public function addition() {
    Assert::equals(2, (new Calculator())->add(1, 1));
  }
}

Security considerations

n/a

Speed impact

Same

Dependencies

https://github.com/xp-framework/reflection

Related documents

The text was updated successfully, but these errors were encountered:

thekid · 2023-01-05T20:40:15Z

First POC @ https://github.com/xp-framework/test

thekid · 2023-01-10T16:14:43Z

Migration script

Implementation: https://gist.github.com/thekid/ad6f908f4badbc3460a74a43e9544a86
Migration utility tracking issue test#2

thekid · 2023-01-15T12:17:25Z

Group (class) and case (method) prerequisites @ xp-framework/test#1

thekid · 2023-01-15T15:43:23Z

Test prerequisites analysis

Usage of test actions:

address/.../RecordOfTest.class.php:73:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")'), Expect(class: Error::class, withMessage: '/Too few arguments .+/')]
address/.../RecordOfTest.class.php:78:  #[Test, Action(eval: 'new RuntimeVersion("<7.1")'), Expect(IllegalArgumentException::class)]
address/.../UsingNextTest.class.php:40:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
address/.../ValueOfTest.class.php:22:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
compiler/.../emit/EnumTest.class.php:8:#[Action(eval: 'new VerifyThat(fn() => function_exists("enum_exists"))')]
compiler/.../emit/IntersectionTypesTest.class.php:50:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
compiler/.../emit/IntersectionTypesTest.class.php:62:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
compiler/.../emit/IntersectionTypesTest.class.php:74:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
compiler/.../emit/InvocationTest.class.php:142:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
compiler/.../emit/InvocationTest.class.php:158:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
compiler/.../emit/LambdasTest.class.php:51:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(LambdaExpression::class, "static"))')]
compiler/.../emit/LambdasTest.class.php:62:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(ClosureExpression::class, "static"))')]
compiler/.../emit/ParameterTest.class.php:76:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
compiler/.../emit/ReflectionTest.class.php:43:  #[Test, Action(eval: 'new RuntimeVersion("<8.1")')]
compiler/.../emit/ReflectionTest.class.php:61:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
compiler/.../emit/TraitsTest.class.php:66:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:74:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:86:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:98:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:110:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../loader/CompilingClassLoaderTest.class.php:141:  #[Test, Action(eval: 'new RuntimeVersion(">=7.3")'), Expect(class: ClassFormatException::class, withMessage: '/Compiler error: Class .+ not found/')]
compression/.../CompressionTest.class.php:54:  #[Test, Values(['gzip', 'GZIP', '.gz', '.GZ']), Action(eval: 'new ExtensionAvailable("zlib")')]
core/.../annotations/BrokenAnnotationTest.class.php:31:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")'), Expect(['class' => ClassFormatException::class, 'withMessage' => '/Unterminated annotation/'])]
core/.../core/EnumTest.class.php:59:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:162:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:170:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:217:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:344:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/ErrorsTest.class.php:115:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new RuntimeVersion("<7.1.0-dev")')]
core/.../core/ErrorsTest.class.php:121:  #[Test, Expect(Error::class), Action(eval: 'new RuntimeVersion(">=7.1.0")')]
core/.../core/ErrorsTest.class.php:127:  #[Test, Expect(ClassCastException::class), Action(eval: 'new RuntimeVersion("<7.4.0-dev")')]
core/.../core/ErrorsTest.class.php:133:  #[Test, Expect(Error::class), Action(eval: 'new RuntimeVersion(">=7.4.0")')]
core/.../core/ExceptionsTest.class.php:140:  #[Test, Action(eval: 'new RuntimeVersion(">=7.0.0")')]
core/.../core/FunctionTypeTest.class.php:275:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../core/NewInstanceTest.class.php:202:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:224:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:246:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:258:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:270:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:282:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:390:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:405:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:420:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:437:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:453:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:468:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:483:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:498:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:513:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:528:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:543:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:559:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:574:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:589:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/ProcessResolveTest.class.php:41:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:46:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:51:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:58:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:65:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:72:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:77:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:87:  #[Test, Action(eval: 'new IsPlatform("WIN")'), Expect(IOException::class)]
core/.../core/ProcessResolveTest.class.php:107:  #[Test, Action(eval: 'new IsPlatform("ANDROID")')]
core/.../core/ProcessResolveTest.class.php:113:  #[Test, Action(eval: 'new IsPlatform("!(WIN|ANDROID)")')]
core/.../core/ProcessResolveTest.class.php:118:  #[Test, Values(['"ls"', "'ls'"]), Action(eval: 'new IsPlatform("!(WIN|ANDROID)")')]
core/.../core/ProcessResolveTest.class.php:123:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:128:  #[Test, Action(eval: 'new IsPlatform("!(WIN|ANDROID)")')]
core/.../core/TypeIntersectionTest.class.php:120:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
core/.../core/TypeIntersectionTest.class.php:127:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
core/.../core/TypeIntersectionTest.class.php:134:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
core/.../core/TypeOfTest.class.php:99:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../core/TypeOfTest.class.php:104:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../core/TypeOfTest.class.php:109:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeOfTest.class.php:114:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:192:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:201:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:210:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:219:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:228:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:234:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:240:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:246:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../io/PathTest.class.php:190:  #[Test, Action(eval: 'new IsPlatform("!^Win")')]
core/.../io/PathTest.class.php:256:  #[Test, Action(eval: 'new IsPlatform("^Win")')]
core/.../io/PathTest.class.php:266:  #[Test, Values(['C:', 'C:/', 'c:', 'C:/']), Action(eval: 'new IsPlatform("^Win")')]
core/.../io/PathTest.class.php:347:  #[Test, Action(eval: 'new IsPlatform("^Win")'), Values([['\\\\remote\\file.txt', true], ['\\\\remote', true]])]
core/.../io/PathTest.class.php:352:  #[Test, Action(eval: 'new IsPlatform("^Win")')]
core/.../io/PathTest.class.php:357:  #[Test, Action(eval: 'new IsPlatform("^Win")')]
core/.../reflection/FieldAccessTest.class.php:75:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldAccessTest.class.php:85:  #[Test, Expect(IllegalAccessException::class), Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldAccessTest.class.php:91:  #[Test, Expect(IllegalAccessException::class), Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldModifiersTest.class.php:28:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldTypeTest.class.php:64:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/FieldTypeTest.class.php:77:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/FieldTypeTest.class.php:89:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/MethodInvocationTest.class.php:46:  #[Test, Expect(TargetInvocationException::class), Action(eval: 'new RuntimeVersion(">=7.0")')]
core/.../reflection/MethodInvocationTest.class.php:52:  #[Test, Expect(TargetInvocationException::class), Action(eval: 'new RuntimeVersion(">=7.0")')]
core/.../reflection/MethodParametersTest.class.php:97:  #[Test, Action(eval: 'new RuntimeVersion(">=7.0")'), Values([['string'], ['int'], ['bool'], ['float']])]
core/.../reflection/MethodParametersTest.class.php:105:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/MethodParametersTest.class.php:114:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")'), Values([['string|int'], ['string|false']])]
core/.../reflection/MethodParametersTest.class.php:270:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:275:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:280:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:285:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:290:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodReturnTypesTest.class.php:87:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/MethodReturnTypesTest.class.php:93:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/MethodReturnTypesTest.class.php:99:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/MethodReturnTypesTest.class.php:105:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")'), Values([['string|int'], ['string|false']])]
core/.../reflection/MethodReturnTypesTest.class.php:169:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/MethodReturnTypesTest.class.php:190:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/RuntimeClassDefinitionTest.class.php:148:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/TypeSyntaxTest.class.php:32:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/TypeSyntaxTest.class.php:53:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/TypeSyntaxTest.class.php:60:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/TypeSyntaxTest.class.php:67:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/TypeSyntaxTest.class.php:74:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../util/MoneyTest.class.php:8:#[Action(eval: 'new ExtensionAvailable("bcmath")')]
core/.../util/OpenSSLSecretTest.class.php:10:#[Action(eval: 'new ExtensionAvailable("openssl")')]
core/.../util/PropertyExpansionTest.class.php:31:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../util/RandomTest.class.php:50:  #[Test, Action(eval: 'new VerifyThat(function() { return (function_exists("random_bytes") || function_exists("openssl_random_pseudo_bytes"));})')]
core/.../util/RandomTest.class.php:55:  #[Test, Action(eval: 'new ExtensionAvailable("openssl")')]
core/.../util/RandomTest.class.php:60:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:93:  #[Test, Action(eval: 'new ExtensionAvailable("openssl")')]
core/.../util/RandomTest.class.php:99:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:116:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
core/.../util/RandomTest.class.php:121:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
core/.../util/SodiumSecretTest.class.php:10:#[Action(eval: 'new ExtensionAvailable("sodium")')]
coverage/.../coverage/tests/RecordCoverageTest.class.php:7:#[Action(eval: 'new VerifyThat(function() { return interface_exists(\unittest\Listener::class); })')]
ftp/.../IntegrationTest.class.php:15:#[Action(eval: 'new StartServer("peer.ftp.unittest.TestingServer", "connected", "shutdown")')]
imaging/.../ExifDataTest.class.php:14:#[Action(eval: 'new ExtensionAvailable("exif")')]
imaging/.../GifImageReaderTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("GIF")]')]
imaging/.../GifImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("GIF")]')]
imaging/.../ImageReaderTest.class.php:16:#[Action(eval: 'new ExtensionAvailable("gd")')]
imaging/.../JpegImageReaderTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("JPEG")]')]
imaging/.../JpegImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("JPEG")]')]
imaging/.../PngImageReaderTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("PNG")]')]
imaging/.../PngImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("PNG")]')]
imaging/.../WebpImageReaderTest.class.php:14:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("WEBP")]')]
imaging/.../WebpImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("WEBP")]')]
mail/.../ImapStoreTest.class.php:13:#[Action(eval: 'new ExtensionAvailable("imap")')]
networking/.../sockets/BSDSocketTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("sockets"), new StartServer(TestingServer::class, "connected", "shutdown")]')]
networking/.../sockets/SocketTest.class.php:12:#[Action(eval: 'new StartServer(TestingServer::class, "connected", "shutdown")')]
rdbms/.../ConfiguredConnectionManagerTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../CriteriaTest.class.php:15:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../DataSetTest.class.php:16:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../DBTest.class.php:10:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../FinderTest.class.php:15:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../ProjectionTest.class.php:19:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../QueryTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../QueuedConnectionManagerTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../RegisteredConnectionManagerTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../sqlite3/SQLite3ConnectionTest.class.php:14:#[Action(eval: 'new ExtensionAvailable("sqlite3")')]
rdbms/.../sqlite3/SQLite3CreationTest.class.php:16:#[Action(eval: 'new ExtensionAvailable("sqlite3")')]
rdbms/.../StatementTest.class.php:13:#[Action(eval: 'new RegisterMockConnection()')]
reflection/.../ConstantsTest.class.php:30:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../ConstantsTest.class.php:56:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../ConstantsTest.class.php:62:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../ConstantsTest.class.php:68:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../InstantiationTest.class.php:89:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:89:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../MethodsTest.class.php:95:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:101:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../MethodsTest.class.php:107:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../MethodsTest.class.php:113:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2")'), Values(['true', 'false'])]
reflection/.../MethodsTest.class.php:125:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../MethodsTest.class.php:228:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:237:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:246:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../MethodsTest.class.php:285:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:294:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../ParametersTest.class.php:143:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../PropertiesTest.class.php:106:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")'), Expect(AccessingFailed::class)]
reflection/.../PropertiesTest.class.php:148:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:157:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:166:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:175:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../PropertiesTest.class.php:184:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../PropertiesTest.class.php:193:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2")'), Values(['true', 'false'])]
reflection/.../PropertiesTest.class.php:220:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:229:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../PropertiesTest.class.php:238:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../TypeTest.class.php:81:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2")')]
reflection/.../TypeTest.class.php:111:  #[Test, Action(eval: 'new VerifyThat(fn() => !self::$ENUMS)')]
reflection/.../TypeTest.class.php:117:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:204:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:210:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
rest-client/.../CompressionHandlingTest.class.php:43:  #[Test, Action(eval: 'new ExtensionAvailable("zlib")')]
rest-client/.../CompressionHandlingTest.class.php:48:  #[Test, Action(eval: 'new ExtensionAvailable("brotli")')]
rest-client/.../CompressionHandlingTest.class.php:74:  #[Test, Values([1, 6, 9]), Action(eval: 'new ExtensionAvailable("zlib")')]
rest-client/.../CompressionHandlingTest.class.php:83:  #[Test, Values([0, 6, 11]), Action(eval: 'new ExtensionAvailable("brotli")')]
sequence/.../EachTest.class.php:85:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1.0")')]
sequence/.../PeekTest.class.php:76:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1.0")')]
text-encode/.../Base64InputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]
text-encode/.../Base64OutputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]
webtest/.../web/tests/FormsTest.class.php:19:    $this->assertEquals($action, $form->getAction());
xml/.../XslCallbackTest.class.php:16:#[Action(eval: '[new ExtensionAvailable("dom"), new ExtensionAvailable("xsl")]')]
zip/.../AbstractZipFileTest.class.php:13:#[Action(eval: 'new ExtensionAvailable("zlib")')]
zip/.../vendors/SevenZipFileTest.class.php:36:  #[Test, Action(eval: 'new ExtensionAvailable("zlib")')]
zip/.../vendors/SevenZipFileTest.class.php:41:  #[Test, Action(eval: 'new ExtensionAvailable("bz2")')]

Usage of VerifyThat:

compiler/.../emit/EnumTest.class.php:8:#[Action(eval: 'new VerifyThat(fn() => function_exists("enum_exists"))')]
compiler/.../emit/LambdasTest.class.php:51:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(LambdaExpression::class, "static"))')]
compiler/.../emit/LambdasTest.class.php:62:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(ClosureExpression::class, "static"))')]
core/.../core/EnumTest.class.php:59:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:162:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:170:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:217:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:344:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/FunctionTypeTest.class.php:275:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../core/NewInstanceTest.class.php:202:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:224:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:246:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:258:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:270:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:282:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:390:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:405:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:420:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:437:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:453:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:468:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:483:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:498:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:513:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:528:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:543:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:559:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:574:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:589:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../util/PropertyExpansionTest.class.php:31:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../util/RandomTest.class.php:50:  #[Test, Action(eval: 'new VerifyThat(function() { return (function_exists("random_bytes") || function_exists("openssl_random_pseudo_bytes"));})')]
core/.../util/RandomTest.class.php:60:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:99:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:116:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
core/.../util/RandomTest.class.php:121:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
coverage/.../RecordCoverageTest.class.php:7:#[Action(eval: 'new VerifyThat(function() { return interface_exists(\unittest\Listener::class); })')]
reflection/.../TypeTest.class.php:111:  #[Test, Action(eval: 'new VerifyThat(fn() => !self::$ENUMS)')]
reflection/.../TypeTest.class.php:117:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:204:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:210:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
text-encode/.../Base64InputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]
text-encode/.../Base64OutputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]

thekid · 2023-01-16T18:15:21Z

Transform values before comparison @ xp-framework/test#3

thekid · 2023-01-16T18:30:19Z

https://github.com/xp-forge/assert has been set to public archive.

thekid · 2023-01-16T18:52:17Z

Submitted to Packagist -> https://packagist.org/packages/xp-framework/test

thekid · 2023-01-21T10:07:23Z

Generalized provider concept @ xp-framework/test#4

See xp-framework/rfc#344

thekid · 2023-01-29T19:35:04Z

First migration for Dialog done partially by migration script. The integration test needed to be migrated manually, as constructor arguments are not passed by default - see xp-framework/test#7

thekid · 2023-01-29T21:33:53Z

Current library is less than 1000 LOC, compared to 2974 for xp-framework/unittest:

$ cloc-1.82.pl src/main/php/
      39 text files.
      39 unique files.
       9 files ignored.

github.com/AlDanial/cloc v 1.82  T=0.10 s (404.0 files/s, 15444.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
PHP                             39            221            368            902
-------------------------------------------------------------------------------
SUM:                            39            221            368            902
-------------------------------------------------------------------------------

thekid · 2023-02-09T19:42:04Z

First release @ https://github.com/xp-framework/test/releases/tag/v1.0.0 🎉

thekid · 2023-02-10T21:00:32Z

Early adapters:

thekid · 2023-02-12T12:31:32Z

Project created for tracking progress: https://github.com/orgs/xp-framework/projects/3/views/1?layout=board

thekid · 2023-02-12T16:39:47Z

All of xp-lang is now migrated, see here

thekid · 2023-03-10T10:16:44Z

All of xp-forge is now migrated, see here

thekid · 2023-06-25T16:41:10Z

Finally, XP Core is migrated 🎉

thekid · 2023-06-25T16:54:15Z

The repository https://github.com/xp-framework/unittest is now archived

See xp-framework/rfc#344

Implements xp-framework/rfc#344

See xp-framework/rfc#344

thekid · 2024-03-10T10:34:34Z

Remaining libraries:

$ grep 'xp-framework/unittest' */composer.json | grep -v ^unittest
caching/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
coverage/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.7",
ftp/composer.json:    "xp-framework/unittest": "^11.0 | ^10.1"
imaging/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
io-collections/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0 | ^6.5"
ldap/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
mail/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
measure/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
meilisearch/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
mocks/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 |^7.0",
patterns/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
sql-parser/composer.json:    "xp-framework/unittest": "^11.1"
text-encode/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
webtest/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0",

See xp-framework/rfc#344

thekid added discussion technology labels Jan 5, 2023

thekid mentioned this issue Jan 15, 2023

Implement test prerequisites xp-framework/test#1

Merged

6 tasks

thekid added a commit to thekid/dialog that referenced this issue Jan 29, 2023

Migrate to new testing library

b920516

See xp-framework/rfc#344

thekid mentioned this issue Jan 29, 2023

Migrate to new testing library thekid/dialog#53

Merged

thekid added accepted and removed discussion labels Jan 29, 2023

This was referenced Jan 31, 2023

Migrate to new testing library thekid/cas#19

Merged

Migrate to new testing library xp-framework/compiler#153

Merged

thekid added implemented and removed accepted labels Feb 10, 2023

This was referenced Feb 11, 2023

Migrate to new testing library xp-framework/ast#41

Merged

Migrate to new testing library xp-framework/reflection#30

Merged

This was referenced Feb 17, 2023

Migrate test suite xp-forge/sequence#55

Merged

Migrate to new testing library xp-forge/web#99

Merged

Migrate to new testing library xp-forge/yaml#10

Merged

thekid mentioned this issue Apr 7, 2023

Migrate to new testing library xp-framework/zip#3

Merged

This was referenced Apr 16, 2023

Migrate to new testing library xp-framework/tokenize#2

Merged

Migrate to new testing library xp-framework/math#4

Merged

Migrate to new testing library xp-framework/collections#3

Merged

Migrate to new testing library xp-framework/logging#17

Merged

thekid mentioned this issue May 18, 2023

Migrate to new testing library xp-framework/networking#27

Merged

thekid added a commit to xp-framework/http that referenced this issue Jul 30, 2023

Migrate to new testing library

47df319

See xp-framework/rfc#344

thekid added a commit to xp-framework/rdbms that referenced this issue Nov 19, 2023

Migrate unit and integration tests to new testing library

b76569b

See xp-framework/rfc#344

thekid added a commit to xp-framework/csv that referenced this issue Feb 4, 2024

Migrate to new testing library, add PHP 8.3 & PHP 8.4

a43b628

See xp-framework/rfc#344

thekid added a commit to xp-framework/xml that referenced this issue Mar 1, 2024

Migrate to new testing library

2dfefe0

Implements xp-framework/rfc#344

thekid added a commit to xp-framework/command that referenced this issue Mar 10, 2024

Migrate to new testing library

f1e1bb8

See xp-framework/rfc#344

thekid added a commit to xp-framework/imaging that referenced this issue Mar 28, 2024

Migrate to new testing library

258ddaa

See xp-framework/rfc#344

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New testing library #344

New testing library #344

thekid commented Jan 5, 2023 •

edited

Loading

thekid commented Jan 5, 2023

thekid commented Jan 10, 2023 •

edited

Loading

thekid commented Jan 15, 2023

thekid commented Jan 15, 2023 •

edited

Loading

thekid commented Jan 16, 2023

thekid commented Jan 16, 2023

thekid commented Jan 16, 2023

thekid commented Jan 21, 2023

thekid commented Jan 29, 2023

thekid commented Jan 29, 2023 •

edited

Loading

thekid commented Feb 9, 2023

thekid commented Feb 10, 2023

thekid commented Feb 12, 2023 •

edited

Loading

thekid commented Feb 12, 2023

thekid commented Mar 10, 2023

thekid commented Jun 25, 2023

thekid commented Jun 25, 2023

thekid commented Mar 10, 2024

New testing library #344

New testing library #344

Comments

thekid commented Jan 5, 2023 • edited Loading

Scope of Change

Rationale

Functionality

Naming

Assert DSL

Extended assertions

Fluent assertions

Transformation

Values and providers

Provider implementation example

Test prerequisites

Security considerations

Speed impact

Dependencies

Related documents

thekid commented Jan 5, 2023

thekid commented Jan 10, 2023 • edited Loading

Migration script

thekid commented Jan 15, 2023

thekid commented Jan 15, 2023 • edited Loading

Test prerequisites analysis

thekid commented Jan 16, 2023

thekid commented Jan 16, 2023

thekid commented Jan 16, 2023

thekid commented Jan 21, 2023

thekid commented Jan 29, 2023

thekid commented Jan 29, 2023 • edited Loading

thekid commented Feb 9, 2023

First release @ https://github.com/xp-framework/test/releases/tag/v1.0.0 🎉

thekid commented Feb 10, 2023

thekid commented Feb 12, 2023 • edited Loading

thekid commented Feb 12, 2023

thekid commented Mar 10, 2023

thekid commented Jun 25, 2023

Finally, XP Core is migrated 🎉

thekid commented Jun 25, 2023

thekid commented Mar 10, 2024

thekid commented Jan 5, 2023 •

edited

Loading

thekid commented Jan 10, 2023 •

edited

Loading

thekid commented Jan 15, 2023 •

edited

Loading

thekid commented Jan 29, 2023 •

edited

Loading

thekid commented Feb 12, 2023 •

edited

Loading