You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In order to support testing patterns where randomized input generation is used to explore a program's state space (such as fuzz, differential, property-based, or mutation testing) we need to add some affordance to developers to specify the following:
The next value to pass into a @Test
A way to receive the prior value and test result as a result of executing @Test
A way to receive additional information, such as changes in code coverage, from the last @Test
A way to stop input generation based on some heuristic or predicate.
Having these 4 items would be the essential set of primitives to really streamline the creation of tools like QuickCheck, libFuzzer, etc. with Swift Testing.
Perhaps this could take the form of a protocol, potentially named TestCaseInputGenerator, that has the following shape:
AdditionalContext can encapsulate data of interest to the test; previously tried values, their results, and other data like code coverage, crashes, etc.
I'm filing this issue rather quickly, in between tasks, so apologizes in advance for oversights on the type signatures and compilability of the proposed API. :)
The text was updated successfully, but these errors were encountered:
SeanROlszewski
changed the title
Add a type for complex input generator that's perform exploratory actions
Add a type for complex input generator that can perform exploratory testing
Sep 25, 2023
SeanROlszewski
changed the title
Add a type for complex input generator that can perform exploratory testing
Add a type for complex input generation that can perform exploratory testing
Sep 25, 2023
SeanROlszewski
changed the title
Add a type for complex input generation that can perform exploratory testing
Add a type for complex input generation that performs exploratory testing
Sep 25, 2023
Description
In order to support testing patterns where randomized input generation is used to explore a program's state space (such as fuzz, differential, property-based, or mutation testing) we need to add some affordance to developers to specify the following:
@Test
@Test
@Test
Having these 4 items would be the essential set of primitives to really streamline the creation of tools like QuickCheck, libFuzzer, etc. with Swift Testing.
Perhaps this could take the form of a protocol, potentially named
TestCaseInputGenerator
, that has the following shape:AdditionalContext
can encapsulate data of interest to the test; previously tried values, their results, and other data like code coverage, crashes, etc.I'm filing this issue rather quickly, in between tasks, so apologizes in advance for oversights on the type signatures and compilability of the proposed API. :)
The text was updated successfully, but these errors were encountered: