Skip to content

Latest commit

 

History

History
218 lines (164 loc) · 17.4 KB

Arbitraries.md

File metadata and controls

218 lines (164 loc) · 17.4 KB
Raw

🏠 Arbitraries

Arbitraries are responsible for the random - but deterministic - generation and shrink of datatypes. They can be combined together to build more complex datatypes.

This documentation lists all the built-in arbitraries provided by fast-check.

You can refer to the generated TypeDoc for more details.

Boolean (:boolean)

  • fc.boolean() either true or false

Numeric (:number)

Integer values:

  • fc.integer() all possible integers ie. from -2147483648 (included) to 2147483647 (included)
  • fc.integer(max: number) all possible integers between -2147483648 (included) and max (included)
  • fc.integer(min: number, max: number) all possible integers between min (included) and max (included)
  • fc.nat() all possible positive integers ie. from 0 (included) to 2147483647 (included)
  • fc.nat(max: number) all possible positive integers between 0 (included) and max (included)
  • fc.maxSafeInteger() all possible positive integers between Number.MIN_SAFE_INTEGER (included) and Number.MAX_SAFE_INTEGER (included)
  • fc.maxSafeNat() all possible positive integers between 0 (included) and Number.MAX_SAFE_INTEGER (included)

Floating point numbers:

  • fc.float() uniformly distributed float value between 0.0 (included) and 1.0 (excluded)
  • fc.float(max: number) uniformly distributed float value between 0.0 (included) and max (excluded)
  • fc.float(min: number, max: number) uniformly distributed float value between min (included) and max (excluded)
  • fc.double()uniformly distributed double value between 0.0 (included) and 1.0 (excluded)
  • fc.double(max: number)uniformly distributed double value between 0.0 (included) and max (excluded)
  • fc.double(min: number, max: number)uniformly distributed double value between min (included) and max (excluded)

BigInt (if supported by your JavaScript interpreter):

  • fc.bigIntN(n: number) all possible bigint between -2^(n-1) (included) and 2^(n-1)-1 (included)
  • fc.bigInt() uniformly distributed bigint values
  • fc.bigInt(min: bigint, max: bigint) all possible bigint between min (included) and max (included)
  • fc.bigUintN(n: number) all possible bigint between 0 (included) and 2^n -1 (included)
  • fc.bigUint() uniformly distributed bigint positive values
  • fc.bigUint(max: bigint) all possible bigint between 0 (included) and max (included)

String (:string)

Single character only:

  • fc.hexa() one character in 0123456789abcdef (lower case)
  • fc.base64() one character in A-Z, a-z, 0-9, + or /
  • fc.char() between 0x20 (included) and 0x7e (included) , corresponding to printable characters (see https://www.ascii-code.com/)
  • fc.ascii() between 0x00 (included) and 0x7f (included)
  • fc.unicode() between 0x0000 (included) and 0xffff (included) but excluding surrogate pairs (between 0xd800 and 0xdfff). Generate any character of UCS-2 which is a subset of UTF-16 (restricted to BMP plan)
  • fc.char16bits() between 0x0000 (included) and 0xffff (included). Generate any 16 bits character. Be aware the values within 0xd800 and 0xdfff which constitutes the surrogate pair characters are also generated meaning that some generated characters might appear invalid regarding UCS-2 and UTF-16 encoding
  • fc.fullUnicode() between 0x0000 (included) and 0x10ffff (included) but excluding surrogate pairs (between 0xd800 and 0xdfff). Its length can be greater than one as it potentially contains multiple UTF-16 characters for a single glyph

Multiple characters:

  • fc.hexaString(), fc.hexaString(maxLength: number) or fc.hexaString(minLength: number, maxLength: number) string based on characters generated by fc.hexa()
  • fc.base64String(), fc.base64String(maxLength: number) or fc.base64String(minLength: number, maxLength: number) string based on characters generated by fc.base64(). Provide valid base64 strings: length always multiple of 4 padded with '=' characters. When using minLength and maxLength make sure that they are compatible together. For instance: asking for minLength=2 and maxLength=4 is impossible for base64 strings as produced by the framework
  • fc.string(), fc.string(maxLength: number) or fc.string(minLength: number, maxLength: number) string based on characters generated by fc.char()
  • fc.asciiString(), fc.asciiString(maxLength: number) or fc.asciiString(minLength: number, maxLength: number) string based on characters generated by fc.ascii()
  • fc.unicodeString(), fc.unicodeString(maxLength: number) or fc.unicodeString(minLength: number, maxLength: number) string based on characters generated by fc.unicode()
  • fc.string16bits(), fc.string16bits(maxLength: number) or fc.string16bits(minLength: number, maxLength: number) string based on characters generated by fc.char16bits(). Be aware that the generated string might appear invalid regarding the unicode standard as it might contain incomplete pairs of surrogate
  • fc.fullUnicodeString(), fc.fullUnicodeString(maxLength: number) or fc.fullUnicodeString(minLength: number, maxLength: number) string based on characters generated by fc.fullUnicode(). Be aware that the length is considered in terms of the number of glyphs in the string and not the number of UTF-16 characters
  • fc.stringOf(charArb: Arbitrary<string>), fc.stringOf(charArb: Arbitrary<string>, maxLength: number) or fc.stringOf(charArb: Arbitrary<string>, minLength: number, maxLength: number) string based on characters generated by charArb. minLength and maxLength define bounds of the number of elements generated using charArb that can be found in the final string

Strings that mimic real strings, with words and sentences:

  • fc.json() or fc.json(maxDepth: number) json strings having keys generated using fc.string(). String values are also produced by fc.string()
  • fc.unicodeJson() or fc.unicodeJson(maxDepth: number) json strings having keys generated using fc.unicodeString(). String values are also produced by fc.unicodeString()
  • fc.lorem(), fc.lorem(maxWordsCount: number) or fc.lorem(maxWordsCount: number, sentencesMode: boolean) lorem ipsum strings. Generator can be configured by giving it a maximum number of characters by using maxWordsCount or switching the mode to sentences by setting sentencesMode to true in which case maxWordsCount is used to cap the number of sentences allowed

Combinors of arbitraries (:T)

  • fc.constant<T>(value: T): Arbitrary<T> constant arbitrary only able to produce value: T
  • fc.constantFrom<T>(...values: T[]): Arbitrary<T> randomly chooses among the values provided. It considers the first value as the default value so that in case of failure it will shrink to it. It expects a minimum of one value and throws whether it receives no value as parameters. It can easily be used on arrays with fc.constantFrom(...myArray) (or fc.constantFrom.apply(null, myArray) for older versions of TypeScript/JavaScript)
  • fc.clonedConstant<T>(value: T): Arbitrary<T> constant arbitrary only able to produce value: T. If it exists, it called its [fc.cloneMethod] at each call to generate
  • fc.oneof<T>(...arbs: Arbitrary<T>[]): Arbitrary<T> randomly chooses an arbitrary at each new generation. Should be provided with at least one arbitrary. All arbitraries are equally probable and shrink is still working for the selected arbitrary. fc.oneof is able to shrink inside the failing arbitrary but not accross arbitraries (contrary to fc.constantFrom when dealing with constant arbitraries)
  • fc.option<T>(arb: Arbitrary<T>): Arbitrary<T | null> or fc.option<T>(arb: Arbitrary<T>, freq: number): Arbitrary<T | null> arbitrary able to nullify its generated value. When provided a custom freq value it changes the frequency of null values so that they occur one time over freq tries (eg.: freq=5 means that 20% of generated values will be null and 80% would be produced through arb). By default: freq=5
  • fc.subarray<T>(originalArray: T[]): Arbitrary<T[]>, or fc.subarray<T>(originalArray: T[], minLength: number, maxLength: number): Arbitrary<T[]> subarray of originalArray. Values inside the subarray are ordered the same way they are in originalArray. By setting the parameters minLength and/or maxLength, the user can change the minimal (resp. maximal) size allowed for the generated subarray. By default: minLength=0 and maxLength=originalArray.length
  • fc.shuffledSubarray<T>(originalArray: T[]): Arbitrary<T[]>, or fc.shuffledSubarray<T>(originalArray: T[], minLength: number, maxLength: number): Arbitrary<T[]> subarray of originalArray. Values within the subarray are ordered randomly. By setting the parameters minLength and/or maxLength, the user can change the minimal (resp. maximal) size allowed for the generated subarray. By default: minLength=0 and maxLength=originalArray.length
  • fc.array<T>(arb: Arbitrary<T>): Arbitrary<T[]>, fc.array<T>(arb: Arbitrary<T>, maxLength: number): Arbitrary<T[]> or fc.array<T>(arb: Arbitrary<T>, minLength: number, maxLength: number): Arbitrary<T[]> array of random length containing values generated by arb. By setting the parameters minLength and/or maxLength, the user can change the minimal (resp. maximal) size allowed for the generated array. By default: minLength=0 and maxLength=10
  • fc.set<T>(arb: Arbitrary<T>): Arbitrary<T[]>, fc.set<T>(arb: Arbitrary<T>, maxLength: number): Arbitrary<T[]> or fc.set<T>(arb: Arbitrary<T>, minLength: number, maxLength: number): Arbitrary<T[]> set of random length containing unique values generated by arb. All the values in the set are unique given the default comparator = (a: T, b: T) => a === b which can be overriden by giving another comparator function as the last argument on previous signatures
  • fc.tuple<T1,T2,...>(arb1: Arbitrary<T1>, arb2: Arbitrary<T2>, ...): Arbitrary<[T1,T2,...]> tuple generated by aggregating the values of arbX like generate: () => [arb1.generate(), arb2.generate(), ...]. This arbitrary perfectly handle shrinks and is able to shink on all the generators
  • fc.dictionary<T>(keyArb: Arbitrary<string>, valueArb: Arbitrary<T>): Arbitrary<{[Key:string]:T}> dictionary containing keys generated using keyArb and values gneerated by valueArb
  • fc.record<T>(recordModel: {[Key:string]: Arbitrary<T>}): Arbitrary<{[Key:string]: T}> or fc.record<T>(recordModel: {[Key:string]: Arbitrary<T>}, constraints: RecordConstraints): Arbitrary<{[Key:string]: T}> record using the incoming arbitraries to generate its values. It comes very useful when dealing with settings. It takes an optional parameter of type RecordConstraints to configure some of its properties. The setting withDeletedKeys=true instructs the record generator that it can omit some keys
  • fc.infiniteStream<T>(arb: Arbitrary<T>): Arbitrary<Stream<T>> infinite Stream of values generated by arb. The Stream structure provided by fast-check implements IterableIterator<T> and comes with useful helpers to manipulate it

Objects (:any)

The framework is able to generate totally random objects in order to adapt to programs that do not requires any specific data structure. All those custom types can be parametrized using ObjectConstraints.Settings.

export module ObjectConstraints {
    export interface Settings {
        maxDepth?: number;         // maximal depth allowed for this object
        key?: Arbitrary<string>;   // arbitrary for key
        values?: Arbitrary<any>[]; // arbitrary responsible for base value
    };
};

Default for key is: fc.string().

Default for values are: fc.boolean(), fc.integer(), fc.double(), fc.string() and constants among null, undefined, Number.NaN, +0, -0, Number.EPSILON, Number.MIN_VALUE, Number.MAX_VALUE , Number.MIN_SAFE_INTEGER, Number.MAX_SAFE_INTEGER, Number.POSITIVE_INFINITY or Number.NEGATIVE_INFINITY.

  • fc.anything() or fc.anything(settings: ObjectConstraints.Settings) generate a possible values coming from Settings and all objects or arrays derived from those same settings
  • fc.object() or fc.object(settings: ObjectConstraints.Settings) generate an object
  • fc.jsonObject() or fc.jsonObject(maxDepth: number) generate an object that is eligible to be stringified and parsed back to itself (object compatible with json stringify)
  • fc.unicodeJsonObject() or fc.unicodeJsonObject(maxDepth: number) generate an object with potentially unicode characters that is eligible to be stringified and parsed back to itself (object compatible with json stringify)

Functions

  • compareBooleanFunc() generate a comparison function taking two parameters a and b and producing a boolean value. true means that a < b, false that a = b or a > b
  • compareFunc() generate a comparison function taking two parameters a and b and producing an integer value. Output is zero when a and b are considered to be equivalent. Output is strictly inferior to zero means that a should be considered strictly inferior to b (similar for strictly superior to zero)
  • func(arb: Arbitrary<TOut>) generate a function of type (...args: TArgs) => TOut outputing values generated using arb

Extended tools

  • context() generate a Context instance for each predicate run. Context can be used to log stuff within the run itself. In case of failure, the logs will be attached in the counterexample and visible in the stack trace

Model based testing

Model based testing approach extends the power of property based testing to state machines - eg.: UI, data-structures.

See section Model based testing or UI test in Tips for an in depth explanation.

Commands

The approach relies on commands. Commands can be seen as operations a user can run on the system. Those commands have:

  • pre-condition - implemented by check - confirming whether or not the command can be executed given the current context
  • execution - implemented by run - responsible to update a simplified context while updating and checking the real system

Commands can either be synchronous - fc.Command<Model, Real> - or asynchronous - fc.AsyncCommand<Model, Real> or fc.AsyncCommand<Model, Real, true>.

// Real : system under test
// Model: simplified state for the system
export interface Command<Model extends object, Real> {
  // Check if the model is in the right state to apply the command
  // WARNING: does not change the model
  check(m: Readonly<Model>): boolean;

  // Execute on r and perform the checks - Throw in case of invalid state
  // Update the model - m - accordingly
  run(m: Model, r: Real): void;

  // Name of the command
  toString(): string;
}

export interface AsyncCommand<Model extends object, Real> {
  check(m: Readonly<Model>): boolean;
  run(m: Model, r: Real): Promise<void>;
  toString(): string;
}

export interface AsyncCommand<Model extends object, Real, true> {
  check(m: Readonly<Model>): Promise<boolean>;
  run(m: Model, r: Real): Promise<void>;
  toString(): string;
}

Arbitrary

While fc.array or any other array arbitrary could be used to generate such data, it is highly recommended to rely on fc.commands to generate arrays of commands. Its shrinker would be more adapted for such cases.

WARNING: fc.commands cannot be replayed for the moment

Possible signatures:

  • fc.commands<Model, Real>(commandArbs: Arbitrary<Command<Model, Real>>[], maxCommands?: number) arrays of Command that can be ingested by fc.modelRun
  • fc.commands<Model, Real>(commandArbs: Arbitrary<Command<Model, Real>>[], settings: CommandsSettings) arrays of Command that can be ingested by fc.modelRun
  • fc.commands<Model, Real>(commandArbs: Arbitrary<AsyncCommand<Model, Real>>[], maxCommands?: number) arrays of AsyncCommand that can be ingested by fc.asyncModelRun
  • fc.commands<Model, Real>(commandArbs: Arbitrary<AsyncCommand<Model, Real>>[], settings: CommandsSettings) arrays of AsyncCommand that can be ingested by fc.asyncModelRun

Possible settings:

interface CommandsSettings {
  maxCommands?: number;       // optional, maximal number of commands to generate per run: 10 by default
  disableReplayLog?: boolean; // optional, do not show replayPath in the output: false by default
  replayPath?: string;        // optional, hint for replay purposes only: '' by default
                              // should be used in conjonction with {seed, path} of fc.assert
}

Model runner

In order to execute the commands properly a call to either fc.modelRun or fc.asyncModelRun as to be done within classical runners - ie. fc.assert or fc.check.

Simplified structure

type Model = { /* stuff */ };
type Real  = { /* stuff */ };

class CommandA extends Command { /* stuff */ };
class CommandB extends Command { /* stuff */ };
// other commands

const CommandsArbitrary = fc.commands([
  fc.constant(new CommandA()),        // no custom parameters
  fc.nat().map(s => new CommandB(s)), // with custom parameter
  // other commands
]);

fc.assert(
  fc.property(
    CommandsArbitrary,
    cmds => {
      const s = () => ({ // initial state builder
          model: /* new model */,
          real:  /* new system instance */
      });
      fc.modelRun(s, cmds);
    }
  )
);