DSPV - Data-serialisation Processing and Validation

DSPV stands for “Data-serialisation, Processing, and Validation”. DSPV is a module for working with data-serialisation formats such as JSON in JavaScript, and making it easy to validate and process data straight into objects. DSPV uses TypeScript to ensure that the core is built faithfully, and it is recommended that DSPV be used with TypeScript for best results.

DSPV is currently focused on JSON, but the addition of other formats such as YAML is on the radar for future.

Why use this module?

This module has various benefits, including:

• ability to smartly parse JSON directly into values based on type-like specifications
• ability to override how basic types (e.g., booleans, numbers) are parsed
• support for arbitrarily complex schemas that specify how to read JSON into non-standard data structures

Usage

Schemas map specification names to specifiers that determine how to process data. The default schema provides support for a host of standard types of data (see Builtin Specifiers).

You can add new specifications to a schema by using the addSchema method, as in the following example:

class Person {
    age: number;
    address: string;

    constructor(age: number, address: string) {
        this.age = age;
        this.address = address;
    }
}

const schemas = Schemas.emptySchemas();

schemas.addSpec(Person, {
    description: "A person with an age and an address",
    load: JsonSchema.objectSchema({
        age: Number,
        address: String
    })
});

This adds a specification that can be referenced as Person, which tries to read (load) an object with an age field (a number), and an address field (a string). We can test this out by creating a parser and running as follows:

const parser = new JsonParser(schemas);

console.log(parser.parseAsOrThrow('{"age": 20, "address": "somewhere on Earth"}'));
// Person { age: 20, address: "somewhere on Earth" }

If we try to give an incorrect age (e.g., a string), or address (e.g., a number,) or gave different fields, we’d be presented with a useful error message that incorporates the description:

function tryItOut(text: string) {
    // 'parseAs' returns an Either value, so we can grab the error message here
    console.log(parser.parseAs(text, Person).either(err => err.message, _ => ""));
}

// wrong type for age
tryItOut('{"age": "not a number", "address": "somewhere on Earth"}');
// When trying to read a value for specification: A person with an age and an address
// I saw: {"age":"not a number","address":"somewhere on Earth"}
// In key: "age"
// When trying to read a value for specification: number
// I saw: "not a number"
// But this is a string

// wrong type for address
tryItOut('{"age": 20, "address": 7}');
// When trying to read a value for specification: A person with an age and an address
// I saw: {"age":20,"address":7}
// In key: "address"
// When trying to read a value for specification: string
// I saw: 7
// But this is a number

// missing a key
tryItOut('{"address": "somewhere on Earth"}');
// When trying to read a value for specification: A person with an age and an address
// I saw: {"address":"somewhere on Earth"}
// But the following keys are required and were not specified: "age"

// extra key
tryItOut('{"age": 20, "address": "somewhere on Earth", "other key": 1}');
// When trying to read a value for specification: A person with an age and an address
// I saw: {"age":20,"address":"somewhere on Earth","other key":1}
// But I saw the following keys which are not accepted by the specification: "other key"

Specifiers with arguments

Specifiers can take arguments, a common example of this is Array which takes a single spec as an argument which determines what kinds of things can be in the array. For example, if you wanted to parse an array of booleans, you could do:

new JsonParser().parseAsOrThrow("[true, false]", [Array, Boolean]);
// [ true, false ]

You can define your own specifiers that takes arguments by defining description and load to be functions. Note that if either of these functions takes optional arguments, you should specify the maxArgs option, which helps provide nicer messages when an incorrect call is made.

const myCustomArray = Symbol('myCustomArray');

const parser = new JsonParser(Schemas.emptySchema().addSpec(myCustomArray, {
    maxArgs: 2,
    description: (getDesc) => (t1, t2 = Boolean) => `Array of ${getDesc(t1)} and ${getDesc(t2)}`,
    load: (t1, t2 = Boolean) => JsonSchema.arraySchema([anyOf, t1, t2], a => a)
}));

parser.parseAsOrThrow("[true, 1, false]", [myCustomArray, Number]);
// [ true, 1, false ]

parser.parseAsOrThrow('["bad"]', [myCustomArray, Number]);
// When trying to read a value for specification: Array of Number and Boolean
// I saw: ["bad"]
// When trying to read a value for specification: [Symbol(anyOf), number, boolean]
// I saw: "bad"
// But this is a string

parser.parseAsOrThrow('["ok", 7]', [myCustomArray, Number, String]);
// [ "ok", 7 ]

parser.parseAsOrThrow("[true, 1, false]", [myCustomArray, Number, String]);
// When trying to read a value for specification: Array of Number and String
// I saw: [true,1,false]
// When trying to read a value for specification: [Symbol(anyOf), number, string]
// I saw: true
// But this is a boolean

Builtin Specifiers

The following specifiers are supported out-of-the-box:

• anyOf (...TS) matches if any of the argument specifiers matches. The result is from the first specifier that matches
• AnyTy matches any type of value and returns it literally
• Array (T?) if T is not provided, matches an array of any type of value. If T is provided as an argument, it matches an array of the given type instead
• Boolean matches a boolean
• Map (String, T2?)? matches an object whose values match T2, and returns this as a Map from String keys to values produced by T2. If T2 is not provided, it defaults to AnyTy
• null matches null
• Number matches a number
• Object (T?) matches an object whose values match T (or AnyTy if T is not provided)
• Set (T?) the same as Array(T) but converts the value to a Set
• String matches a string
• tuple (...TS) matches an array whose length must be the same as TS and whose every ith element matches the ith element of TS

Development

Coverage

To generate code test coverage, make sure you have access to the genhtml tool (e.g., via the lcov package on AUR), then run make coverage in the top-level of the project.

Testing

Run make test in the top-level of the project to run the tests. If you need to see results for tests that passed, run make test_verbose or deno test instead.