flow-degen README

flow-degen

This is a deserialization generator for JavaScript objects that are under Flow. Some deserializer/validator projects use $ObjMap and some clever casting to achieve a type safe means of deserialization/validation. flow-degen aims to leverage Flow itself in ensuring type safety by generating all of that ugly deserialization/validation code you would have written by hand.

Pros:

1. flow-degen introduces no runtime dependencies to consumers, other than itself.
2. flow-degen emits generators that use plain Flow type checking. There is no any casting internally, and there are no magic types.

Cons:

1. flow-degen does not currently provide a list of deserializer errors, but instead bails on the first error.
2. There is potentially a memory/storage footprint concern for non-trivial sizes and amounts of deserializers. A minifier may significantly mitigate this con. Using degenRefiner to reference other refiners can also reduce the footprint by calling other refiners instead of duplicating refinement logic in multiple places.

installation

yarn add -E -D flow-degen

config structure

You’ll need a config file to work as inputs to flow-degen. It has the structure below. Any paths or file names that start with ./ are intended to show a relative directory.

{
  "baseDir": "",
  "generatedPreamble": "",
  "generators": [
    {
      "exports": {
        "fooGenerator": "fooRefiner"
      },
      "inputFile": "./dir/generator-input-file.js",
      "outputFile": "deserializer-output-file.js"
    }
  ],
  "importLocations": {
    "importName": "./dir/runtime.js"
  },
  "typeLocations": {
    "TypeName": "./dir/type.js"
  }
}

baseDir

baseDir is the directory that the generated file will be assumed to be living out of relative to the imports.

generatedPreamble

generatedPreamble is the code or text you want to appear at the top of the file. You can use this to insert a copyright comment block, include linter rules (such as disabling ESLint’s no-used-expressions rule which can be an issue for disjoint unions).

generators

This is a list of generators and how they produce refiners. Generators here have the following structure:

{
  "exports": {
    "fooGenerator": "fooRefiner",
    "barGenerator": "barRefiner",
    "bazGenerator": "bazRefiner"
  },
  "inputFile": "foo-generator.js",
  "outputFile": "foo-refiner.js"
}

exports

exports is a mapping of identifiers exported from the generator file that flow-degen can find, and it maps to identifiers it will generate as refiners for that associated generator. In the sample configuration, fooGenerator is found in foo-generator.js, and it will emit a fooRefiner to foo-refiner.js that you can then import or require to use.

exports is also implicitly added to importLocations such that your refiners can refer to each other, and even achieve recursive calls if your structure requires recursion.

importLocations

This is a mapping of import names (which must be valid JavaScript identifiers) to files. The identifiers must map to export entities inside of your module. These will be hoisted to the top of the generated deserializer files if they are used. Including entries in here does not mean it will be used in your files, it is simply a lookup for flow-degen to use.

typeLocations

Just like importLocations, typeLocations is a reference for export type identifiers so the generated deserializer can find them if any of the combined deserializers use it.

usage

internal deserializer generators

import from flow-degen to get deserialization generators.

degenField

degenField is meant to be used in conjunction with degenObject.

degenFilePath

This is just an alias for degenString currently, but could one day encompass a Flow opaque type that, while represented by a string, is ensured to be a valid file path.

degenList

Requires a deserializer to be used for the element type, which is provided as its only argument. This will produce an Array<T>.

Suppose we have a foos-generator.js:

import { degenList, degenNumber} from 'flow-degen'

const numberType = { name: 'number', typeParams: [] }
export const foosGenerator = () => degenList(numberType, degenNumber())

Upon importing the emitted file, you can now refine into an Array of number:

import { deFoos } from './foos-refiner.js'

deFoos([1, 2, 3]) // Produces [1, 2, 3].
deFoos('farsnaggle') // Produces Error object.

declare var someInput: mixed

const eitherResult = deFoos(someInput)

if(eitherResult instanceof Error) {
  // Here the result did not refine correctly.
  console.error('How did this happen?', eitherResult)
} else {
  // Now you have an Array of number.
  console.log(eitherResult.map(x => x + 1))
}

degenMapping

A “mapping” is of the type {[A]: B} although usually it will be {[string]: mixed}. It takes the key meta type, the value meta type, a key deserializer, and a value deserializer for A and B respectively.

degenMaybe

The degenMaybe generator is for creating refiners for maybe types (e.g. type Foo = ?string). The maybe type will still require additional refinement after passing through the refiner. For example, given the type:

export type Foo = {
  bar: ?string,
}

And generator:

import { degenObject, degenField, degenMaybe, degenString } from 'flow-degen'

const fooType = { name: 'Foo' }
const stringType = { name: 'string' }
export const fooGenerator = () => degenObject(fooType, [
  degenField('bar', degenMaybe(stringType, degenString())),
])

The refiner would be used like so:

import { deFoo } from './foo-refiner.js'

declare var someInput: mixed

const eitherResult = deFoo(someInput)

if(eitherResult instanceof Error) {
  // Here the result did not refine correctly.
  console.error('How did this happen?', eitherResult)
} else {
  // We have a foo, but bar may not have been present
  if (eitherResult.bar != null) {
    console.log(eitherResult.bar + ' was refined')
  } else {
    console.log('result had a null bar')
  }
}

degenNumber

The degenNumber deserializer simply deserializes a value as a number.

degenObject

An “Object” can be thought of as a collection of “fields”. See degenField as these go together except for empty objects. degenObject takes the type of the object and a list of required fields that degenField can emit, and a second list of degenField results that represent the optional fields.

Assume the object Cat.

export type Cat = {
  // Cats always have demands.
  demands: number,
  // Cats can have no love sometimes.
  love?: number,
}

const catType = { name: 'Cat' }
const catGenerator = () => degenObject(catType, [
  degenField('demands', degenNumber()),
], [
  degenField('love', degenNumber()),
])

It is well known that cats always have demands but only sometimes have love. It is fallacious to assume love will always be present.

import { catRefiner } from './cat-refiner.js'

// It's pretty easy to get an unsanitized cat from anywhere, really.
handleUnsanitizedCat((input) => {
  const catOrError: string | Error = catRefiner(input)
  if(catOrError instanceof Error) {
    goGetADog()
  } else {
    // We have a cat! But we can't expect love.
    // Flow will also settle for a null check for love.
    if(catOrError.hasOwnProperty('love')) {
      console.log(`My cat loves me ${catOrError.love} love units!`)
    } else {
      console.log('My cat does not have any love for me at all...')
    }
  }
})

degenString

The degenString deserializer simply deserializes a value as a string.

Say we have a name-generator.js:

import { degenString } from 'flow-degen'
export const nameGenerator = () => degenString()

And this is configured to produce a name-refiner.js, this is how it would be used:

import { nameRefiner } from './name-refiner.js'

// This could be an HTTP POST handler on a server, or a form handler on a UI
handleUnsanitizedInput((input) => {
  const nameOrError: string | Error = nameRefiner(input)
  if(nameOrError instanceof Error) {
    console.error(nameOrError)
  } else {
    // Here can we use the name.
    storeName(nameOrError)
  }
})

degenSentinelValue

This deserializer is to be used in conjunction with degenSum to produce deserializers for a sum type. This represents one member of the union. It needs a key, which is a string value for the sentinel value, and the object deserializer itself, which will likely be degenObject.

degenSum

The degenSum deserializer handles sum type objects. It takes the type of the union, the sentinel field name, the sentinel field type, and a list of sentinel object deserializers (which can just come from degenObject) from degenSentinelValue.

degenValue

The degenValue deserializer takes a type (as a string) and a value (which could be anything). It checks for the literal equivalence of that value. This can be helpful when using Flow’s sentinel properties for sum types of objects.

degenRefiner

The degenRefiner refiner simply imports a symbol for use. This allows recursion to work when the refined data structure is recursive. Also it allows for reuse of other refiners of any kind. This reduces the size of generated refiners significantly. Otherwise the refiners are inlined.

Suppose we have a foo-generator.js whose generator builds the deFoo refiner:

import { degenObject, degenField, degenString } from 'flow-degen'

const fooType = { name: 'Foo' }
export const fooGenerator = () => degenObject(fooType, [
  degenField('first', degenString()),
  degenField('last', degenString()),
])

And we have a bar-generator.js:

import {
  degenObject,
  degenField,
  degenString,
  degenRefiner,
} from 'flow-degen'

// This is the same fooType in foo-generator.js, and could be imported.
const fooType = { name: 'Foo' }
const barType = { name: 'Bar' }
export const fooGenerator = () => degenObject(barType, [
  degenField('foo', degenRefiner(fooType, 'deFoo')),
])

Here the generator will simply invoke deFoo to refine the foo field. Any import, type, and hoist information will be made available in this refiner.

Note that this symbol must be one that is managed by flow-degen in your configuration file, or your configuration file must specify in the imports how to find this symbol.

creating meta types

Objects of type MetaType are passed into many generator functions and contain information flow-degen uses to build imports and type signatures in the generated code. The MetaType type can be found in src/generator.js but at a minimum contains the type name:

type Foo = {
  bar: string,
}

const fooType = { name: 'Foo' }

In the case of generic types, the optional typeParams field in MetaType can be used to list the meta types to be specified in the type signature:

type Foo<K: string, V: string> = {
  [K]: V,
}

const stringType = { name: 'string' }
const fooType = { name: 'Foo', typeParams: [stringType, stringType]}

Some types (for example flow utility types like $PropertyType) take literal strings or numbers instead of a type. The MetaType has an optional literal boolean to indicate these usages:

type Foo = {
  bar: {
    baz: string,
  },
}

const fooType = { name: 'Foo' }
const bazPropertyType = { name: "'baz'", literal: true }
const barType = { name: '$PropertyType', typeParams: [fooType, bazPropertyType] }

Note that string literals (and other literals with delimiters) need to include the delimiters in the name (e.g. “‘baz’” instead of “baz” or ‘baz’).

building custom deserializer generators

All deserializers must satisfy the following contract:

• They must be a function.
• The function returns a DeserializerGenerator<CustomType: string, CustomImport: string>, which is a tuple of a function that returns a string (the code) and a CodeGenDep<CustomType: string, CustomImport: string>. The exacts of these types can be found in ./src/generator.js.
• The code returned by the function must accept a mixed as a parameter. This is your input provided from your mystery variable. It is assumed to be “deserialized” already in the sense that it is not a string of JSON but perhaps the result of JSON.parse.
• If any imports are used, they must be enumerated in the imports list of the CodeGenDep. Any imports used by the generated function will also need to be part of the CustomImport type parameter of the generator as well as included in importLocations in your flow-degen configuration file (adding an import to importLocations is not necessary if the import is an export from a refiner defined in your flow-degen config).
• If any type imports are used, they must be enumerated in the types list of the CodeGenDep. Any types used by the generated function will also need to be part of the CustomType type parameter of the generator as well as included in typeLocations in your flow-degen configuration file.
• Consider that your generated code could likely be embedded deep within a function chain. If you need some “root” access to the module to declare things such as throw-away types, use the hoists list to place code.
• If your generator delegates to other generators (such as degenList delegating to a deserializer for the elements), you must honor the results of its CodeGenDep when you call the generator. This could mean merging the CodeGenDep with your own. The mergeDeps function in ./src/generator.js does this for you. It is found by flow-degen consumers as a top-level export (=import { mergeDeps } from ‘flow-degen’=).
• Try testing your refiner with an opaque type. This seems to be a good way to ensure Flow cannot run into issues with type inferencing. We suspect this is a good test because opaque types can never be inferred, and therefore will always need explicit types at the call site of a refiner.

Let’s create an custom generator example where we have an uppercase string.

import {
  degenString,
  mergeDeps,
  type DeserializerGenerator,
} from 'flow-degen'
import {
  type UppercaseString,
  uppercase,
} from './my-string-utils.js'

type UppercaseGeneratorType =
  | 'UppercaseString'

type UppercaseGeneratorImport =
  | 'uppercase'

export const degenUppercaseString = (
): DeserializerGenerator<UppercaseGeneratorType, UppercaseGeneratorImport> => {
  const [ stringGenerator, stringDeps ] = degenString()
  return [
    () => {
      return `(x: mixed): UppercaseString => {
         return uppercase(${stringGenerator()})
      }`
    },
    mergeDeps(
      stringDeps,
      {
        hoists: [],
        imports: [ 'uppercase' ],
        types: [ { name: 'UppercaseString' } ],
      },
    ),
  ]
}

Custom generators are no different from the built-in generators.

import {
  degenUppercaseString,
} from './custom-degens.js'

export const generateUppercaseStringRefiner = () => degenUppercaseString()

The built-in generators in src/generators.js can be used as more complex examples for building your own generators.

command line

Once installed, you can use the flow-degen script to generate your deserializers:

yarn flow-degen degen-config.json

consuming generated deserializers

The output files you indicate will export refiner functions defined in the exports config for the generator. The refiner functions take the form of (mixed) => T | Error.

import fs from 'fs'
import { fooDeserializer } from './foo.deserializer.js'

const unvalidatedFoo = JSON.parse(fs.readFileSync('foo.json', 'utf8'))
const fooOrError = fooDeserializer(unvalidatedFoo)

// Refine the result.
if(fooOrError instanceof Error) {
  console.error('Error deserializing foo:', fooOrError)
} else {
  doStuffWithFoo(fooOrError)
}

editing generated deserializers

Do not edit these files directly except for debugging purposes. The files will be overwritten on subsequent runs of the generator. Also, the code written there is not designed with human maintainability as its chief concern.

source control

Tooling could be built to make the generation process opaque to a consumer, but at the time that method is not known to flow-degen maintainers. It is fine and even recommended to check your generated deserializers into source control.

known issues

no-unused-expressions

When using degenSum, ESLint has a no-unused-expressions rule that fails during a cast in the default case. This expression doesn’t do anything in the runtime, but Flow needs it to tie the “everything else” match to the default case. This makes Flow flag an error when a member of the union isn’t enumerated in the switch. To work around this issue, you can add // eslint-disable no-unused-expressions to your configuration’s generatedPreamble.

bragging rights

The config object above is generated from config-generator.js which in turn must deserialize itself in order to build the generator. mind-blown.gif