Proposal: a tool for generating machine-modules (schema, creator, helpers, exhaustive and sound typing)

[MrNovado]

So I've been playing around with TS and XState for a while now and... I have to say, the value TS brings to the table here for the end-user is very limited.

Like, it's cool to be able to somewhat competently build schema' artifacts, and see actions' descriptors and whatnot, but when it comes to implementing a schema into a machine -- you feel almost completely disconnected. Working with big-enough schemas feels unsafe and forces you to grock and glance over the schema over and over until implementation is 'done'.

Questions which are very hard to answer without looking at the schema of your machine:

• which actions/guards/activities do you need to implement? (no suggestions)
• did we implement everything that is needed to create a working instance? (runtime)
• which state-combinations are you able to match against? (no suggestions)
• are there any typos in transition-targets? (runtime)
• which events are we able to send at the state we are in now? (no suggestions)
• what is the context of execution (all possible variants of (state, event)) for the selected action/condition? (currently assumes all actions could be executed at any state-node)
• etc.

Proposal

In order to answer at least some of those questions and enhance the overall dev-experience, it seems one have to up the abstraction level and generate machine-schema and all the helpers (something like folks from apollo[graphql-impl] did) first, before consuming.

https://github.com/MrNovado/xstate-gentype-rfc

/* Machine' description to compile into a module */

function Machine(props) {
    return (
        <Actor
            context={
                {
                    /* context obj -- types? */
                }
            }
            initial="init"
            states={[
                <StateNode
                    name="init"
                    on={[
                        <StateEvent name="START">
                            <Transition
                                target="pending"
                                cond="start-cond"
                                actions="start-action"
                            />
                        </StateEvent>,
                    ]}
                />,
                <StateNode name="pending"></StateNode>,
                <StateNode name="resolved"></StateNode>,
                <StateNode name="rejected500"></StateNode>,
                <StateNode name="rejectedUnknown"></StateNode>,
            ]}
            on={[
                <StateEvent name="STOP">
                    <Transition target="init" />
                </StateEvent>,
            ]}
        />
    );
}

/* Compiled machine module */

type Event = ... | ...;
type Context = ...;

type Schema = ...;
const machineSchema: Schema = ...;

type Implementation = { guards: ..., actions: ..., };

export default crm
     = (impl: Implementation)
     => createMachine<Context, Schema, Event>(machineSchema, impl);

/* This way you won't need to worry about missing impl artifacts. */

/* Generating an exhaustive strict state-matching interface: */

states: {
   a: { ab: {}, ac: {}, ad: {} },
   b: { bc: {} },
   d: { dd: {} }
}
 
type ExhaustivePatterns = [a] | [a, ab] | [a, ac] | [a, ad] | [b] | [b, bc] | ...;
export default matches 
   = (pattern: ExhaustivePatterns)
   => xstate.matches(pattern.join("."));

---

What do you think?

In my proposal, I used a jsx-like syntax to show how the machine-description might look like, but the syntax could be exactly the same as the one xstate already uses (or anything else really). Moreover, machine-description could be generated from a visual constructor of sorts, like folks from yakindu have shown.

[Andarist]

This is already, somewhat, on our roadmap (exhaustive type generation). It will take some time before we deliver this though as we are focused on some other work right now. Would you maybe be interested in helping us out with this?

[MrNovado]

@Andarist, sure, do you have a particular vision on how something like this should be done? Or any proof of concept would suffice for now?

[Andarist]

I have not yet thought much about technical details on how this could be implemented. I think it would be the best to reuse existing tooling as much as possible, so I would start with exploring ESLint + TypeScript programmatic APIs to see what's possible. Most likely we'd need to actually write generated types onto the file system to make TS actually type check against that generated schema.

Any proof of concept would be good - you can also reach out to me on Twitter or some other channel so we could discuss this further if you want.

[MrNovado]

@Andarist, progress report. Finally been able to play with xstate a little more on this one.

I've made a repo here for now:
https://github.com/MrNovado/xstate-gentype-proof

I entertained the idea of generating a ts-machine-module from a simple machine config in JSON, using xstate-machines 😋.

The example here is just a simple tree-traversal machine
https://github.com/MrNovado/xstate-gentype-proof/blob/v1.0.0-simple/gentype/simple.schema.json
https://github.com/MrNovado/xstate-gentype-proof/blob/v1.0.0-simple/gentype/simple.gentype.js

Here's the resulting module:
https://github.com/MrNovado/xstate-gentype-proof/blob/v1.0.0-simple/src/simple.machine.gen.ts

Currently, it generates:

• exhaustive path type, listing all possible state-paths
• exhaustive event type, listing all possible events
• createMachine helper with options (only simple entry actions are listed currently)
• state2EventMap listing related events for all state-paths
• hasEvent and matches enhanced by path and event typings

And here's how to consume a generated module (with react):
https://github.com/MrNovado/xstate-gentype-proof/blob/v1.0.0-simple/src/index.tsx

---

There's still probably a lot more stuff possible to do with just a simple JSON-schema, but complex stuff requires a source to be typed as well 🤔 Thoughts?

[Andarist]

@MrNovado I just want to let you know that I've seen this! Thanks for doing this ❤️ . I'm still thinking about various things regarding the whole topic so I can make sure that I give a thoughtful comment on this.

[Andarist]

@MrNovado this is a really cool experiment! It IMHO showcases nicely that this can be done.

The one problem I have with the used approach is that it requires defining schema separately and I believe that the experience of authoring machines should not change (not by much at least). Otherwise, we force people to learn more APIs, we need to document more, etc. Schemas are useful, but for the ideal developer experience, this should be as seamless as we can make it. At least that should be the default - more capabilities can, of course, be added.

After thinking about it some more - I think we could start from writing a dedicated VScode plugin that would generate those typings based on existing configs passed to createMachine. I believe we could leverage its TS language service to do some hard work for us, we'd "only" need to traverse the obtained config and output types. I don't know all the technical details how this could be done just yet, but I've found a sign that it should be possible to hook into its TS language service here: microsoft/vscode@575e163 . This needs further exploration though, to figure out this API as it's not clearly contained in that commit (there is no example, nor tests showcasing the usage). Later we could try to extract abstract logic from this work to enable integrations with other systems like ESLint etc. A VScode extension seems like a good MVP though.

The tricky part (which I have not figured out yet) is how to trick TS compiler to use those generated types instead of builtin ones. We definitely need to alter the source code, otherwise, TS won't be able to resolve and type-check this. But how to do it? What's the user input? What's the generated output?

One, very rough, possibility which comes to my mind is this:

import { createMachine } from 'xstate'
+import { LightMachineType } from './light-machine.xstate-types.ts'

const lightMachine = createMachine({
  initial: 'green',
  /* ... */
-})
+}) as any as LightMachineType

There are also some unknowns here. For example - how actual work in IDE will look like when updating the input machine? Should we try to regenerate types when writing? Should we only regenerate on save? If latter - won't it be confusing to see errors reported based on the outdated types before user hits save?

[MrNovado]

@Andarist

The one problem I have with the used approach is that it requires defining schema separately

Technically, xstate already asks its users to define their schemas separately. Not necessarily into a separate module right away, but once a schema is completely declarative, decoupled from its implementations (which is the way it should be), you can move it around no problem. And let us not forget about scxml-schemas which, one could argue, will always be sourced as their own completely separate files (at least when it comes to importing them in a static-ish way)? With this approach, it is very easy to see how anyone can generate type-enhanced schemas from any source what-so-ever.

Otherwise, we force people to learn more APIs, we need to document more, etc.

I personally think it is very much inevitable. Type-enhanced schemas, by definition, provide new APIs, simply because they give you new types to play with. The more problems you face, the more tools you would want, the more APIs you would get.

Trying it out and seeing how actually easy it is to create a generator around a static schema I think it is reasonable to envision several competing generators to happen at some point anyway. So, you should embrace it and not worry much about it. Good API's will happen through pain and suffering as per usual, xstate should simply choose its own -- some -- way of doing this just as an example to show to and to inspire others who wish for more?

I think we could start from writing a dedicated VScode plugin

Plugin or a library -- to me it feels like a delivery method more than anything. IDE integration seems like a neat feature for those who have the product, but a library-package is a lot more universal in terms of usability (or accessibility even). VScode is a nice tool indeed, but I fail to see the necessity of funneling your userbase into this specific product in such a way. xState does not require you to have any prerequisites in terms of tooling, why this particular case should be different?

What's the generated output?

As for the generated output, the usual case with generators, at least from my experience, is to output stuff in a separate module (with a .gen. or any other tag) as per request. It is a fairly easy and time-tested approach: apollo did this, protobuf did, reason-gentypes, and so on... With TS it's also easy and there's no need in tricking a compiler into anything. It just works 😄.

Hot-schema-regeneration is a neat QoL, but is not required at first and still can be introduced later on. There's no need to complicate this so much. Let's just make a library-package tool that generates an enhanced schema in some way, with some API and see where it would lead?

Cheers,