Simple auto-generated parsers

[jonaslagoni]

The future I want to speed up is the following:

No matter which programming language or AsyncAPI version you use or want to support, these simple parsers will always be there to let you easily interact with AsyncAPI documents (instead of just iterating the raw JSON).

What are simple parsers?

• 100% auto-generated models with Modelina and the AsyncAPI JSON Schema documents.
• Limited functionality from the start (compared to our JS parser and its nice-to-have features)
  • Only has the ability to convert to and from JSON.
  • No reference resolvent (you can check if it's a reference or not and if it is you have to do the resolvent yourself through a third-party library)
  • No parser-api (does not handle breaking changes for you)
  • Limited schema parsing (Only support what is auto-generated and what the spec-json-schema documents define, otherwise it will be a generic any type)
  • No validation (Use a third-party library before parsing the document)
  • No spectral (Same as validation)
  • No traits will be applied

Why would we want this?

1. They come with a 1% running maintenance cost (shot in the dark 😉), meaning we can slowly progress by adding more "complete" parsers such as our JS parser.
2. Because the AsyncAPI JSON Schema documents are part of our specification contribution pipeline, as soon as the specification comes out, the parser is updated (even before that because of pre-releases).
3. Give a broader group of developers faster access to parsers to create tooling in other languages.
4. If Modelina can't generate AsyncAPI models for a parser it cant generate payload models 🤷 So this is a way to self-validate, assess and improve Modelina.
5. Long-term simplify spec contribution workflow, to only needing to change the spec-json-schema documents.

How to achieve it

We start with two parsers Rust and TypeScript/JavaScript.

These are the tasks for getting started:

1. Setup basic project structure
   • For TypeScript, we need to set it up to transpile to both CJS, ESM, and types, so we can use one of the AsyncAPI project setups for this.
   • Setup with conventional release to make the auto-generation phase easier.
2. Set up the Modelina generator script to generate the models to our desired output.
   • We need to validate that Modelina generates the appropriate models or whether we need to tweak it, both in terms of options and presets.
3. Setup auto-checking spec-json-schema for new releases, and automatically create the appropriate changes with GitHub CI through conventional release. This is somewhat similar to what I did here: https://eventstack.tech/posts/automated-utopia-typescript

And that's actually it for an initial setup.

Future

Here are some of the future ideas:

• Create simple parsers for Python, C#, Go, and Kotlin/Java, see this list of dependent issues in Modelina that need to be solved:

• We could look into how we can incorporate the parser-api to auto-generate parsers and mapping 👀 Regardless these initial models would be needed.

• We could look into how we can replicate this for code-first tools as well 🧐 As the only change would be what Modelina generates, which is easy through presets.

• Once set up, we could always start the conversation about how many nice-to-have features the parser should have, as long as it does not go against the auto-generated part and keeps maintenance at a minimum.

Discussion

Although they will have a very low running cost, they do take a bit of effort to set up, so that's why I wanted to ask if these would be useful and worth the upfront investment 🤔 Or whether you see it as a waste because the full features are needed to be usable (those we have without JS parser).

Happy to answer questions as well, don't hold back 😄

[derberg]

There are some serious limits there 😄

I see this proposal as a super useful solution for tooling providers to provide parsers in different languages, even our parser would benefit. My only question is: is it simple auto-generated parser that we need and will get or rather auto-generated AsyncAPI models that anyone can import into their projects? parsing we can just have as examples in tests for the generated models. It would still be beneficial if we care about regular updates of those, improvement of modelina, and official releases (just making sure you do not think that in this case anyone can take and use modelina for it on their own)

[jonaslagoni]

There are some serious limits there 😄

Yep, is there any one thing in particular you feel would make you not want to use it? 🤔

is it simple auto-generated parser that we need and will get or rather auto-generated AsyncAPI models that anyone can import into their projects

Well, the "problem" is that each implementation needs very specific models, it's not enough with the plain old classes alone. For example, our JS parser uses the new parser API, and the underlying models for each major version map the API to the underlying data structure, so they wouldn't really want to use such exposed models.

I don't think they are mutually exclusive tbh, we can have as many variants as we need, i.e. provide a pure AsyncAPI model that is nothing but a plain old class, a simple parser that can properly serialize and deserialize AsyncAPI documents, we can even think about adding fluent interfaces for building AsyncAPI documents with code first.

Or maybe we can bundle all those features into one set of models. It's kinda hard to know for sure what works and does not.

This is why I just proposed the simplest model approach to start with and get the auto-generated part nailed down first before diving into what the underlying models should contain.

It would still be beneficial if we care about regular updates of those, improvement of modelina, and official releases (just making sure you do not think that in this case anyone can take and use modelina for it on their own)

I don't understand you 100% here, do you mind clarifying what you mean? 🙂