feat: add binding validation

[jonaslagoni]

Description

This PR is a suggestion on how to handle binding versions as well as integrate them into the specification.

Binding versioning

Each binding should now be located as the following folder structure for the json-schema files:

bindings:
  amqp:
    0.2.0:
    x.x.x:
  kafka:
    ...

The binding $ids will now contain the accurate version, so instead of "$id": "http://asyncapi.com/bindings/amqp/channel.json" it will have "$id": "http://asyncapi.com/bindings/amqp/0.2.0/channel.json".

This setup will allow us to work on bindings alongside the spec itself, and continuously release new binding versions.

Integrating with spec

One of the core issues with bindings is that some binding versions might only fit together with specific versions of the spec. To allow for this finetuned control, we can now reference the binding versions exactly as they are compatible and leave out versions when necessary:

    "amqp": {
      "oneOf": [
        {
          "$ref": "http://asyncapi.com/bindings/amqp/0.2.0/channel.json"
        }
      ]
    },

Related issues

Related to asyncapi/spec#590
Related to asyncapi/parser-js#315
Related to asyncapi/bindings#113
Blocked by #289

[jonaslagoni]

Before converting all the bindings to this exact setup, I think we need to agree on the approach beforehand. Tagging code owners and known interested parties: @fmvilas @smoya @derberg @dalelane @magicmatatjahu

[fmvilas]

We considered this setup back then and ditched this idea because it allows having more than one version of a single binding, which is weird. However, since you're using oneOf and the version key is part of the binding, it may make more sense 🤔 Love it because it makes the tooling easier.

Referencing asyncapi/bindings#62 because it's related.

[smoya]

+1 to this.

A quick clarification, something we should have in mind from now on:

With this approach, the only hard requirement is that JSON Schemas for bindings should contain bindingVersion property. WIthout it, none of the versions included in oneOf could be determined.

As a side note, maybe, we could do a similar script as you @jonaslagoni did for creating a new JSON Schema file for new AsyncAPI spec version but for creating a new binding or a new version of a binding. This could go into a new issue/feature request.

[derberg]

This looks so simple 😄 I waited few day with review as I was afraid of the complexity...that is not here. Clean 💪🏼

[jonaslagoni]

With this approach, the only hard requirement is that JSON Schemas for bindings should contain bindingVersion property. WIthout it, none of the versions included in oneOf could be determined.

We default to the latest binding version if nothing is specified.

As a side note, maybe, we could do a similar script as you @jonaslagoni did for creating a new JSON Schema file for new AsyncAPI spec version but for creating a new binding or a new version of a binding. This could go into a new issue/feature request.

We should definitely do that yes 🙂

[smoya]

We default to the latest binding version if nothing is specified.

That could potentially break documents if a new major version of the binding is released. Documents that were working with binding A without specifying the version, assuming the latest was 1.0.0 but suddenly 2.0.0 is released

[jonaslagoni]

That could potentially break documents if a new major version of the binding is released. Documents that were working with binding A without specifying the version, assuming the latest was 1.0.0 but suddenly 2.0.0 is released

Yea you are right, might need to redefine that behavior when introducing this.

[dalelane]

looks like a good idea to me 👍

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[jonaslagoni]

Alright adapted all the bindings and other changes. Here are the changes I made:

1. Fixed an issue with binding schemas using an incorrect regex that we solved earlier for all the other core definitions.
2. Fixed the references of bindings to target 3.0.0 i.e. for AsyncAPI Schema and specification extensions. This might become a problem if a binding version targets multiple major versions i.e. 2.0.0 and 3.0.0 (if the referenced schemas change that is). When referencing version 3.0.0 files we can be sure that the specification extension does not contain any major changes that would resolve in the schemas no longer working for something like 3.3.0.
3. I have added a specific sunshine test that loads the 3.0.0 schema and an example AsyncAPI document with different binding protocols to test if it actually validates "correctly".
4. I have adapted the bundler to now load all the binding version schemas so they are accurately bundled together.
5. I have committed the bundled 3.0.0 schema for the sole purpose that you can see the bundling behavior in effect, even though each release automatically does this anyway.
6. I have adapted the components section to reference the new binding schemas.

This PR is ready to be reviewed and merged 🙂

[derberg]

why it targets next-major-spec and not master? I don't see it relying on any changes related to 3.0 spec changes, or did I miss something? we can release it sooner right? just another major, or?

[jonaslagoni]

why it targets next-major-spec and not master? I don't see it relying on any changes related to 3.0 spec changes, or did I miss something? we can release it sooner right? just another major, or?

I actually did not count on integrating the bindings for version 2, which is why I targeted next-major-spec.

The only reason for this is to err on caution, as we are removing a schema bindingsObject.json if we targeted version 2, as it might already be in use.

[derberg]

@jonaslagoni sorry, I mean that we do not need to wait for 3.0 as we can just release spec-json-schemas 4.0.0

[jonaslagoni]

@jonaslagoni sorry, I mean that we do not need to wait for 3.0 as we can just release spec-json-schemas 4.0.0

I know er can do it, but as i said, I did not plan to backtrack this to AsyncAPI 2.x, that said its definitely possible, using the same setup as here.

Are you implying that is what we should do? Or? 😃

[derberg]

Are you implying that is what we should do? Or? 😃

yeah, kinda 😄

• it is a great feature, and it would be nice if we ship it sooner to people, get feedback, adjust
• it is a breaking change. I'm in a club of people that prefer not to introduce too many changes at once, so it is easier to debug errors

but I'm definitely not planning to die on that hill if others don't share my opinion 😃

[jonaslagoni]

Alright so here are the options:

1. We keep the PR as it is now, and only introduce this change for 3.0.0 (for now, we can decide to do it later if we want to). It will mean we delay when users get access to the binding schemas and it prevents a new major version.
2. We re-create this PR to target the master branch and release it as a new major version.

@fmvilas @dalelane @derberg those are your possibilities in short 😅 Let me know what you would like to do, I have no strong feelings either other then option 1 is the easiest from my point of view 😆

[smoya]

I also see more value in releasing this sooner rather than waiting for 3.0. +1 to @derberg 's reasoning. It provides a lot of value.

[fmvilas]

Yeah, don't have a strong opinion either but Lukasz's point makes sense.

[dalelane]

sooner sounds good to me too 👍

[jonaslagoni]

I have updated the PR with the latest changes on next and added the new pulsar bindings, should be ready for final review ✌️

[dalelane]

👍

[jonaslagoni]

@fmvilas @derberg @smoya @char0n do you want to review it or can we mark it as ready to merge?

[derberg]

just to clarify the previous discussion. So the final decision is to put it in 3.0 and not sooner?

[jonaslagoni]

So the final decision is to put it in 3.0 and not sooner?

Tbh... I just don't want to spend the time splitting up the PR into one for next and wait for it to be released, and then add support for it in next-major-spec 😅 Especially considering the time this PR took.

[jonaslagoni]

We can merge this and backtrack it to next if anybody wants to drive it 👍

[jonaslagoni]

And we still dont know what to do about the specification extension, as it's currently explicitly linked to spec 3.0 in each binding version. So it's not as easy as saying "let's do it", problems will arise when we try to backport it.

Of course, it's something we still have to figure out for the next version anyway, but at least it's not right now that will further delay it.

But in the end it's up to you 🤷

[derberg]

I do not have super hard feelings about it.
I know that there are people using bindings as they want (not following binding schemas entirely) on production, so releasing these before spec 3.0 will cause them some headaches.

So yeah, let's just merged it for 3.0 only for now

[jonaslagoni]

@derberg I will let you folks decide when adequate enough time has passed for reviewers to respond and for merge it.

[fmvilas]

LGTM 👍

[smoya]

🚀🌔 LGTM!

[derberg]

@char0n we are just missing your approval

[jonaslagoni]

So yea

[jonaslagoni]

/rtm

[asyncapi-bot]

🎉 This PR is included in version 5.0.0-next-major-spec.1 🎉

The release is available on:

• npm package (@next-major-spec dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[char0n]

Great work here! 🎉

Regarding AsyncAPI 2.x.y - I'm assuming that every binding version that is available is allowed to be used for any AsyncAPI 2.x.y version right? Or do we have somewhere the matching map of which binding versions is allowed for what AsyncAPI 2.x.y spec?

[jonaslagoni]

@char0n yea there is still no consensus or progress in the discussion regarding that, as the spec 3.0 changes need to be done first before figuring out what to do about the compatibility between versions of bindings: asyncapi/bindings#182

[asyncapi-bot]

🎉 This PR is included in version 6.0.0-next-major-spec.1 🎉

The release is available on:

• npm package (@next-major-spec dist-tag)
• GitHub release

Your semantic-release bot 📦🚀