Pitch: Better reference handling for AsyncAPI #2
jonaslagoni
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
For too long we have run into problems with references, could not use private references, could not resolve non-JSON references (Protobuf, etc), did not correctly work well when different standards were involved, and the list goes on. This pitch proposes a way to solve these things, while still providing the flexibility and extensibility to support any of your use cases.
Problem
These are the problems that this pitch aims to solve.
TL;DR:
We have always used third-party tools to handle references for AsyncAPI, but nothing so far has ever solved all the needs that we have, on top of this the reference handling that it did support, does not allow the extensibility needed for AsyncAPI.
Problem 1: Context Dependant
Depending on where in the AsyncAPI document you use references, it needs to behave differently following different standards.
If you define a payload with an OpenAPI schema object, subsequent references in that object need to be resolved based on their standard. Same for any other schema format used.
This is one of the major problems we need to solve, cause no other tool we have found so far can handle this.
Problem 2: Non-JSON/YAML References
In AsyncAPI, we have had multiple discussions in multiple places about wanting to use payload models such as Protobuf and XSD, but that requires that the reference tool can support these non-JSON/YAML files.
No other tool we have found so far can handle these different reference formats.
Problem 3: Load It From Anywhere
A problem we have had for a long time is that users want to load references, that are usually behind closed doors.
No other tool we have found so far, can handle or allow us to extend existing handlers to support loading references from anywhere.
Solution
The way to build this solution is through extensibility, assuming that we cannot know all the use cases, but allow the user to customize the behavior as they see fit while still providing support for the major use cases.
We need to remember that this solution should be scalable across multiple languages, especially when we join it together with the Scaling AsyncAPI parsers pitch. Although it's out of scope to include multiple languages, documenting the different use cases and the solution to each is important!
Do have a look at existing solutions and whether we can contribute to them instead, however, if there is none, don't hesitate to create it from scratch.
Solution Guidelines
I suggest that we create the first implementation in TypeScript/JavaScript, so we can immediately benefit from it in our parser and potential upcoming parser.s
Requirements that the first solution should handle:
There are different ways that users should be able to interact with the library, i.e. its API surface (I have left out some things for long-term instead of now):
{$ref: ./test}
becomes{...}
)For example in our parser, we need to be able to control the flow of resolving references, where the resolved object changes over time.
Main inputs that should be supported:
Secondary inputs that you can encounter in the main input and that should be supported:
Boundaries
Risks
1. Too Complicated
Standards are notoriously complicated, and when you are implementing something where you have multiple standards interacting with each other, it becomes even more complicated. This case is no different. To mitigate the risk try to re-use from other libraries as much as possible and even incorporate utility libraries where possible.
Long-term
Secondary inputs that you can encounter in the main input and that we should support long-term:
There are different ways that you should be able to interact with the library, i.e. its API surface:
{$ref: ./test}
becomes{$ref: {...}}
). This is needed for newer versions of JSON SchemaRelated resources
Related issues that this pitch will help solve:s
validate
andgenerate
Commands asyncapi/generator#1057schemaFormat
asyncapi/spec#930Other related resources:
$ref
/$id
json-schema-org/json-schema-spec#724$id
conform to RFC 3986 suggestion for base URI elements json-schema-org/json-schema-spec#729Beta Was this translation helpful? Give feedback.
All reactions