Defining a collection of applications #658
Comments
|
@jonaslagoni Well thought out and I like all of the examples! I want to bring up the discovery usecase. The approach we took for discovery is two phased: 1. grab all of the data you can from the running system (system-centric) 2. use this data to define the application boundaries of your eda (application-centric). The output of phase 1 is AsyncAPI (using the discovery tool). In this case the "discoverer" doesn't necessarily know the application boundaries, but is just accumulating data to feed into phase 2 where these boundaries are defined. Phase 1 discovery is a transitory use of the spec but I think worthy of consideration. I guess one could argue that AsyncAPI may not be the right output format of phase 1, but no one wants yet another specification. |
|
@gregmeldrum wouldn't the discovery use case be the same as the dictionary use case defined above? You just throw message definitions on an AsyncAPI file but don't know who's publishing or subscribing, i.e., you don't specify |
|
@jonaslagoni , I apologize for this long overdue response to this. It is a very good analysis.
This works in the context of files stored in github--you can group the files however you want, and the application==file scoping makes sense. But for non-file implementations, such as registries, things get trickier. I don't see a way to address the following use cases if there is a registry with a REST API:
Looking at @boyney123 , @dalelane and @SolaceProducts work, I think it would be a mistake to take only a file-exclusive approach. If we don't have a spec object to represent Not having an |
|
I think a live discussion is in order to try and deep dive into some of the perspectives of this. There is a lot to cover, so one might not be enough, but let's start with that and see where it takes us. So let's do a regular 1-hour live stream where we can discuss the different aspects and what it can mean to define multiple applications. I suggest we do it a week from now on Monday 28. February 16 UTC. The goal of the discussion is to reach a solid suggestion (RFC-1) about what should be done. I outlined my own thought in the above post, but many of the things are abstract and maybe hard to grasp, so this would be a platform to raise concerns and have a friendly discussion 🙂 @fmvilas @derberg these loose meetings that we have from time to time that is not recurring, anything I need to do to set that up? |
|
Good perspective @jessemenning 👏
But yea, it is definitely a valid argument, supporting it explicitly, would probably make sense in a lot of cases 🤔 |
The recording is out: https://www.youtube.com/watch?v=Vl8Nn3GOJiE Recap:
Monday the 7 of March at 16 UTC, we are gonna have another meeting to continue the discussion! |
|
I just watched the recording. Thanks for all the discussion you folks drove in there 👏 💯. I wanted to make a couple of suggestions on the approach for declaring applications. 1. Introduce a new key for defining system infoInstead of using the I'm suggesting this because:
Example: asyncapi: 3.0.0
system:
id: "com.mycompany.payments"
name: "Payment gateway"
description: "The main Payment gateway available on example.com"
metadata:
team: "platform"2. To create a new object for each application instead of directly using an array of references.This is a little bit more tricky and controversial. But I want to drop some of my thoughts (not super elaborated) here. Define a new application object on the I'm suggesting this because:
Example: asyncapi: 3.0.0
# ...
applications:
- name: "Core payment service" # maybe we can remove this, just use the `info.name`
application:
- $ref: './backend.asyncapi.yaml'
- name: "Analytics server"
description: "The service where all the payments are recorded to. This application is widely shared across the company"
application:
- $ref: './analytics.asyncapi.yaml' |
|
Just watched the video nice work and thanks for recording. Here are some of my thoughts/questions whilst watching When I saw the use of You asked "How could the system.yml" be useful to users, here were my thoughts
You showed examples of Versioning ha... yeah looks interesting, hard to know if systems would be versioned or not, from my own experience I would version my frontend and backend in your example but probably not the domain/system.... as @fmvilas was saying version concept is too loose IMO (in this context) @smoya I like your idea in number 2 of having additional fields in the applications array rather than just a direct pointer to the asyncapi files, allowing us to extend Also |
As no one was able to join there is no recording as there were no meeting 😆 Let's see when in the future it makes sense to hold another one.
I think this makes sense, IF and only IF there is a huge difference in which properties for the information about the system vs application. I.e. metadata I don't believe makes sense, as it most likely varies a lot between use-cases and cant be well defined, so extensions fit better there in my opinion. That said, creating a separate
There is something interesting about this cause it actually solves one of the problems that I recently faced. How do you group, specific applications together (and should you)? For example, if I had 3 applications that comprise my payment system would I want to group these 3 together? In your suggestion, it enables at least one level of groupings, but not systems in systems (as I read it). I don't like the
@boyney123 the |
|
I am gonna stop pursuing this issue for now as it seems we do not have enough clear use-cases for it, so it would not make sense to force this change as of now. I still believe this is eventually needed, and if you ever encounter such a use case please let us know below. If you wish to champion this issue, feel free to take it over 🙂 |
|
This issue has been automatically marked as stale because it has not had recent activity 😴 It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation. There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model. Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here. Thank you for your patience ❤️ |
This is an extension to The many meanings of an AsyncAPI file that tries to answer the question, should we have a way to define a larger system i.e. the need to specify applications in the context of the system.
It is a combination of the following conversations #601, comments in 618, comments in #594 and #628.
This issue tries to capture the possibility of introducing such a feature, it should not necessarily be seen as a complete feature request. Even though the issue is "small", it is a complex discussion and it can be solved in many ways. I am gonna try to break it down into more digestible chunks with examples.
Just to make sure we stay on track in the discussion, these are the main scope of the discussion. Beyond that, feel free to start a new issue or discussion focusing on a single part of the problem if you want to solve it.
Scope of discussion
Outside the scope of discussion
System context
There are a couple of ways I see we can address this, and it all depends on what we see as most fitting for AsyncAPI. I will be building upon Frans syntax from the proposal to solve the publish/subscribe confusion as this restructure is necessary to further the discussion.
I will be using the social media example as a baseline and show how the suggested changes will affect such an example.
Adapting the syntax from #618 to the examples, will make this our starting point:
Common definitions (common.asyncapi.yaml)
Even though the current social media example have these definitions completely separated, having everything gathered in one file makes more sense.
The backend - WebSocket server (backend.asyncapi.yaml)
The frontend - WebSocket client (frontend.asyncapi.yaml)
Notification service (notifications-service.asyncapi.yaml)
Comments service (comments-service.asyncapi.yaml)
Public API (public.asyncapi.yaml)
Through no change
Since a system is nothing but a collection of applications, how you group those applications together does not necessarily have to be defined with AsyncAPI.
A very simple example could be a physical directory where all the related AsyncAPI applications are located within the same directory. A complex solution could be something like a developed platform that groups them in some way such as in a database.
Q/A
I am gonna try to answer the questions asked in #628 as to how the changes affect the answers:
Can an application belong to multiple systems?
As there are no restrictions on how you define a collection of applications, it is possible.
How many separate applications are described?
Depends on how you interpret what a collection of applications is.
Which channels are used by which applications?
Through the reusability, we can make a basic search for it.
What messages are read/sent onto the channels by each application?
Depending on the structure, you can look at the applications themselves.
Where are those channels located?
By default, channels are located in all servers unless specifically defined as only related to one server.
If I want to send/receive a message from a specific application, which protocol/server/channel do I connect to?
As each application "know" which operations others can perform, and over what channels. Indirectly we know which servers those operations are performed on.
Can I define servers that all applications must connect to?
No, however, if you can use references for servers you "can".
Through a new simple keyword: Stay with application-centric
The same way we allow you to define a dictionary, we can allow the definition of a system. Say we introduce a simple
applicationskeyword on the root level of the specification, based on your example @nictownsend.The spec will keep its application-centric approach, where it does not know anything about a system context.
The new structure of the spec will look like this:
Building upon the existing types mentioned #628 (comment), we extend it with a system type. With the new keyword, we add some system types similar to the application and dictionary types.
The system as a non-dictionary
One of the ways to define the system, is by only including applications. The other files still use the
common.asyncapi.yamlfile.This means we introduce a new system file:
The system (system.asyncapi.yaml)
System as a dictionary
You can also use the system definition like a dictionary, meaning we replace the
common.asyncapi.yamlfile withsystem.asyncapi.yaml.Each individual application would then reference the system file instead of the
common.asyncapi.yaml.The system (system.asyncapi.yaml)
The backend - WebSocket server (backend.asyncapi.yaml)
The system as a mix
This type is just a mix between the two approaches, where the system for some channels/messages, etc, is used as a dictionary, other times applications define it themselves.
Q/A
Can an application belong to multiple systems?
As there are no restrictions on how you define a collection of applications, so it can be possible.
How many separate applications are described?
This can be looked up directly in the system definition.
Which channels are used by which applications?
You would have to go through each application to see what channels they use.
What messages are read/sent onto the channels by each application?
You would still have to walk through each application to find this information. From a design point, through references you could have this information present (if used as a dictionary) in the system file.
Where are those channels located?
By default, channels are located in all servers unless specifically defined as only related to one server. Nothing changes here. You would still need to look up each application how they define it.
If I want to send/receive a message from a specific application, which protocol/server/channel do I connect to?
As each application hold the information about which operations is performed, and over what channels. Nothing changes here.
Can I define servers that all applications must connect to?
No.
Through a change of perspective: Moving from application-centric to system-centric
To move from an application-centric to system-centric, would require a breaking change in the specification. This would mean that we would not be able to describe a standalone application.
The new structure of the spec will look like this:
I will center the system around each "server". Meaning I will have two system definitions, one for the MQTT broker and its applications and one centered around the WebSocket server. There are other ways you can structure this, but I will focus on these two.
This would change all our applications to be defined like so:
System definitions for MQTT (system.mqtt.asyncapi.yaml)
Notice the MQTT system has the
backend.asyncapi.yamlapplication referenced as it is part of both the MQTT and WebSocket system.The backend - WebSocket server (websocket.system.asyncapi.yaml)
Notice that the backend server is now both seen as an application and a system. This system comprises of backend server itself and the frontend application.
The backend server is also seen as an application to the MQTT broker - as part of the
mqtt.system.asyncapi.yaml.The frontend - WebSocket client (frontend.asyncapi.yaml)
Notification service (notifications-service.asyncapi.yaml)
Comments service (comments-service.asyncapi.yaml)
Public API (public.asyncapi.yaml)
Q/A
Can an application belong to multiple systems?
Yes, as we can see with the backend application, it is a system in itself for WS, and part of the MQTT system.
It does however require that channel ID's never overlap.
How many separate applications are described?
This can be looked up directly in the system definition.
Which channels are used by which applications?
You would still have to go through each application to see which operations they perform and over what channels.
What messages are read/sent onto the channels by each application?
Since the system defines the channels and their message definitions this can be looked up there. No need to access the application definitions, they only define operations.
Where are those channels located?
By default, channels are located in all servers unless specifically defined as only related to one server. Nothing changes here.
If I want to send/receive a message from a specific application, which protocol/server/channel do I connect to?
As each application hold the information about which operations is performed, and over what channels. Nothing changes here.
Can I define servers that all applications must connect to?
No, because in many cases depending on the application, they need to define specific information, such as for MQTT, `clientId`.
My conclusion
Even though something like a system-centric approach could be an interesting concept, my answer would be we should stay with an application-centric approach. It would then be up to the individuals, how they group multiple applications together. We just provide the building blocks for you to define applications.
Yes, it will mean you need to duplicate many of the definitions, but through
$refwe can limit it, at least enough to where I don't see it as a massive disadvantage.What do you think?
The text was updated successfully, but these errors were encountered: