Drag&drop AsyncAPI block builder

[magicmatatjahu]

Currently we have in the studio Visualiser which renders the flow of operations in an application described using AsyncAPI. Unfortunately it only works in readonly mode.

A very good solution would be to add the ability to create AsyncAPI specifications from scratch (or update existing one) using drag and drop blocks, which could be connected, similar to the current blocks in visualiser.

Project assumptions:

• new tool should be written in React (in TS) using https://github.com/wbkd/react-flow library.
• there should be a registry of blocks that can be dragged
• the blocks should reflect the real structure of AsyncAPI, but they should not be too scattered, i.e. blocks for Message Object, Operation Object are ok, but a block for Description etc is not acceptable.
• each block should be able to edit the data it contains, e.g. Operation block should have possibility to choose kind of operation and corresponding message(s) etc.
• should be information that something is badly designed or missing required data in the block.
• to make generic solution (support 2.X.X and 3.X.X and up versions) project should be based on JSON Schema of AsyncAPI spec if it's possible

Whether that new tool should be standalone (as a separate project) or integrated in the Studio - to be discussed.

Scope of the project may change by June, the above is just an overview of what needs to be done. Feel free to start discussion if you have questions :)

To open the current Visualiser please go to the https://studio.asyncapi.com/ and choose fourth node on the left navigation and you should see blocks :)

[fmvilas]

Initially, I'd add just one kind of block, for Message. There would be an initial block you can't remove (the app, as it is right now). Then you can connect the app with multiple message blocks. The connections are the operations (publish or subscribe for now). When you create a new connection, it asks you which channel and which operation you want to perform. In the modal/form asking you for channel and operation information, we can ask for more details about the channel and the operation, like description, summary, etc. You should also be able to create a new channel instead of choosing one from the existing list.

Another kind of block that comes to my mind is the Broker block. This way you can connect the app to multiple brokers. When creating operations like described in my first paragraph, we could also give the user a list of brokers to select from. Depending on the broker you select, a different set of channels might be offered. That works for broker-based architectures but not for client/server APIs where our application is the server. Therefore, the application block should be configurable and let you choose between client, server, or mixed. If it's a client or "mixed", it can connect to a broker. If it's a server, it can't but it offers its own set of "channels".

Happy to create example diagrams if needed.

[mcturco]

@magicmatatjahu @fmvilas I actually would love to discuss this over a video call some time (maybe we should live stream it as well so others can join) but I am wondering how this idea can tie in somehow the work that is needed for the design system

[magicmatatjahu]

@mcturco No problem, we can make discussion but better will be before GSOC, April/May?

@fmvilas Yeah, I agree that we need main block, the App block. Also I like that idea with Message. However, is it a good approach to combine operations with a broker? It seems to me that connecting the application to the broker should immediately connect all the operations (with messages) to the broker and it is a "simpler" solution to the problem. Well, unless some operations are performed for a particular broker, and others for another (under the same channel) then the user should be able to do so.

From my suggestions: A nice option would be to have a Components block where someone could connect Message, Channels etc and then this Components block could be connected to a given application/broker and each message/channel etc would be available for selection in the corresponding blocks connected to the App block.

The question is what to do with Servers, should they be treated as separate blocks or connected/merged with the application as it is now? That and other concerns should be discussed :)

[magicmatatjahu]

but I am wondering how this idea can tie in somehow the work that is needed for the design system

I don't think this is a problem, because the most serious problem here is how to code the logic and test everything, changing styles can be done at the end of GSOC once we have some components from another GSOC task :)

[asrient]

This sounds interesting, will like to work on this for gsoc.

[fmvilas]

This is what I meant:

You start with the application block (can't be deleted). We offer only two types of blocks: message and broker. You should be able to connect messages with the application (probably only with the server interface, if there is one). You should be able to connect the app with the broker and it will ask you to fill in the needed information like what message, channel, operation, etc.

However, is it a good approach to combine operations with a broker?

I think it would be cool for the user to just "link" the app to a broker (like you do in Miro or any other diagramming tool) and it will immediately ask you for information about this "link". For instance: what message, what operation (are you sending to or receiving from the broker?), what channel, etc. Then once the information is filled, the diagram is updated and shows the new message being published to the broker. With this information, we can generate (or update) an AsyncAPI file containing a channel with an operation and the message in this operation. And IMHO it's intuitive for the user to just say "I want to make my app publish to the broker" and then we ask for more details.

If we let people drop message blocks and connect them to the app (not the server interface), they'll still be missing one step, which is answering the question "where is this message going? or coming from?". In other words, we would still have to connect the message to the broker, which I think is less intuitive because it's a two-step process, leaving the diagram in a weird or incomplete step in between.

It seems to me that connecting the application to the broker should immediately connect all the operations (with messages) to the broker and it is a "simpler" solution to the problem.

Are we talking about the same thing here? 🤔 I'm understanding this issue as a way to create (and edit) new AsyncAPI files. What do you mean by "immediately connect all the operations"?

The question is what to do with Servers, should they be treated as separate blocks or connected/merged with the application as it is now?

I think it can be represented as part of the application (because it really is). I'm talking about servers, not brokers. I mean, the ones you create with code like WebSocket or HTTP servers.

[wh0sumit]

Hey everyone,

I have gone through this issue tread and found it pretty interesting to work on. Something that I am able to understand by going through all of your messages. I have worked with React Beautiful DnD before thus I am able somehow to get what we are trying to do here.

This is what I think we are trying to do

So we need to create a visual interface to manage and create APIs with EDA Architecture. So that developer doesn't have to spend a lot of time writing YAML code, right?

Please guide me through this, and let me know if I am going in the right direction with this.

Thank you 🙂

[mcturco]

Just want to mention that this is also a GREAT project for product design contributors! I think before we jump into code, it would be useful to do some foundational research and UX ideation

[magicmatatjahu]

@wh0sumit Hi!

So we need to create a visual interface to manage and create APIs with EDA Architecture. So that developer doesn't have to spend a lot of time writing YAML code, right?

Yes, exactly :) The user will create specifications using blocks rather than writing YAML/JSON. It's hard for me to say what needs to be done in a sequence, that's more or less how GSOC looks like and that's how our mentoring program will look like, you have to give the execution plan and we as mentors can help you with it :)

We can't forget about @asrient. You were the first one still interested in GSOC. Will you let us know about your desire to participate?

[wh0sumit]

In order to better understand how we are going to implement the blocks and what other things are going on, let's start with the frontend first.

• The current response coming from the Kafka broker can be shown to the application part, along with a button to add a message.

• Whenever the user clicks on the button, a modal will be opened/or we can open a sidebar, whichever way we like, where the user can choose the type of node he/she wants to implement; the way we can do that is to give a modal to add a message.

• The message will be sent directly to the broker after saving and the broker will check the message ID and user ID of the user who sent the message. after receiving the data, the broker will update the previous state with the current state and send it back to the response to the application.

• When the application gets the response, it'll check whether anything's been updated or not. If anything's been updated, the application updates the previous state and adds the new response which is coming directly from the broker. As far as I'm concerned that we are going to use react-flow to implement this right?

Though I am unfamiliar with the Kafka broker, I am willing to work on the frontend part of this project.

CC: @magicmatatjahu @fmvilas @mcturco

[asrient]

We can't forget about @asrient. You were the first one still interested in GSOC. Will you let us know about your desire to participate?

Hi, thanks for mentioning me. I will keep an eye to see if I can find something suitable for me.

[magicmatatjahu]

@asrient Hi! So you don't want to handle that task?

[magicmatatjahu]

@wh0sumit You only partially understand task. You should not focus on brokers/kafka/business logic etc, but only on structure of AsyncAPI spec - https://github.com/asyncapi/spec/blob/master/spec/asyncapi.md Here we should create tool to build AsyncAPI by blocks like here

As you can see one operation people can use and send operation to the application and also subscribe to the application and wait for 3 messages. We should move all objects in AsyncAPI to the "block language".

[magicmatatjahu]

@wh0sumit @asrient this idea was selected for AsyncAPI Mentorship 2022 -> asyncapi/community#376 (comment)

The problem is, I don't know who to pick. @asrient you was first, but you @wh0sumit described own approach (so I guess you thought longer about this project). Let me know if you're still interested, because one of you might not be anymore and it will be easier to choose.

[asrient]

Hi,@wh0sumit can go forward with it, he has put more effort on it till now. I'm also currently bit busy with some personal stuff so I have no issues, I will give this program a try next year. All the best @wh0sumit!

[wh0sumit]

@magicmatatjahu, I am interested in this project and would love to be a part of it, thank you for considering me.

@asrient, I appreciate that you consider me for this project and I would love to connect with you :)

[magicmatatjahu]

@wh0sumit So welcome on a board! :)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[magicmatatjahu]

Still valid.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[Shurtu-gal]

@magicmatatjahu Can I work on this issue?

[fmvilas]

@Shurtu-gal we're probably going to use a different approach to this. I'm drafting the new Studio here and here.

[Shurtu-gal]

Great @fmvilas I would love to help out, wherever I can.

[fmvilas]

Awesome, I'll ping you on Slack to coordinate :)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[Amzani]

This will be part of #634