feat: update aqmp config #310

VaishnaviNandakumar · 2023-06-13T13:31:00Z

Description

This implementation makes use of the Direct Exchange type of the AMQP protocol where a routing key is used to route messages to a queue. The configuration file has been structured to include the Exchange information under the publisher binding while both the Queue and Routing Key are specified under the subscriber section.
Provides a functional setup of AMQP configuration that supports a publisher service and a consumer listener.
This configuration is flexible to maintain the exchanges, queues and routing keys of more than one channel.
Test config files are added under the test/user_examples folder.

Test

Single Channel
Message passed to RabbitMQ via command line.

python.exe rabbitmqadmin.exe publish exchange=lightMeasuredExchange routing_key=lightMeasuredRoutingKey payload="{ \"id\":1, \"lumens\": 20}
Message published

Output from App Listener

Multiple Channels
Message passed to RabbitMQ via command line.

python.exe rabbitmqadmin.exe publish exchange=lightMeasuredExchange_Streetlight1 routing_key=lightMeasuredRoutingKey_Streetlight1 payload="{ \"id\":1, \"lumens\": 25}
Message published

python.exe rabbitmqadmin.exe publish exchange=lightMeasuredExchange_Streetlight2 routing_key=lightMeasuredRoutingKey_Streetlight2 payload="{ \"id\":2, \"lumens\": 50}
Message published

Output from App Listener

Related issue(s)
Resolves #307

VaishnaviNandakumar · 2023-06-13T13:33:07Z

@Tenischev Hi, please have a look. Thanks.

Tenischev · 2023-06-17T11:27:43Z

Hi @VaishnaviNandakumar
Did you check, maybe some improvements also required in rabbitmq.yml?

VaishnaviNandakumar · 2023-06-17T16:17:28Z

Hi @Tenischev, I am not very familiar with docker images and configs so I didn't want to touch that.

VaishnaviNandakumar · 2023-06-25T08:09:40Z

Hi @Tenischev , any update on this?

Tenischev · 2023-06-25T08:37:46Z

Hi @VaishnaviNandakumar ,sorry I was busy this week, but I'm remember about pr. Will check soon

Tenischev

Sorry for such long delay

partials/AmqpPublisher.java

Tenischev · 2023-07-28T17:04:20Z

template/src/main/java/com/asyncapi/service/MessageHandlerService.java

+
+    {% elif asyncapi | isProtocol('amqp')  %}
+    {% for channelName, channel in asyncapi.channels() %}
+    {%- set schemaName = channel.subscribe().message().payload().uid() | camelCase | upperFirst %}


I think channel.publish().message() should be there, not subscribe

From my understanding of AMQP, the messages are published into an exchange and listens from a queue. This queue is defined under the subscriber/consumer module.

There's a small logical/conceptual change required here. Will go through it once again and revise it.

"the messages are published into an exchange and listens from a queue" - agree.
But please have a look to explanation of AyncAPI publish\subscribe meaning - https://www.asyncapi.com/docs/tutorials/getting-started/hello-world

In this example, you only have one channel called hello. The sample application subscribes to this channel to receive hello {name} messages.

asyncapi: 2.6.0 info: title: Hello world application version: '0.1.0' channels: hello: publish: message: payload: type: string pattern: '^hello .+$'

By means, if AsyncAPI define publish, that means that our generated application should be able to receive messages from the publish. And vice-versa, if AsyncAPI define subscribe, that means that our generated application should be able to sned messages to the subscribe.

The MessageHandlerService should be responsible for receiving messages to do so, listeners for the AsyncAPI publish channels should be defined.

I had earlier referred to this doc from which my understanding of publish/subscribe was the opposite of your explanation. That is, Publish sends messages and Subscriber receives it. Following this, I thought a listener would be most appropriate for a subscriber instead.

With the upcoming v3 changes to AsyncAPI, publish/subscribe will be replaced by send/receive. Should I hold on to the current changes until then? It will require a rewrite of the template generation.

Sorry, I do not find the doc is opposite to what I wrote.
It is the same meaning:

If in AsyncAPI the publish defined, then we should generate Subscriber\Consumer\MessageHandler.

Updating to AsyncAPI v3 is a global task and need to be implemented in whole template, not a part.
I would suggest to you to finish current PR for v2

I think I got where I went wrong. I was trying to validate the doc from a server perspective while it should've been the client instead. Sorry about that. So just to confirm, publish defines the consumer and subscribe defines the producer.
In that case for AMQP, queue and routing key should be defined under publish while exchange should come under subscribe.

In that case for AMQP, queue and routing key should be defined under publish while exchange should come under subscribe.

Yes, this is also my understanding

Also, AMQP channel binding might be interesting for you in perspective of this PR

Thanks. Will update accordingly.

template/src/main/java/com/asyncapi/service/MessageHandlerService.java

Tenischev · 2023-07-28T17:08:22Z

template/src/main/resources/application.yml

+  {{channelName}}:
+    exchange: {{channel.publish().binding('amqp').exchange}}
+    queue: {{channel.subscribe().binding('amqp').queue}}
+    routingKey: {{channel.subscribe().binding('amqp').routingKey}}


I'm not sure that this is a good idea to generate exchange and queue for all channels without taking into account if API really has publish\subscribe for this channel

@Tenischev So would the best way to go about it be to put in a check to see if the publish/ subscribe has been defined for that channel and then define exchange/queues etc?

I think yes, of course this is not so critical for the configuration file, but it will keep it clean

Tenischev · 2023-07-28T17:09:12Z

partials/AmqpConfig.java

+
+    {% for channelName, channel in asyncapi.channels() %}
+    @Value("${amqp.{{- channelName -}}.exchange}")
    private String {{channelName}}Exchange;

-    {% endif %}{% endfor %}
-    {% for channelName, channel in asyncapi.channels() %}{% if channel.hasPublish() %}
-    @Value("${amqp.queue.{{- channelName -}}}")
+    @Value("${amqp.{{- channelName -}}.queue}")
    private String {{channelName}}Queue;

-    {% endif %}{% endfor %}
+    @Value("${amqp.{{- channelName -}}.routingKey}")
+    private String {{channelName}}RoutingKey;
+    {% endfor %}


I'm not sure that this is a good idea to generate exchange and queue for all channels without taking into account if API really has publish\subscribe for this channel

Tenischev · 2023-07-28T20:49:35Z

partials/AmqpConfig.java

+                    {{channelName}}_Exchange,
+                    {{channelName}}_Binding
+                {% else %}
+                    {{channelName}}_Queue,


Also here, you could use {% if not loop.last %},{% endif %} to add coma after _Binding instead of counting i

VaishnaviNandakumar · 2023-09-01T17:22:54Z

@Tenischev Hi, sorry I missed out on these comments. Slightly packed rn but I will try to get back to resolve these comments in a week. Thanks!

…spring-template into feat/update-aqmp-config

…iNandakumar/java-spring-template into feat/update-aqmp-config

VaishnaviNandakumar

I have made the below changes in this update

Restructured config yaml files to fit AMQP bindings as defined here. It wasn't done as per standard before.
Added a routingKey key under bindings in subscriber. The previous implementation had it hard coded.
Removed BindingBuilder from AmqpConfig. Initially it binded all the exchanges, queues and routing keys that came under a single channel together which wasn't conceptually right.
Added hasPublisher() and hasSubscriber() checks.

Output Validation

Single Channel
Multiple Channels

Tenischev · 2023-09-05T20:47:21Z

@VaishnaviNandakumar, please check failed tests.
Also I see you added new API to user examples which is great, but now we a lack of example for amqp in tests/mocks/ if you find it possible, it would be very good to place small and simple amqp API in tests/mocks/ and create snapshoot test for it.
As ref you could have a look on tests/mocks/kafka.yml or tests/mocks/mqtt.yml there just one publish, one subscribe channel and simple payload component.

VaishnaviNandakumar · 2023-09-06T07:57:38Z

@Tenischev which editor are you using? I'm working so far on IntelliJ but I am not able to run any tests. Need some help with the setup.

Tenischev · 2023-09-06T21:02:13Z

@VaishnaviNandakumar I also use IntelliJ, but template test requires npm. Just run npm install and then npm test from the template folder, to update snapshoots npm test -- -u.
By looking to GitHub report most issues are just new empty lines or removed empty lines.
Additionally, you could find standard Java unit test that in folder template/src/test/java/com/asyncapi, but there is no special unit test for AMQP.

VaishnaviNandakumar · 2023-09-07T13:37:03Z

@Tenischev I have updated the PR to fix the failing tests. However, I do not have the bandwidth at the moment to add more for AMQP. Can that be taken up as a separate issue instead?

Tenischev · 2023-09-07T18:08:50Z

@VaishnaviNandakumar sure it could be done in a separate PR.

Tenischev

@VaishnaviNandakumar there is one veeeery small typo, could you please fix and i believe we will merge it

Tenischev · 2023-09-28T22:31:56Z

template/build.gradle

@@ -15,6 +15,7 @@ repositories {
 dependencies {
 	{%- if asyncapi | isProtocol('amqp') %}
 	implementation('org.springframework.integration:spring-integration-amqp')
+	implementation('org.springframework.integration:spring-integration-amqp')


Why it's doubled?

# Conflicts: # template/src/main/java/com/asyncapi/service/MessageHandlerService.java # tests/__snapshots__/kafka.test.js.snap # tests/__snapshots__/mqtt.test.js.snap # tests/__snapshots__/oneOf.test.js.snap # tests/__snapshots__/parameters.test.js.snap

asyncapi-bot · 2023-10-07T20:47:48Z

🎉 This PR is included in version 1.3.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

VaishnaviNandakumar added 4 commits June 8, 2023 22:43

Updated config file for amqp

1aa8103

Added publisher service

de6040e

Added listeners and publisher code

aaec4c3

Updated AMQP config and added sample config files to test

eeab14e

VaishnaviNandakumar requested review from Tenischev, derberg and asyncapi-bot-eve as code owners June 13, 2023 13:31

Tenischev changed the title ~~Feat/update aqmp config~~ feat: update aqmp config Jun 17, 2023

Merge branch 'master' into feat/update-aqmp-config

4176cd5

Tenischev requested changes Jul 28, 2023

View reviewed changes

Tenischev reviewed Jul 28, 2023

View reviewed changes

Merge branch 'master' into feat/update-aqmp-config

77787f1

VaishnaviNandakumar and others added 5 commits September 3, 2023 19:54

Merge branch 'master' of https://github.com/VaishnaviNandakumar/java-…

0ddfba0

…spring-template into feat/update-aqmp-config

Updated amqp config

34cbafd

Merge branch 'asyncapi:master' into feat/update-aqmp-config

6e5b555

Merge branch 'feat/update-aqmp-config' of https://github.com/Vaishnav…

eaa0dee

…iNandakumar/java-spring-template into feat/update-aqmp-config

Segregated declarables for exchanges and queues

d46f4cb

VaishnaviNandakumar commented Sep 5, 2023

View reviewed changes

Updated test snapshots

03f7758

Tenischev requested changes Sep 28, 2023

View reviewed changes

Tenischev added 3 commits October 7, 2023 23:30

fix double dependency

73e0e93

fix snapshots

ab81bbd

Tenischev approved these changes Oct 7, 2023

View reviewed changes

Tenischev merged commit 6c611bc into asyncapi:master Oct 7, 2023
8 checks passed

asyncapi-bot added the released label Oct 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: update aqmp config #310

feat: update aqmp config #310

VaishnaviNandakumar commented Jun 13, 2023

VaishnaviNandakumar commented Jun 13, 2023

Tenischev commented Jun 17, 2023

VaishnaviNandakumar commented Jun 17, 2023

VaishnaviNandakumar commented Jun 25, 2023

Tenischev commented Jun 25, 2023

Tenischev left a comment

Tenischev Jul 28, 2023

VaishnaviNandakumar Sep 3, 2023

VaishnaviNandakumar Sep 3, 2023

Tenischev Sep 4, 2023

VaishnaviNandakumar Sep 4, 2023

Tenischev Sep 4, 2023 •

edited

VaishnaviNandakumar Sep 4, 2023

Tenischev Sep 4, 2023

VaishnaviNandakumar Sep 4, 2023

Tenischev Jul 28, 2023

VaishnaviNandakumar Sep 3, 2023

Tenischev Sep 4, 2023

Tenischev Jul 28, 2023

Tenischev Jul 28, 2023

VaishnaviNandakumar commented Sep 1, 2023

VaishnaviNandakumar left a comment

Tenischev commented Sep 5, 2023 •

edited

VaishnaviNandakumar commented Sep 6, 2023

Tenischev commented Sep 6, 2023

VaishnaviNandakumar commented Sep 7, 2023

Tenischev commented Sep 7, 2023

Tenischev left a comment

Tenischev Sep 28, 2023

asyncapi-bot commented Oct 7, 2023

feat: update aqmp config #310

feat: update aqmp config #310

Conversation

VaishnaviNandakumar commented Jun 13, 2023

VaishnaviNandakumar commented Jun 13, 2023

Tenischev commented Jun 17, 2023

VaishnaviNandakumar commented Jun 17, 2023

VaishnaviNandakumar commented Jun 25, 2023

Tenischev commented Jun 25, 2023

Tenischev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Tenischev Sep 4, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VaishnaviNandakumar commented Sep 1, 2023

VaishnaviNandakumar left a comment

Choose a reason for hiding this comment

Tenischev commented Sep 5, 2023 • edited

VaishnaviNandakumar commented Sep 6, 2023

Tenischev commented Sep 6, 2023

VaishnaviNandakumar commented Sep 7, 2023

Tenischev commented Sep 7, 2023

Tenischev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

asyncapi-bot commented Oct 7, 2023

Tenischev Sep 4, 2023 •

edited

Tenischev commented Sep 5, 2023 •

edited