Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: first support of bindings for AVRO key #77

Conversation

M3lkior
Copy link
Collaborator

@M3lkior M3lkior commented Sep 9, 2021

see the #67 for the details ;

this implementation only support bindings.[protocol].key but if you have any feedback for a better support of general bindings for any property ; i'm listening

Here is the result with the support of bindings for my kafka key definition that was previously not correctly displayed in the html generation (test with the last version of the generator)

image

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to AsyncAPI. Thanks a lot for creating your first pull request. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.

Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

@M3lkior M3lkior force-pushed the feat/67-first-support-of-kafka-key-bindings branch from dcfcc72 to 721731f Compare September 9, 2021 15:17
@M3lkior M3lkior changed the title feat: first support of bindings for kafka key feat: first support of bindings for AVRO key Sep 9, 2021
@M3lkior M3lkior force-pushed the feat/67-first-support-of-kafka-key-bindings branch from 721731f to 6cf8ca3 Compare September 9, 2021 15:18
@M3lkior M3lkior marked this pull request as ready for review September 9, 2021 15:18
index.js Outdated
@@ -7,6 +7,19 @@ module.exports.parse = async ({ message, defaultSchemaFormat }) => {
message['x-parser-original-payload'] = message.payload;
message.payload = transformed;
delete message.schemaFormat;

async function handleProtocolKey() {
for (const protocol of Object.keys(message.bindings)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can go straight to Kafka protocol instead of iterating. I don't know any other protocol that has a key property that can be defined with Avro.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if fact this was my first approach but i guess maybe new EDI technologies in future can handle message key like nats.io ... but ok i will change this implementation to only support the kafka protocol

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done @fmvilas

@magicmatatjahu
Copy link
Member

magicmatatjahu commented Sep 10, 2021

Maybe as a temporary solution to this problem it is ok, but in the long term it is not a good solution. We can now hardcode with that, but what if someone defines a key in kafka with raml instead of avro, or in a different format? The problem is much deeper, because not only kafka.key, but also other values in other bindings can be described by "schema format". There is a link to the similar issue. We should allow things that are schemas to be defined in different formats - currently we only allow payload in messages (but we don't, for example, allow schemas to be defined in components/schemas).

EDIT: Related issue with proposal asyncapi/spec#622

@M3lkior
Copy link
Collaborator Author

M3lkior commented Sep 10, 2021

Maybe as a temporary solution to this problem it is ok, but in the long term it is not a good solution. We can now hardcode with that, but what if someone defines a key in kafka with raml instead of avro, or in a different format? The problem is much deeper, because not only kafka.key, but also other values in other bindings can be described by "schema format". There is a link to the similar issue. We should allow things that are schemas to be defined in different formats - currently we only allow payload in messages (but we don't, for example, allow schemas to be defined in components/schemas).

EDIT: Related issue with proposal asyncapi/spec#622

Yes i agree with you. i assume that the parser-js only use the avro-parser if a dedicated avro format is defined. In your proposal, i agree with the fact to set a schemaFormat per property like headers, key and value to tune the way we process the specific format

@fmvilas
Copy link
Member

fmvilas commented Sep 13, 2021

Looks good to me so far. I think we should update the Kafka binding before we proceed to merge this one: https://github.com/asyncapi/bindings/tree/master/kafka.

@M3lkior
Copy link
Collaborator Author

M3lkior commented Sep 13, 2021

Looks good to me so far. I think we should update the Kafka binding before we proceed to merge this one: https://github.com/asyncapi/bindings/tree/master/kafka.

By indicating that the key can also handle an Avro reference file in addition to a Schema Object ?

@fmvilas
Copy link
Member

fmvilas commented Sep 13, 2021

Yes, that's what I was thinking. As the binding is right now, this library would not be compliant but since I think it's a great thing to have for Kafka devs, maybe it's just a matter of updating the Kafka binding definition.

@M3lkior
Copy link
Collaborator Author

M3lkior commented Sep 13, 2021

Yes, that's what I was thinking. As the binding is right now, this library would not be compliant but since I think it's a great thing to have for Kafka devs, maybe it's just a matter of updating the Kafka binding definition.

Here is the related PR @fmvilas asyncapi/bindings#81

@M3lkior
Copy link
Collaborator Author

M3lkior commented Sep 17, 2021

Guys (@fmvilas / @magicmatatjahu ) , what about this PR status linked to the merge asyncapi/bindings#81 ? Could you merge it ?

@magicmatatjahu
Copy link
Member

@fmvilas You requested changes one time :) Please look again.

@fmvilas
Copy link
Member

fmvilas commented Sep 17, 2021

@M3lkior there are some linter errors. Otherwise, it looks good to me.

@M3lkior
Copy link
Collaborator Author

M3lkior commented Sep 17, 2021

@M3lkior there are some linter errors. Otherwise, it looks good to me.

oups sorry, it is fixed !

fmvilas
fmvilas previously approved these changes Sep 17, 2021
Copy link
Member

@fmvilas fmvilas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@fmvilas
Copy link
Member

fmvilas commented Sep 17, 2021

@magicmatatjahu can you review and approve/reject?

magicmatatjahu
magicmatatjahu previously approved these changes Sep 17, 2021
index.js Outdated Show resolved Hide resolved
Co-authored-by: Maciej Urbańczyk <urbanczyk.maciej.95@gmail.com>
@M3lkior M3lkior dismissed stale reviews from magicmatatjahu and fmvilas via 07716bf September 17, 2021 11:51
@sonarcloud
Copy link

sonarcloud bot commented Sep 17, 2021

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@magicmatatjahu magicmatatjahu merged commit be78b76 into asyncapi:master Sep 17, 2021
@asyncapi-bot
Copy link
Contributor

🎉 This PR is included in version 0.5.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants