Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixed Kafka integration #150

Merged
merged 3 commits into from May 17, 2022
Merged

Fixed Kafka integration #150

merged 3 commits into from May 17, 2022

Conversation

oskardudycz
Copy link
Owner

@oskardudycz oskardudycz commented May 17, 2022 •
edited

Fixed Kafka integration

  • added the initialiser to docker-compose to set the Kafka topics upfront.
  • fixed EventEnvelope creation to use a generic one. Changed the non-generic from record to interface to make sure that it won't be used.
  • made sure that EventBus takes only EventEnvelope.
  • created EventEnvelope in the external publishing.
  • used handler in KafkaConsumer to deserialise contracts with private constructors,
  • changed KafkaConsumer setting to commit offset manually.
  • enforced EventEnvelope on in-memory bus.
  • added EventEnvelopeExtensions for Kafka message deserialisation and aligned with EventStoreDB subscription to all to ignore messages that are not able to deserialise

TODO in follow up PRs:

  • - use Marten command bus stream for command bus outbox pattern implementation. The HTTP one is too volatile, especially that's used only for distributed processes,
  • - allow to pass regular events and take trace information to EventEnvelope internally inside the event bus.

Thanks, @jchannon for catching that, and apologies for promising you bluntly that it'll work fine 😅

@oskardudycz oskardudycz marked this pull request as draft May 17, 2022 15:16
@oskardudycz
Fixed Kafka integration:
6485ea9 
- added the initializer to docker compose to set the Kafka topics upfront.
- fixed EventEnvelope creation to use generic one. Changed the non-generic from record to interface to make sure that it won't be used.
- made sure that EventBus takes only EventEnvelope.
- created EventEnvelope in the external publishing.
- used handler in KafkaConsumer to deserialise contracts with private constructors,
- changed KafkaConsumer setting to manually commit offset.
@oskardudycz oskardudycz marked this pull request as ready for review May 17, 2022 16:33
@jchannon
Copy link
Contributor

🤘

@oskardudycz
Added EventEnvelopeExtensions for Kafka message deserialisation and a…
f26c430 
…ligned with EventStoreDB subscription to all to ignore messages that's not able to deserialise
@oskardudycz oskardudycz merged commit 89c7d1d into main May 17, 2022
@oskardudycz oskardudycz deleted the fixes_to_kafka_integration branch May 17, 2022 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug enhancement Kafka Marten
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@oskardudycz @jchannon