scala-kafka-cqrs-es

This is a demo application showing a potential use-case for Apache Kafka with at-least-once, transactional message processing.

Architecture

The following applications are available:

• stock service
• report service
• notification service

Stock Service

• provides HTTP endpoints for managing stock (pending)
• stores stock state in its database ✅
• sends a StockEvent.Created to stock-event topic - smuggled through the database (mailbox pattern) for at-least-once delivery and consistency
• run a concurrent process to send events pending in the database (mailbox pattern part 2)

Report Service

• reads StockEvents ✅
• builds its own read model of stock for reports (pending)
• builds reports (pending)
• sends ReportEvent.Created to report-event topic (pending)
• serves reports over HTTP (pending)

Notification Service (pending)

• consumes ReportEvents
• notifies customers that a new report is available (in the demo case, that'll be just printing to logs)
• keeps track of the amount of notifications sent
• serves stats (notifications sent) over HTTP

Setup

1. Run Kafka with Schema Registry. This docker-compose can be used, but additional options need to be added to the broker's environment variables to enable transactional processing:

   KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
   KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1

2. Build the apps: sbt stage

3. Run the apps:

   # stock service - port 9001
   ./applications/stock/target/universal/stage/bin/stock
   
   # report service - port 9002
   ./applications/reports/target/universal/stage/bin/reports
   
   # notification service - port 9003
   ./applications/notifications/target/universal/stage/bin/notifications

4. Use swagger to call application endpoints. All apps expose a Swagger UI at /swagger. (pending)