./gradlew clean build bootRun --args='--spring.profiles.active=local' # Kafka Traveler Microservices Demo: Accounts
Accounts Service
Spring Boot/Kafka/Mongo Microservice, part of a set of microservices for this project. Services use Spring Kafka 2.1.6 to maintain eventually consistent data between their different Customer domain objects.
Originally code based on the post, Spring Kafka - JSON Serializer Deserializer Example, from the CodeNotFound.com Blog.
Development
For Kakfa, use garystafford/kafka-docker project, a clone of the wurstmeister/kafka-docker project. The garystafford/kafka-docker local docker-compose file builds a Kafka, Kafka Manager, ZooKeeper, MongoDB, Eureka Server, and Zuul.
Commands
I develop and debug directly from JetBrains IntelliJ. The default Spring profile will start the three services on different ports.
./gradlew clean build bootRun --args='--spring.profiles.active=local'Swagger UI: http://localhost:8085/swagger-ui/index.html
Creating Sample Data
Create sample data for each service. Requires Kafka is running. Endpoints for Zuul, when using Docker Swarm/Stack, are different. See this Python script for Zuul endpoints.
# accounts - create sample customer accounts
http http://localhost:8085/customers/sample
# orders - add sample orders to each customer
http http://localhost:8090/customers/sample/orders
# orders - send approved orders to fulfillment service
http http://localhost:8090/customers/sample/fulfill
# fulfillment - change fulfillment requests from approved to processing
http http://localhost:8095/fulfillments/sample/process
# fulfillment - change fulfillment requests from processing to shipping
http http://localhost:8095/fulfillments/sample/ship
# fulfillment - change fulfillment requests from processing to in transit
http http://localhost:8095/fulfillments/sample/in-transit
# fulfillment - change fulfillment requests from in transit to in received
http http://localhost:8095/fulfillments/sample/receiveContainer Infrastructure
$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7a1382353e96 garystafford/storefront-orders:3.0.0 "java -jar -Djava.se…" About an hour ago Up About an hour 8080/tcp storefront_orders.1.8w90p3sdxl3wb837vfy7w382j
fd3e9bb45c6f garystafford/storefront-accounts:3.0.0 "java -jar -Djava.se…" About an hour ago Up About an hour 8080/tcp storefront_accounts.1.ma2kv9v3ulzq1xqziutht3631
264a7ba2dd8a garystafford/storefront-fulfillment:3.0.0 "java -jar -Djava.se…" About an hour ago Up About an hour 8080/tcp storefront_fulfillment.1.qwb64lv2dgvyoqkm2aaf22exj
0862ecb5daa6 hlebalbau/kafka-manager:3.0.0.5 "/kafka-manager/bin/…" 2 hours ago Up 2 hours 0.0.0.0:9000->9000/tcp storefront_kafka_manager.1.fqk7i3k8c71mmzwp4d8ink5fv
53a661cb17cf wurstmeister/kafka:2.13-2.7.0 "start-kafka.sh" 2 hours ago Up 2 hours 0.0.0.0:9092->9092/tcp storefront_kafka.1.ka95csztjcfnz5lozz2e32onl
550d68864b6f zookeeper:3.7.0 "/docker-entrypoint.…" 2 hours ago Up 2 hours 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, 8080/tcp storefront_zookeeper.1.b0sde5og2cc74j1ssjmqdi3e3
6358b43b9af1 garystafford/storefront-eureka:3.0.0 "java -jar -Djava.se…" 2 hours ago Up 2 hours 0.0.0.0:8761->8761/tcp storefront_eureka.1.deqfb9ibcb6p84qg84b14d5vz
bb6b52740b2d mongo-express:0.54.0 "tini -- /docker-ent…" 2 hours ago Up 2 hours 0.0.0.0:8081->8081/tcp storefront_mongo_express.1.he0sj16ofyd8xke8t887fvlkc
c6e435d70a48 mongo:4.4.6 "docker-entrypoint.s…" 2 hours ago Up 2 hours 0.0.0.0:27017->27017/tcp storefront_mongo.1.pv0grikr4yjdipafpdvrg56uf
2e09b154bdc8 grafana/grafana:7.5.6-ubuntu "/run.sh" 2 hours ago Up 2 hours 0.0.0.0:3000->3000/tcp storefront_grafana.1.zlhn98glmxgfoiyl8ce7yvjb9
43fcb19f38c0 garystafford/storefront-zuul:1.0.0 "java -jar -Djava.se…" 2 hours ago Up 2 hours 0.0.0.0:8080->8080/tcp, 8761/tcp storefront_zuul.1.6q7kbj6zo0qx8ccbom7m1uetr
f1ee73eec8b5 prom/prometheus:v2.27.0 "/bin/prometheus --c…" 2 hours ago Up 2 hours 0.0.0.0:9090->9090/tcp storefront_prometheus.1.bubfxjy30auua8hlwgc9k221v
Orders Customer Object in MongoDB
docker exec -it $(docker ps | grep storefront_mongo | awk '{print $NF}') sh
mongo
db.customer.accounts.find().pretty()
db.customer.orders.remove({})
{
"_id" : ObjectId("5b18661ebe417602a48132ed"),
"name" : {
"title" : "Ms.",
"firstName" : "Susan",
"lastName" : "Blackstone"
},
"contact" : {
"primaryPhone" : "433-544-6555",
"secondaryPhone" : "223-445-6767",
"email" : "susan.m.blackstone@emailisus.com"
},
"addresses" : [
{
"type" : "BILLING",
"description" : "My CC billing address",
"address1" : "33 Oak Avenue",
"city" : "Nowhere",
"state" : "VT",
"postalCode" : "444556-9090"
},
{
"type" : "SHIPPING",
"description" : "Home Sweet Home",
"address1" : "33 Oak Avenue",
"city" : "Nowhere",
"state" : "VT",
"postalCode" : "444556-9090"
}
],
"creditCards" : [
{
"type" : "PRIMARY",
"description" : "Master Card",
"number" : "4545-5656-7878-9090",
"expiration" : "4/19",
"nameOnCard" : "Susan M. Blackstone"
}
],
"_class" : "com.storefront.model.Customer"
}
Current Results
Output from application, on the accounts.customer.change topic
2018-06-03 03:17:36.510 INFO [-,aee18dd362da06b4,aee18dd362da06b4,false] 12 --- [nio-8080-exec-1] o.a.kafka.common.utils.AppInfoParser : Kafka version : 1.0.1
2018-06-03 03:17:36.510 INFO [-,aee18dd362da06b4,aee18dd362da06b4,false] 12 --- [nio-8080-exec-1] o.a.kafka.common.utils.AppInfoParser : Kafka commitId : c0518aa65f25317e
2018-06-03 03:17:37.163 INFO [-,aee18dd362da06b4,aee18dd362da06b4,false] 12 --- [nio-8080-exec-1] c.storefront.handler.AfterSaveListener : event='org.springframework.data.mongodb.core.mapping.event.AfterSaveEvent[source=Customer(id=5b135dd0be4176000cf30283, name=Name(title=Ms., firstName=Mary, middleName=null, lastName=Smith, suffix=null), contact=Contact(primaryPhone=456-789-0001, secondaryPhone=456-222-1111, email=marysmith@yougotmail.com), addresses=[Address(type=BILLING, description=My CC billing address, address1=1234 Main Street, address2=null, city=Anywhere, state=NY, postalCode=45455-66677), Address(type=SHIPPING, description=Home Sweet Home, address1=1234 Main Street, address2=null, city=Anywhere, state=NY, postalCode=45455-66677)], creditCards=[CreditCard(type=PRIMARY, description=VISA, number=4545-6767-8989-0000, expiration=7/21, nameOnCard=Mary Smith)])]'
2018-06-03 03:17:37.163 INFO [-,aee18dd362da06b4,aee18dd362da06b4,false] 12 --- [nio-8080-exec-1] com.storefront.kafka.Sender : sending payload='Customer(id=5b135dd0be4176000cf30283, name=Name(title=Ms., firstName=Mary, middleName=null, lastName=Smith, suffix=null), contact=Contact(primaryPhone=456-789-0001, secondaryPhone=456-222-1111, email=marysmith@yougotmail.com), addresses=[Address(type=BILLING, description=My CC billing address, address1=1234 Main Street, address2=null, city=Anywhere, state=NY, postalCode=45455-66677), Address(type=SHIPPING, description=Home Sweet Home, address1=1234 Main Street, address2=null, city=Anywhere, state=NY, postalCode=45455-66677)], creditCards=[CreditCard(type=PRIMARY, description=VISA, number=4545-6767-8989-0000, expiration=7/21, nameOnCard=Mary Smith)])' to topic='accounts.customer.change'
2018-06-03 03:17:37.166 INFO [-,aee18dd362da06b4,aee18dd362da06b4,false] 12 --- [nio-8080-exec-1] c.storefront.handler.AfterSaveListener : event='org.springframework.data.mongodb.core.mapping.event.AfterSaveEvent[source=Customer(id=5b135dd0be4176000cf30284, name=Name(title=Ms., firstName=Susan, middleName=null, lastName=Blackstone, suffix=null), contact=Contact(primaryPhone=433-544-6555, secondaryPhone=223-445-6767, email=susan.m.blackstone@emailisus.com), addresses=[Address(type=BILLING, description=My CC billing address, address1=33 Oak Avenue, address2=null, city=Nowhere, state=VT, postalCode=444556-9090), Address(type=SHIPPING, description=Home Sweet Home, address1=33 Oak Avenue, address2=null, city=Nowhere, state=VT, postalCode=444556-9090)], creditCards=[CreditCard(type=PRIMARY, description=Master Card, number=4545-5656-7878-9090, expiration=4/19, nameOnCard=Susan M. Blackstone)])]'
2018-06-03 03:17:37.166 INFO [-,aee18dd362da06b4,aee18dd362da06b4,false] 12 --- [nio-8080-exec-1] com.storefront.kafka.Sender : sending payload='Customer(id=5b135dd0be4176000cf30284, name=Name(title=Ms., firstName=Susan, middleName=null, lastName=Blackstone, suffix=null), contact=Contact(primaryPhone=433-544-6555, secondaryPhone=223-445-6767, email=susan.m.blackstone@emailisus.com), addresses=[Address(type=BILLING, description=My CC billing address, address1=33 Oak Avenue, address2=null, city=Nowhere, state=VT, postalCode=444556-9090), Address(type=SHIPPING, description=Home Sweet Home, address1=33 Oak Avenue, address2=null, city=Nowhere, state=VT, postalCode=444556-9090)], creditCards=[CreditCard(type=PRIMARY, description=Master Card, number=4545-5656-7878-9090, expiration=4/19, nameOnCard=Susan M. Blackstone)])' to topic='accounts.customer.change'
Output from Kafka container using the following command.
kafka-console-consumer.sh \
--bootstrap-server localhost:9092 \
--from-beginning --topic accounts.customer.change \
| jqKafka Consumer Output
{"id":"5b1be010a8d05620a3d7efe8","name":{"title":"Mr.","firstName":"John","middleName":"S.","lastName":"Doe","suffix":"Jr."},"contact":{"primaryPhone":"555-666-7777","secondaryPhone":"555-444-9898","email":"john.doe@internet.com"},"addresses":[{"type":"BILLING","description":"My cc billing address","address1":"123 Oak Street","address2":null,"city":"Sunrise","state":"CA","postalCode":"12345-6789"},{"type":"SHIPPING","description":"My home address","address1":"123 Oak Street","address2":null,"city":"Sunrise","state":"CA","postalCode":"12345-6789"}]}
{"id":"5b1be010a8d05620a3d7efe9","name":{"title":"Ms.","firstName":"Mary","middleName":null,"lastName":"Smith","suffix":null},"contact":{"primaryPhone":"456-789-0001","secondaryPhone":"456-222-1111","email":"marysmith@yougotmail.com"},"addresses":[{"type":"BILLING","description":"My CC billing address","address1":"1234 Main Street","address2":null,"city":"Anywhere","state":"NY","postalCode":"45455-66677"},{"type":"SHIPPING","description":"Home Sweet Home","address1":"1234 Main Street","address2":null,"city":"Anywhere","state":"NY","postalCode":"45455-66677"}]}
{"id":"5b1be010a8d05620a3d7efea","name":{"title":"Ms.","firstName":"Susan","middleName":null,"lastName":"Blackstone","suffix":null},"contact":{"primaryPhone":"433-544-6555","secondaryPhone":"223-445-6767","email":"susan.m.blackstone@emailisus.com"},"addresses":[{"type":"BILLING","description":"My CC billing address","address1":"33 Oak Avenue","address2":null,"city":"Nowhere","state":"VT","postalCode":"444556-9090"},{"type":"SHIPPING","description":"Home Sweet Home","address1":"33 Oak Avenue","address2":null,"city":"Nowhere","state":"VT","postalCode":"444556-9090"}]}
To manually create accounts.customer.change topic
kafka-topics.sh --create \
--zookeeper zookeeper:2181 \
--replication-factor 1 --partitions 1 \
--topic accounts.customer.changeClear messages from accounts.customer.change topic
Delete the topic from the Kafka Manager UI, or from the commandline:
kafka-configs.sh --zookeeper zookeeper:2181 \
--alter --entity-type topics --entity-name accounts.customer.change \
--add-config retention.ms=1000
# wait ~ 2-3 minutes to clear...check if clear
kafka-console-consumer.sh \
--bootstrap-server localhost:9092 \
--from-beginning --topic accounts.customer.change
kafka-configs.sh --zookeeper zookeeper:2181 \
--alter --entity-type topics --entity-name accounts.customer.change \
--delete-config retention.ms