Refactor the microservice architecture for Vert.x 3.5

[sczyh30]

Some plans:

• Adapt for API changes
• Kafka integration Event sourcing integration with Kafka #2
• Enhance global authentication (JWT based) Enhance authentication #5
• Enhance API Gateway (e.g. file upload, WebSocket, fault-tolerance, performance)
• Enhance integration with Vert.x Config
• More functional and reactive code (with RxJava 2)
• Improve fault-tolerance architecture (e.g. Circuit Breaker for each microservice component) Improve health check and fault-tolerance mechanism #19
• Redesign for distributed transactions Redesign for distributed transactions #22
• Improve performance / throughput test
• Robust health check mechanism (Maybe integrate with Vert.x Health Check) Improve health check and fault-tolerance mechanism #19
• Database middleware layer Database middleware abstraction #26
• Initial design for service orchestration
• Distributed tracing
• Service mesh for cloud native architecture

This can be a huge change so it may take a long time to refactor.

Update (Feb 13, 2018): Redesigning...

[Romeh]

please let me know if you need any help or contribution , I have worked with vertx 3 recently with ignite in banking project , so i can help you as well

[sczyh30]

@Romeh Hi, can you offer some advice about the microservice architecture? (both bussiness layer and infrastructure layer is ok, e.g. the progress of checkout->payment->order changes, event sourcing with Kafka, API Gateway, module design and division, performance). Thanks a lot! :)

Any contributions are welcome!

[Romeh]

Sure , i am checking out the source code locally now and I will go through it into details to be able to provide useful feedback , i will focus on event sourcing part as i have implemented the CQRS pattern over vertx and ignite plus kafka for my last project , then we can go through the API gateway .

[Romeh]

@sczyh30 you have done an excellent job so far !! , first checks i think you have part in ur plan already , but to be more aligned with cloud mircroservices design , we need the following :

1. distributed configuration service for your services catalog that handle service configuration per target environment , configuration should be completely separated from the service code, vertx config can be a good start.

2. circuit breaker should cover service to service communications as well to avoid cascading domino error effect in case one of ur services is not up or down (hystrix can be used here with Rx java or reactive vertx calling style , as hystrix provide more control about the load and metrics than standard vertx CB)

3. health checks i think needed to be replaced by vertx health check and should be located into the service discovery , plus checking into the future if we can use one of service discovery services bridges supported by vertx .

4. we need to introduce API documentation for the REST verticals , ex swagger

5. separation of the common dependencies and having clean BOM managment

6. add performance logging via ELK stack in place already

7. use kafka or file beats as the logging events shipper instead of the default TCP shipper as it will be bad performer

8. API trust between services , using JWT could cover that

I will start now going through the event sourcing part now , and if you have any branch for that please let me know.

Thx a lot

[Romeh]

and Small question , if i need to start the same twice with different hosts as a matter of Scalability, the current service discovery implementation will support it ?

[eciuca]

Hi,

Is anyone still working on this? I will be building something on the current version but I am interested in the future 3.5 version too. I think I could even contribute myself but it looks to me like no one is active anymore on this repo.

Emanuel

[sczyh30]

@eciuca Hi, I might be free soon and I've been planning to redesign and refactor the whole architecture. The new version is expected to be based on Vert.x 3.5 and RxJava 2. I'll start working on it once I have time.

[rdinkel]

any update on the schedule? :) happy xmas

[blukhand]

hah, it seems that working in alibaba makes you busy. thank you for your blog about vertx.

[eciuca]

Meanwhile, I have started adding stuff to the BaseMicroserviceVerticle and RestMicroserviceVerticle in a private repo which I intend to make public. I added stuff like jax-rs annotation support, AWS hazelcast configuration and docker hazelcast configuration. I will maybe post an url here when I make them public. Unfortunately it won't be a fork but I'll specify that it's derived from here. I created even an archetype for a DDD oriented microservice using these.

[gvsbr74]

@sczyh30
Thanks for the fantastic stuff you created.
Are you planning to create a scala version of this java application? I am learning to use Vert.x by using scala and using your app as a base and I am encountering a strange issue with ServiceDiscovery getRecords or getRecord or getRecordFuture. All these methods fail with the below error. The same code works fine in Java.

ava.lang.NullPointerException
at io.vertx.servicediscovery.types.HttpLocationConverter.fromJson(HttpLocationConverter.java:30)
at io.vertx.servicediscovery.types.HttpLocation.(HttpLocation.java:64)
at io.vertx.servicediscovery.types.impl.HttpEndpointImpl$HttpEndpointReference.(HttpEndpointImpl.java:69)
at io.vertx.servicediscovery.types.impl.HttpEndpointImpl.get(HttpEndpointImpl.java:53)
at io.vertx.servicediscovery.impl.DiscoveryImpl.getReferenceWithConfiguration(DiscoveryImpl.java:162)
at io.vertx.servicediscovery.impl.DiscoveryImpl.getReference(DiscoveryImpl.java:157)
at io.vertx.scala.servicediscovery.ServiceDiscovery.getReference(ServiceDiscovery.scala:82)