Do not allow relationship between entities in different microservices

[deepu105]

Seems like currently, we allow relationships to be created between entities belonging to different microservices(Need to be double checked, but based on #8920 we allow it at least for the user entity)

IMO we shouldn't allow this as it has many architectural issues as well as practical tech issues

• This creates a tight coupling between microservices and beats the purpose
• We won't be able to generate an app working out of the box for these(without having to do weird workarounds fix relationship with user with oauth2 and microservices #9845)
  -The entity data gets duplicated across services and it's hard to keep them in sync.

@jhipster/developers WDYT?

If we decide not to allow this we need to do below

• Validate and restrict in entity sub gen & JDL
• Document some solutions to achieve those use cases
  • Use soft relationships(Add a userId filed on the entity and write business code to populate it may be using feign clients)
  • other solutions??

[atomfrede]

Totally agree we should not allow relations between entities of different services/apps.

[PierreBesson]

Personally I think it is a bad idea to allow relationship between different microservices. This would qualify as an instance of the "entity service anti pattern" : https://www.michaelnygard.com/blog/2017/12/the-entity-service-antipattern/

[Tcharl]

If we would allow that, the good pattern would be really complicated to be abstracted in JH:

Let's take a service A with an entity a
A gateway or another service B with b related to a.

When saving 'a', it should send a message in a queue catched by B which duplicates the data in it's own 'a' table...
Also, I'm totally inline with pbesson: even with this pattern, we are not business centric but model centric.

When I have to make that kind of antipattern with JH, I usually duplicate the same entity twice with a different name, do not generate entities on the consumer side and then make the join with a feign call in service.

In order to ease this procedure (if requested by users), it would be nice to introduce some new behaviour on entity generation:

• Introduce a 'noentity' or 'dtoonly' keyword or something like that, generating endpoints, dtos and service handling elements with collections instead of mappers and entities.
• Duplicating the entity when referenced in two microservices or gateways and guessing that the only one without the dtoonly option is the master one: so that we'll know where to generate the unique model class in the front-end.

[deepu105]

@atomfrede I hope you meant "not allow"?

[atomfrede]

Of course 😃 it's to warm for such things.

…

On Fri, Jun 21, 2019, 19:33 Deepu K Sasidharan ***@***.***> wrote: @atomfrede <https://github.com/atomfrede> I hope you meant "not allow"? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9949?email_source=notifications&email_token=AABRVCN2SJPJKEZ7PBHAQDTP3UGGXA5CNFSM4HZQ3DO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYJC6JY#issuecomment-504508199>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABRVCKH7D3IMHUPP3MIJT3P3UGGXANCNFSM4HZQ3DOQ> .

[mraible]

FWIW, our current OAuth + microservices implementation does not create relationships between microservices. Each microservice has its own User entity.

[pascalgrimaud]

Agree with you, I think we should "not allow"
Another solution would be to push Kafka support further
See #9567 (comment)

[MathieuAA]

The JDL already forbids this, so this is a big +1 from me.

[colameo]

I've a different opinion about this topic. From a model perspective it's not an anti pattern sharing entities between micro services as long as they are used only to "consume" data provided by another service. The bad practice is sharing entities and maintaining such code manually which is against the DRY principal and creates dependencies between APIs. But when code replication is made by a generator (which is one of the purpose of having a “model-based-generation” approach) it's not a bad practice at all. The generator could generate the code in a separate folder (generated/... or src-gen/... etc.) and a service can so consume data provided by another API without having a dependency on code level.

[laurentsorin]

Hi,
I have been working for a year on a jhipster monolithic project and now we are studying a microservices migration

I agree to distinguish between a 'A' consumer (dto + feign client) part and a 'B' provider part (which is the micoservice which manage the B entity itself) as an option provided by the generator. It should be useful in many business cases (cf @Tcharl post)
But the main constraint is that we cannot do "join" queries to filter/sort 'A' entity with a property which is in a 'B' entity, so we should provide another option to handle this use case.

3 solutions :

1. I saw solutions where each micro service has its own schema on the same database and data is shared via versioned view to limit the coupling (however i'm still wondering how to handle the cache properly...). Here you can do join queries, but away from microservices pattern.

2. Replicate 'B' model (or some of the 'B' properties) in 'A' data model perimeter and listen to a message from 'B' service for each 'B' update. 'A' microservice is responsible to update 'B' data in its own perimeter. For me its the best solution as data is always up to date and you can create queries as usual (sort, filter). This can be achieved via kafka as @pascalgrimaud suggest. But I don't know how to properly use kafka to handle this specific use case.

3. The A microservice has not the good perimeter (but when all your micro services need a referential data managed by a microservice how do you achieve this properly... ?)

As a conclusion I suggest that the generator provides 2 options :

• By direct communication (feign client) - Generator must provide a DTO version of the distant entity + the feign client generated (however this case we cannot create join queries)
• Via Message broker (solution 2 described above) - Generator must provide a persisted version of the distant entity and also preconfigured messages/channels/services to communicate

[pascalgrimaud]

As mentionned here #9287 (comment) :

• for my current project, with @nonomoho and the help from @hdurix, we implemented the sync data between 2 microservices, using Kafka
• so, we have a relationship between microservices
• it is not so complex :-)

So yes, it's possible with Kafka
I'll open a ticket soon to discuss about all steps, and improvement of Kafka support, then I'll take the lead on this

[MathieuAA]

@pascalgrimaud so, does this mean we're gonna allow relationshps as long as the architecture is event-driven and forbid when it is not?

[pascalgrimaud]

@MathieuAA : my idea is not really to allow relationships between microservices but to duplicate the common table between 2 microservices.
1 would be the master, so the producer
The other same tables would be the consumers, no Rest API, only consumer + service + repository + table

does this mean we're gonna allow relationshps as long as the architecture is event-driven and forbid when it is not?

Exact. Right now, I'm short on time, but I'll detail my idea in a specific ticket with Kafka soon, so we can discuss about it

[dean0bambin0]

Just coming here from #7424 From a bit of googling it looks like Debezium might be a good fit here with Kafka. It could be used not only to keep entities in sync across microservices but also for the Elasticsearch data replication which would fit nicely into jHipster too as it provides the elasticsearch layer. Maybe it's not good fit as i'm not super familiar with jHipster architecture but just said I'd throw it out there.

[PierreBesson]

I think especially Embedded Debezium could be a good fit for JHipster : https://debezium.io/docs/embedded/
ping @gunnarmorling who seem to be silently following this discussion 😄

[dean0bambin0]

Cool, i might give it a shot during the week.

[pascalgrimaud]

It's opened for too long and no more discussion on this ticket.

Then, to resume:

• it's not a good idea to allow relationship between entities in different microservices
• to have this use case, the advise is to use Kafka for that
• there is a very nice module https://github.com/fdelbrayelle/generator-jhipster-kafka maintained by @fdelbrayelle

So let's close this

[enesdogan27]

Definitely what i was looking. Thank you for amazing work guys.