Support for disabling any kind of entity mapping and caching

[francescortiz]

Is your feature request related to a problem? Please describe.
We use em.fork() compulsively for the following reasons:

1. We work with large ammounts of data.
2. In consequence, we carefully craft our queries, and we never query duplicate data in our processes.
3. The entity mapper is highly innefficent with large datasets. When we discovered this, I reduced a process from more than one day (i didn't let it finish) to just 15 minutes just by doing em.fork() on every loop iteration.
4. Also, according to the docs, em.upsert has different behavior depending on wether the entity mapper has seen the entity or not, which makes it hard to predict how it willl behave. Doc quoted below:

https://mikro-orm.io/docs/entity-manager
If the entity is already present in current context, there won't be any queries - instead, the entity data will be assigned and an explicit flush will be required for those changes to be persisted.

Describe the solution you'd like
It would be great if we could totally disable the entitiy mapper and/or any other kind of peristence managed by mikro-orm beyond the operation we are executing.

Describe alternatives you've considered
For now we have decided to use query builders as much as possible because they work in a more comprehensive way, while still giving use the power of relational mapping of the database (code is easier and safer to write).

Additional context

• We use transactions when we need atomicity.
• We love the relational mapping of queries and results that mikro-orm gives us, so we don't consider using alternatives.

[B4nan]

This is very abstract, you will need to be much more concrete if we should take this forward. Let's start with what you mean by "entity mapper", you mean the hydration layer?

The entity mapper is highly innefficent with large datasets. When we discovered this, I reduced a process from more than one day (i didn't let it finish) to just 15 minutes just by doing em.fork() on every loop iteration.

If this is a performance problem, you should start with a complete reproduction for that, so I can understand what problem are you dealing with.

edit: moving to discussions before we settle on what should be done

[B4nan]

Btw I guess you know about the disableIdentityMap option?

https://mikro-orm.io/docs/entity-manager#disabling-identity-map-and-change-set-tracking

[francescortiz]

Thanks for the guidance. I didn't know exactly where to point to int his topic. I didn't know about the disableIdentityMap and it looks exactly as the behavior that I look for. Is there a way to make it the default? Thank you

[B4nan]

Now you can: ef32b14

[francescortiz]

Wow, this is great. Thank you very much!

[francescortiz]

After revieweing the source code a bit, I am wondering that what we need might be em.clear(); because I don't know how disableIdentityMap: true affects transactional entity managers. Is disableIdentityMap: true making queries inside the transactional em's transaction or is it making the queries outside the transaction? Thank you very much

[B4nan]

The context is cleared automatically when you use this flag:

https://github.com/mikro-orm/mikro-orm/blob/master/packages/core/src/EntityManager.ts#L152-L159

This option is about reading, so em.find and similar, and that itself is not automatically wrapped in any transaction.