Incrementally add mappings without "invalidating" EntityDescriptorsCache #4269
Replies: 2 comments 3 replies
-
That's an interesting issue. Do you really have perfomance issues without entity descriptors pre-generation? In any case, with your approach I would suggest to try two-phase startup and see how it works for you:
I would like to look into this issue more, so I would appreciate more details on your model:
|
Beta Was this translation helpful? Give feedback.
-
Hi MaceWindu, let me first apologize for my delayed response to your questions. I've been preoccupied with some other urgent matters. I would also like to thank you for your help, I greatly appreaciate that.
Yes, because when certain processes are executed for the first time, they have to do a bunch of distinct queries (with different POCO classes). The whole process takes about 5 seconds and approximately 1.5 second gets consumed by various linq2db methods:
I am also adding a dotTrace screenshot which shows that when I exclude the above-mentioned linq2db methods, the consumed time drops by about 1.5s: It is true that 1.5s is less than a half of the process which I need to optimize, but I have to start somewhere. I am also not sure if 1.5s all comes from entity descriptor generation or also some other stuff. Even if I solve all other things and I am left with 1.5 seconds from linq2db, it is still too much. I would like to reduce the reflection and compiling time as much as possible by performing it during application startup phase. I hope you see my point ...
There are 124 classes, each class has in average 33 properties.
We don't configure linq2db with associations.
I didn't think of this one, but it makes sense what you say, thanks for the hint!
I actually came up with an interesting solution which seems to work nicely and makes it possible to parallelize also the registration of mappings. I created my own MappingAttribute[] IMetadataReader.GetAttributes(Type type)
{
lock (_lockObject)
{
DefineMappingsIfNecessary(type);
return GetCachedMappings(type);
}
} I will also test your improvements once they are released. So, I would say that this case is solved. Thanks again for your assistance and best regards! Boris |
Beta Was this translation helpful? Give feedback.
-
Hi,
I have a rather exotic performance issue with linq2db version 5.2.2. I am working on a large ERP system where we have about 150 POCO classes. If I get it correctly it is best practice to create a single MappingSchema (or a few of them) and fill it with appropriate mappings before executing the first linq2db query. I use fluent API to do that (via
EntityMappingBuilder<>
class).Here comes the first issue: our method which defines the mappings for a single POCO class takes about 300 milliseconds to execute for a typical POCO class, which is a bit slow. I profiled a release build with DotTrace. This screenshot clearly shows that most of the time is lost in various linq2db internal methods (e.g.,
GetMemberEx
,GetMappingAttributes
, etc):I also realized that the garbage collector is firing up frequently while defining the mappings, which is in a way reasonable (plenty of short lived objects being allocated). This all leads me to the conclusion that I simply cannot speed up significantly the initial mapping configuration. With about 150 POCO classes it looks like we are destined to wait more than 30 seconds before the mapping configuration is done with its work.
Our ERP system is a desktop application and customers want to have the startup as fast as possible. That's why I tried to parallelize the mapping configuration, i.e., split it to chunks of POCO classes and define on parallel threads the mappings for each chunk.
This leads to my main problem. We use a single MappingSchema through the whole application. It is supposed to hold a cache of all
EntityDescriptor
instances (one EntityDescriptor for one POCO class). I forgot to say before that I also force the building ofEntityDescriptor
instances on a parallel thread during system startup, since the creation of those classes is also slow (reflection, delegate compiling). But as soon as I call theEntityMappingBuilder<>.Build
method to "flush" the mappings of one POCO class, theMappingSchema.ConfigurationID
is reset through this call chain:This basically means that all of the
EntityDescriptor
instances built so far and cached in theEntityDescriptorsCache
structure are no longer accessible (the changedMappingSchema.ConfigurationID
is part of the cache key).Long story short: is there a way to incrementally fill the mappings into
MappingSchema
instance while at the same time keeping itsEntityDescriptorsCache
structure intact?Thanks a lot for any kind of help / suggestions. Best regards,
Boris Turk
Beta Was this translation helpful? Give feedback.
All reactions