A better assessment of Oak.DynamicDb

[amirrajan]

For the FetchSalesOrderHeaderOakDynamicDb measurement, I moved the stop watch to include the dynamic binding of Oak's DynamicModel. This is a much more accurate picture of the net time it takes to:

• retrieve the records
• attach methods associated with change tracking
• attach methods associated with validation
• attach methods associated with retrieving relationships

The numbers that you are seeing is accurate. The lazy binding of the dynamic methods happens the first time a dynamic property is accessed, but as the benchmarks show, this process is slow.

I've added another method FetchSalesOrderHeaderOakDynamicDbDto that hydrates a dynamic object without change tracking. This option is supported by the framework. And can be seen here: https://github.com/amirrajan/RawDataAccessBencher/compare/oak-dynamicdb?expand=1#diff-880238b8ae29809c5eb1471325d8f291R18

[FransBouma]

Are you sure you want this? The total time will be 2x the time of NHibernate, which is already ridiculously slow.

[amirrajan]

I stated earlier that Oak's dynamic model is doing change tracking, validations and associations. I can do a pull request that shows just change tracking. I think on twitter we exchanged a few tweets stating you didn't think that was fair. I do believe if I do just change tracking, it would be a better apples to apples comparison. But I'll leave that up to you.

[amirrajan]

Would it be okay if I show just change tracking?

[FransBouma]

I think how it is now, is OK with me. The thing is that there are always things postponed till after the entity is materialized and actually used. For example, collections for related entities are created in LLBLGen Pro when the property is accessed for the first time (for memory reasons). Other frameworks might create them with every entity, which might be a bit slower (not much, but still). The very slow enumeration is perhaps solely the fact it's a dynamic type and therefore very slow. It's not slow because the rows are read when the collection is enumerated. If that would be the case, it should be taken into account.

It's of course a thing of note. I refactored the code currently (not committed yet) which allows you to emit the enumeration time as well, to show whether that's fast or not.

So my question to you is: the enumeration over the collection, is it solely slow because dynamic types suck so much or is it doing a lot under the hood which is otherwise done by other frameworks when the entity is materialized? (like actually reading the row, e.g. you previously stored the row in the entity object and you now materialize the entity with it, guessing here).

[amirrajan]

Here are the results I've gotten (with current pull request code). Are you sure it's 2x slower? I'm using SqlServer2008r2 express

https://gist.github.com/amirrajan/7913390

[FransBouma]

I did a simple 7500+1200=~2x4300 calculation, (as it takes 7500 ms to enumerate the 31K entities). I'm not sure whether the enumeration should be inside the timing... It should then be the same for all frameworks, even if that's just 2-10ms for the rest.

I'll run dottrace on it to see why the enumeration is so slow.

[amirrajan]

So my question to you is: the enumeration over the collection, is it solely slow because dynamic types suck so much or is it doing a lot under the hood which is otherwise done by other frameworks when the entity is materialized?

It is slow because I'm doing a lot under the hood, dynamic types are slower (but I do think that's negligible). It's doing 4 things under the hood. Attaching introspection capabilities, attaching change tracking, attaching validations and attaching association capabilities.

I think it would be a better apples to apples comparison if I only attach introspection and change tracking capabilities (as opposed to additionally attaching validation and associations). What do you think?

The thing is that there are always things postponed till after the entity is materialized and actually used

Yes the DynamicModel type is postponing the materialization of additional features.

For example, collections for related entities are created in LLBLGen Pro when the property is accessed for the first time (for memory reasons).

Ah, good to know. DynamicModel defers the hydration of introspection, change tracking, association, and validation features until a dynamic property is accessed.

The very slow enumeration is perhaps solely the fact it's a dynamic type and therefore very slow

No, the slowness is because of the deferred hydration (not because it's dynamic).

If that would be the case, it should be taken into account.

If I'm reading that right, that is not the case.

It's of course a thing of note. I refactored the code currently (not committed yet) which allows you to emit the enumeration time as well, to show whether that's fast or not.

I think that's a good idea. It will definitely give a better picture of what's going on.

[FransBouma]

It's a bug in your code. :) Gemini.cs, line 667, that's the line which takes 67% of the time, due to the Contains. It will call down to the Equals on the element, which is called.... 6.8 million times. See screenshot:

I've saved the snapshot of dottrace, it's 40MB.

[amirrajan]

That's incredible! Let me fix that and see what the numbers look like. I'll submit another pull request with the updated bits.