Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EntityLite #13

Closed
wants to merge 1 commit into from
Closed

Add EntityLite #13

wants to merge 1 commit into from

Conversation

jesuslpm
Copy link

EntityLite is the fastest

@FransBouma
Copy link
Owner

Thanks Jesus. I'll look at it shortly, I have time later this week to check out your additions so I can merge them :)

@FransBouma
Copy link
Owner

It's not the fastest:

Averaged total results per framework. Fastest and slowest result omited

Non-change tracking fetches, set fetches (10 runs), no caching

Entity Framework v6.0.0.0 (v6.0.21211.0) : 468,50ms. Enumeration average: 3,00ms
EntityLite 1.0.10-Beta : 497,13ms. Enumeration average: 2,00ms
Handcoded materializer using DbDataReader : 499,25ms. Enumeration average: 2,00ms
PetaPoco Fast v4.0.3 : 500,00ms. Enumeration average: 2,00ms
PetaPoco v4.0.3 : 513,25ms. Enumeration average: 2,75ms
Dapper : 522,75ms. Enumeration average: 2,88ms
Linq to Sql v4.0.0.0 (v4.0.30319.18408) : 563,50ms. Enumeration average: 2,88ms
ServiceStack OrmLite v4.0.5.0 (v4.0.5.0) : 611,13ms. Enumeration average: 2,00ms
LLBLGen Pro v4.1.0.0 (v4.1.13.1213), typed view : 733,13ms. Enumeration average: 5,13ms
Oak.DynamicDb using dynamic Dto class : 1.222,25ms. Enumeration average: 208,13ms

Change tracking fetches, set fetches (10 runs), no caching

DataTable, using DbDataAdapter : 530,88ms. Enumeration average: 51,13ms
Linq to Sql v4.0.0.0 (v4.0.30319.18408) : 629,50ms. Enumeration average: 3,00ms
LLBLGen Pro v4.1.0.0 (v4.1.13.1213) : 707,00ms. Enumeration average: 11,00ms
Oak.DynamicDb using typed dynamic class : 1.237,00ms. Enumeration average: 1.675,00ms
Entity Framework v6.0.0.0 (v6.0.21211.0) : 3.025,63ms. Enumeration average: 3,00ms
NHibernate v3.3.1.4000 (v3.3.3.4001) : 4.042,63ms. Enumeration average: 4,00ms

Non-change tracking individual fetches (100 elements, 10 runs), no caching

EntityLite 1.0.10-Beta : 0,55ms per individual fetch
Dapper : 0,60ms per individual fetch
Oak.DynamicDb using dynamic Dto class : 0,62ms per individual fetch
ServiceStack OrmLite v4.0.5.0 (v4.0.5.0) : 0,69ms per individual fetch
Handcoded materializer using DbDataReader : 0,81ms per individual fetch
PetaPoco Fast v4.0.3 : 1,04ms per individual fetch
LLBLGen Pro v4.1.0.0 (v4.1.13.1213), typed view : 1,61ms per individual fetch
Entity Framework v6.0.0.0 (v6.0.21211.0) : 1,86ms per individual fetch
Linq to Sql v4.0.0.0 (v4.0.30319.18408) : 2,54ms per individual fetch
PetaPoco v4.0.3 : 3,83ms per individual fetch

Change tracking individual fetches (100 elements, 10 runs), no caching

Oak.DynamicDb using typed dynamic class : 0,66ms per individual fetch
DataTable, using DbDataAdapter : 0,71ms per individual fetch
LLBLGen Pro v4.1.0.0 (v4.1.13.1213) : 1,28ms per individual fetch
NHibernate v3.3.1.4000 (v3.3.3.4001) : 1,38ms per individual fetch
Entity Framework v6.0.0.0 (v6.0.21211.0) : 2,61ms per individual fetch
Linq to Sql v4.0.0.0 (v4.0.30319.18408) : 2,67ms per individual fetch

Change tracking fetches, set fetches (10 runs), caching

LLBLGen Pro v4.1.0.0 (v4.1.13.1213) : 213,25ms. Enumeration average: 7,13ms

Anyway, there are a couple of problems I have with your contribution. The first is that the benchmark is overly crowded with microORMs and I want to keep those at a minimum. The second is that your framework is barely used elsewhere.

There's also a problem with the referenced assemblies (there's a version conflict, but vs.net doesn't say which one :( ), although I find that minor, compared to the other two problems.

So it's not the code, which is fine, it's that I don't want the benchmark to become a test with tools which are not used by many. Therefore I'm going to decline your PR. I hope you don't take it the wrong way. It is quite fast, though it is in the same range as other micro ORMs, so the IL generated at runtime likely won't differ much between them. ;)

@FransBouma FransBouma closed this Jan 31, 2014
@jesuslpm
Copy link
Author

Hi Frans,

Thank you for spending your time on taking a look on my pull request.

I hoped you accepted the pull request. EntityLite is new, still in beta, and threrefore nobody knows it yet. It would have been great that EntityLite was in your benchmark because more people would know it.

Thank you anyway

The first time I executed your benchmak, EntityLite was the fastest. Subsequent benchmaks, wasn't, sometimes was the fastest, sometimes was not. Your benchmark not always shows the same results.

Please, can you tell me what is the problem with the referenced assemblies? I'm not aware of it. Could you help me to solve it?

Thank you again.

@jesuslpm
Copy link
Author

I just wanted to add a comment on these numbers:

Entity Framework v6.0.0.0 (v6.0.21211.0) : 468,50ms. Enumeration average: 3,00ms
EntityLite 1.0.10-Beta : 497,13ms. Enumeration average: 2,00ms
Handcoded materializer using DbDataReader : 499,25ms. Enumeration average: 2,00ms

How can Handcoded materializer using DbDataReader be slower than EntityFramework and EntityLite?

And these:
EntityLite 1.0.10-Beta : 0,55ms per individual fetch
Dapper : 0,60ms per individual fetch
Oak.DynamicDb using dynamic Dto class : 0,62ms per individual fetch
ServiceStack OrmLite v4.0.5.0 (v4.0.5.0) : 0,69ms per individual fetch
Handcoded materializer using DbDataReader : 0,81ms per individual fetch

How can Handcoded materializer using DbDataReader be on the 5th position?

It seems that something is wrong

@jesuslpm
Copy link
Author

I also would like to thank you for including the benchmark results in your comments. Although I liked to see Handcoded materializer using DbDataReader the fastest as is actually is.

I'm writing a CodeProject article introducing EntityLite that I'm going to send to CodeProject next weeks.

May I reference these comments on the article?

@FransBouma
Copy link
Owner

The first time I executed your benchmak, EntityLite was the fastest. Subsequent benchmaks, wasn't, sometimes was the fastest, sometimes was not. Your benchmark not always shows the same results.

It might differ a lot if you have the db locally as the db cpu time is then not available for the ORM code. It also might be due to a garbage collect or other things. The differences are very minor between runs though.

The reference problem is something I couldn't solve, it's the error you get when one project in the solution references assembly X v1.0.0.0 and another project references X v1.0.1.0 or something, so a different version for the same assembly X. VS.NET doesn't tell which one it is, sadly.

How can Handcoded materializer using DbDataReader be slower than EntityFramework and EntityLite?

It's a tiny bit slower than entitylight (3ms), which might be due to the IL you generate is perhaps a bit more efficient, e.g. your loop might be more efficient than the loop generated by the C# compiler. EF generates other code which is more efficient. I don't know what trick they're using, haven't looked into their codebase yet for that.

The handcoded code is close to what people would write when they would write raw ado.net code, there might be some inefficiency in one or two of the statements, but it's rather minor.

May I reference these comments on the article?

sure, no problem :) Good luck with the article. :)

@jesuslpm
Copy link
Author

your loop might be more efficient than the loop generated by the C# compiler .

The handcoded code is close to what people would write when they would write raw ado.net code, there might be some inefficiency in one or two of the statements, but it's rather minor.

The IL code I generate is not more efficient than the code generated by the C#, and the handcoded materializer code is more efficient than EntityLite. In EntityLite I call IsDBNull for each field, handcoded materializer doesn't, the IL generated DynamicMethod is just for create an object and setting its properties from the datareader, the datareader looping code is outside the DynamicMethod. The DynamicMethod is cached, but there is an overhead to locate it in the cache, mostly because the cached method must conform with the datareader schema.

@FransBouma
Copy link
Owner

  your loop might be more efficient than the loop generated by the C# compiler <<
  The handcoded code is close to what people would write when they would write raw ado.net

code, there might be some inefficiency in one or two of the statements, but it's rather minor. <<

The IL code I generate is not more efficient than the code generated by the C#, and the handcoded
materializer code is more efficient than EntityLite. In EntityLite I call IsDBNull for each field,
handcoded materializer doesn't

I do call IsDBNull :) https://github.com/FransBouma/RawDataAccessBencher/blob/master/RawBencher/Benchers/HandCodedBencher.cs

In previous versions I didn't but profiling it suggested that the boxing/unboxing going on in GetValues() was very performance heavy so I switched to IsDBNull, although IsDBNull is not a great method as it's slower than comparing to DBNull.Value. However to avoid boxing/unboxing, it's not feasible to use DBNull.Value comparisons, so it's overall the better choice. 

, the IL generated DynamicMethod is just for create an object and
setting its properties from the datareader, the datareader looping code is outside the DynamicMethod.
The DynamicMethod is cached, but there is an overhead to locate it in the cache, mostly because the
cached method must conform with the datareader schema.

but that's just once per query, right? It can loop over the results with the same method, so the overall time delay to look things up is minimal compared to the time lost looping over the results. I still don't know why the EF non-changetracking is faster, as they clearly skip statements which are in the handwritten code which they don't (always) execute in their code. Profiling their code doesn't show what's so special about it. the only thing I can think of is that somewhere there's still boxing going on in the handwritten bencher and not in theirs...

    FB

@FransBouma
Copy link
Owner

I found the reason why it's faster: there were no FK fields present in the EF code. When adding them, their materializer isn't magically faster, but actually of normal performance and slower than the handwritten one.

@jesuslpm
Copy link
Author

Sorry from come back here so late.

I do call IsDBNull <<

But only for nullable columns. I call it for every column.

but that's just once per query, right? <<

But in the "Non-change tracking individual fetches" benchmark, which shows EntityLite faster than hand-coded materializer, It is for every row as well, since the query returns just one row.

I believe there should be something that is causing different results on each benchmark run, such as other processes running on the machine.

I found the reason why it's faster <<

I'm glad to hear that.

@jesuslpm
Copy link
Author

I also would like to add a last comment

It's not the fastest:

Based on your results It is.

It's the fastest in Non-change tracking individual fetches, and the second in Non-change tracking fetches, set fetches. But Entity Framework which is the winner in Non-change tracking fetches, set feches, it is the 8th in Non-change tracking fetches. Therefore EntityLite is the fastest in non-change tracking fetches. EntityLite doesn't support change tracking.

@jesuslpm
Copy link
Author

jesuslpm commented Apr 9, 2014

I published the CodeProject article where I referenced your blog post and mentioned your ORM:
http://www.codeproject.com/Articles/753796/i-nercya-EntityLite-A-Lightweight-Database-First-M
I would love to hear your comments.

@FransBouma
Copy link
Owner

Article looks nice, lots of info! Looks like a good showcase for your framework :)

@jesuslpm
Copy link
Author

jesuslpm commented Apr 9, 2014

Thanks Frans :-). I think It's very dense, it's not very easy to read. I'm not going to ask my girlfriend to read it.

@FransBouma
Copy link
Owner

Haha :) nah, it has a lot of info, so that tends to get more dense than light articles which only describe one thing, but for people interested in the tool it covers most of the things to describe so why limit it :)

FB

Thanks Frans :-). I think It's very dense, it's not very easy to read. I'm not going to ask my
girlfriend to read it.


Reply to this email directly or view it on GitHub
#13 (comment) .
<https://github.com/notifications/beacon/3628530__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcx
MjY4MDg4OSwiZGF0YSI6eyJpZCI6MjM0OTc3Mzl9fQ==--4c7c31da76331120533d9b8b77842d51dbf498e1.gif>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants