Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update benchmarks with latest results. #2016

Merged
merged 2 commits into from
Dec 18, 2023
Merged

Update benchmarks with latest results. #2016

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a752a0c
Update benchmarks with latest results.
kant2002 Dec 18, 2023
2a2f126
Address PR feedback
kant2002 Dec 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions Directory.Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
<PackageVersion Include="Iesi.Collections" Version="4.0.5" />
<PackageVersion Include="linq2db.SqlServer" Version="3.7.0" />
<PackageVersion Include="Microsoft.Data.SqlClient" Version="5.1.1" />
<PackageVersion Include="Microsoft.Data.Sqlite" Version="7.0.10" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.SqlServer" Version="5.0.0" />
<PackageVersion Include="Microsoft.Data.Sqlite" Version="8.0.0" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.SqlServer" Version="8.0.0" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.7.1" />
<PackageVersion Include="Mighty" Version="3.2.0" />
<PackageVersion Include="MySqlConnector" Version="2.2.7" />
Expand Down
99 changes: 54 additions & 45 deletions Readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,56 +156,65 @@ A key feature of Dapper is performance. The following metrics show how long it t

The benchmarks can be found in [Dapper.Tests.Performance](https://github.com/DapperLib/Dapper/tree/main/benchmarks/Dapper.Tests.Performance) (contributions welcome!) and can be run via:
```bash
dotnet run --project .\benchmarks\Dapper.Tests.Performance\ -c Release -f netcoreapp3.1 -- -f * --join
dotnet run --project .\benchmarks\Dapper.Tests.Performance\ -c Release -f net8.0 -- -f * --join
```
Output from the latest run is:
``` ini
BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19041.208 (2004/?/20H1)
Intel Core i7-7700HQ CPU 2.80GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.1.201
[Host] : .NET Core 3.1.3 (CoreCLR 4.700.20.11803, CoreFX 4.700.20.12001), X64 RyuJIT
ShortRun : .NET Core 3.1.3 (CoreCLR 4.700.20.11803, CoreFX 4.700.20.12001), X64 RyuJIT
BenchmarkDotNet v0.13.7, Windows 10 (10.0.19045.3693/22H2/2022Update)
Intel Core i7-3630QM CPU 2.40GHz (Ivy Bridge), 1 CPU, 8 logical and 4 physical cores
.NET SDK 8.0.100
[Host] : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX
ShortRun : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX

```
| ORM | Method | Return | Mean | StdDev | Error | Gen 0 | Gen 1 | Gen 2 | Allocated |
|--------------- |------------------------------ |-------- |----------:|----------:|----------:|--------:|-------:|-------:|----------:|
| Belgrade | ExecuteReader | Post | 94.46 μs | 8.115 μs | 12.268 μs | 1.7500 | 0.5000 | - | 8.42 KB |
| Hand Coded | DataTable | dynamic | 105.43 μs | 0.998 μs | 1.508 μs | 3.0000 | - | - | 9.37 KB |
| Hand Coded | SqlCommand | Post | 106.58 μs | 1.191 μs | 1.801 μs | 1.5000 | 0.7500 | 0.1250 | 7.42 KB |
| Dapper | QueryFirstOrDefault&lt;dynamic&gt; | dynamic | 119.52 μs | 1.320 μs | 2.219 μs | 3.6250 | - | - | 11.39 KB |
| Dapper | &#39;Query&lt;dynamic&gt; (buffered)&#39; | dynamic | 119.93 μs | 1.943 μs | 2.937 μs | 2.3750 | 1.0000 | 0.2500 | 11.73 KB |
| Massive | &#39;Query (dynamic)&#39; | dynamic | 120.31 μs | 1.340 μs | 2.252 μs | 2.2500 | 1.0000 | 0.1250 | 12.07 KB |
| Dapper | QueryFirstOrDefault&lt;T&gt; | Post | 121.57 μs | 1.564 μs | 2.364 μs | 1.7500 | 0.7500 | - | 11.35 KB |
| Dapper | &#39;Query&lt;T&gt; (buffered)&#39; | Post | 121.67 μs | 2.913 μs | 4.403 μs | 1.8750 | 0.8750 | - | 11.65 KB |
| PetaPoco | &#39;Fetch&lt;T&gt; (Fast)&#39; | Post | 124.91 μs | 4.015 μs | 6.747 μs | 2.0000 | 1.0000 | - | 11.5 KB |
| Mighty | Query&lt;T&gt; | Post | 125.23 μs | 2.932 μs | 4.433 μs | 2.2500 | 1.0000 | - | 12.6 KB |
| LINQ to DB | Query&lt;T&gt; | Post | 125.76 μs | 2.038 μs | 3.081 μs | 2.2500 | 0.7500 | 0.2500 | 10.62 KB |
| PetaPoco | Fetch&lt;T&gt; | Post | 127.48 μs | 4.283 μs | 6.475 μs | 2.0000 | 1.0000 | - | 12.18 KB |
| LINQ to DB | &#39;First (Compiled)&#39; | Post | 128.89 μs | 2.627 μs | 3.971 μs | 2.5000 | 0.7500 | - | 10.92 KB |
| Mighty | Query&lt;dynamic&gt; | dynamic | 129.20 μs | 2.577 μs | 3.896 μs | 2.0000 | 1.0000 | - | 12.43 KB |
| Mighty | SingleFromQuery&lt;T&gt; | Post | 129.41 μs | 2.094 μs | 3.166 μs | 2.2500 | 1.0000 | - | 12.6 KB |
| Mighty | SingleFromQuery&lt;dynamic&gt; | dynamic | 130.59 μs | 2.432 μs | 3.677 μs | 2.0000 | 1.0000 | - | 12.43 KB |
| Dapper | &#39;Contrib Get&lt;T&gt;&#39; | Post | 134.74 μs | 1.816 μs | 2.746 μs | 2.5000 | 1.0000 | 0.2500 | 12.29 KB |
| ServiceStack | SingleById&lt;T&gt; | Post | 135.01 μs | 1.213 μs | 2.320 μs | 3.0000 | 1.0000 | 0.2500 | 15.27 KB |
| LINQ to DB | First | Post | 151.87 μs | 3.826 μs | 5.784 μs | 3.0000 | 1.0000 | 0.2500 | 13.97 KB |
| EF 6 | SqlQuery | Post | 171.00 μs | 1.460 μs | 2.791 μs | 3.7500 | 1.0000 | - | 23.67 KB |
| DevExpress.XPO | GetObjectByKey&lt;T&gt; | Post | 172.36 μs | 3.758 μs | 5.681 μs | 5.5000 | 1.2500 | - | 29.06 KB |
| Dapper | &#39;Query&lt;T&gt; (unbuffered)&#39; | Post | 174.40 μs | 3.296 μs | 4.983 μs | 2.0000 | 1.0000 | - | 11.77 KB |
| Dapper | &#39;Query&lt;dynamic&gt; (unbuffered)&#39; | dynamic | 174.45 μs | 1.988 μs | 3.340 μs | 2.0000 | 1.0000 | - | 11.81 KB |
| DevExpress.XPO | FindObject&lt;T&gt; | Post | 181.76 μs | 5.554 μs | 9.333 μs | 8.0000 | - | - | 27.15 KB |
| DevExpress.XPO | Query&lt;T&gt; | Post | 189.81 μs | 4.187 μs | 8.004 μs | 10.0000 | - | - | 31.61 KB |
| EF Core | &#39;First (Compiled)&#39; | Post | 199.72 μs | 3.983 μs | 7.616 μs | 4.5000 | - | - | 13.8 KB |
| NHibernate | Get&lt;T&gt; | Post | 248.71 μs | 6.604 μs | 11.098 μs | 5.0000 | 1.0000 | - | 29.79 KB |
| EF Core | First | Post | 253.20 μs | 3.033 μs | 5.097 μs | 5.5000 | - | - | 17.7 KB |
| NHibernate | HQL | Post | 258.70 μs | 11.716 μs | 17.712 μs | 5.0000 | 1.0000 | - | 32.1 KB |
| EF Core | SqlQuery | Post | 268.89 μs | 19.349 μs | 32.516 μs | 6.0000 | - | - | 18.5 KB |
| EF 6 | First | Post | 278.46 μs | 12.094 μs | 18.284 μs | 13.5000 | - | - | 44.18 KB |
| EF Core | &#39;First (No Tracking)&#39; | Post | 280.88 μs | 8.192 μs | 13.765 μs | 3.0000 | 0.5000 | - | 19.38 KB |
| NHibernate | Criteria | Post | 304.90 μs | 2.232 μs | 4.267 μs | 11.0000 | 1.0000 | - | 60.29 KB |
| EF 6 | &#39;First (No Tracking)&#39; | Post | 316.55 μs | 7.667 μs | 11.592 μs | 8.5000 | 1.0000 | - | 50.95 KB |
| NHibernate | SQL | Post | 335.41 μs | 3.111 μs | 4.703 μs | 19.0000 | 1.0000 | - | 78.86 KB |
| NHibernate | LINQ | Post | 807.79 μs | 27.207 μs | 45.719 μs | 8.0000 | 2.0000 | - | 53.65 KB |

| ORM | Method | Return | Mean | StdDev | Error | Gen0 | Gen1 | Gen2 | Allocated |
|-------------------- |------------------------------- |------------- |---------:|----------:|----------:|--------:|-------:|-------:|----------:|
| Dapper cache impact | QueryFirstParameters_Cache | Void | 104.0 us | 0.64 us | 0.97 us | 0.8750 | - | - | 2824 B |
| Dapper cache impact | ExecuteParameters_Cache | Void | 106.6 us | 1.24 us | 2.08 us | 0.6250 | - | - | 2184 B |
| Hand Coded | SqlCommand | Post | 129.1 us | 0.60 us | 0.91 us | 1.2500 | 1.0000 | - | 7584 B |
| Hand Coded | DataTable | dynamic | 133.4 us | 1.55 us | 2.34 us | 3.0000 | - | - | 9576 B |
| SqlMarshal | SqlCommand | Post | 141.9 us | 2.56 us | 3.88 us | 1.7500 | 1.5000 | - | 11529 B |
| LINQ to DB | Query<T> | Post | 144.3 us | 2.14 us | 3.24 us | 1.7500 | 1.2500 | - | 10904 B |
| Mighty | SingleFromQuery<T> | Post | 144.5 us | 1.30 us | 2.19 us | 2.2500 | 1.2500 | - | 12201 B |
| Mighty | SingleFromQuery<dynamic> | dynamic | 145.5 us | 2.50 us | 3.79 us | 2.0000 | 1.7500 | - | 12710 B |
| Mighty | Query<dynamic> | dynamic | 146.3 us | 1.38 us | 2.32 us | 2.0000 | 1.7500 | - | 12710 B |
| Dapper | 'Query<dynamic> (buffered)' | dynamic | 148.4 us | 1.19 us | 2.00 us | 2.0000 | 1.5000 | - | 11968 B |
| Mighty | Query<T> | Post | 149.1 us | 2.63 us | 3.98 us | 1.7500 | 1.5000 | - | 12201 B |
| LINQ to DB | 'First (Compiled)' | Post | 150.1 us | 1.97 us | 3.31 us | 2.0000 | 1.5000 | - | 12128 B |
| Dapper | QueryFirstOrDefault<dynamic> | dynamic | 151.0 us | 5.48 us | 8.29 us | 3.5000 | - | - | 11648 B |
| ServiceStack | SingleById<T> | Post | 154.1 us | 2.24 us | 3.38 us | 2.5000 | 1.2500 | 0.2500 | 15248 B |
| Dapper | QueryFirstOrDefault<T> | Post | 154.9 us | 4.10 us | 6.20 us | 1.7500 | 1.5000 | - | 11608 B |
| Massive | 'Query (dynamic)' | dynamic | 156.9 us | 4.05 us | 6.12 us | 2.0000 | 1.5000 | - | 12342 B |
| Norm | 'Read<> (tuples)' | ValueTuple`8 | 157.2 us | 3.16 us | 5.31 us | 2.0000 | 1.7500 | - | 12745 B |
| Dapper | 'Contrib Get<T>' | Post | 157.9 us | 3.14 us | 4.75 us | 2.0000 | 1.5000 | - | 12440 B |
| Dapper | 'Query<T> (buffered)' | Post | 158.0 us | 4.28 us | 6.47 us | 2.0000 | 1.0000 | - | 11888 B |
| Norm | 'Read<()> (named tuples)' | ValueTuple`8 | 163.3 us | 1.03 us | 1.56 us | 2.2500 | 2.0000 | 1.2500 | 14562 B |
| Dapper cache impact | ExecuteNoParameters_Cache | Void | 177.0 us | 6.37 us | 9.64 us | - | - | - | 760 B |
| Dapper cache impact | QueryFirstNoParameters_NoCache | Void | 177.6 us | 2.62 us | 3.96 us | 0.5000 | - | - | 1576 B |
| DevExpress.XPO | FindObject<T> | Post | 182.0 us | 3.41 us | 5.73 us | 8.5000 | - | - | 28099 B |
| LINQ to DB | First | Post | 183.3 us | 5.28 us | 7.98 us | 2.5000 | 1.5000 | - | 14041 B |
| EF 6 | SqlQuery | Post | 185.9 us | 3.08 us | 4.66 us | 4.5000 | 0.5000 | - | 24210 B |
| Dapper cache impact | ExecuteNoParameters_NoCache | Void | 187.5 us | 3.17 us | 4.80 us | - | - | - | 760 B |
| Dapper cache impact | QueryFirstNoParameters_Cache | Void | 187.6 us | 7.54 us | 11.40 us | - | - | - | 1520 B |
| DevExpress.XPO | GetObjectByKey<T> | Post | 188.0 us | 2.66 us | 4.46 us | 4.5000 | 1.0000 | - | 30114 B |
| Norm | 'Read<> (class)' | Post | 199.9 us | 1.98 us | 3.79 us | 3.0000 | 0.5000 | - | 17579 B |
| Dapper | 'Query<dynamic> (unbuffered)' | dynamic | 204.3 us | 2.79 us | 4.22 us | 1.5000 | 1.0000 | - | 12048 B |
| DevExpress.XPO | Query<T> | Post | 213.9 us | 7.82 us | 13.15 us | 10.0000 | - | - | 32083 B |
| Dapper | 'Query<T> (unbuffered)' | Post | 217.6 us | 5.62 us | 8.50 us | 1.5000 | 1.0000 | - | 12008 B |
| EF Core | 'First (Compiled)' | Post | 228.1 us | 5.69 us | 8.60 us | 2.0000 | - | - | 7521 B |
| NHibernate | HQL | Post | 239.4 us | 6.76 us | 10.21 us | 6.5000 | 0.5000 | - | 31885 B |
| NHibernate | Get<T> | Post | 247.1 us | 6.55 us | 9.91 us | 6.0000 | 0.5000 | - | 29885 B |
| EF 6 | First | Post | 297.6 us | 28.83 us | 48.45 us | 13.0000 | - | - | 43309 B |
| NHibernate | Criteria | Post | 300.1 us | 13.37 us | 25.57 us | 14.0000 | 1.0000 | - | 57562 B |
| EF Core | 'First (No Tracking)' | Post | 314.2 us | 43.80 us | 73.60 us | 3.0000 | 1.0000 | - | 17987 B |
| EF Core | First | Post | 319.8 us | 32.72 us | 54.99 us | 3.0000 | - | - | 11306 B |
| EF 6 | 'First (No Tracking)' | Post | 327.2 us | 33.05 us | 55.54 us | 12.0000 | 1.0000 | - | 50237 B |
| NHibernate | SQL | Post | 327.7 us | 16.66 us | 28.00 us | 21.0000 | 1.0000 | - | 80007 B |
| EF Core | SqlQuery | Post | 334.2 us | 33.86 us | 56.90 us | 5.0000 | - | - | 18195 B |
| Belgrade | FirstOrDefault | Task`1 | 384.4 us | 20.30 us | 34.11 us | 4.0000 | 1.0000 | - | 20556 B |
| Dapper cache impact | QueryFirstParameters_NoCache | Void | 677.3 us | 5.55 us | 8.40 us | 3.0000 | 2.0000 | - | 10640 B |
| NHibernate | LINQ | Post | 936.0 us | 168.35 us | 254.52 us | 10.0000 | - | - | 46181 B |
| Dapper cache impact | ExecuteParameters_NoCache | Void | 986.8 us | 198.13 us | 299.55 us | 3.0000 | 2.0000 | - | 10001 B |

Feel free to submit patches that include other ORMs - when running benchmarks, be sure to compile in Release and not attach a debugger (<kbd>Ctrl</kbd>+<kbd>F5</kbd>).

Expand Down
3 changes: 3 additions & 0 deletions benchmarks/Dapper.Tests.Performance/Benchmarks.PetaPoco.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using PetaPoco;
using System.ComponentModel;
using System.Linq;

namespace Dapper.Tests.Performance
{
#if !NET5_0_OR_GREATER
[Description("PetaPoco")]
public class PetaPocoBenchmarks : BenchmarkBase
{
Expand Down Expand Up @@ -38,4 +40,5 @@ public Post FetchFast()
return _dbFast.Fetch<Post>("SELECT * from Posts where Id=@0", i).First();
}
}
#endif
}
3 changes: 3 additions & 0 deletions benchmarks/Dapper.Tests.Performance/Benchmarks.RepoDB.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
using System.ComponentModel;
using System.Linq;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using RepoDb;

namespace Dapper.Tests.Performance
{
#if !NET5_0_OR_GREATER
mgravell marked this conversation as resolved.
Show resolved Hide resolved
[Description("RepoDB")]
public class RepoDbBenchmarks : BenchmarkBase
{
Expand Down Expand Up @@ -51,4 +53,5 @@ public Post ExecuteQuery()
return _connection.ExecuteQuery<Post>("select * from Posts where Id = @Id", new { Id = i }).First();
}
}
#endif
}
2 changes: 1 addition & 1 deletion benchmarks/Dapper.Tests.Performance/Dapper.Tests.Performance.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<AssemblyName>Dapper.Tests.Performance</AssemblyName>
<Description>Dapper Core Performance Suite</Description>
<OutputType>Exe</OutputType>
<TargetFrameworks>net462;net5.0</TargetFrameworks>
<TargetFrameworks>net462;net8.0</TargetFrameworks>
<IsTestProject>false</IsTestProject>
<NoWarn>$(NoWarn);IDE0063;IDE0034;IDE0059;IDE0060</NoWarn>
</PropertyGroup>
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/Dapper.Tests.Performance/app.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<connectionStrings>
<add name="Main" connectionString="Data Source=.;Initial Catalog=tempdb;Integrated Security=True" providerName="System.Data.SqlClient" />
<add name="Main" connectionString="Data Source=.;Initial Catalog=tempdb;Integrated Security=True;TrustServerCertificate=True" providerName="System.Data.SqlClient" />
</connectionStrings>
</configuration>