Significant overhead for a large number of rows

[nenadnoveljic]

There's a significant overhead for large datasets. Some operations could, perhaps, be executed just once for all rows. The profiled stacks below might give a clue:

Thread #4 • 44'828 ms 100.00% ThreadStart • 44'828 ms • 1 call • System.Threading.ThreadHelper.ThreadStart 99.97% ExecuteTests • 44'817 ms • 1 call • TestingSlapper.Program.ExecuteTests 99.95% ExecuteSlapper • 44'807 ms • 10 calls • TestingSlapper.Program.ExecuteSlapper 91.46% MapDynamic • 41'001 ms • 10 calls • Slapper.AutoMapper.MapDynamic(IEnumerable, Boolean) 91.46% MapDynamic • 40'998 ms • 10 calls • Slapper.AutoMapper.MapDynamic(Type, IEnumerable, Boolean) 90.58% Map • 40'603 ms • 10 calls • Slapper.AutoMapper.Map(Type, IEnumerable, Boolean) 38.06% Map • 17'062 ms • 1'000'000 calls • Slapper.AutoMapper+InternalHelpers.Map(IDictionary, Object, Object) ►28.58% CreateTypeMap • 12'811 ms • 1'000'000 calls • Slapper.AutoMapper+InternalHelpers.CreateTypeMap(Type) ►4.99% SetMemberValue • 2'239 ms • 1'000'000 calls • Slapper.AutoMapper+InternalHelpers.SetMemberValue(Object, Object, Object) 1.34% TryGetValue • 599 ms • 1'000'000 calls • System.Collections.Generic.Dictionary2.TryGetValue(TKey, out TValue)
0.94% GetOrAdd • 423 ms • 1'000'000 calls • System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey, TValue) 0.54% ToLower • 242 ms • 1'000'000 calls • System.String.ToLower(CultureInfo) 0.47% get_CurrentCulture • 213 ms • 1'000'000 calls • System.Threading.Thread.get_CurrentCulture 0.22% IsPrimitiveImpl • 99 ms • 1'000'000 calls • System.RuntimeType.IsPrimitiveImpl 0.17% MoveNext • 75 ms • 2'000'000 calls • System.Collections.Generic.Dictionary+Enumerator2.MoveNext
0.01% [Garbage collection] • 5 ms • 1 call
33.23% GetIdentifierHash • 14'895 ms • 1'000'000 calls • Slapper.AutoMapper+InternalHelpers.GetIdentifierHash(Type, IDictionary)
30.53% GetIdentifiers • 13'687 ms • 1'000'000 calls • Slapper.AutoMapper+InternalHelpers.GetIdentifiers(Type)
►28.54% CreateTypeMap • 12'793 ms • 1'000'000 calls • Slapper.AutoMapper+InternalHelpers.CreateTypeMap(Type)
0.93% GetOrAdd • 416 ms • 1'000'000 calls • System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey, TValue) 0.74% Any • 333 ms • 1'000'000 calls • System.Linq.Enumerable.Any(IEnumerable) 2.21% NewGuid • 989 ms • 1'000'000 calls • System.Guid.NewGuid 0.08% GetHashCode • 35 ms • 1'000'000 calls • System.Guid.GetHashCode ►0.00% AutoMapper+Cache..cctor • 0 ms • 1 call • Slapper.AutoMapper+Cache..cctor ►4.99% GetInstanceCache • 2'238 ms • 1'000'000 calls • Slapper.AutoMapper+Cache.GetInstanceCache 3.71% Insert • 1'662 ms • 1'999'872 calls • System.Collections.Generic.Dictionary2.Insert(TKey, TValue, Boolean)
►2.76% Dictionary2..ctor • 1'237 ms • 1'000'000 calls • System.Collections.Generic.Dictionary2..ctor(IDictionary, IEqualityComparer)
2.60% TryGetValue • 1'166 ms • 1'000'000 calls • System.Collections.Generic.Dictionary2.TryGetValue(TKey, out TValue) 1.89% FindEntry • 848 ms • 1'000'000 calls • System.Collections.Generic.Dictionary2.FindEntry(TKey)
►1.48% CreateInstance • 661 ms • 999'884 calls • Slapper.AutoMapper+InternalHelpers.CreateInstance(Type)
0.09% MoveNext • 41 ms • 1'000'010 calls • System.Collections.Generic.List+Enumerator1.MoveNext 0.09% [Garbage collection] • 41 ms • 11 calls 0.09% GetHashCode • 40 ms • 1'000'000 calls • System.RuntimeType.GetHashCode 0.07% get_Current • 33 ms • 1'000'000 calls • System.Collections.Generic.List+Enumerator1.get_Current
0.00% 5 functions hidden • 1 ms total • 41 calls total
►0.88% ToList • 393 ms • 10 calls • System.Linq.Enumerable.ToList(IEnumerable)
0.00% Select • 0 ms • 10 calls • System.Linq.Enumerable.Select(IEnumerable, Func)
0.00% 2 functions hidden • 0 ms total • 11 calls total
►7.05% Query • 3'161 ms • 10 calls • Dapper.SqlMapper.Query(IDbConnection, String, Object, IDbTransaction, Boolean, Nullable, Nullable)
►1.13% ToList • 508 ms • 10 calls • System.Linq.Enumerable.ToList(IEnumerable)
►0.29% SqlMapper..cctor • 131 ms • 1 call • Dapper.SqlMapper..cctor
►0.00% SqlMapper+<>c..cctor • 0 ms • 1 call • Dapper.SqlMapper+<>c..cctor
0.01% 21 functions hidden • 3 ms total • 282 calls total`

The tests programs are here: https://github.com/nenadnoveljic/blogs/tree/master/slapper-overhead .

[Dolfik1]

You can try to use Mappy it should be faster

[randyburden]

Hi, we've just released version 2.0.3 release which has some performance improvements yielding a 57% mapping speed improvement.

• v2.0.2: Mapped 120,000 deeply nested complex objects in 30333 ms.
• v2.0.3: Mapped 120,000 deeply nested complex objects in 13046 ms.

Hopefully, this will help folks out and without any loss of backward compatibility they've come to enjoy with Slapper.AutoMapper.

Reflection is inherently slow, especially using the straight-forward pattern that Slapper.AutoMapper uses. If we were to use advanced techniques such as compiling lambdas or emitting IL I'm sure we could massively improve the performance but it would require a total rewrite, which is what Mappy seems to be.