Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic serialize/deserialize benchmarks #107

Merged
merged 3 commits into from Dec 12, 2019

Conversation

Turnerj
Copy link
Collaborator

@Turnerj Turnerj commented Dec 10, 2019

As I mentioned in #100 , it would be a good idea to add some benchmarks to Schema.NET, specifically around the serialization/deserialization process.

While the core serialization processing might be in a third-party library (either Newtonsoft.Json or eventually System.Text.Json), there are multiple custom converters used in the library which may have their own performance bottlenecks.

The benchmarks, however, are also useful for gauging the performance improvement when going to System.Text.Json too.

I've specifically included 2 benchmarks, one testing WebSite with a fairly basic model (taken from the WebSiteTest) and one for Book (taken from BookTest). While there isn't anything to really base these numbers on at the moment, it can help for relative improvements.

I may add more specialised benchmarks, specifically targeting the ValuesJsonConverter in the future as that is where I believe the biggest gains are in the library.

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
Intel Core i7-6700HQ CPU 2.60GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=5.0.100-alpha1-015710
  [Host]     : .NET Core 3.0.0 (CoreCLR 4.700.19.46205, CoreFX 4.700.19.46214), X64 RyuJIT
  Job-MDWQYW : .NET Core 3.0.0 (CoreCLR 4.700.19.46205, CoreFX 4.700.19.46214), X64 RyuJIT

Runtime=.NET Core 3.0

Book Benchmark

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
Serialize 175.9 us 1.05 us 0.88 us 15.3809 - - 47.27 KB
Deserialize 445.8 us 3.48 us 3.26 us 39.0625 - - 120.96 KB

WebSite Benchmark

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
Serialize 26.90 us 0.252 us 0.236 us 2.9297 - - 9.16 KB
Deserialize 47.89 us 0.300 us 0.280 us 4.9438 - - 15.17 KB

using BenchmarkDotNet.Jobs;
using Newtonsoft.Json;

[SimpleJob(RuntimeMoniker.NetCoreApp30)]
Copy link
Owner

@RehanSaeed RehanSaeed Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few more useful atrributes that could be added here to generate some graphs and more stats. See https://github.com/Dotnet-Boxed/Framework/blob/master/Benchmarks/Boxed.Mapping.Benchmark/MapListBenchmark.cs

Copy link
Collaborator Author

@Turnerj Turnerj Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, happy to add those. Do you think it is worth benchmarking multiple versions of .NET like your linked example or just leave it still as .NET Core 3?

Copy link
Owner

@RehanSaeed RehanSaeed Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.NET Framework is still used a fair bit, so I added it. The results can be very different.

@RehanSaeed
Copy link
Owner

RehanSaeed commented Dec 11, 2019

Only one comment. Looks good apart from that.

@Turnerj
Copy link
Collaborator Author

Turnerj commented Dec 12, 2019

I've added the additional benchmark settings that you've linked to. Yeah, there is a bit of a difference between .NET 4.7.2 and .NET Core 3:

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
Intel Core i7-6700HQ CPU 2.60GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
  [Host]     : .NET Framework 4.8 (4.8.4075.0), X64 RyuJIT
  Job-VTGAFA : .NET Framework 4.8 (4.8.4075.0), X64 RyuJIT
  Job-ZVNHQX : .NET Core 3.0.0 (CoreCLR 4.700.19.46205, CoreFX 4.700.19.46214), X64 RyuJIT
Method Runtime Mean Error StdDev Min Max Gen 0 Gen 1 Gen 2 Allocated
Serialize .NET 4.7.2 160.3 us 3.04 us 2.84 us 157.4 us 165.6 us 15.3809 - - 47.52 KB
Deserialize .NET 4.7.2 439.2 us 3.93 us 3.49 us 434.2 us 447.0 us 38.0859 - - 119.51 KB
Serialize .NET Core 3.0 152.5 us 2.35 us 1.96 us 147.1 us 155.8 us 15.3809 - - 47.27 KB
Deserialize .NET Core 3.0 376.8 us 5.63 us 4.99 us 368.5 us 384.9 us 37.5977 - - 116.11 KB

@RehanSaeed RehanSaeed merged commit 8f7ac2d into RehanSaeed:master Dec 12, 2019
2 checks passed
@RehanSaeed
Copy link
Owner

RehanSaeed commented Dec 12, 2019

Thanks!

@Turnerj Turnerj deleted the add-benchmarks branch Dec 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants