Skip to content

Protobuf-net Performance #754

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
dceravigupta opened this issue Jan 4, 2021 · 9 comments
Open

Protobuf-net Performance #754

dceravigupta opened this issue Jan 4, 2021 · 9 comments

Comments

@dceravigupta
Copy link

I was doing some benchmarking to compare the performance of Protobuf-net with Google.Protobuf and MessagePack serializers and noticed that both Google.Protobuf and MessagePack are significantly faster specially in case of serialization time than Protobuf-net.

I was wondering whether its due any limitation with the code first nature of Protobuf-net or its just matter of code optimization?

Benchmarking Code:
https://github.com/geeknoid/SerializationBenchmark

Here's the result of a benchmark that compares various serialization options.

  • The *Compressed versions uses MessagePack's built-in LZ4 compression.
  • The *Pickled versions use an external LZ4 compression library.
  • The *Plain versions are just the serializer proper, using efficient buffer management to avoid allocs.
  • ProtobufNet_* versions uses the Protobuf-net library.
  • Protobuf_* version uses the Google.Protobuf library.
Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
MessagePack_SerializePlain 292.1 ms 5.40 ms 6.21 ms 2000.0000 - - 9.61 MB
MessagePack_SerializeCompressed 492.8 ms 7.35 ms 10.06 ms 2000.0000 - - 9.61 MB
MessagePack_SerializePickled 751.7 ms 14.62 ms 29.87 ms 19000.0000 - - 76.52 MB
ProtobufNet_SerializePlain 1,056.8 ms 19.53 ms 19.19 ms 11000.0000 - - 44.25 MB
ProtobufNet_SerializePickled 1,641.1 ms 30.66 ms 28.67 ms 28000.0000 - - 114.67 MB
Protobuf_SerializePlain 484.8 ms 6.40 ms 5.67 ms 3000.0000 - - 13.73 MB
Protobuf_SerializePickled 966.9 ms 17.26 ms 26.87 ms 20000.0000 - - 82.4 MB
SystemTextJson_SerializePlain 803.1 ms 15.89 ms 28.24 ms 56000.0000 - - 226.82 MB
SystemTextJson_SerializePickled 1,417.1 ms 18.51 ms 15.45 ms 24000.0000 - - 99.49 MB
Newtonsoft_SerializePlain 1,718.5 ms 33.79 ms 54.56 ms 285000.0000 - - 1153.03 MB
Newtonsoft_SerializePickled 2,288.0 ms 25.30 ms 21.13 ms 307000.0000 43000.0000 - 1232.83 MB
MessagePack_DeserializePlain 586.0 ms 11.58 ms 11.89 ms 113000.0000 2000.0000 - 452.13 MB
MessagePack_DeserializeCompressed 620.9 ms 12.37 ms 20.32 ms 113000.0000 2000.0000 - 452.12 MB
MessagePack_DeserializePickled 705.9 ms 19.34 ms 52.60 ms 139000.0000 20000.0000 - 557.4 MB
ProtobufNet_DeserializePlain 771.3 ms 12.71 ms 18.23 ms 121000.0000 - - 482.79 MB
ProtobufNet_DeserializePickled 845.0 ms 14.73 ms 13.06 ms 148000.0000 4000.0000 - 593.8 MB
Protobuf_DeserializePlain 545.1 ms 7.08 ms 5.91 ms 129000.0000 1000.0000 - 518.34 MB
Protobuf_DeserializePickled 743.8 ms 28.11 ms 82.01 ms 154000.0000 22000.0000 - 617.75 MB
SystemTextJson_DeserializePlain 1,882.1 ms 27.80 ms 21.70 ms 125000.0000 16000.0000 - 498.96 MB
SystemTextJson_DeserializePickled 2,075.7 ms 43.62 ms 122.31 ms 176000.0000 1000.0000 - 706.18 MB
Newtonsoft_DeserializePlain 3,098.0 ms 60.45 ms 97.61 ms 255000.0000 54000.0000 - 1021.27 MB
Newtonsoft_DeserializePickled 3,301.8 ms 65.39 ms 114.53 ms 307000.0000 38000.0000 - 1228.71 MB
@mgravell
Copy link
Member

mgravell commented Jan 5, 2021 via email

@dceravigupta
Copy link
Author

@mgravell there is a link to the benchmark code in the original post. Anyway here it is:

https://github.com/geeknoid/SerializationBenchmark

@dceravigupta
Copy link
Author

Hey @mgravell, did you get a chance to look at the code I shared? Is there anything I missed which I should do to improve performance of protobuf-net?

@mgravell
Copy link
Member

mgravell commented Jan 9, 2021 via email

@Neo-vortex
Copy link

Here are the results :
BenchmarkDotNet=v0.12.1, OS=ubuntu 20.04
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=5.0.102
[Host] : .NET Core 5.0.2 (CoreCLR 5.0.220.61120, CoreFX 5.0.220.61120), X64 RyuJIT
DefaultJob : .NET Core 5.0.2 (CoreCLR 5.0.220.61120, CoreFX 5.0.220.61120), X64 RyuJIT

Method Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated
MessagePack_SerializePlain 250.9 ms 0.59 ms 0.53 ms 250.9 ms 1500.0000 - - 9.61 MB
MessagePack_SerializeCompressed 403.8 ms 1.93 ms 1.80 ms 403.9 ms 1000.0000 - - 9.61 MB
MessagePack_SerializePickled 666.4 ms 1.70 ms 1.51 ms 666.0 ms 12000.0000 - - 76.6 MB
ProtobufNet_SerializePlain 879.3 ms 4.87 ms 4.32 ms 878.8 ms 7000.0000 - - 44.25 MB
ProtobufNet_SerializePickled 1,323.7 ms 5.09 ms 3.98 ms 1,323.3 ms 19000.0000 - - 114.67 MB
Protobuf_SerializePlain 384.3 ms 1.29 ms 1.01 ms 384.1 ms 2000.0000 - - 13.73 MB
Protobuf_SerializePickled 810.4 ms 1.64 ms 1.53 ms 810.7 ms 13000.0000 - - 82.48 MB
SystemTextJson_SerializePlain 623.8 ms 6.78 ms 5.29 ms 621.9 ms 37000.0000 - - 226.98 MB
SystemTextJson_SerializePickled 1,150.4 ms 2.44 ms 2.04 ms 1,150.0 ms 16000.0000 - - 99.64 MB
Newtonsoft_SerializePlain 1,312.3 ms 5.71 ms 4.77 ms 1,312.4 ms 192000.0000 24000.0000 - 1152.73 MB
Newtonsoft_SerializePickled 1,861.3 ms 5.94 ms 4.96 ms 1,861.1 ms 205000.0000 22000.0000 - 1232.53 MB
MessagePack_DeserializePlain 444.4 ms 2.15 ms 1.79 ms 444.5 ms 75000.0000 10000.0000 - 452.12 MB
MessagePack_DeserializeCompressed 456.7 ms 1.53 ms 1.43 ms 456.3 ms 75000.0000 10000.0000 - 452.12 MB
MessagePack_DeserializePickled 486.2 ms 2.42 ms 2.27 ms 485.8 ms 93000.0000 12000.0000 - 557.41 MB
ProtobufNet_DeserializePlain 573.4 ms 11.39 ms 11.18 ms 567.7 ms 80000.0000 10000.0000 - 482.79 MB
ProtobufNet_DeserializePickled 678.8 ms 13.10 ms 15.09 ms 676.2 ms 99000.0000 14000.0000 - 593.8 MB
Protobuf_DeserializePlain 428.2 ms 5.33 ms 4.16 ms 427.0 ms 86000.0000 12000.0000 - 518.34 MB
Protobuf_DeserializePickled 474.9 ms 8.83 ms 9.07 ms 470.5 ms 103000.0000 14000.0000 - 617.83 MB
SystemTextJson_DeserializePlain 1,245.4 ms 13.05 ms 12.21 ms 1,241.8 ms 83000.0000 11000.0000 - 498.96 MB
SystemTextJson_DeserializePickled 1,414.1 ms 28.17 ms 58.17 ms 1,370.8 ms 117000.0000 16000.0000 - 706.18 MB
Newtonsoft_DeserializePlain 2,301.1 ms 17.45 ms 13.62 ms 2,298.0 ms 169000.0000 28000.0000 - 1021.43 MB
Newtonsoft_DeserializePickled 2,385.4 ms 3.36 ms 2.81 ms 2,386.1 ms 204000.0000 40000.0000 - 1228.33 MB

Results are not good for protobuf-net :(

@InCerryGit
Copy link

InCerryGit commented Jul 27, 2021 •
edited
Loading

maybe we can use source generator
https://devblogs.microsoft.com/dotnet/introducing-c-source-generators/

@reyang reyang mentioned this issue Apr 19, 2023
Closed
@AmeyMhatre96
Copy link

Tried to update the packages and run benchmarks for .NET 7 & .NET 8. Results:

image

Method .NET 7 Mean .NET 8 Mean Allocated (MB) .NET 7 Allocated (MB) .NET 8 % Change (Mean)
MessagePack_SerializePlain 238.6 ms 235.9 ms 10.08 MB 10.08 MB -1.13%
MessagePack_SerializeCompressed 421.0 ms 381.0 ms 10.08 MB 10.08 MB -9.50%
MessagePack_SerializePickled 613.3 ms 491.7 ms 10.08 MB 10.08 MB -19.80%
ProtobufNet_SerializePlain 803.6 ms 713.4 ms 46.40 MB 46.40 MB -11.21%
ProtobufNet_SerializePickled 1,187.0 ms 983.8 ms 46.40 MB 46.40 MB -17.06%
Protobuf_SerializePlain 288.6 ms 211.5 ms 0.42 MB 1.07 MB -26.70%
Protobuf_SerializePickled 678.3 ms 469.9 ms 0.84 MB 1.07 MB -30.68%
SystemTextJson_SerializePlain 554.7 ms 497.0 ms 236.48 MB 236.48 MB -10.42%
SystemTextJson_SerializePickled 1,001.0 ms 774.4 ms 20.48 MB 20.48 MB -22.55%
Newtonsoft_SerializePlain 1,058.6 ms 783.3 ms 1209.12 MB 1209.36 MB -26.02%
Newtonsoft_SerializePickled 1,510.6 ms 1,140.0 ms 991.92 MB 991.84 MB -24.53%
MessagePack_DeserializePlain 415.0 ms 373.2 ms 474.08 MB 474.08 MB -10.08%
MessagePack_DeserializeCompressed 446.4 ms 397.5 ms 474.08 MB 474.08 MB -10.97%
MessagePack_DeserializePickled 466.1 ms 400.7 ms 474.08 MB 474.08 MB -14.00%
ProtobufNet_DeserializePlain 565.1 ms 492.1 ms 506.24 MB 506.24 MB -12.91%
ProtobufNet_DeserializePickled 605.1 ms 517.5 ms 506.24 MB 506.24 MB -14.50%
Protobuf_DeserializePlain 341.7 ms 286.1 ms 535.30 MB 535.30 MB -16.30%
Protobuf_DeserializePickled 377.1 ms 315.0 ms 535.30 MB 535.30 MB -16.48%
SystemTextJson_DeserializePlain 1,099.0 ms 911.0 ms 523.20 MB 523.20 MB -17.10%
SystemTextJson_DeserializePickled 1,227.5 ms 960.3 ms 523.20 MB 523.20 MB -21.75%
Newtonsoft_DeserializePlain 2,127.7 ms 1,507.3 ms 1053.68 MB 1053.68 MB -29.17%
Newtonsoft_DeserializePickled 2,259.7 ms 1,535.6 ms 1053.52 MB 1053.76 MB -32.27%

@mgravell
Copy link
Member

mgravell commented Oct 6, 2023

I know what we need to do about this, and an experiment has been started with great results - it just needs a pile of my time, and right now my focus is Dapper/AOT. A lot of the learnings are shared, note - lots of generators goodness on the horizon.

@prpercival
Copy link

prpercival commented Oct 28, 2024 •
edited
Loading

Any updates on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@mgravell @AmeyMhatre96 @dceravigupta @InCerryGit @Neo-vortex @prpercival