Protobuf-net Performance

[dceravigupta]

I was doing some benchmarking to compare the performance of Protobuf-net with Google.Protobuf and MessagePack serializers and noticed that both Google.Protobuf and MessagePack are significantly faster specially in case of serialization time than Protobuf-net.

I was wondering whether its due any limitation with the code first nature of Protobuf-net or its just matter of code optimization?

Benchmarking Code:
https://github.com/geeknoid/SerializationBenchmark

Here's the result of a benchmark that compares various serialization options.

• The *Compressed versions uses MessagePack's built-in LZ4 compression.
• The *Pickled versions use an external LZ4 compression library.
• The *Plain versions are just the serializer proper, using efficient buffer management to avoid allocs.
• ProtobufNet_* versions uses the Protobuf-net library.
• Protobuf_* version uses the Google.Protobuf library.

Method	Mean	Error	StdDev	Gen 0	Gen 1	Gen 2	Allocated
MessagePack_SerializePlain	292.1 ms	5.40 ms	6.21 ms	2000.0000	-	-	9.61 MB
MessagePack_SerializeCompressed	492.8 ms	7.35 ms	10.06 ms	2000.0000	-	-	9.61 MB
MessagePack_SerializePickled	751.7 ms	14.62 ms	29.87 ms	19000.0000	-	-	76.52 MB
ProtobufNet_SerializePlain	1,056.8 ms	19.53 ms	19.19 ms	11000.0000	-	-	44.25 MB
ProtobufNet_SerializePickled	1,641.1 ms	30.66 ms	28.67 ms	28000.0000	-	-	114.67 MB
Protobuf_SerializePlain	484.8 ms	6.40 ms	5.67 ms	3000.0000	-	-	13.73 MB
Protobuf_SerializePickled	966.9 ms	17.26 ms	26.87 ms	20000.0000	-	-	82.4 MB
SystemTextJson_SerializePlain	803.1 ms	15.89 ms	28.24 ms	56000.0000	-	-	226.82 MB
SystemTextJson_SerializePickled	1,417.1 ms	18.51 ms	15.45 ms	24000.0000	-	-	99.49 MB
Newtonsoft_SerializePlain	1,718.5 ms	33.79 ms	54.56 ms	285000.0000	-	-	1153.03 MB
Newtonsoft_SerializePickled	2,288.0 ms	25.30 ms	21.13 ms	307000.0000	43000.0000	-	1232.83 MB
MessagePack_DeserializePlain	586.0 ms	11.58 ms	11.89 ms	113000.0000	2000.0000	-	452.13 MB
MessagePack_DeserializeCompressed	620.9 ms	12.37 ms	20.32 ms	113000.0000	2000.0000	-	452.12 MB
MessagePack_DeserializePickled	705.9 ms	19.34 ms	52.60 ms	139000.0000	20000.0000	-	557.4 MB
ProtobufNet_DeserializePlain	771.3 ms	12.71 ms	18.23 ms	121000.0000	-	-	482.79 MB
ProtobufNet_DeserializePickled	845.0 ms	14.73 ms	13.06 ms	148000.0000	4000.0000	-	593.8 MB
Protobuf_DeserializePlain	545.1 ms	7.08 ms	5.91 ms	129000.0000	1000.0000	-	518.34 MB
Protobuf_DeserializePickled	743.8 ms	28.11 ms	82.01 ms	154000.0000	22000.0000	-	617.75 MB
SystemTextJson_DeserializePlain	1,882.1 ms	27.80 ms	21.70 ms	125000.0000	16000.0000	-	498.96 MB
SystemTextJson_DeserializePickled	2,075.7 ms	43.62 ms	122.31 ms	176000.0000	1000.0000	-	706.18 MB
Newtonsoft_DeserializePlain	3,098.0 ms	60.45 ms	97.61 ms	255000.0000	54000.0000	-	1021.27 MB
Newtonsoft_DeserializePickled	3,301.8 ms	65.39 ms	114.53 ms	307000.0000	38000.0000	-	1228.71 MB

[mgravell]

Any version information here, for context? Or better: the actual code? I am indeed overdue another optimization pass, but hard to comment beyond that without seeing them.

…

On Mon, 4 Jan 2021, 21:33 Ravi Gupta, ***@***.***> wrote: I was doing some benchmarking to compare the performance of Protobuf-net with Google.Protobuf and MessagePack serializers and noticed that both Google.Protobuf and MessagePack are significantly faster specially in case of serialization time than Protobuf-net. I was wondering whether its due any limitation with the code first nature of Protobuf-net or its just matter of code optimization? Benchmarking Code: https://github.com/geeknoid/SerializationBenchmark Here's the result of a benchmark that compares various serialization options. - The *Compressed versions uses MessagePack's built-in LZ4 compression. - The *Pickled versions use an external LZ4 compression library. - The *Plain versions are just the serializer proper, using efficient buffer management to avoid allocs. - ProtobufNet_* versions uses the Protobuf-net library. - Protobuf_* version uses the Google.Protobuf library. Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated MessagePack_SerializePlain 292.1 ms 5.40 ms 6.21 ms 2000.0000 - - 9.61 MB MessagePack_SerializeCompressed 492.8 ms 7.35 ms 10.06 ms 2000.0000 - - 9.61 MB MessagePack_SerializePickled 751.7 ms 14.62 ms 29.87 ms 19000.0000 - - 76.52 MB ProtobufNet_SerializePlain 1,056.8 ms 19.53 ms 19.19 ms 11000.0000 - - 44.25 MB ProtobufNet_SerializePickled 1,641.1 ms 30.66 ms 28.67 ms 28000.0000 - - 114.67 MB Protobuf_SerializePlain 484.8 ms 6.40 ms 5.67 ms 3000.0000 - - 13.73 MB Protobuf_SerializePickled 966.9 ms 17.26 ms 26.87 ms 20000.0000 - - 82.4 MB SystemTextJson_SerializePlain 803.1 ms 15.89 ms 28.24 ms 56000.0000 - - 226.82 MB SystemTextJson_SerializePickled 1,417.1 ms 18.51 ms 15.45 ms 24000.0000 - - 99.49 MB Newtonsoft_SerializePlain 1,718.5 ms 33.79 ms 54.56 ms 285000.0000 - - 1153.03 MB Newtonsoft_SerializePickled 2,288.0 ms 25.30 ms 21.13 ms 307000.0000 43000.0000 - 1232.83 MB MessagePack_DeserializePlain 586.0 ms 11.58 ms 11.89 ms 113000.0000 2000.0000 - 452.13 MB MessagePack_DeserializeCompressed 620.9 ms 12.37 ms 20.32 ms 113000.0000 2000.0000 - 452.12 MB MessagePack_DeserializePickled 705.9 ms 19.34 ms 52.60 ms 139000.0000 20000.0000 - 557.4 MB ProtobufNet_DeserializePlain 771.3 ms 12.71 ms 18.23 ms 121000.0000 - - 482.79 MB ProtobufNet_DeserializePickled 845.0 ms 14.73 ms 13.06 ms 148000.0000 4000.0000 - 593.8 MB Protobuf_DeserializePlain 545.1 ms 7.08 ms 5.91 ms 129000.0000 1000.0000 - 518.34 MB Protobuf_DeserializePickled 743.8 ms 28.11 ms 82.01 ms 154000.0000 22000.0000 - 617.75 MB SystemTextJson_DeserializePlain 1,882.1 ms 27.80 ms 21.70 ms 125000.0000 16000.0000 - 498.96 MB SystemTextJson_DeserializePickled 2,075.7 ms 43.62 ms 122.31 ms 176000.0000 1000.0000 - 706.18 MB Newtonsoft_DeserializePlain 3,098.0 ms 60.45 ms 97.61 ms 255000.0000 54000.0000 - 1021.27 MB Newtonsoft_DeserializePickled 3,301.8 ms 65.39 ms 114.53 ms 307000.0000 38000.0000 - 1228.71 MB — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#754>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAEHMDX5ZJ6MDV4RLLHZ7TSYIX2RANCNFSM4VTTK47Q> .

[dceravigupta]

@mgravell there is a link to the benchmark code in the original post. Anyway here it is:

https://github.com/geeknoid/SerializationBenchmark

[dceravigupta]

Hey @mgravell, did you get a chance to look at the code I shared? Is there anything I missed which I should do to improve performance of protobuf-net?

[mgravell]

I haven't had chance to look. Still catching up on Xmas backlog, plus: all the distractions that come from whatever crazy world we're currently in.

…

On Sat, 9 Jan 2021, 20:07 Ravi Gupta, ***@***.***> wrote: Hey @mgravell <https://github.com/mgravell>, did you get a chance to look at the code I shared? Is there anything I missed which I should do to improve performance of protobuf-net? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#754 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAEHMHQKW3UX7WF5MQN7H3SZCZQPANCNFSM4VTTK47Q> .

[Neo-vortex]

Here are the results :
BenchmarkDotNet=v0.12.1, OS=ubuntu 20.04
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=5.0.102
[Host] : .NET Core 5.0.2 (CoreCLR 5.0.220.61120, CoreFX 5.0.220.61120), X64 RyuJIT
DefaultJob : .NET Core 5.0.2 (CoreCLR 5.0.220.61120, CoreFX 5.0.220.61120), X64 RyuJIT

Method	Mean	Error	StdDev	Median	Gen 0	Gen 1	Gen 2	Allocated
MessagePack_SerializePlain	250.9 ms	0.59 ms	0.53 ms	250.9 ms	1500.0000	-	-	9.61 MB
MessagePack_SerializeCompressed	403.8 ms	1.93 ms	1.80 ms	403.9 ms	1000.0000	-	-	9.61 MB
MessagePack_SerializePickled	666.4 ms	1.70 ms	1.51 ms	666.0 ms	12000.0000	-	-	76.6 MB
ProtobufNet_SerializePlain	879.3 ms	4.87 ms	4.32 ms	878.8 ms	7000.0000	-	-	44.25 MB
ProtobufNet_SerializePickled	1,323.7 ms	5.09 ms	3.98 ms	1,323.3 ms	19000.0000	-	-	114.67 MB
Protobuf_SerializePlain	384.3 ms	1.29 ms	1.01 ms	384.1 ms	2000.0000	-	-	13.73 MB
Protobuf_SerializePickled	810.4 ms	1.64 ms	1.53 ms	810.7 ms	13000.0000	-	-	82.48 MB
SystemTextJson_SerializePlain	623.8 ms	6.78 ms	5.29 ms	621.9 ms	37000.0000	-	-	226.98 MB
SystemTextJson_SerializePickled	1,150.4 ms	2.44 ms	2.04 ms	1,150.0 ms	16000.0000	-	-	99.64 MB
Newtonsoft_SerializePlain	1,312.3 ms	5.71 ms	4.77 ms	1,312.4 ms	192000.0000	24000.0000	-	1152.73 MB
Newtonsoft_SerializePickled	1,861.3 ms	5.94 ms	4.96 ms	1,861.1 ms	205000.0000	22000.0000	-	1232.53 MB
MessagePack_DeserializePlain	444.4 ms	2.15 ms	1.79 ms	444.5 ms	75000.0000	10000.0000	-	452.12 MB
MessagePack_DeserializeCompressed	456.7 ms	1.53 ms	1.43 ms	456.3 ms	75000.0000	10000.0000	-	452.12 MB
MessagePack_DeserializePickled	486.2 ms	2.42 ms	2.27 ms	485.8 ms	93000.0000	12000.0000	-	557.41 MB
ProtobufNet_DeserializePlain	573.4 ms	11.39 ms	11.18 ms	567.7 ms	80000.0000	10000.0000	-	482.79 MB
ProtobufNet_DeserializePickled	678.8 ms	13.10 ms	15.09 ms	676.2 ms	99000.0000	14000.0000	-	593.8 MB
Protobuf_DeserializePlain	428.2 ms	5.33 ms	4.16 ms	427.0 ms	86000.0000	12000.0000	-	518.34 MB
Protobuf_DeserializePickled	474.9 ms	8.83 ms	9.07 ms	470.5 ms	103000.0000	14000.0000	-	617.83 MB
SystemTextJson_DeserializePlain	1,245.4 ms	13.05 ms	12.21 ms	1,241.8 ms	83000.0000	11000.0000	-	498.96 MB
SystemTextJson_DeserializePickled	1,414.1 ms	28.17 ms	58.17 ms	1,370.8 ms	117000.0000	16000.0000	-	706.18 MB
Newtonsoft_DeserializePlain	2,301.1 ms	17.45 ms	13.62 ms	2,298.0 ms	169000.0000	28000.0000	-	1021.43 MB
Newtonsoft_DeserializePickled	2,385.4 ms	3.36 ms	2.81 ms	2,386.1 ms	204000.0000	40000.0000	-	1228.33 MB

Results are not good for protobuf-net :(

[InCerryGit]

maybe we can use source generator
https://devblogs.microsoft.com/dotnet/introducing-c-source-generators/

[AmeyMhatre96]

Tried to update the packages and run benchmarks for .NET 7 & .NET 8. Results:

Method	.NET 7 Mean	.NET 8 Mean	Allocated (MB) .NET 7	Allocated (MB) .NET 8	% Change (Mean)
MessagePack_SerializePlain	238.6 ms	235.9 ms	10.08 MB	10.08 MB	-1.13%
MessagePack_SerializeCompressed	421.0 ms	381.0 ms	10.08 MB	10.08 MB	-9.50%
MessagePack_SerializePickled	613.3 ms	491.7 ms	10.08 MB	10.08 MB	-19.80%
ProtobufNet_SerializePlain	803.6 ms	713.4 ms	46.40 MB	46.40 MB	-11.21%
ProtobufNet_SerializePickled	1,187.0 ms	983.8 ms	46.40 MB	46.40 MB	-17.06%
Protobuf_SerializePlain	288.6 ms	211.5 ms	0.42 MB	1.07 MB	-26.70%
Protobuf_SerializePickled	678.3 ms	469.9 ms	0.84 MB	1.07 MB	-30.68%
SystemTextJson_SerializePlain	554.7 ms	497.0 ms	236.48 MB	236.48 MB	-10.42%
SystemTextJson_SerializePickled	1,001.0 ms	774.4 ms	20.48 MB	20.48 MB	-22.55%
Newtonsoft_SerializePlain	1,058.6 ms	783.3 ms	1209.12 MB	1209.36 MB	-26.02%
Newtonsoft_SerializePickled	1,510.6 ms	1,140.0 ms	991.92 MB	991.84 MB	-24.53%
MessagePack_DeserializePlain	415.0 ms	373.2 ms	474.08 MB	474.08 MB	-10.08%
MessagePack_DeserializeCompressed	446.4 ms	397.5 ms	474.08 MB	474.08 MB	-10.97%
MessagePack_DeserializePickled	466.1 ms	400.7 ms	474.08 MB	474.08 MB	-14.00%
ProtobufNet_DeserializePlain	565.1 ms	492.1 ms	506.24 MB	506.24 MB	-12.91%
ProtobufNet_DeserializePickled	605.1 ms	517.5 ms	506.24 MB	506.24 MB	-14.50%
Protobuf_DeserializePlain	341.7 ms	286.1 ms	535.30 MB	535.30 MB	-16.30%
Protobuf_DeserializePickled	377.1 ms	315.0 ms	535.30 MB	535.30 MB	-16.48%
SystemTextJson_DeserializePlain	1,099.0 ms	911.0 ms	523.20 MB	523.20 MB	-17.10%
SystemTextJson_DeserializePickled	1,227.5 ms	960.3 ms	523.20 MB	523.20 MB	-21.75%
Newtonsoft_DeserializePlain	2,127.7 ms	1,507.3 ms	1053.68 MB	1053.68 MB	-29.17%
Newtonsoft_DeserializePickled	2,259.7 ms	1,535.6 ms	1053.52 MB	1053.76 MB	-32.27%

[mgravell]

I know what we need to do about this, and an experiment has been started with great results - it just needs a pile of my time, and right now my focus is Dapper/AOT. A lot of the learnings are shared, note - lots of generators goodness on the horizon.

[prpercival]

Any updates on this?

[dameng324]

Any body who care about performance could try : https://github.com/dameng324/LightProto which is about 20-50% better performance than protobuf-net with painless migration.

[zeinali-ali]

@dameng324 That's cool, I was just started developing exactly same thing 2 day ago.
Is it production ready?

[dameng324]

@dameng324 That's cool, I was just started developing exactly same thing 2 day ago.
Is it production ready?

@zeinali-ali Not yet. The main feature design will not change， but the serializer api is not stable yet. You can try it and any feedback is appreciate.