Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update SerializerBenchmark #692

Merged
merged 5 commits into from Dec 1, 2019
Merged

Update SerializerBenchmark #692

merged 5 commits into from Dec 1, 2019

Conversation

neuecc
Copy link
Member

@neuecc neuecc commented Nov 30, 2019

  • use latest benchmarkdotnet
  • use Core30
  • add System.Text.Json
  • add Lz4BlockArray in v2
  • add use nativeresolver's v2 benchmark

result of shortrun

Method Serializer Mean Error DataSize Gen 0 Gen 1 Gen 2 Allocated
AnswerDeserialize BinaryFormatter_ 51,762.73 ns NA - 8.2397 - - 34568 B
AnswerDeserialize Ceras_ 3,295.73 ns NA - 0.4311 - - 1816 B
AnswerDeserialize DataContract_ 33,098.32 ns NA - 3.7842 - - 15912 B
AnswerDeserialize FsPickler_ 5,074.11 ns NA - 0.9384 - - 3936 B
AnswerDeserialize Hyperion_ 5,406.63 ns NA - 1.0071 - - 4216 B
AnswerDeserialize Jil_ 10,214.00 ns NA - 1.4343 - - 6024 B
AnswerDeserialize JsonNet 31,869.25 ns NA - 2.4414 - - 10360 B
AnswerDeserialize MessagePack_v1 2,477.16 ns NA - 0.4311 - - 1816 B
AnswerDeserialize MessagePack_v2 3,913.10 ns NA - 0.4311 - - 1816 B
AnswerDeserialize MessagePack_v2_opt 3,274.65 ns NA - 0.4311 - - 1816 B
AnswerDeserialize MessagePackLz4_v1 2,578.78 ns NA - 0.4311 - - 1816 B
AnswerDeserialize MessagePackLz4_v2 3,914.56 ns NA - 0.4272 - - 1816 B
AnswerDeserialize MsgPackCli 8,029.95 ns NA - 0.6866 - - 2920 B
AnswerDeserialize ProtobufNet 5,555.61 ns NA - 0.9689 - - 4064 B
AnswerDeserialize SpanJson_ 4,813.62 ns NA - 0.4425 - - 1864 B
AnswerDeserialize SystemTextJson 17,537.53 ns NA - 0.5188 - - 2208 B
AnswerDeserialize Utf8Json_ 7,941.75 ns NA - 0.4425 - - 1864 B
AnswerSerialize BinaryFormatter_ 63,159.58 ns NA 3.64 KB 10.8643 0.1221 - 45644 B
AnswerSerialize Ceras_ 3,264.91 ns NA 476.00 B 1.1215 - - 4704 B
AnswerSerialize DataContract_ 16,238.46 ns NA 2.74 KB 3.6621 - - 15432 B
AnswerSerialize FsPickler_ 11,868.28 ns NA 793.00 B 1.6479 - - 6952 B
AnswerSerialize Hyperion_ 7,623.49 ns NA 1015.00 B 1.1520 - - 4824 B
AnswerSerialize Jil_ 6,097.93 ns NA 1.78 KB 2.5558 - - 10704 B
AnswerSerialize JsonNet 20,401.35 ns NA 1.76 KB 2.9907 0.0305 - 12584 B
AnswerSerialize MessagePack_v1 2,364.70 ns NA 469.00 B 0.1183 - - 496 B
AnswerSerialize MessagePack_v2 2,298.42 ns NA 469.00 B 0.1183 - - 496 B
AnswerSerialize MessagePack_v2_opt 2,170.86 ns NA 433.00 B 0.1106 - - 464 B
AnswerSerialize MessagePackLz4_v1 3,180.48 ns NA 450.00 B 0.1144 - - 480 B
AnswerSerialize MessagePackLz4_v2 3,365.30 ns NA 451.00 B 0.1144 - - 480 B
AnswerSerialize MsgPackCli 5,539.52 ns NA 515.00 B 0.9232 - - 3872 B
AnswerSerialize ProtobufNet 2,718.64 ns NA 498.00 B 0.3166 - - 1336 B
AnswerSerialize SpanJson_ 3,554.91 ns NA 1.82 KB 0.4501 - - 1888 B
AnswerSerialize SystemTextJson 12,600.88 ns NA 1.75 KB 1.3123 - - 5520 B
AnswerSerialize Utf8Json_ 4,894.59 ns NA 1.82 KB 0.4501 - - 1896 B

There seems to be a point in deserialize that can be improved (due to the complexity of SequenceReader?)
But I think it's a pretty good result!
Unfortunately, the current benchmark cannot measure Contractless (because ContractKey is specified, IntKey takes precedence).
This is the point I want to improve (since many people like Contractless)

System.Text.Json's result is very bad but it is offtopic.

@neuecc neuecc requested a review from AArnott November 30, 2019 20:38
@neuecc
Copy link
Member Author

neuecc commented Nov 30, 2019

Measure contractless is important to check v2 performance..
I've added quick-hack to support contactless only shortrun.

Method Serializer Mean Error DataSize Gen 0 Gen 1 Gen 2 Allocated
AnswerDeserialize BinaryFormatter_ 51.639 us NA - 8.2397 - - 34568 B
AnswerDeserialize Ceras_ 3.125 us NA - 0.4311 - - 1816 B
AnswerDeserialize DataContract_ 32.876 us NA - 3.7842 - - 15896 B
AnswerDeserialize FsPickler_ 4.947 us NA - 0.9384 - - 3936 B
AnswerDeserialize Hyperion_ 5.469 us NA - 1.0071 - - 4216 B
AnswerDeserialize Jil_ 10.306 us NA - 1.4343 - - 6032 B
AnswerDeserialize JsonNet 30.977 us NA - 2.4414 - - 10360 B
AnswerDeserialize MessagePack_v1 2.396 us NA - 0.4311 - - 1816 B
AnswerDeserialize MessagePack_v2 3.171 us NA - 0.4311 - - 1816 B
AnswerDeserialize MessagePackLz4_v1 2.518 us NA - 0.4311 - - 1816 B
AnswerDeserialize MessagePackLz4_v2 3.898 us NA - 0.4272 - - 1816 B
AnswerDeserialize MsgPack_v1_str_lz4 3.637 us NA - 0.4311 - - 1816 B
AnswerDeserialize MsgPack_v1_string 3.420 us NA - 0.4311 - - 1816 B
AnswerDeserialize MsgPack_v2_opt 3.235 us NA - 0.4311 - - 1816 B
AnswerDeserialize MsgPack_v2_str_lz4 6.222 us NA - 0.4272 - - 1816 B
AnswerDeserialize MsgPack_v2_string 5.168 us NA - 0.4272 - - 1816 B
AnswerDeserialize MsgPackCli 7.505 us NA - 0.6943 - - 2920 B
AnswerDeserialize ProtobufNet 5.378 us NA - 0.9689 - - 4065 B
AnswerDeserialize SpanJson_ 4.767 us NA - 0.4425 - - 1864 B
AnswerDeserialize SystemTextJson 17.075 us NA - 0.5188 - - 2208 B
AnswerDeserialize Utf8Json_ 8.463 us NA - 0.4425 - - 1864 B
AnswerSerialize BinaryFormatter_ 73.361 us NA 3.64 KB 10.8643 0.1221 - 45644 B
AnswerSerialize Ceras_ 2.677 us NA 475.00 B 1.1215 - - 4704 B
AnswerSerialize DataContract_ 15.948 us NA 2.75 KB 3.6774 - - 15424 B
AnswerSerialize FsPickler_ 10.949 us NA 793.00 B 1.6479 - - 6952 B
AnswerSerialize Hyperion_ 6.965 us NA 1015.00 B 1.1520 - - 4824 B
AnswerSerialize Jil_ 7.461 us NA 1.79 KB 2.5482 - - 10720 B
AnswerSerialize JsonNet 27.727 us NA 1.76 KB 2.9907 - - 12584 B
AnswerSerialize MessagePack_v1 2.285 us NA 469.00 B 0.1183 - - 496 B
AnswerSerialize MessagePack_v2 2.941 us NA 469.00 B 0.1183 - - 496 B
AnswerSerialize MessagePackLz4_v1 3.687 us NA 450.00 B 0.1144 - - 480 B
AnswerSerialize MessagePackLz4_v2 3.449 us NA 451.00 B 0.1144 - - 480 B
AnswerSerialize MsgPack_v1_str_lz4 4.822 us NA 867.00 B 0.2060 - - 888 B
AnswerSerialize MsgPack_v1_string 3.342 us NA 1.23 KB 0.3052 - - 1288 B
AnswerSerialize MsgPack_v2_opt 1.983 us NA 433.00 B 0.1106 - - 464 B
AnswerSerialize MsgPack_v2_str_lz4 5.469 us NA 868.00 B 0.2136 - - 904 B
AnswerSerialize MsgPack_v2_string 4.340 us NA 1.23 KB 0.3052 - - 1288 B
AnswerSerialize MsgPackCli 6.235 us NA 519.00 B 0.9232 - - 3864 B
AnswerSerialize ProtobufNet 2.972 us NA 497.00 B 0.3128 - - 1320 B
AnswerSerialize SpanJson_ 3.803 us NA 1.83 KB 0.4539 - - 1904 B
AnswerSerialize SystemTextJson 16.202 us NA 1.76 KB 1.3123 - - 5512 B
AnswerSerialize Utf8Json_ 4.815 us NA 1.83 KB 0.4501 - - 1904 B

_str, _string is contractless.
The result is okay, this performance can be shipped.
However, it is still not the best compared to the competition, so I would like to aim a little higher in the next updates.

@neuecc neuecc merged commit 5ba2b5d into v2.0 Dec 1, 2019
@neuecc neuecc deleted the update-benchmark branch December 1, 2019 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants