Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks for jsoniter-scala #30

Merged
merged 2 commits into from
Jun 6, 2019

Conversation

plokhotnyuk
Copy link
Contributor

@plokhotnyuk plokhotnyuk commented May 12, 2019

@sirthias Thanks for the bench suit! So diversive workloads!

JFYK, I have found that Borer 0.9.0 outperforms jsoniter-scala 0.47.0 in parsing of some messages where case classes have too many fields to parse in a hot-loop (~90 or more, like in reddit-scala.json).

Also, it helped to find an issue with case classes that has 32, 64, etc. fields

Bellow are results of both parsers on my notebook with OpenJDK 11:

jsoniter-scala

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt       Score      Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5   21967.906 ± 1746.788  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5   32899.698 ± 2063.908  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5    5373.159 ±  152.063  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5   42421.083 ± 2976.315  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5    7816.742 ±  104.682  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5    3084.256 ±  299.346  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5    3897.806 ±   37.593  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5    9029.286 ±  941.237  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5   45934.193 ± 1965.535  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5    1284.162 ±   52.458  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5     103.252 ±    1.138  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5    2633.520 ±   26.382  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5   31354.180 ± 1351.290  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5   45692.197 ± 1740.369  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5   34854.963 ± 3368.925  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5     621.655 ±   52.564  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5   49388.139 ±  250.832  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5   41175.069 ±  389.151  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 australia-abc.json  thrpt    5   17017.090 ± 2294.860  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       bitcoin.json  thrpt    5   54574.871 ±  891.949  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                      doj-blog.json  thrpt    5    2397.797 ±    8.361  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel              eu-lobby-country.json  thrpt    5   30957.100 ± 4159.665  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel            eu-lobby-financial.json  thrpt    5   11244.430 ±  850.009  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 eu-lobby-repr.json  thrpt    5    3173.156 ±    5.666  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 github-events.json  thrpt    5    4344.217 ±   82.729  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  github-gists.json  thrpt    5    8661.939 ±   56.661  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                json-generator.json  thrpt    5   76583.562 ± 7675.512  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    meteorites.json  thrpt    5    1298.139 ±    2.993  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                        movies.json  thrpt    5      97.220 ±    5.099  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  reddit-scala.json  thrpt    5    5850.978 ±  315.216  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    rick-morty.json  thrpt    5   35537.533 ±  200.995  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  temp-anomaly.json  thrpt    5  130836.224 ± 3779.165  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  thai-cinemas.json  thrpt    5   47765.793 ± 5328.250  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       turkish.json  thrpt    5     680.563 ±   53.296  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel  twitter_api_compact_response.json  thrpt    5   70981.552 ± 8654.732  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel          twitter_api_response.json  thrpt    5   72710.033 ± 8286.549  ops/s

Borer

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                               (fileName)   Mode  Cnt      Score      Error  Units
[info] BorerModelBenchmark.decodeModel                 australia-abc.json  thrpt    5  12973.821 ±  435.743  ops/s
[info] BorerModelBenchmark.decodeModel                       bitcoin.json  thrpt    5  18548.508 ± 2236.431  ops/s
[info] BorerModelBenchmark.decodeModel                      doj-blog.json  thrpt    5   3842.196 ±  271.666  ops/s
[info] BorerModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5  26240.354 ± 1069.575  ops/s
[info] BorerModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5   5050.531 ±   42.634  ops/s
[info] BorerModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5   2094.002 ±   33.674  ops/s
[info] BorerModelBenchmark.decodeModel                 github-events.json  thrpt    5   2873.735 ±  131.852  ops/s
[info] BorerModelBenchmark.decodeModel                  github-gists.json  thrpt    5   6270.788 ±  149.951  ops/s
[info] BorerModelBenchmark.decodeModel                json-generator.json  thrpt    5  24110.860 ±  338.608  ops/s
[info] BorerModelBenchmark.decodeModel                    meteorites.json  thrpt    5    731.396 ±   20.834  ops/s
[info] BorerModelBenchmark.decodeModel                        movies.json  thrpt    5     49.768 ±    1.597  ops/s
[info] BorerModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5   3087.225 ±  200.847  ops/s
[info] BorerModelBenchmark.decodeModel                    rick-morty.json  thrpt    5  16458.631 ± 1934.838  ops/s
[info] BorerModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5  36273.004 ± 2576.495  ops/s
[info] BorerModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5  23447.275 ±  609.492  ops/s
[info] BorerModelBenchmark.decodeModel                       turkish.json  thrpt    5    326.427 ±   33.588  ops/s
[info] BorerModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5  33541.663 ±  815.795  ops/s
[info] BorerModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5  23502.183 ±   96.724  ops/s
[info] BorerModelBenchmark.encodeModel                 australia-abc.json  thrpt    5   7030.249 ±   51.533  ops/s
[info] BorerModelBenchmark.encodeModel                       bitcoin.json  thrpt    5  17588.288 ±  410.173  ops/s
[info] BorerModelBenchmark.encodeModel                      doj-blog.json  thrpt    5   1724.224 ±   12.816  ops/s
[info] BorerModelBenchmark.encodeModel              eu-lobby-country.json  thrpt    5  17913.519 ±  150.094  ops/s
[info] BorerModelBenchmark.encodeModel            eu-lobby-financial.json  thrpt    5   3155.994 ±   30.442  ops/s
[info] BorerModelBenchmark.encodeModel                 eu-lobby-repr.json  thrpt    5   1247.968 ±    8.172  ops/s
[info] BorerModelBenchmark.encodeModel                 github-events.json  thrpt    5   2040.756 ±   10.658  ops/s
[info] BorerModelBenchmark.encodeModel                  github-gists.json  thrpt    5   3503.917 ±   89.063  ops/s
[info] BorerModelBenchmark.encodeModel                json-generator.json  thrpt    5  29309.195 ±   79.318  ops/s
[info] BorerModelBenchmark.encodeModel                    meteorites.json  thrpt    5    564.650 ±   14.809  ops/s
[info] BorerModelBenchmark.encodeModel                        movies.json  thrpt    5     37.810 ±    0.505  ops/s
[info] BorerModelBenchmark.encodeModel                  reddit-scala.json  thrpt    5   2108.860 ±   26.504  ops/s
[info] BorerModelBenchmark.encodeModel                    rick-morty.json  thrpt    5  11376.416 ±  843.751  ops/s
[info] BorerModelBenchmark.encodeModel                  temp-anomaly.json  thrpt    5  66785.734 ± 1263.151  ops/s
[info] BorerModelBenchmark.encodeModel                  thai-cinemas.json  thrpt    5  21973.951 ±  643.827  ops/s
[info] BorerModelBenchmark.encodeModel                       turkish.json  thrpt    5    340.449 ±   20.387  ops/s
[info] BorerModelBenchmark.encodeModel  twitter_api_compact_response.json  thrpt    5  20356.984 ±  620.225  ops/s
[info] BorerModelBenchmark.encodeModel          twitter_api_response.json  thrpt    5  18582.182 ± 2964.082  ops/s

Jackson

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                 (fileName)   Mode  Cnt      Score      Error  Units
[info] JacksonModelBenchmark.decodeModel                 australia-abc.json  thrpt    5   9640.055 ±  154.962  ops/s
[info] JacksonModelBenchmark.decodeModel                       bitcoin.json  thrpt    5  14788.302 ±  177.847  ops/s
[info] JacksonModelBenchmark.decodeModel                      doj-blog.json  thrpt    5   3867.669 ±  325.992  ops/s
[info] JacksonModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5  33629.711 ±  200.833  ops/s
[info] JacksonModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5   5315.855 ±   69.552  ops/s
[info] JacksonModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5   2412.082 ±  224.597  ops/s
[info] JacksonModelBenchmark.decodeModel                 github-events.json  thrpt    5   2967.125 ±   46.540  ops/s
[info] JacksonModelBenchmark.decodeModel                  github-gists.json  thrpt    5   7214.918 ±   78.647  ops/s
[info] JacksonModelBenchmark.decodeModel                json-generator.json  thrpt    5  21823.028 ±  103.701  ops/s
[info] JacksonModelBenchmark.decodeModel                    meteorites.json  thrpt    5    791.469 ±    9.383  ops/s
[info] JacksonModelBenchmark.decodeModel                        movies.json  thrpt    5     24.541 ±    0.641  ops/s
[info] JacksonModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5   2856.821 ±  254.216  ops/s
[info] JacksonModelBenchmark.decodeModel                    rick-morty.json  thrpt    5  19595.194 ±  530.582  ops/s
[info] JacksonModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5  32928.180 ± 1448.088  ops/s
[info] JacksonModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5  25503.427 ±  212.422  ops/s
[info] JacksonModelBenchmark.decodeModel                       turkish.json  thrpt    5    429.388 ±    3.893  ops/s
[info] JacksonModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5  26755.528 ±  883.811  ops/s
[info] JacksonModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5  18988.700 ±  162.400  ops/s
[info] JacksonModelBenchmark.encodeModel                 australia-abc.json  thrpt    5  12177.324 ±  130.271  ops/s
[info] JacksonModelBenchmark.encodeModel                       bitcoin.json  thrpt    5  25324.262 ±  382.139  ops/s
[info] JacksonModelBenchmark.encodeModel                      doj-blog.json  thrpt    5   2310.215 ±   26.993  ops/s
[info] JacksonModelBenchmark.encodeModel              eu-lobby-country.json  thrpt    5  36612.848 ±  966.748  ops/s
[info] JacksonModelBenchmark.encodeModel            eu-lobby-financial.json  thrpt    5   9513.191 ±  483.989  ops/s
[info] JacksonModelBenchmark.encodeModel                 eu-lobby-repr.json  thrpt    5   2235.690 ±    4.474  ops/s
[info] JacksonModelBenchmark.encodeModel                 github-events.json  thrpt    5   3178.480 ±   61.843  ops/s
[info] JacksonModelBenchmark.encodeModel                  github-gists.json  thrpt    5   8310.023 ±  103.032  ops/s
[info] JacksonModelBenchmark.encodeModel                json-generator.json  thrpt    5  38440.709 ± 1083.986  ops/s
[info] JacksonModelBenchmark.encodeModel                    meteorites.json  thrpt    5    940.130 ±   11.318  ops/s
[info] JacksonModelBenchmark.encodeModel                        movies.json  thrpt    5     86.898 ±    2.843  ops/s
[info] JacksonModelBenchmark.encodeModel                  reddit-scala.json  thrpt    5   3864.755 ±   33.595  ops/s
[info] JacksonModelBenchmark.encodeModel                    rick-morty.json  thrpt    5  18922.142 ±  181.314  ops/s
[info] JacksonModelBenchmark.encodeModel                  temp-anomaly.json  thrpt    5  67836.752 ±  302.757  ops/s
[info] JacksonModelBenchmark.encodeModel                  thai-cinemas.json  thrpt    5  38321.765 ±  629.438  ops/s
[info] JacksonModelBenchmark.encodeModel                       turkish.json  thrpt    5    500.478 ±   44.155  ops/s
[info] JacksonModelBenchmark.encodeModel  twitter_api_compact_response.json  thrpt    5  41331.299 ±  265.281  ops/s
[info] JacksonModelBenchmark.encodeModel          twitter_api_response.json  thrpt    5  41818.340 ±  414.329  ops/s

@plokhotnyuk plokhotnyuk force-pushed the add-jsoniter-scala-benchmarks branch from 7d4cd31 to 0877f41 Compare May 12, 2019 21:21
@plokhotnyuk
Copy link
Contributor Author

Another finding that for different JVMs results can differ in ~2 times on some workloads (like serialization of eu-lobby-country.json or parsing of reddit-scala.json):

GraalVM CE 19

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt       Score       Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5   21132.438 ±  1120.826  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5   30181.272 ±  1358.660  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5    3672.356 ±    76.221  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5   49906.195 ±   571.590  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5    4014.865 ±   183.444  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5    1464.669 ±    54.275  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5    2648.096 ±    23.948  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5    6776.827 ±   391.164  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5   33281.192 ±  2118.833  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5    1329.214 ±   107.486  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5     123.610 ±     8.865  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5    1522.018 ±    92.992  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5   28355.746 ±   217.154  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5   84865.886 ±  6899.279  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5   33827.780 ±  2094.253  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5     647.767 ±    13.806  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5   31602.470 ±   708.503  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5   25733.319 ±   747.606  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 australia-abc.json  thrpt    5   25410.040 ±   493.261  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       bitcoin.json  thrpt    5   54712.291 ±  3196.234  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                      doj-blog.json  thrpt    5    3053.943 ±    93.111  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel              eu-lobby-country.json  thrpt    5   64033.956 ±  2321.221  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel            eu-lobby-financial.json  thrpt    5   13872.084 ±   360.730  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 eu-lobby-repr.json  thrpt    5    3140.436 ±    10.352  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 github-events.json  thrpt    5    5060.933 ±   244.374  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  github-gists.json  thrpt    5    8490.624 ±    77.945  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                json-generator.json  thrpt    5   68334.839 ±  3850.692  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    meteorites.json  thrpt    5    1843.812 ±     6.327  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                        movies.json  thrpt    5     144.924 ±     5.338  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  reddit-scala.json  thrpt    5    7454.938 ±   196.525  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    rick-morty.json  thrpt    5   26518.929 ±  1467.802  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  temp-anomaly.json  thrpt    5  153564.730 ± 10423.373  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  thai-cinemas.json  thrpt    5   58097.670 ±  3353.852  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       turkish.json  thrpt    5     895.282 ±    31.152  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel  twitter_api_compact_response.json  thrpt    5   73305.487 ±   427.190  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel          twitter_api_response.json  thrpt    5   72228.330 ±  3680.419  ops/s

GraalVM EE 19

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt       Score      Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5   18581.684 ± 1506.857  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5   28761.964 ±  668.739  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5    4215.075 ±   80.557  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5   49405.739 ± 4788.250  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5    4128.428 ±  311.367  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5    1579.149 ±  136.012  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5    2515.023 ±   14.628  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5    7819.084 ±  222.084  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5   41025.343 ± 3686.300  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5    1485.627 ±   87.586  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5     139.218 ±   12.776  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5    1724.598 ±   10.562  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5   29243.991 ±  162.374  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5   87893.495 ± 3194.644  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5   40759.545 ±  236.671  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5     757.984 ±   13.959  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5   44471.422 ± 3571.524  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5   32695.003 ±  470.492  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 australia-abc.json  thrpt    5   27482.796 ± 2092.938  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       bitcoin.json  thrpt    5   65018.304 ±  311.584  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                      doj-blog.json  thrpt    5    3338.409 ±   85.690  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel              eu-lobby-country.json  thrpt    5   83414.856 ± 7231.398  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel            eu-lobby-financial.json  thrpt    5   18409.610 ±  829.034  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 eu-lobby-repr.json  thrpt    5    4061.998 ±  431.456  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 github-events.json  thrpt    5    5624.209 ±  522.397  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  github-gists.json  thrpt    5    9305.461 ± 1081.649  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                json-generator.json  thrpt    5   85684.715 ± 4870.233  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    meteorites.json  thrpt    5    2051.428 ±   25.754  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                        movies.json  thrpt    5     159.198 ±   15.335  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  reddit-scala.json  thrpt    5    8759.929 ±  780.701  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    rick-morty.json  thrpt    5   30571.710 ±  278.372  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  temp-anomaly.json  thrpt    5  168179.210 ± 3859.957  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  thai-cinemas.json  thrpt    5   78363.229 ± 4341.440  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       turkish.json  thrpt    5     952.839 ±  107.559  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel  twitter_api_compact_response.json  thrpt    5  101484.101 ± 6853.670  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel          twitter_api_response.json  thrpt    5  102556.364 ± 3567.522  ops/s

JDK 11

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt       Score      Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5   22448.763 ±  473.948  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5   32469.933 ±  360.648  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5    5251.357 ±  168.570  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5   41949.971 ± 4255.847  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5    7068.057 ±  154.768  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5    3134.996 ±  122.537  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5    3853.689 ±   40.133  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5   10095.199 ±  234.986  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5   45828.134 ± 3845.074  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5    1414.241 ±    5.780  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5     103.125 ±    1.398  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5    2567.776 ±   15.388  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5   31515.581 ±  681.707  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5   48582.345 ± 4440.882  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5   39555.776 ± 3676.919  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5     628.077 ±    7.550  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5   50067.208 ±  322.819  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5   41817.200 ±  330.464  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 australia-abc.json  thrpt    5   24430.310 ±   96.304  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       bitcoin.json  thrpt    5   55057.365 ±  650.783  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                      doj-blog.json  thrpt    5    2373.921 ±   21.020  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel              eu-lobby-country.json  thrpt    5   31882.819 ±  283.417  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel            eu-lobby-financial.json  thrpt    5   11577.368 ±  831.861  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 eu-lobby-repr.json  thrpt    5    3126.693 ±   38.953  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 github-events.json  thrpt    5    4651.414 ±   70.536  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  github-gists.json  thrpt    5    8042.511 ±  926.432  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                json-generator.json  thrpt    5   77496.766 ±  716.843  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    meteorites.json  thrpt    5    1365.764 ±   10.043  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                        movies.json  thrpt    5     100.098 ±    4.064  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  reddit-scala.json  thrpt    5    5923.117 ±  135.292  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    rick-morty.json  thrpt    5   35020.494 ± 2830.977  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  temp-anomaly.json  thrpt    5  135882.231 ± 1291.044  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  thai-cinemas.json  thrpt    5   48904.478 ± 4494.447  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       turkish.json  thrpt    5     658.008 ±   37.623  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel  twitter_api_compact_response.json  thrpt    5   71777.120 ±  600.985  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel          twitter_api_response.json  thrpt    5   71556.461 ± 1465.476  ops/s

JDK 8

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt       Score      Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5   19110.720 ±  116.072  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5   34525.466 ± 1398.149  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5    5156.803 ±  533.522  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5   45566.057 ± 3685.881  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5    8340.800 ±   92.612  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5    3203.598 ±  698.966  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5    3762.813 ±   67.741  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5    9779.282 ±  310.943  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5   46144.835 ± 2122.228  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5    1317.242 ±   30.327  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5     130.503 ±   11.934  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5    2210.175 ±  121.739  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5   32637.585 ± 3198.561  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5   59589.648 ±  404.774  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5   37384.649 ± 3218.889  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5     619.765 ±   19.944  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5   53310.641 ± 3099.917  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5   42688.956 ± 1402.021  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 australia-abc.json  thrpt    5   21953.720 ± 2522.183  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       bitcoin.json  thrpt    5   59348.149 ±  809.598  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                      doj-blog.json  thrpt    5    2181.393 ±  148.943  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel              eu-lobby-country.json  thrpt    5   70721.544 ±  281.476  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel            eu-lobby-financial.json  thrpt    5   12753.444 ± 1155.267  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 eu-lobby-repr.json  thrpt    5    2899.231 ±  268.092  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 github-events.json  thrpt    5    6526.537 ±  742.161  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  github-gists.json  thrpt    5   12158.038 ±  107.126  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                json-generator.json  thrpt    5   83890.297 ± 1584.441  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    meteorites.json  thrpt    5    1875.279 ±  178.789  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                        movies.json  thrpt    5     149.401 ±    0.821  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  reddit-scala.json  thrpt    5    4470.281 ±  427.286  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    rick-morty.json  thrpt    5   36083.626 ±  147.541  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  temp-anomaly.json  thrpt    5  138283.259 ± 5250.071  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  thai-cinemas.json  thrpt    5   64550.752 ± 1550.467  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       turkish.json  thrpt    5     652.603 ±   46.374  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel  twitter_api_compact_response.json  thrpt    5   82113.538 ±  765.605  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel          twitter_api_response.json  thrpt    5   81321.424 ± 1709.218  ops/s

@plokhotnyuk
Copy link
Contributor Author

plokhotnyuk commented May 19, 2019

I have updated version of jsoniter-scala to 0.48.2.

This version fixes performance loss during parsing of reddit-scala.json on GraalVM:

GraalVM CE 19

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt      Score      Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5  21137.298 ± 2576.091  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5  32038.213 ± 1062.448  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5   3590.197 ±   34.556  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5  51905.397 ± 3671.827  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5   6325.390 ±  716.095  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5   2680.334 ±   48.747  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5   3908.954 ±  231.286  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5   8895.996 ±  899.005  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5  46490.659 ± 1023.848  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5   1606.525 ±  137.692  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5    130.139 ±    2.728  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5   3914.164 ±  278.650  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5  31559.665 ±  432.116  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5  84685.130 ± 5112.514  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5  43454.440 ± 2561.089  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5    612.673 ±   10.274  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5  49290.528 ± 3639.866  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5  35394.890 ± 2596.423  ops/s

GraalVM EE 19

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt      Score      Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5  18649.927 ± 2948.764  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5  29339.551 ± 1193.023  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5   4190.885 ±  185.643  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5  49946.681 ± 1461.469  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5   7783.302 ±  978.287  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5   3032.458 ±  289.140  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5   3649.764 ±   81.827  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5   8104.469 ±  524.968  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5  40762.430 ± 4475.728  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5   1567.960 ±  216.720  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5    134.412 ±   16.538  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5   4673.060 ±  387.273  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5  29784.859 ± 1993.939  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5  83022.313 ± 2952.670  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5  41986.723 ±  429.288  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5    708.004 ±   25.669  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5  50103.576 ± 1009.380  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5  34276.330 ± 3464.259  ops/s

JDK 11

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt      Score      Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5  21168.759 ± 2146.859  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5  33388.169 ±  751.156  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5   5229.167 ±  552.448  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5  42660.861 ±  374.974  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5   7142.656 ±  747.711  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5   2921.801 ±  918.981  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5   3838.424 ±  105.571  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5   9271.434 ±   60.011  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5  45726.664 ± 2602.086  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5   1247.787 ±   12.668  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5    102.880 ±    4.364  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5   2549.818 ±  219.909  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5  30783.044 ± 2674.652  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5  47272.620 ± 4121.415  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5  39719.397 ± 2208.625  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5    611.501 ±   54.177  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5  52161.023 ± 2568.518  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5  40325.277 ± 3500.389  ops/s

JDK 8

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt      Score      Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5  18874.172 ± 3265.137  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5  33745.441 ± 2516.454  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5   5157.572 ±   90.678  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5  45987.342 ± 3651.595  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5   7719.530 ±  228.556  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5   3199.296 ±  274.966  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5   3638.730 ±  578.177  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5  10266.163 ±  273.609  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5  42273.638 ±  918.631  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5   1301.057 ±   73.119  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5    126.565 ±   10.843  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5   2227.216 ±  141.893  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5  28228.600 ± 3065.062  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5  56663.190 ± 4314.984  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5  41301.253 ± 3574.101  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5    634.201 ±   67.767  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5  51299.431 ± 5953.871  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5  40794.099 ± 3223.575  ops/s

@plokhotnyuk
Copy link
Contributor Author

plokhotnyuk commented May 21, 2019

Yet another finding is that turning off of hyper-threading improves results of some benchmarks (see encodeModel eu-lobby-country.json, and encodeModel movies.json):

JDK 11

[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark                                                       (fileName)   Mode  Cnt       Score      Error  Units
[info] JsoniterScalaModelBenchmark.decodeModel                 australia-abc.json  thrpt    5   20589.366 ± 3053.148  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       bitcoin.json  thrpt    5   32358.282 ± 4419.401  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                      doj-blog.json  thrpt    5    5219.061 ±  212.971  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel              eu-lobby-country.json  thrpt    5   42481.348 ± 4229.312  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel            eu-lobby-financial.json  thrpt    5    7356.698 ±  271.804  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 eu-lobby-repr.json  thrpt    5    3056.842 ±  636.388  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                 github-events.json  thrpt    5    3743.474 ±  209.539  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  github-gists.json  thrpt    5    8945.164 ± 1219.664  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                json-generator.json  thrpt    5   44836.845 ± 1748.093  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    meteorites.json  thrpt    5    1295.215 ±   34.269  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                        movies.json  thrpt    5     100.992 ±   12.058  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  reddit-scala.json  thrpt    5    2578.642 ±  216.200  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                    rick-morty.json  thrpt    5   31200.330 ± 1319.421  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  temp-anomaly.json  thrpt    5   48484.662 ± 2514.038  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                  thai-cinemas.json  thrpt    5   38467.634 ± 3397.670  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel                       turkish.json  thrpt    5     602.343 ±   66.217  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel  twitter_api_compact_response.json  thrpt    5   50350.910 ± 5763.956  ops/s
[info] JsoniterScalaModelBenchmark.decodeModel          twitter_api_response.json  thrpt    5   40822.327 ± 1029.571  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 australia-abc.json  thrpt    5   17698.090 ± 2277.074  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       bitcoin.json  thrpt    5   57129.450 ± 1969.983  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                      doj-blog.json  thrpt    5    2469.409 ±  134.204  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel              eu-lobby-country.json  thrpt    5   58869.590 ± 2676.087  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel            eu-lobby-financial.json  thrpt    5   11627.932 ±  504.421  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 eu-lobby-repr.json  thrpt    5    3148.936 ±  314.772  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                 github-events.json  thrpt    5    4187.138 ±  546.933  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  github-gists.json  thrpt    5    8413.957 ±  691.462  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                json-generator.json  thrpt    5   76648.070 ± 6566.122  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    meteorites.json  thrpt    5    1312.493 ±  132.494  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                        movies.json  thrpt    5     128.314 ±    6.244  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  reddit-scala.json  thrpt    5    5844.320 ±  519.633  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                    rick-morty.json  thrpt    5   35733.880 ± 4177.861  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  temp-anomaly.json  thrpt    5  135392.142 ±  367.454  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                  thai-cinemas.json  thrpt    5   48290.094 ± 4850.387  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel                       turkish.json  thrpt    5     688.823 ±   17.964  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel  twitter_api_compact_response.json  thrpt    5   70972.539 ± 2960.590  ops/s
[info] JsoniterScalaModelBenchmark.encodeModel          twitter_api_response.json  thrpt    5   72124.368 ± 2794.050  ops/s

@sirthias sirthias merged commit 8d6dc68 into sirthias:master Jun 6, 2019
@plokhotnyuk plokhotnyuk deleted the add-jsoniter-scala-benchmarks branch June 6, 2019 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants