2021 08 30 bench results
Andrea edited this page Sep 2, 2021
·
2 revisions
Benchmarked on: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz, 32G RAM
Config:
1ae5a57 Thu, 26 Aug 2021 11:18:44 +0200 LesnyRumcajs Use released versions for Akka / Akka HTTP
- GRPC_BENCHMARK_DURATION=120s
- GRPC_BENCHMARK_WARMUP=30s
- GRPC_SERVER_CPUS=2
- GRPC_SERVER_RAM=512m
- GRPC_CLIENT_CONNECTIONS=50
- GRPC_CLIENT_CONCURRENCY=1000
- GRPC_CLIENT_QPS=0
- GRPC_CLIENT_CPUS=9
- GRPC_REQUEST_PAYLOAD=100B
1 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| rust_tonic_st | 50173 | 19.86 ms | 23.77 ms | 25.18 ms | 27.54 ms | 100.94% | 15.97 MiB |
| rust_thruster_st | 43902 | 22.71 ms | 27.69 ms | 29.02 ms | 31.90 ms | 102.48% | 11.72 MiB |
| rust_tonic_mt | 41817 | 23.84 ms | 10.06 ms | 12.15 ms | 594.18 ms | 101.55% | 15.26 MiB |
| scala_akka | 41636 | 23.87 ms | 56.16 ms | 66.41 ms | 90.88 ms | 101.97% | 239.43 MiB |
| rust_grpcio | 36535 | 27.31 ms | 30.37 ms | 31.05 ms | 33.57 ms | 102.6% | 24.35 MiB |
| rust_thruster_mt | 36506 | 27.32 ms | 11.68 ms | 13.70 ms | 696.06 ms | 101.78% | 12.71 MiB |
| java_quarkus | 36324 | 27.45 ms | 48.86 ms | 53.13 ms | 77.79 ms | 100.65% | 104.45 MiB |
| dotnet_grpc | 35070 | 28.39 ms | 40.22 ms | 67.19 ms | 80.80 ms | 100.78% | 92.62 MiB |
| java_hotspot_grpc_pgc | 34474 | 28.92 ms | 53.24 ms | 58.33 ms | 82.85 ms | 100.78% | 196.98 MiB |
| java_hotspot_grpc_g1gc | 30514 | 32.68 ms | 58.48 ms | 65.18 ms | 91.50 ms | 101.21% | 174.54 MiB |
| cpp_grpc_mt | 30186 | 33.06 ms | 36.72 ms | 38.09 ms | 40.42 ms | 101.91% | 8.3 MiB |
| java_hotspot_grpc_zgc | 30015 | 33.23 ms | 60.40 ms | 69.22 ms | 85.09 ms | 101.35% | 117.71 MiB |
| cpp_grpc_st | 29962 | 33.30 ms | 37.61 ms | 38.69 ms | 41.44 ms | 102.37% | 8.37 MiB |
| java_hotspot_grpc_she | 28284 | 35.27 ms | 65.76 ms | 82.02 ms | 96.56 ms | 100.96% | 404.66 MiB |
| java_openj9_grpc_gencon | 28116 | 35.47 ms | 63.07 ms | 68.81 ms | 119.22 ms | 95.47% | 121.48 MiB |
| java_hotspot_grpc_sgc | 25025 | 39.86 ms | 68.15 ms | 74.96 ms | 87.99 ms | 101.93% | 83.42 MiB |
| crystal_grpc | 22026 | 45.28 ms | 52.73 ms | 55.53 ms | 58.97 ms | 113.45% | 33.41 MiB |
| kotlin_grpc | 17461 | 57.14 ms | 101.14 ms | 107.14 ms | 177.55 ms | 100.9% | 218.98 MiB |
| go_grpc | 14259 | 69.94 ms | 110.94 ms | 182.79 ms | 205.82 ms | 101.25% | 36.54 MiB |
| java_micronaut | 13372 | 74.64 ms | 101.19 ms | 109.45 ms | 190.25 ms | 102.47% | 471.31 MiB |
| java_micronaut_workstealing | 13256 | 75.29 ms | 98.92 ms | 108.55 ms | 189.83 ms | 101.77% | 449.36 MiB |
| node_grpc_st | 12059 | 82.75 ms | 93.19 ms | 97.66 ms | 104.41 ms | 99.9% | 51.06 MiB |
| swift_grpc | 11276 | 88.50 ms | 90.27 ms | 90.94 ms | 92.41 ms | 101.37% | 6.1 MiB |
| haskell_mu | 11187 | 89.23 ms | 106.46 ms | 111.98 ms | 126.49 ms | 101.86% | 125.64 MiB |
| scala_zio | 6126 | 162.88 ms | 215.91 ms | 291.87 ms | 398.72 ms | 100.16% | 273.31 MiB |
| node_grpcjs_st | 5348 | 186.55 ms | 203.22 ms | 208.40 ms | 216.77 ms | 100.35% | 43.45 MiB |
| csharp_grpc | 5337 | 186.94 ms | 209.74 ms | 215.23 ms | 266.33 ms | 102.21% | 105.09 MiB |
| elixir_grpc | 5141 | 194.09 ms | 219.17 ms | 224.50 ms | 239.16 ms | 101.67% | 74.79 MiB |
| scala_fs2 | 2961 | 336.63 ms | 388.28 ms | 496.40 ms | 583.95 ms | 102.2% | 221.82 MiB |
| python_grpc | 2191 | 454.50 ms | 484.32 ms | 488.37 ms | 494.40 ms | 101.97% | 33.66 MiB |
| ruby_grpc | 1957 | 508.63 ms | 563.00 ms | 579.65 ms | 638.52 ms | 102.11% | 32.93 MiB |
| java_aot | 1796 | 554.03 ms | 912.01 ms | 1.09 s | 1.39 s | 100.51% | 184.96 MiB |
| php_grpc | 1585 | 627.29 ms | 693.03 ms | 694.95 ms | 698.36 ms | 101.48% | 66.54 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
2 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| scala_akka | 81435 | 12.08 ms | 22.33 ms | 29.06 ms | 57.40 ms | 180.39% | 242.08 MiB |
| cpp_grpc_mt | 67635 | 14.55 ms | 27.80 ms | 32.75 ms | 42.48 ms | 203.79% | 27.89 MiB |
| rust_tonic_mt | 63485 | 15.66 ms | 38.40 ms | 69.30 ms | 108.06 ms | 200.99% | 16.17 MiB |
| java_hotspot_grpc_pgc | 62621 | 15.78 ms | 30.53 ms | 36.22 ms | 59.03 ms | 204.14% | 198.26 MiB |
| dotnet_grpc | 61093 | 16.21 ms | 25.18 ms | 31.93 ms | 41.54 ms | 200.33% | 153.28 MiB |
| java_hotspot_grpc_g1gc | 58060 | 17.11 ms | 35.04 ms | 42.28 ms | 62.13 ms | 203.86% | 170.57 MiB |
| rust_grpcio | 57807 | 17.20 ms | 24.49 ms | 26.93 ms | 33.02 ms | 203.84% | 52.66 MiB |
| rust_thruster_mt | 55439 | 17.95 ms | 47.31 ms | 80.62 ms | 132.57 ms | 200.9% | 14.09 MiB |
| java_openj9_grpc_gencon | 53357 | 18.60 ms | 45.18 ms | 54.86 ms | 68.34 ms | 203.69% | 152.89 MiB |
| java_hotspot_grpc_she | 53227 | 18.57 ms | 41.42 ms | 54.96 ms | 68.61 ms | 203.3% | 394.35 MiB |
| java_hotspot_grpc_zgc | 52976 | 18.67 ms | 39.78 ms | 50.68 ms | 66.25 ms | 203.04% | 119.31 MiB |
| java_quarkus | 50172 | 19.79 ms | 41.43 ms | 44.71 ms | 51.40 ms | 164.53% | 100.96 MiB |
| rust_tonic_st | 49874 | 19.98 ms | 21.77 ms | 22.65 ms | 24.80 ms | 102.53% | 14.15 MiB |
| kotlin_grpc | 49567 | 19.98 ms | 50.10 ms | 63.53 ms | 73.81 ms | 204.39% | 218.71 MiB |
| java_hotspot_grpc_sgc | 48845 | 20.34 ms | 37.15 ms | 42.62 ms | 59.24 ms | 197.71% | 92.89 MiB |
| rust_thruster_st | 44112 | 22.60 ms | 27.11 ms | 28.51 ms | 31.58 ms | 101.68% | 11.69 MiB |
| go_grpc | 39332 | 25.20 ms | 64.97 ms | 72.22 ms | 86.06 ms | 204.3% | 34.91 MiB |
| cpp_grpc_st | 30181 | 33.06 ms | 36.53 ms | 37.99 ms | 39.99 ms | 102.71% | 8.73 MiB |
| crystal_grpc | 22007 | 45.32 ms | 52.86 ms | 55.65 ms | 59.28 ms | 113.88% | 33.86 MiB |
| java_micronaut | 21801 | 45.74 ms | 67.27 ms | 74.39 ms | 88.34 ms | 164.08% | 508.42 MiB |
| swift_grpc | 21332 | 46.76 ms | 49.16 ms | 50.10 ms | 52.85 ms | 201.29% | 6.19 MiB |
| java_micronaut_workstealing | 20084 | 49.65 ms | 72.47 ms | 79.85 ms | 91.48 ms | 155.03% | 503.14 MiB |
| csharp_grpc | 18374 | 54.30 ms | 85.16 ms | 89.07 ms | 95.94 ms | 204.91% | 110.1 MiB |
| haskell_mu | 18073 | 55.23 ms | 70.78 ms | 74.24 ms | 80.96 ms | 202.94% | 176.19 MiB |
| node_grpc_st | 12668 | 78.77 ms | 87.03 ms | 92.78 ms | 97.27 ms | 103.21% | 48.58 MiB |
| scala_zio | 11180 | 89.27 ms | 123.90 ms | 186.14 ms | 210.50 ms | 202.07% | 264.27 MiB |
| elixir_grpc | 9722 | 102.68 ms | 119.18 ms | 124.50 ms | 135.16 ms | 204.3% | 86.41 MiB |
| node_grpcjs_st | 7016 | 142.25 ms | 154.18 ms | 157.18 ms | 163.81 ms | 114.98% | 43.5 MiB |
| java_aot | 6975 | 143.06 ms | 274.86 ms | 302.85 ms | 396.52 ms | 194.78% | 226.29 MiB |
| scala_fs2 | 6534 | 152.75 ms | 184.26 ms | 189.07 ms | 208.78 ms | 207.6% | 233.63 MiB |
| php_grpc | 3995 | 249.63 ms | 288.68 ms | 289.95 ms | 292.05 ms | 205.18% | 66.78 MiB |
| python_grpc | 3978 | 250.72 ms | 276.09 ms | 280.59 ms | 290.77 ms | 151.57% | 36.15 MiB |
| ruby_grpc | 2627 | 379.08 ms | 410.69 ms | 418.88 ms | 453.59 ms | 133.43% | 34.24 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
3 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| scala_akka | 86775 | 11.30 ms | 19.51 ms | 23.87 ms | 34.58 ms | 192.35% | 234.54 MiB |
| java_hotspot_grpc_pgc | 84981 | 11.41 ms | 23.07 ms | 30.19 ms | 46.77 ms | 256.99% | 209.76 MiB |
| cpp_grpc_mt | 83642 | 11.61 ms | 20.97 ms | 27.18 ms | 44.43 ms | 267.62% | 38.03 MiB |
| java_hotspot_grpc_g1gc | 81762 | 11.91 ms | 23.38 ms | 30.03 ms | 44.68 ms | 276.55% | 174.71 MiB |
| java_hotspot_grpc_she | 81579 | 11.92 ms | 23.58 ms | 30.33 ms | 45.59 ms | 274.45% | 403.76 MiB |
| java_openj9_grpc_gencon | 81508 | 11.92 ms | 24.09 ms | 31.19 ms | 45.80 ms | 283.39% | 154.64 MiB |
| java_hotspot_grpc_zgc | 80576 | 12.07 ms | 23.81 ms | 31.39 ms | 46.47 ms | 290.39% | 118.17 MiB |
| rust_tonic_mt | 79908 | 12.22 ms | 23.85 ms | 31.57 ms | 48.69 ms | 278.67% | 16.69 MiB |
| dotnet_grpc | 79251 | 12.40 ms | 18.82 ms | 22.19 ms | 31.62 ms | 240.36% | 127.01 MiB |
| rust_grpcio | 78930 | 12.18 ms | 22.02 ms | 29.35 ms | 51.00 ms | 270.26% | 37.47 MiB |
| go_grpc | 78685 | 12.46 ms | 22.91 ms | 27.55 ms | 38.50 ms | 306.83% | 34.3 MiB |
| kotlin_grpc | 76695 | 12.73 ms | 24.58 ms | 33.25 ms | 47.89 ms | 293.5% | 219.94 MiB |
| rust_thruster_mt | 73594 | 13.34 ms | 26.20 ms | 33.88 ms | 51.38 ms | 283.2% | 14.08 MiB |
| java_hotspot_grpc_sgc | 61419 | 16.06 ms | 30.63 ms | 36.54 ms | 46.77 ms | 240.59% | 106.81 MiB |
| java_quarkus | 58550 | 16.87 ms | 33.79 ms | 42.86 ms | 52.81 ms | 202.7% | 112.18 MiB |
| rust_tonic_st | 50164 | 19.86 ms | 24.19 ms | 25.40 ms | 28.02 ms | 101.16% | 13.9 MiB |
| rust_thruster_st | 44035 | 22.64 ms | 27.19 ms | 28.37 ms | 31.35 ms | 101.19% | 11.55 MiB |
| csharp_grpc | 35428 | 28.11 ms | 50.50 ms | 55.11 ms | 63.32 ms | 306.65% | 110.46 MiB |
| java_micronaut | 33469 | 29.73 ms | 45.56 ms | 52.44 ms | 65.02 ms | 220.63% | 510.64 MiB |
| java_micronaut_workstealing | 33420 | 29.75 ms | 45.36 ms | 52.44 ms | 66.86 ms | 240.82% | 511.55 MiB |
| swift_grpc | 30182 | 33.04 ms | 35.80 ms | 36.59 ms | 38.45 ms | 301.21% | 6.46 MiB |
| cpp_grpc_st | 29616 | 33.69 ms | 37.47 ms | 38.83 ms | 41.12 ms | 102.55% | 8.73 MiB |
| haskell_mu | 23473 | 42.51 ms | 81.02 ms | 91.68 ms | 109.77 ms | 300.07% | 219.71 MiB |
| crystal_grpc | 22005 | 45.33 ms | 52.97 ms | 55.84 ms | 59.65 ms | 112.97% | 33.44 MiB |
| scala_zio | 17546 | 56.87 ms | 93.46 ms | 107.04 ms | 145.90 ms | 306.77% | 293.41 MiB |
| elixir_grpc | 13881 | 71.91 ms | 85.20 ms | 90.05 ms | 100.09 ms | 304.26% | 92.0 MiB |
| java_aot | 13095 | 76.20 ms | 150.97 ms | 185.55 ms | 233.36 ms | 281.38% | 251.49 MiB |
| scala_fs2 | 12870 | 77.50 ms | 94.34 ms | 105.58 ms | 165.09 ms | 318.32% | 238.89 MiB |
| node_grpc_st | 12551 | 79.50 ms | 87.49 ms | 93.12 ms | 98.00 ms | 103.5% | 49.91 MiB |
| node_grpcjs_st | 7025 | 142.07 ms | 153.22 ms | 155.53 ms | 164.58 ms | 114.76% | 43.18 MiB |
| php_grpc | 6654 | 149.99 ms | 181.08 ms | 182.45 ms | 185.74 ms | 308.07% | 66.2 MiB |
| python_grpc | 4058 | 245.82 ms | 273.83 ms | 277.90 ms | 287.41 ms | 150.51% | 33.99 MiB |
| ruby_grpc | 2604 | 382.61 ms | 413.96 ms | 422.51 ms | 450.65 ms | 133.48% | 31.27 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------