2022 03 15 bench results
Andrea edited this page Mar 25, 2022
·
3 revisions
Benchmarked on: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz, 32G RAM
Config:
f86892d Mon, 14 Mar 2022 17:04:36 +0100 GitHub =akka update akka-http version to 10.2.9 (#207)
- GRPC_BENCHMARK_DURATION=120s
- GRPC_BENCHMARK_WARMUP=30s
- GRPC_SERVER_RAM=512m
- GRPC_CLIENT_CONNECTIONS=50
- GRPC_CLIENT_CONCURRENCY=1000
- GRPC_CLIENT_QPS=0
- GRPC_CLIENT_CPUS=9
- GRPC_REQUEST_SCENARIO=complex_proto
1 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| rust_tonic_st | 48427 | 20.37 ms | 25.23 ms | 26.27 ms | 28.42 ms | 100.87% | 13.36 MiB |
| rust_thruster_st | 44557 | 22.19 ms | 24.45 ms | 28.63 ms | 31.50 ms | 100.61% | 9.2 MiB |
| rust_tonic_mt | 41447 | 23.91 ms | 10.37 ms | 11.67 ms | 585.66 ms | 101.08% | 14.9 MiB |
| rust_thruster_mt | 37927 | 26.16 ms | 11.39 ms | 12.71 ms | 653.28 ms | 100.76% | 10.65 MiB |
| scala_akka | 35289 | 27.08 ms | 62.93 ms | 72.65 ms | 92.49 ms | 102.12% | 228.74 MiB |
| dotnet_grpc | 34512 | 26.58 ms | 44.07 ms | 63.72 ms | 80.59 ms | 100.35% | 110.56 MiB |
| rust_grpcio | 33434 | 29.73 ms | 32.93 ms | 34.04 ms | 36.51 ms | 101.8% | 18.0 MiB |
| java_quarkus | 33361 | 29.56 ms | 52.31 ms | 56.11 ms | 75.22 ms | 100.21% | 107.22 MiB |
| java_hotspot_grpc_pgc | 32881 | 29.72 ms | 56.07 ms | 60.60 ms | 73.43 ms | 101.55% | 190.3 MiB |
| java_hotspot_grpc_g1gc | 29764 | 32.99 ms | 61.18 ms | 71.49 ms | 96.73 ms | 101.77% | 163.01 MiB |
| cpp_grpc_mt | 28514 | 34.96 ms | 39.73 ms | 40.76 ms | 43.29 ms | 102.45% | 9.59 MiB |
| cpp_grpc_st | 28453 | 35.04 ms | 40.02 ms | 40.96 ms | 43.60 ms | 102.78% | 8.91 MiB |
| java_hotspot_grpc_she | 28181 | 34.91 ms | 63.60 ms | 73.15 ms | 89.82 ms | 102.18% | 403.17 MiB |
| java_hotspot_grpc_zgc | 28131 | 34.95 ms | 61.54 ms | 72.90 ms | 90.50 ms | 101.54% | 111.59 MiB |
| cpp_asio_grpc | 26479 | 37.66 ms | 42.98 ms | 44.13 ms | 46.78 ms | 102.16% | 8.81 MiB |
| java_openj9_grpc_gencon | 25071 | 39.04 ms | 70.87 ms | 86.85 ms | 275.74 ms | 95.94% | 116.7 MiB |
| java_hotspot_grpc_sgc | 22914 | 43.06 ms | 78.47 ms | 86.06 ms | 112.83 ms | 101.35% | 90.15 MiB |
| crystal_grpc | 21628 | 45.41 ms | 54.45 ms | 55.92 ms | 60.24 ms | 124.66% | 35.34 MiB |
| java_quarkus_native | 20436 | 48.68 ms | 53.50 ms | 55.13 ms | 60.60 ms | 100.86% | 88.13 MiB |
| scala_fs2 | 18812 | 52.42 ms | 90.38 ms | 98.79 ms | 167.79 ms | 101.7% | 224.79 MiB |
| kotlin_grpc | 17610 | 56.11 ms | 93.23 ms | 100.94 ms | 162.03 ms | 101.22% | 210.4 MiB |
| go_vtgrpc | 14673 | 64.26 ms | 101.45 ms | 109.55 ms | 191.09 ms | 96.16% | 29.76 MiB |
| java_micronaut | 12666 | 78.54 ms | 100.83 ms | 110.65 ms | 188.25 ms | 101.52% | 468.29 MiB |
| swift_grpc | 12048 | 82.78 ms | 84.32 ms | 84.89 ms | 86.12 ms | 101.24% | 5.91 MiB |
| java_micronaut_workstealing | 11744 | 84.70 ms | 101.60 ms | 110.55 ms | 196.25 ms | 101.92% | 458.23 MiB |
| go_grpc | 10112 | 95.47 ms | 195.52 ms | 211.53 ms | 306.24 ms | 96.53% | 30.21 MiB |
| scala_zio | 5813 | 171.22 ms | 249.22 ms | 301.15 ms | 404.30 ms | 101.2% | 265.96 MiB |
| erlang_grpcbox | 5111 | 195.19 ms | 203.83 ms | 207.54 ms | 221.08 ms | 102.06% | 94.76 MiB |
| csharp_grpc | 4750 | 209.95 ms | 264.37 ms | 276.06 ms | 287.48 ms | 102.11% | 102.38 MiB |
| elixir_grpc | 4606 | 216.59 ms | 243.07 ms | 249.33 ms | 262.76 ms | 101.84% | 129.12 MiB |
| node_grpcjs_st | 4212 | 236.60 ms | 264.86 ms | 270.63 ms | 280.78 ms | 100.24% | 45.33 MiB |
| python_async_grpc | 2580 | 386.14 ms | 413.36 ms | 423.25 ms | 454.68 ms | 101.86% | 32.81 MiB |
| python_grpc | 2177 | 457.41 ms | 486.49 ms | 491.82 ms | 500.16 ms | 102.12% | 33.61 MiB |
| ruby_grpc | 1898 | 524.30 ms | 583.15 ms | 595.06 ms | 668.94 ms | 101.23% | 35.54 MiB |
| java_aot | 1649 | 602.26 ms | 1.01 s | 1.18 s | 1.49 s | 100.39% | 189.89 MiB |
| d_grpc | 1499 | 659.46 ms | 503.44 ms | 4.21 s | 5.80 s | 104.51% | 27.27 MiB |
| php_grpc | 858 | 1.15 s | 1.20 s | 1.20 s | 1.28 s | 101.3% | 65.49 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
2 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| scala_akka | 74192 | 10.97 ms | 21.24 ms | 27.24 ms | 55.99 ms | 195.8% | 233.54 MiB |
| dotnet_grpc | 68616 | 12.10 ms | 20.69 ms | 25.30 ms | 34.67 ms | 199.74% | 179.23 MiB |
| cpp_asio_grpc | 67126 | 12.99 ms | 25.25 ms | 30.04 ms | 38.41 ms | 203.15% | 34.35 MiB |
| rust_tonic_mt | 65457 | 14.26 ms | 37.82 ms | 59.56 ms | 88.95 ms | 199.02% | 18.35 MiB |
| cpp_grpc_mt | 63885 | 14.00 ms | 25.47 ms | 29.98 ms | 36.93 ms | 204.29% | 30.66 MiB |
| rust_thruster_mt | 60303 | 15.54 ms | 42.20 ms | 62.07 ms | 88.96 ms | 200.39% | 12.28 MiB |
| java_hotspot_grpc_pgc | 58789 | 15.02 ms | 33.75 ms | 42.63 ms | 61.69 ms | 203.0% | 208.68 MiB |
| rust_grpcio | 55047 | 17.49 ms | 23.04 ms | 24.60 ms | 28.26 ms | 202.79% | 39.34 MiB |
| java_hotspot_grpc_g1gc | 52677 | 17.24 ms | 38.80 ms | 50.12 ms | 68.21 ms | 203.16% | 198.45 MiB |
| java_hotspot_grpc_she | 51144 | 17.80 ms | 43.70 ms | 57.69 ms | 71.46 ms | 203.52% | 402.76 MiB |
| java_hotspot_grpc_zgc | 49187 | 18.59 ms | 41.64 ms | 49.76 ms | 64.52 ms | 203.67% | 116.63 MiB |
| rust_tonic_st | 48210 | 20.48 ms | 22.06 ms | 22.85 ms | 24.29 ms | 102.57% | 12.94 MiB |
| java_openj9_grpc_gencon | 48204 | 18.54 ms | 51.99 ms | 60.43 ms | 69.34 ms | 203.21% | 138.29 MiB |
| java_hotspot_grpc_sgc | 46041 | 20.21 ms | 38.52 ms | 44.56 ms | 72.97 ms | 197.61% | 91.99 MiB |
| java_quarkus | 45264 | 20.69 ms | 41.78 ms | 45.77 ms | 53.21 ms | 167.21% | 116.59 MiB |
| rust_thruster_st | 44341 | 22.29 ms | 27.54 ms | 28.92 ms | 31.46 ms | 102.17% | 9.4 MiB |
| kotlin_grpc | 41517 | 22.03 ms | 59.20 ms | 69.02 ms | 80.04 ms | 203.47% | 215.38 MiB |
| go_vtgrpc | 40974 | 20.47 ms | 55.66 ms | 62.29 ms | 73.06 ms | 205.34% | 29.87 MiB |
| scala_fs2 | 34588 | 27.02 ms | 66.79 ms | 73.71 ms | 84.19 ms | 203.71% | 252.48 MiB |
| java_quarkus_native | 34436 | 28.69 ms | 33.74 ms | 35.61 ms | 40.31 ms | 195.35% | 88.9 MiB |
| cpp_grpc_st | 28691 | 34.74 ms | 38.92 ms | 40.34 ms | 42.42 ms | 102.24% | 9.35 MiB |
| go_grpc | 27319 | 32.82 ms | 79.73 ms | 86.99 ms | 100.43 ms | 202.97% | 30.75 MiB |
| java_micronaut_workstealing | 26814 | 36.40 ms | 67.61 ms | 72.68 ms | 83.20 ms | 189.64% | 505.03 MiB |
| java_micronaut | 24395 | 40.01 ms | 68.41 ms | 74.28 ms | 85.31 ms | 184.19% | 502.54 MiB |
| swift_grpc | 22737 | 43.77 ms | 45.93 ms | 47.28 ms | 49.12 ms | 200.57% | 6.15 MiB |
| crystal_grpc | 21610 | 45.45 ms | 54.22 ms | 55.80 ms | 60.28 ms | 126.07% | 36.37 MiB |
| csharp_grpc | 14543 | 68.13 ms | 94.29 ms | 98.59 ms | 105.72 ms | 205.36% | 105.23 MiB |
| scala_zio | 10448 | 94.47 ms | 161.10 ms | 189.39 ms | 267.95 ms | 201.17% | 268.59 MiB |
| erlang_grpcbox | 9427 | 105.88 ms | 116.97 ms | 120.85 ms | 130.15 ms | 202.89% | 81.6 MiB |
| elixir_grpc | 8427 | 118.45 ms | 137.29 ms | 143.22 ms | 155.71 ms | 203.86% | 123.85 MiB |
| node_grpcjs_st | 6450 | 154.41 ms | 162.49 ms | 166.92 ms | 173.39 ms | 131.25% | 46.63 MiB |
| java_aot | 6257 | 158.15 ms | 295.06 ms | 331.06 ms | 423.77 ms | 195.38% | 227.31 MiB |
| python_async_grpc | 4736 | 210.67 ms | 233.06 ms | 235.90 ms | 240.59 ms | 146.69% | 34.95 MiB |
| python_grpc | 3892 | 256.27 ms | 279.70 ms | 283.71 ms | 290.99 ms | 150.6% | 33.46 MiB |
| d_grpc | 3474 | 285.26 ms | 833.01 ms | 957.24 ms | 1.10 s | 211.8% | 29.21 MiB |
| ruby_grpc | 2493 | 399.70 ms | 433.73 ms | 445.27 ms | 487.76 ms | 128.15% | 38.79 MiB |
| php_grpc | 2262 | 440.39 ms | 485.49 ms | 486.87 ms | 489.06 ms | 204.98% | 66.53 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
3 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| scala_akka | 78492 | 9.78 ms | 17.30 ms | 21.55 ms | 31.34 ms | 217.45% | 247.59 MiB |
| cpp_grpc_mt | 78200 | 9.17 ms | 17.08 ms | 22.11 ms | 34.90 ms | 255.42% | 41.55 MiB |
| cpp_asio_grpc | 77755 | 9.26 ms | 17.18 ms | 22.06 ms | 34.60 ms | 259.55% | 41.32 MiB |
| java_hotspot_grpc_pgc | 77171 | 9.67 ms | 19.84 ms | 24.72 ms | 35.70 ms | 265.72% | 210.33 MiB |
| dotnet_grpc | 77111 | 10.13 ms | 16.03 ms | 19.56 ms | 28.12 ms | 220.87% | 173.13 MiB |
| rust_tonic_mt | 76680 | 10.70 ms | 20.73 ms | 27.36 ms | 40.09 ms | 269.36% | 19.38 MiB |
| go_vtgrpc | 76411 | 9.32 ms | 17.28 ms | 20.61 ms | 28.72 ms | 296.13% | 29.27 MiB |
| java_hotspot_grpc_g1gc | 75420 | 9.95 ms | 20.40 ms | 26.59 ms | 39.60 ms | 274.24% | 179.62 MiB |
| java_hotspot_grpc_zgc | 74683 | 10.11 ms | 20.73 ms | 27.18 ms | 40.25 ms | 289.73% | 125.57 MiB |
| java_hotspot_grpc_she | 73951 | 10.32 ms | 21.66 ms | 28.46 ms | 42.52 ms | 286.25% | 429.26 MiB |
| java_openj9_grpc_gencon | 73575 | 10.33 ms | 20.88 ms | 27.10 ms | 39.48 ms | 294.21% | 135.22 MiB |
| rust_grpcio | 73549 | 11.31 ms | 18.66 ms | 23.29 ms | 35.03 ms | 273.59% | 30.8 MiB |
| rust_thruster_mt | 71890 | 11.74 ms | 23.30 ms | 29.70 ms | 42.79 ms | 270.6% | 12.85 MiB |
| kotlin_grpc | 67397 | 11.65 ms | 32.03 ms | 47.00 ms | 62.76 ms | 303.45% | 226.32 MiB |
| scala_fs2 | 59752 | 13.87 ms | 31.16 ms | 42.31 ms | 59.67 ms | 307.88% | 249.68 MiB |
| java_hotspot_grpc_sgc | 56537 | 15.11 ms | 31.28 ms | 36.67 ms | 45.20 ms | 242.91% | 98.74 MiB |
| java_quarkus | 53713 | 16.31 ms | 29.13 ms | 41.40 ms | 49.73 ms | 209.11% | 109.59 MiB |
| go_grpc | 50784 | 16.00 ms | 36.81 ms | 44.99 ms | 59.99 ms | 307.53% | 30.32 MiB |
| rust_tonic_st | 48347 | 20.42 ms | 25.09 ms | 26.29 ms | 28.72 ms | 101.35% | 12.79 MiB |
| rust_thruster_st | 44483 | 22.22 ms | 27.67 ms | 28.96 ms | 31.45 ms | 100.73% | 9.39 MiB |
| java_quarkus_native | 43723 | 22.14 ms | 28.03 ms | 30.70 ms | 37.03 ms | 285.09% | 89.95 MiB |
| java_micronaut | 36876 | 25.34 ms | 43.93 ms | 51.82 ms | 64.12 ms | 275.35% | 510.89 MiB |
| java_micronaut_workstealing | 34734 | 27.05 ms | 48.60 ms | 55.73 ms | 67.88 ms | 264.43% | 509.09 MiB |
| swift_grpc | 32165 | 30.79 ms | 32.91 ms | 33.83 ms | 35.16 ms | 301.72% | 6.49 MiB |
| csharp_grpc | 30186 | 31.69 ms | 54.77 ms | 59.54 ms | 67.31 ms | 307.12% | 113.88 MiB |
| cpp_grpc_st | 29016 | 34.35 ms | 37.39 ms | 39.47 ms | 41.82 ms | 102.35% | 8.99 MiB |
| crystal_grpc | 21456 | 45.78 ms | 55.21 ms | 56.20 ms | 60.72 ms | 127.08% | 37.67 MiB |
| scala_zio | 16519 | 59.56 ms | 89.10 ms | 100.37 ms | 158.40 ms | 306.25% | 306.21 MiB |
| elixir_grpc | 12368 | 80.71 ms | 95.42 ms | 101.16 ms | 113.42 ms | 305.57% | 132.42 MiB |
| java_aot | 11829 | 82.88 ms | 169.03 ms | 194.44 ms | 249.44 ms | 284.64% | 211.96 MiB |
| erlang_grpcbox | 10560 | 94.50 ms | 103.60 ms | 106.75 ms | 115.04 ms | 286.71% | 95.07 MiB |
| d_grpc | 6464 | 153.11 ms | 298.32 ms | 311.44 ms | 389.97 ms | 329.29% | 27.08 MiB |
| node_grpcjs_st | 6426 | 154.94 ms | 162.05 ms | 168.06 ms | 174.19 ms | 131.67% | 45.89 MiB |
| python_async_grpc | 4796 | 208.05 ms | 230.37 ms | 233.82 ms | 238.56 ms | 149.05% | 32.86 MiB |
| python_grpc | 3875 | 257.40 ms | 279.74 ms | 283.37 ms | 293.27 ms | 151.05% | 33.56 MiB |
| php_grpc | 3567 | 279.58 ms | 295.18 ms | 296.22 ms | 298.99 ms | 309.69% | 66.15 MiB |
| ruby_grpc | 2532 | 393.38 ms | 424.90 ms | 440.98 ms | 493.04 ms | 132.8% | 37.41 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------