Moderate request rate

• Moderate TPS
  • High-performance (C, C++, Rust)
    • Regular percentiles (p50,90,99)
    • Outlier latency (p99.9 and p99.99)
    • Trimmed mean and standard deviation
    • CPU and Memory consumption
    • Summary
  • Memory-safe languages (Rust, Golang, Java, Python)
    • Regular percentiles (p50,90,99)
    • Outlier latency (p99.9 and p99.99)
    • Trimmed mean and standard deviation
    • Summary
  • Total summary
  • Conclusion

Moderate TPS

Payload starts at 10,000 rps with 1,000 increment until it reaches 25,000 rps. Then stay there for 10 minutes.

High-performance (C, C++, Rust)

Regular percentiles (p50,90,99)

We can see that all three add 50-100µs on top of the baseline, with slightly better latency for C++.

Outlier latency (p99.9 and p99.99)

For outlier latency, the results are somewhat mixed. While for p99.9 the best results are shown by C++, for p99.99 `C++' is the worst, and Rust is the best:

Trimmed mean and standard deviation

All three are nearly identical, adding ~50µs to the mean, and ~15µs to standard deviation:

CPU and Memory consumption

We can see that CPU utilization for all C, C++, and Rust are close, however, C++ consumes slightly more than Rust, and Rust consumes slightly more than C.

Memory consumption is on par for C++ and Rust, but HAProxy consumes more memory. However, the difference is really negligible and on the order of 0.1% of the total memory.

Summary

	p50	p90	p99	p99.9	p99.99	max	tm99.9	stddev
Baseline	190	295	459	718	945	8,160	202	83
C (HAProxy)	240	359	591	987	1,460	15,880	258	98
C++ (draft-http-tunnel)	246	368	554	797	1,620	16,820	263	95
Rust (http-tunnel)	245	366	594	1,020	1,380	12,310	265	97

Memory-safe languages (Rust, Golang, Java, Python)

Regular percentiles (p50,90,99)

Please note that for Java and Python, the max TPS was somewhat lowered, because of the inability of the TCP Proxies to handle more.

So the comparison was between:

• Rust - 25k rps
• Golang - 25k rps
• Java - 15k rps
• Python - 10k rps

Now we can see that at p50-p90 level, Golang is somewhat comparable to Rust, but quickly deviates at p99 level, adding a whole millisecond.

Java and Python with lower RPS exhibit substantially higher latencies:

Outlier latency (p99.9 and p99.99)

Outlier latency shows an even larger difference with Rust, and for Java is the worst of all four:

Trimmed mean and standard deviation

While the trimmed mean is not dramatically worse for Golang compared to Rust. But the standard deviation is. Again, Java and Python are well behind both Rust and Golang:

Summary

	p50	p90	p99	p99.9	p99.99	max	tm99.9	stddev
Rust (http-tunnel)	245	366	594	1,020	1,380	12,310	265	97
Tcp-Proxy (Golang)	259	387	1,510	4,910	7,970	16,160	304	350
NetCrusher (Java)	287	665	1,700	8,200	35,100	44,800	377	718
pproxy (Python)	485	949	2,240	4,690	6,740	16,630	585	398

Total summary

	p50	p90	p99	p99.9	p99.99	max	tm99.9	stddev
Baseline	190	295	459	718	945	8,160	202	83
C (HAProxy)	240	359	591	987	1,460	15,880	258	98
C++ (draft-http-tunnel)	246	368	554	797	1,620	16,820	263	95
Rust (http-tunnel)	245	366	594	1,020	1,380	12,310	265	97
Tcp-Proxy (Golang)	259	387	1,510	4,910	7,970	16,160	304	350
NetCrusher (Java)	287	665	1,700	8,200	35,100	44,800	377	718
pproxy (Python)	485	949	2,240	4,690	6,740	16,630	585	398

Conclusion

The Rust solution is on par with C/C++ solutions at all levels. Golang is fine at p50-90, but the tail latencies and standard deviation are not as good. NetCrusher and pproxy are unlikely suitable for latency-sensitive applications.