This is a simple benchmark for Apache Mina Core TCP Client

Intro

Current implementation of IoServiceStatistics is blocking - it locks on throughputCalculationLock for almost all operation.

However, IoServiceStatistics is used by all threads which writes to IoSession and by all NioProcessor threads.

There is a concern that IoServiceStatistics slows down performance

Goals

The aim of the benchmark is to compare performance of:

1. Mina TCP Client with blocking IoServiceStatistics implementation (original)
2. Mina TCP Client with non-blocking IoServiceStatistics implementation (patched)

Benchmark details

• There are N IoSession's between Client and Server

• There are N threads, each thread has its own IoSession and they write messages simultaneously

• The measurements are taken between the time the message was written to IoSession and the time when it was actually sent to the server by NioProcessor

• The rate of sending messages is regulated by a Semaphore:

  • acquire Semaphore before sening message
  • release Semaphore when the message was actually sent

• Java reflection is used for imitating non-blocking IoServiceStatistics (benchmark.Client.makeIoServiceStatisticsNonBlocking)

• The only difference between 2 benchmarks is here benchmark.ClientBenchmark.ClientBenchmark:

  if (config.blockingClientIoServiceStats) {
      System.out.println("BLOCKING Client IoServiceStatistics implementation is used");
  } else {
      client.makeIoServiceStatisticsNonBlocking();
      System.out.println("NON-BLOCKING Client IoServiceStatistics implementation is used");
  }
  

• Benchmark parameters are taken from benchmark.ClientBenchmark.Config

• The results:

  • for each iteration: calculated percentiles for sent messages during the iteration
  • for summary: calculate mean for all percentiles for all iterations (warmup iterations excluded)

• By default, the configuration is:

  • 3 warmup iterations which are not used in summary result
  • 10 real iterations which are used in summary result
  • 50 threads (N=50), each thread sends 50000 messages for each iteration

How to run

1. Run benchmark with blocking IoServiceStatistics:

   mvn clean compile exec:exec@blocking
   

2. Run benchmark with non-blocking IoServiceStatistics:

   mvn clean compile exec:exec@non-blocking
   

Results

1. blocking IoServiceStatistics (original):

================================================
SUMMARY RESULTS
================================================
Mean Result for 10 iterations: 
p25: 50 mcs, p50: 140 mcs, p75: 400 mcs, p90: 905 mcs, p95: 1418 mcs, p99: 11485 mcs, p100: 505313 mcs, 
================================================

2. non-blocking IoServiceStatistics (patched):

================================================
SUMMARY RESULTS
================================================
Mean Result for 10 iterations: 
p25: 44 mcs, p50: 85 mcs, p75: 150 mcs, p90: 239 mcs, p95: 319 mcs, p99: 1311 mcs, p100: 1374162 mcs, 
================================================

Summary

The results show that IoServiceStatistics introduces huge latencies for the benchmark condition:

# patched vs original:

p50: 85mcs vs 140mcs
p75: 150mcs vs 400mcs
p90: 239mcs vs 905mcs
p95: 319mcs vs 1418mcs
p99: 1311mcs vs 11485mcs