Benchmark

kazeburo edited this page Nov 13, 2014 · 8 revisions

Gazelle, Starlet and Starman Benchmark (behind nginx reverse proxy)

gazelle_benchmark

nginx.conf

worker_processes 4; 
daemon off;
events {
  worker_connections  10000;
}

http {
  include     mime.types;
  access_log  off;
  sendfile    on;
  tcp_nopush  on;
  tcp_nodelay on;
  etag        off;
  upstream app {
    server unix:/dev/shm/app.sock;
  }

  server {
    location / {
      proxy_pass http://app;
    }
    location = /index.html {
      root html;
    }
  }
}

app.psgi

simple counter app

use Plack::Builder;
use Plack::Request;
use Cache::Memcached::Fast;
use JSON::XS;

my $_JSON = JSON::XS->new->utf8;
my $mem = Cache::Memcached::Fast->new({servers=>[qw/127.0.0.1:11211/]});
$mem->set('counter:1',0);
$mem->disconnect_all;
builder {
    enable 'AccessLog', logger => sub { };
    sub {
        my $env = shift;
        my $req = Plack::Request->new($env);
        my $id = $req->parameters->{id};
        my $counter = $mem->incr('counter:'.$id);
        [200, ['Content-Type'=>'application/json; charset=UTF-8'], [$_JSON->encode({counter=>$counter})]];
    }
};

Software Version

$ perl -v

This is perl 5, version 20, subversion 1 (v5.20.1) built for x86_64-linux

Copyright 1987-2014, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

$ cpanm Starman Starlet HTTP::Parser::XS JSON::XS Cache::Memcached::Fast
Starman is up to date. (0.4011)
Starlet is up to date. (0.24)
HTTP::Parser::XS is up to date. (0.16)
JSON::XS is up to date. (3.01)
Cache::Memcached::Fast is up to date. (0.22)

nginx

static html benchmark

$ ./wrk -c 32 -t 8 -d 10 'http://10.x.x.x/index.html'
Running 10s test @ http://10.x.x.x/index.html
  8 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   254.60us  132.28us   5.70ms   94.56%
    Req/Sec    16.81k     1.99k   21.00k    84.58%
  1274525 requests in 10.00s, 274.64MB read
Requests/sec: 127461.87
Transfer/sec:     27.47MB

Gazelle

hello world

$ start_server --path=/dev/shm/app.sock  -- plackup -I./lib -Iblib/arch -s Gazelle \
   --max-workers 8 --max-reqs-per-child 500000 -E prod \
   -e 'sub{[200,[],["hello world\n"]]}'

result

$ ./wrk -c 32 -t 8 -d 10 http://10.x.x.x/
Running 10s test @ http://10.x.x.x/
  8 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   303.88us  214.59us  12.30ms   96.37%
    Req/Sec    14.19k     2.03k   19.44k    76.71%
  1060196 requests in 10.00s, 152.62MB read
Requests/sec: 106028.19
Transfer/sec:     15.26MB

counter app

$ start_server --path=/dev/shm/app.sock  -- plackup -I./lib -Iblib/arch -s Gazelle \
   --max-workers 16 --max-reqs-per-child 500000 -E prod -a app.psgi

result

$ ./wrk -c 32 -t 8 -d 10 'http://10.x.x.x/counter?id=1'
Running 10s test @ http://10.x.x.x/counter?id=1
  8 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   758.10us  248.64us   6.94ms   82.82%
    Req/Sec     5.58k   452.80     7.44k    75.70%
  422825 requests in 10.00s, 83.71MB read
Requests/sec:  42284.67
Transfer/sec:      8.37MB

Starlet

$ start_server --path=/dev/shm/app.sock  -- plackup -I./lib -Iblib/arch -s Starlet \
   --max-workers 8 --max-reqs-per-child 500000 -E prod \
   -e 'sub{[200,[],["hello world\n"]]}'

result

$ ./wrk -c 32 -t 8 -d 10 http://10.x.x.x/
Running 10s test @ http://10.x.x.x/
  8 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   521.41us  588.16us  20.54ms   97.07%
    Req/Sec     8.18k     1.18k   12.33k    74.46%
  620658 requests in 10.00s, 89.35MB read
Requests/sec:  62068.97
Transfer/sec:      8.94MB

counter app

$ start_server --path=/dev/shm/app.sock  -- plackup -I./lib -Iblib/arch -s Starlet \
   --max-workers 16 --max-reqs-per-child 500000 -E prod -a app.psgi

result

$ ./wrk -c 32 -t 8 -d 10 'http://10.x.x.x/counter?id=1'
Running 10s test @ http://10.x.x.x/counter?id=1
  8 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.13ms  226.63us   5.23ms   73.81%
    Req/Sec     3.72k   265.06     4.78k    65.91%
  282912 requests in 10.00s, 55.84MB read
Requests/sec:  28291.73
Transfer/sec:      5.58MB

Starman

$ starman --listen /dev/shm/app.sock --workers 8 --max-requests 500000 -E prod --preload-app \
  --disable-keepalive -e 'sub{[200,[],["hello world\n"]]}'

result

$ ./wrk -c 32 -t 8 -d 10 http://10.x.x.x/
Running 10s test @ http://10.x.x.x/
  8 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.09ms    1.36ms  54.15ms   94.86%
    Req/Sec     4.38k     1.28k    9.56k    66.15%
  332977 requests in 10.00s, 47.93MB read
Requests/sec:  33299.82
Transfer/sec:      4.79MB

counter app

$ starman --listen /dev/shm/app.sock --workers 40 --max-requests 500000 -E prod --preload-app \
  --disable-keepalive -a app.psgi

result

./wrk -c 32 -t 8 -d 10 'http://10.x.x.x/counter?id=1'
Running 10s test @ http://10.x.x.x/counter?id=1
  8 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.66ms    1.07ms  56.18ms   95.12%
    Req/Sec     2.63k   474.91     4.11k    73.39%
  200993 requests in 10.00s, 39.67MB read
Requests/sec:  20100.21
Transfer/sec:      3.97MB

Server Environment

A benchmark tool and web servers were executed at different hosts. Two hosts are connected via 1Gbps ethernet and a switch. Both server have a Intel Xeon E3-1240v3 3.40GHz (4core/8thread) CPU

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.