Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blog post on D8 Performance with HA/HP Infrastructure? #13

Closed
geerlingguy opened this Issue Feb 19, 2015 · 8 comments

Comments

Projects
None yet
1 participant
@geerlingguy
Copy link
Owner

geerlingguy commented Feb 19, 2015

See: http://buytaert.net/making-drupal-8-fly

D8 offers a lot more integration with caching logic, pumping data through the right places at the right times, etc. Maybe write about the opportunities offered by D8 with Redis, Nginx caching, etc.

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

geerlingguy commented Feb 23, 2015

Configuration as of 0.9.0 is working decently, and nginx is serving up pages pretty fast:

$ ab -n 10000 -c 200 http://pidramble.com/about
This is ApacheBench, Version 2.3 <$Revision: 1554214 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking pidramble.com (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        nginx/1.2.1
Server Hostname:        pidramble.com
Server Port:            80

Document Path:          /about
Document Length:        6388 bytes

Concurrency Level:      200
Time taken for tests:   7.261 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      74310000 bytes
HTML transferred:       63880000 bytes
Requests per second:    1377.24 [#/sec] (mean)
Time per request:       145.218 [ms] (mean)
Time per request:       0.726 [ms] (mean, across all concurrent requests)
Transfer rate:          9994.39 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        5   69  14.9     70    1121
Processing:    31   74  18.3     73     541
Waiting:       30   74  17.7     72     396
Total:         36  144  19.5    144    1202

Percentage of the requests served within a certain time (ms)
  50%    144
  66%    144
  75%    145
  80%    145
  90%    146
  95%    147
  98%    153
  99%    154
 100%   1202 (longest request)

I need to still get D8 to use Redis (though this won't have any effect on proxied requests...), and I still need to do more benchmarking with MySQL on microSD vs SSD, etc.

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

geerlingguy commented Feb 23, 2015

Also of note, that 9994.39 transfer rate is pretty close to maxing out the 100 Mbps interface. The gigabit interface might be able to afford slightly faster cached throughput, maybe even up to 2200 req/s or so!

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

geerlingguy commented Feb 26, 2015

With some further configuration tweaks to Nginx on the balancer, I can eke out a few more req/s:

$ wrk -t4 -c200 -d10 http://pidramble.com/
Running 10s test @ http://pidramble.com/
  4 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   110.25ms   11.29ms 185.02ms   92.70%
    Req/Sec   453.20     10.61   500.00     80.90%
  18029 requests in 10.00s, 110.11MB read
Requests/sec:   1802.39
Transfer/sec:     11.01MB

I'm definitely maxing out the built-in LAN at this point. I think I need to switch back to the GigE adapter again and see if that gives me another few req/s, or if we're hitting CPU limitations. iotop was showing a max of 95.5 Mbps!

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

geerlingguy commented Feb 26, 2015

Woohoo, now that's more like it!

$ wrk -t4 -c100 -d10 http://pidramble.com/
Running 10s test @ http://pidramble.com/
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    32.14ms    1.52ms  47.92ms   67.83%
    Req/Sec   783.40     37.90     0.89k    65.94%
  30799 requests in 10.00s, 188.10MB read
Requests/sec:   3079.86
Transfer/sec:     18.81MB

That's over the GigE USB 3.0 TRENDnet adapter (needs no drivers), configured as the eth1 interface.

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

geerlingguy commented Mar 2, 2015

Also working with PHP FPM configuration optimization, I'm getting 15+ uncached requests/second (using Redis caching):

$ wrk -t4 -c48 -d300 http://pidramble.com/?nocache=true
Running 5m test @ http://pidramble.com/?nocache=true
  4 threads and 48 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.02s   603.74ms   4.49s    66.99%
    Req/Sec     3.44      0.72     5.00     86.20%
  4753 requests in 5.00m, 24.84MB read
  Socket errors: connect 0, read 0, write 0, timeout 2451
Requests/sec:     15.83
Transfer/sec:     84.72KB

I think I've hit a point where the only optimizations I can make are hardware-related:

  • For cached traffic: the main bottleneck is the network interface.
  • For uncached traffic: the main bottleneck is the number of processes I can run on backend webservers without hitting CPU and memory limits.

Going further on uncached traffic (assuming full onslaught of requests):

  • MySQL is using 12-15% CPU, only about 25% RAM. Everything is cached in memory for reads.
  • Redis is using 18-20% CPU, only about 40% RAM.
  • Balancer is using 2-4% CPU, only about 40% RAM.
  • Webservers are using 100% (load ~= 4-5) CPU, about 93% RAM.

This is with the 1.1.0 configuration/version of this project.

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

geerlingguy commented Mar 3, 2015

Over in #40 I got the cluster running on DigitalOcean, and it seems the exact same configuration runs about 62% faster on a cluster of 1GB Droplets.

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

geerlingguy commented Mar 11, 2015

Over on the Wiki I've been compiling some benchmarks for easier digestion: https://github.com/geerlingguy/raspberry-pi-dramble/wiki/Dramble-D8-Benchmarks

I think I have enough to go on to write at least a blog post or two—maybe a dozen! Drupal 8 has some nice new features to make caching easier and more powerful... but it's also quite slow (in terms of raw PHP performance) at this point in time (in comparison to Drupal 7's OOTB experience, at least). Hopefully that'll change.

@geerlingguy

This comment has been minimized.

Copy link
Owner Author

geerlingguy commented Jan 6, 2016

I'm going to close this issue and simply continue blogging on http://www.jeffgeerling.com/ and http://www.pidramble.com/ about D8 performance and HA/HP.

@geerlingguy geerlingguy closed this Jan 6, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.