New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better VM preformance when using 1 CPU core #410

Closed
primozcigler opened this Issue Nov 12, 2015 · 12 comments

Comments

Projects
None yet
4 participants
@primozcigler
Contributor

primozcigler commented Nov 12, 2015

Hi there,

I had an issue with the local VM (VirtualBox), when the server was responding slowly. I moved the WP network from my previous VM.

The TTFB on the previous VM was around 500ms for the HTML to get back. The stack was pretty much the same (nignx + php-fpm, mariadb).

When I copies the files to the Trellis and dumped the DB, the TTFB increased to around 1500ms - 2000ms. This was unexpected and I couldn't tolerate such deplay during dev, as page is refreshed frequently. I decided to compare the config to find the problem that introduced the delay. Started with php-fpm, went to nginx, mariadb and at the end even tried to disable some extensions like xdebug. Nothing made any significant difference.

I almost gave up at the end, when I remembered to check the Vagrantfile as well. I experimented a bit with various settings and at the end I figured out very interesting thing: if I comment out the vb.customize ['modifyvm', :id, '--cpus', cpus] which sets the number of cores for VM, the TTFB drops down to 500-600ms for each request => ~4x perf. boost.

I would like other to get some feedback regarding this and would like to see if anyone else can test this out and confirm the same strange behavior?
The problem is more apparent in more complex WP stacks, with multiple heavy plugins like WooCommerce, Page Builder by SiteOrigin and all the plugins that heavily rely on filters, like simple lightbox.

I am on Linux now, but I experienced the same problem on Mac as well.

Related #62.

@swalkinshaw

This comment has been minimized.

Show comment
Hide comment
@swalkinshaw

swalkinshaw Nov 13, 2015

Member

Obviously this seems counter to what we'd expect. I don't really have anything to say about this unless other people are also experiencing this.

Member

swalkinshaw commented Nov 13, 2015

Obviously this seems counter to what we'd expect. I don't really have anything to say about this unless other people are also experiencing this.

@primozcigler

This comment has been minimized.

Show comment
Hide comment
@primozcigler

primozcigler Nov 13, 2015

Contributor

I hope other people try this out and let us know. If not, feel free to close the ticket, but at least you've been warned in case that someone else reports this in the future.

Contributor

primozcigler commented Nov 13, 2015

I hope other people try this out and let us know. If not, feel free to close the ticket, but at least you've been warned in case that someone else reports this in the future.

@QWp6t

This comment has been minimized.

Show comment
Hide comment
@QWp6t

QWp6t Nov 20, 2015

Member

Intel or AMD?

Member

QWp6t commented Nov 20, 2015

Intel or AMD?

@primozcigler

This comment has been minimized.

Show comment
Hide comment
@primozcigler

primozcigler Nov 22, 2015

Contributor

Intel. I hope to find some time to do consistent testing with Apache's ab tool this week on multiple machines to either confirm or decline this issue.

Contributor

primozcigler commented Nov 22, 2015

Intel. I hope to find some time to do consistent testing with Apache's ab tool this week on multiple machines to either confirm or decline this issue.

@primozcigler

This comment has been minimized.

Show comment
Hide comment
@primozcigler

primozcigler Dec 13, 2015

Contributor

I benchmarked roots example project with Apache Benchmark and I can confirm this (OSX 10.10.5, VirtualBox 4.3.30).

Results (run with command $ ab -n 20 -w http://roots-example-project.dev/):

With vb.customize ['modifyvm', :id, '--cpus', cpus]

Server Software:nginx
Server Hostname:roots-example-project.dev
Server Port:80
Document Path:/
Document Length:6416 bytes
Concurrency Level:1
Time taken for tests:5.822 seconds
Complete requests:20
Failed requests:0
Total transferred:135000 bytes
HTML transferred:128320 bytes
Requests per second:3.44
Transfer rate:22.64 kb/s received
Connnection Times (ms)
  min avg max
Connect: 0 0 3
Processing: 263 291 566
Total: 263 291 569

With vb.customize ['modifyvm', :id, '--cpus', 1]

Server Software:nginx
Server Hostname:roots-example-project.dev
Server Port:80
Document Path:/
Document Length:6416 bytes
Concurrency Level:1
Time taken for tests:3.330 seconds
Complete requests:20
Failed requests:0
Total transferred:135000 bytes
HTML transferred:128320 bytes
Requests per second:6.01
Transfer rate:39.59 kb/s received
Connnection Times (ms)
  min avg max
Connect: 0 0 2
Processing: 151 166 358
Total: 151 166 360

More real-world example (more complex theme, couple of plugins installed) was even more problematic:

With vb.customize ['modifyvm', :id, '--cpus', cpus]

Connnection Times (ms)
  min avg max
Connect: 0 0 2
Processing: 873 930 1194
Total: 873 930 1196

With vb.customize ['modifyvm', :id, '--cpus', 1]

Connnection Times (ms)
  min avg max
Connect: 0 0 2
Processing: 230 333 687
Total: 230 333 689
Contributor

primozcigler commented Dec 13, 2015

I benchmarked roots example project with Apache Benchmark and I can confirm this (OSX 10.10.5, VirtualBox 4.3.30).

Results (run with command $ ab -n 20 -w http://roots-example-project.dev/):

With vb.customize ['modifyvm', :id, '--cpus', cpus]

Server Software:nginx
Server Hostname:roots-example-project.dev
Server Port:80
Document Path:/
Document Length:6416 bytes
Concurrency Level:1
Time taken for tests:5.822 seconds
Complete requests:20
Failed requests:0
Total transferred:135000 bytes
HTML transferred:128320 bytes
Requests per second:3.44
Transfer rate:22.64 kb/s received
Connnection Times (ms)
  min avg max
Connect: 0 0 3
Processing: 263 291 566
Total: 263 291 569

With vb.customize ['modifyvm', :id, '--cpus', 1]

Server Software:nginx
Server Hostname:roots-example-project.dev
Server Port:80
Document Path:/
Document Length:6416 bytes
Concurrency Level:1
Time taken for tests:3.330 seconds
Complete requests:20
Failed requests:0
Total transferred:135000 bytes
HTML transferred:128320 bytes
Requests per second:6.01
Transfer rate:39.59 kb/s received
Connnection Times (ms)
  min avg max
Connect: 0 0 2
Processing: 151 166 358
Total: 151 166 360

More real-world example (more complex theme, couple of plugins installed) was even more problematic:

With vb.customize ['modifyvm', :id, '--cpus', cpus]

Connnection Times (ms)
  min avg max
Connect: 0 0 2
Processing: 873 930 1194
Total: 873 930 1196

With vb.customize ['modifyvm', :id, '--cpus', 1]

Connnection Times (ms)
  min avg max
Connect: 0 0 2
Processing: 230 333 687
Total: 230 333 689
@primozcigler

This comment has been minimized.

Show comment
Hide comment
@primozcigler

primozcigler Dec 13, 2015

Contributor

I pinged @swrobel over twitter. It would be great to have him elaborate on this, as AFAIK you guys added these Vagrantfile 'improvements' after this blogpost.

Contributor

primozcigler commented Dec 13, 2015

I pinged @swrobel over twitter. It would be great to have him elaborate on this, as AFAIK you guys added these Vagrantfile 'improvements' after this blogpost.

@primozcigler

This comment has been minimized.

Show comment
Hide comment
@primozcigler

primozcigler Dec 13, 2015

Contributor

Moreover, I experience the same issues on linux machine as well.

Contributor

primozcigler commented Dec 13, 2015

Moreover, I experience the same issues on linux machine as well.

@swrobel

This comment has been minimized.

Show comment
Hide comment
@swrobel

swrobel Dec 13, 2015

@primozcigler really interesting findings. I suppose in my post, I take for granted that more cpus = better performance, or at least that it won't cause worse performance. One thing to note is that you didn't test concurrency - not that is a typical concern in development contexts. I used wrk for my NFS vs VirtualBox sharing benchmarks because it's supposed to better emulate real-world traffic than ab but I still have no explanation for why you're seeing reduced performance.

swrobel commented Dec 13, 2015

@primozcigler really interesting findings. I suppose in my post, I take for granted that more cpus = better performance, or at least that it won't cause worse performance. One thing to note is that you didn't test concurrency - not that is a typical concern in development contexts. I used wrk for my NFS vs VirtualBox sharing benchmarks because it's supposed to better emulate real-world traffic than ab but I still have no explanation for why you're seeing reduced performance.

@primozcigler

This comment has been minimized.

Show comment
Hide comment
@primozcigler

primozcigler Dec 14, 2015

Contributor

@swrobel thanks for stopping by 😃

I could use wrk as well, but I can tell the performance slowdown is noticeable even when developing - without benchmarks.
So there's no need for me to test this again. It would be great if anyone else takes couple of minutes of time and does independent test. Hopefully guys here at Roots, maybe this is particular to WP - I haven't bother to benchmark other stacks.

Contributor

primozcigler commented Dec 14, 2015

@swrobel thanks for stopping by 😃

I could use wrk as well, but I can tell the performance slowdown is noticeable even when developing - without benchmarks.
So there's no need for me to test this again. It would be great if anyone else takes couple of minutes of time and does independent test. Hopefully guys here at Roots, maybe this is particular to WP - I haven't bother to benchmark other stacks.

@swalkinshaw

This comment has been minimized.

Show comment
Hide comment
@swalkinshaw

swalkinshaw Jan 17, 2016

Member

Figured I'd test this out myself. I'm a bit dubious of how much these benchmarks matter and they definitely aren't rigorous or anything.

I timed vagrant up and then the ab bench you posted:

CPUS: 4 (sysctl -n hw.ncpu)

  • vagrant up: 7:45.16
  • ab -n 20 -w http://example.dev/: ~3.3 req/s

CPU: 1

  • vagrant up: 9:17.21
  • ab -n 20 -w http://example.dev/: ~4.2 req/s

I ran the ab bench a few times.

Funny enough I also found this: rdsubhas/vagrant-faster#5

Anecdotal but I'm leaning towards removing this cpu configuration.

Member

swalkinshaw commented Jan 17, 2016

Figured I'd test this out myself. I'm a bit dubious of how much these benchmarks matter and they definitely aren't rigorous or anything.

I timed vagrant up and then the ab bench you posted:

CPUS: 4 (sysctl -n hw.ncpu)

  • vagrant up: 7:45.16
  • ab -n 20 -w http://example.dev/: ~3.3 req/s

CPU: 1

  • vagrant up: 9:17.21
  • ab -n 20 -w http://example.dev/: ~4.2 req/s

I ran the ab bench a few times.

Funny enough I also found this: rdsubhas/vagrant-faster#5

Anecdotal but I'm leaning towards removing this cpu configuration.

swalkinshaw added a commit that referenced this issue Feb 15, 2016

@swalkinshaw

This comment has been minimized.

Show comment
Hide comment
@swalkinshaw

swalkinshaw Feb 15, 2016

Member

@primozcigler thanks for bringing this up. Finally went with 1 as the default.

Member

swalkinshaw commented Feb 15, 2016

@primozcigler thanks for bringing this up. Finally went with 1 as the default.

@primozcigler

This comment has been minimized.

Show comment
Hide comment
@primozcigler

primozcigler Feb 15, 2016

Contributor

My pleasure.

Contributor

primozcigler commented Feb 15, 2016

My pleasure.

cameroncf added a commit to cameroncf/vagrant-centos-lucee that referenced this issue Jun 13, 2016

Reduce CPU count to 1
My recent experience is that reducing the Virtual CPU setting speeds up performance considerably. This appears to be supported by other articles out there on the interwebs. I'm proposing it as a change to your template since I have used it as a reference point a lot. Test the change and see if you see a performance improvement too.

For more information see:
https://ruin.io/2014/benchmarking-virtualbox-multiple-core-performance/
roots/trellis#410

cameroncf added a commit to cameroncf/vagrant-ubuntu-lucee that referenced this issue Jun 13, 2016

Reduce CPU count to 1
My recent experience is that reducing the Virtual CPU setting speeds up performance considerably. This appears to be supported by other articles out there on the interwebs. I'm proposing it as a change to your template since I have used it as a reference point a lot. Test the change and see if you see a performance improvement too.

For more information see:
https://ruin.io/2014/benchmarking-virtualbox-multiple-core-performance/
roots/trellis#410
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment