Update Chef cookbook

[mfenner]

Update the existing Chef cookbook/Vagrantfile to use Ubuntu 14.04, nginx and puma. Most of this code can be shared with the alm application, and if fact any rails application running under nginx/puma.

[jure]

Somewhat related: I really like what Discourse is doing, where they're using https://github.com/discourse/discourse/blob/master/docs/VAGRANT.md (with Chef) for development, but Docker (standalone, all services in one docker) for easy deployment to production. They also have individual Docker components if you want to deploy a bigger installation, but the simple case works for 99% of users.

The vagrant image that they use is created like this: https://github.com/discourse/discourse/blob/master/docs/DEVELOPER-ADVANCED.md

It's quite nice as the process of working on the software involves only vagrant up and bundle exec rails s. No need for capistrano (helps #2) or other indirection.

With regards to updating the Vagrantfile, if the code to bring up nginx+puma on Ubuntu 14.04 is identical across the two projects, perhaps it would make sense to create a separate project that creates a VM image, and then use that image as the base in both projects.

[jure]

#8 switches to Ubuntu 14.04 and Apache 2.4. I also made some changes to https://github.com/articlemetrics/alm-report-cookbook, because it was failing in a few areas due to the OS and Apache updates (things like RewriteLog no longer existing on Apache 2.4, etc.)

That gives us a much better base for updating to nginx/puma. I saw that you assigned yourself, is there any particular reason you want to do this? If not, I can also do it.

[mfenner]

Cool. I have spent some time with Vagrant/Docker yesterday, but I'm still learning. It sems that the Phusion/Passenger folks are docker experts and have built some great docker images, e.g. https://hub.docker.com/u/phusion/baseimage/ and https://hub.docker.com/u/phusion/passenger-ruby21/.

The reason I assigned myself to the Vagrant/Chef work is that it is essentially the same code in alm and alm-report. We can discuss how best to do this when we talk next time. Questions:

• Move to Ubuntu 14.04? Some Chef recipes have not been updated yet, but I think it is time
• Use Docker? I think it would be great both for development and deployment. My host is a Mac, so the process is a bit more involved, but it would be great if I don't have to generate a full VM for every alm project.
• What web/application server to use? I'm fine with nginx, but would also consider passenger if going with docker as the phusion folks seem to have done great work.

[jure]

Move to Ubuntu 14.04? Some Chef recipes have not been updated yet, but I think it is time

In #8 I've actually switched to 14.04 (12aadd5) and it works just fine, so that can be done.

Use Docker? I think it would be great both for development and deployment. My host is a Mac, so the process is a bit more involved, but it would be great if I don't have to generate a full VM for every alm project

Yes, Docker sounds good. What we have now (with #8 changes) is probably good enough for Iteration 1, but a Docker-based system (in combination with Vagrant) will be better, mostly because we can also handle production deployment with Docker. The https://hub.docker.com/u/phusion/passenger-ruby21/ image sounds like a great start for any kind of Dockerization, and from what I gathered the name doesn't imply it's meant for Phusion Passenger, only that it should be "as easy as being a passenger along for the ride" :). So it's "merely" a good base image with ruby and node.

Comparing Puma to Phusion Passenger, I found this interesting point by the makers of Passenger:

The fact that the open source version [of Phusion Passenger] only supports multi-process I/O is problematic for people who have web apps with lots of I/O bound, slow-running requests. For those people, running hundreds of Phusion Passenger processes to get the same sort of concurrency as Puma's multithreading is probably not doable, so Puma's multithreading support becomes the most important. (https://github.com/phusion/passenger/wiki/Puma-vs-Phusion-Passenger)

The open source/free version of Passenger is multi-process single-threaded, and Puma is multi-threaded all the way. For an application like alm-reports, where most of the requests are held back by network I/O, it looks like the makers of Passenger are telling us to use either Puma or the enterprise version of their server. I vote for Puma.

On the other hand, doesn't seem to be a lot of support out there for Docker/Puma combination, though I didn't spend a lot of time looking yet.

[jure]

Very interesting discussion about the baseimage-docker mentioned above: https://news.ycombinator.com/item?id=7258009

The gist:

1. Don't build a monolithic VM with Docker, separate services into individual containers and use Docker to manage them
2. Don't forget about the init process, running in a minimal container does not absolve your app from being a good citizen in a Linux system

[mfenner]

@jure I will change the deployment with a pull request with a deployment tomorrow, but if you need the cookbook before, be aware that it is renamed to alm_report-cookbook (the underscore). The hyphen can apparently create problems.

[jure]

Cool, I'll give it a shot.

[jure]

Had a few issues with dependencies, but it looks like I've solved them now:

# Our cookbooks
cookbook 'ruby',
  git: 'git@github.com:articlemetrics/ruby-cookbook.git'

cookbook 'passenger_nginx',
  git: 'git@github.com:articlemetrics/passenger_nginx-cookbook.git'

cookbook 'capistrano',
  git: 'git@github.com:articlemetrics/capistrano-cookbook.git'

cookbook 'mysql_rails',
  git: 'git@github.com:articlemetrics/mysql_rails-cookbook.git'

cookbook "alm_report",
  path: '/Users/juretriglav/src/plos/alm_report-cookbook'
  #:git => "git@github.com:articlemetrics/alm_report-cookbook.git",
  #:ref => "mode"

Now running into an issue with the alm_report cookbook:

==> default: [2014-09-02T09:26:44+00:00] ERROR: passenger_nginx[alm-report] (alm_report::default line 15) had an error: Chef::Exceptions::ResourceNotFound: resource apt_repository[brightbox-ruby-ng-trusty] is configured to notify resource execute[apt-get update] with action run, but execute[apt-get update] cannot be found in the resource collection. apt_repository[brightbox-ruby-ng-trusty] is defined in /tmp/vagrant-chef-1/chef-solo-1/cookbooks/ruby/recipes/default.rb:3:in `from_file'

[mfenner]

You are too fast. I'm currently doing some final testing and tweaking. There is a change in the Vagrantfile (removing the shell commands), and an update to the Cheffile that I will put into a pull request shortly.

[mfenner]

The two major pain points are file permissions, and ordering of actions (Chef doesn't run top to bottom, but precompiles, so you have to be careful what to expect).

[jure]

Looks like it's related to this:

https://github.com/articlemetrics/ruby-cookbook/blob/master/recipes/default.rb#L10

It tries to call apt-get update, but the new apt, 2.5.3 ( > 2.4.0), which alm_report-cookbook installs by default, only has apt-get-update. Discussion here: sous-chefs/apt#84

I've submitted a pull request to the ruby-cookbook repository.

Edit: Ah! Yes, I'm almost there as well. Didn't know you would be working on the alm-report repository too.

[jure]

Can we close this, Martin, seeing as the revamped Vagrant/Chef stuff is working great?