Migrating an existing Alaveteli site from ruby 1.8.7

Louise Crow edited this page Jan 25, 2016 · 84 revisions

Note: Throughout this guide we assume that the Alaveteli deployment user is called alaveteli. If this is not true for your install, you will need to modify the home directory paths referenced to the home directory of your deployment user. We also assume that your Alaveteli application is installed in /var/www/alaveteli. Again, if this is not true, adjust the paths in this guide to match the location of your Alaveteli install.

1. Preparation

Upgrade from Squeeze to Wheezy

If you're running Debian Squeeze, it's a good idea to upgrade to Wheezy as Squeeze will only be supported until February 2016. Squeeze only has ruby 1.8.7 installed by default.

Upgrade to Alaveteli version 0.22

If you’re currently running Alaveteli under ruby 1.8.7, it’s a good idea to upgrade to release 0.22 before you switch to running under a later version of ruby. A big difference between ruby 1.8.7 and later versions is the difference in handling string encodings. Alaveteli version 0.22 resolves a number of bugs related to this and adds some features to handle issues with existing data in the database and elsewhere, including a script for finding poorly encoded strings and cleaning them up prior to upgrading.

2. Install a new ruby version (optional)

Alaveteli currently supports ruby 1.9.3 and 2.0.0. Although 1.9.3 is end-of-life, our supported OS versions - Ubuntu Precise and Debian Wheezy - are both LTS (Long Term Support) versions, meaning that the maintainers will continue to provide security updates for the packaged ruby versions. Both package ruby 1.8.7 and 1.9.3 as alternatives. If you want to run Alaveteli under ruby 2.0, you will need to install it and make sure you keep it up-to-date with security patches.

3. Install a ruby version manager (optional)

You can switch ruby system-wide, or just for the user that runs your Alaveteli site. Switching ruby system-wide is a bit simpler. However, one advantage of setting the ruby version on a per-user basis is that you can run the script to detect poorly encoded data under your new ruby version before upgrading your Alaveteli install. You may also have other ruby code running on your server that you want to keep running under 1.8.7. These instructions cover switching ruby on a per-user basis using the ruby version manager rbenv - this is packaged in both our supported OSes and is less invasive than RVM. At mySociety, we set the ruby version per user for the Alaveteli installs that we host using rbenv.

Debian Wheezy

Install rbenv:

sudo apt-get install rbenv

Then follow the instructions from https://wiki.debian.org/Ruby, as your deployment user:

$ ruby -v
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]
$ rbenv init 
# Load rbenv automatically by adding
# the following to ~/.bash_profile:

eval "$(rbenv init -)"
$ echo 'eval "$(rbenv init -)"' >> ~/.bash_profile # or ~/.bashrc, depends on your setup
$ rbenv versions
$ rbenv alternatives 
$ rbenv versions
  1.8.7-debian
  1.9.3-debian

Logout and login again and you'll be able to switch the default user back and forth.

Ubuntu Precise

For Ubuntu, to install rbenv to allow you to set the default ruby interpreter on a per-user basis, run the following as your deployment user:

$ ruby -v
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]
$ rbenv init
The program 'rbenv' is currently not installed.  To run 'rbenv' please ask your administrator to install the package 'rbenv'
$ sudo apt-get install rbenv
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed
  rbenv
0 to upgrade, 1 to newly install, 0 to remove and 45 not to upgrade.
Need to get 15.3 kB of archives.
After this operation, 79.9 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ precise/universe rbenv all 0.1.2+git20100922-1 [15.3 kB]
Fetched 15.3 kB in 0s (150 kB/s) 
Selecting previously unselected package rbenv.
(Reading database ... 81080 files and directories currently installed.)
Unpacking rbenv (from .../rbenv_0.1.2+git20100922-1_all.deb) ...
Processing triggers for man-db ...
Setting up rbenv (0.1.2+git20100922-1) ...
$ rbenv init
# Load rbenv automatically by adding
# the following to ~/.bash_profile:

eval "$(rbenv init -)"

$ echo 'eval "$(rbenv init -)"' >> ~/.bashrc
$ rbenv versions
$ rbenv alternatives
$ rbenv versions
  1.8.7-debian
  1.9.3-debian

Logout and login again and you'll be able to switch the default user back and forth.

4. Bring your site down

Make sure your webserver config redirects traffic to a down.html maintenance page if one exists in the public directory of Alaveteli.

###Nginx example

https://github.com/mysociety/alaveteli/blob/release/0.22/config/nginx.conf.example#L25

###Apache example

https://github.com/mysociety/alaveteli/blob/release/0.22/config/httpd.conf-example#L56-76

Create the down.html page in public (there's an example down.default.html that you can customise.)

5. Defer incoming email

Make sure incoming email will be deferred by your MTA.

Create a defer script:

cat >> /var/www/alaveteli/script/defer << EOF
#!/bin/bash
exit 75
EOF

Make it executable:

chmod a+x /var/www/alaveteli/script/defer

Exim

In /var/www/alaveteli/config/aliases, comment out the line

^foi\\+.*: "|/home/vagrant/alaveteli/script/mailin"

and add a line

^foi\\+.*: "|/home/vagrant/alaveteli/script/defer"

If you have Alaveteli set up to filter messages to site admin addresses, redirect that to the defer script too. Comment out the line

user-support:     |/var/www/alaveteli/script/handle-mail-replies

and add a line

user-support:     |/var/www/alaveteli/script/defer

Postfix

In /etc/postfix/master.cf, comment out the line

  flags=R user=alaveteli argv=/var/www/alaveteli/script/mailin```

and add a line below it:

```alaveteli unix  - n n - 50 pipe
  flags=R user=alaveteli argv=/var/www/alaveteli/script/defer```

If you have Alaveteli set up to filter messages to site admin addresses, redirect that to the `defer` script too. Comment out the line

alaveteli_replies unix - n n - 50 pipe flags=R user=alaveteli argv=/var/www/alaveteli/script/handle-mail-replies


and add a line below it:

```alaveteli_replies unix  - n n - 50 pipe
  flags=R user=alaveteli argv=/var/www/alaveteli/script/defer```

Reload postfix:

```postfix reload```

## 6. Switch ruby version 

### System wide switch 
####Debian Wheezy
If you’re running on Debian, you can switch the default ruby version with `ruby-switch` as follows (instructions taken from https://wiki.debian.org/Ruby):

ruby -v

ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]

ruby-switch --list

ruby1.8 ruby1.9.1

ruby-switch --set ruby1.9.1

update-alternatives: using /usr/bin/gem1.9.1 to provide /usr/bin/gem (gem) in manual mode. update-alternatives: using /usr/bin/ruby1.9.1 to provide /usr/bin/ruby (ruby) in manual mode.

ruby -v

ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux]

ruby-switch --auto

update-alternatives: using /usr/bin/ruby1.8 to provide /usr/bin/ruby (ruby) in auto mode. update-alternatives: using /usr/bin/gem1.8 to provide /usr/bin/gem (gem) in auto mode.

ruby -v

ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]

####Ubuntu Precise
On Ubuntu Precise, you can switch the default ruby version for the system from 1.8.7 to 1.9.3 with:

`sudo update-alternatives --set ruby /usr/bin/ruby1.9.1`



### Per-user switch 

If you're using `rbenv` you need to make sure that the correct ruby version will be used in all contexts in which Alaveteli runs code: 

#### App

Set the new ruby version globally for the Alaveteli deployment user. From their account: 

rbenv global ruby1.9.1


Uncomment [the line](https://github.com/mysociety/alaveteli/blob/develop/config/httpd.conf-example#L30) in `httpd.conf` that sets `PassengerRuby` to the `rbenv` ruby. If the name of your deployment user is not `alaveteli`, you will need to update the line to reference the home directory of your own deployment user.

#### Cron

Uncomment [the line](https://github.com/mysociety/alaveteli/blob/develop/config/crontab-example#L10) that adds the `rbenv` shims to the cron file's `PATH`.

#### Daemons 

Uncomment [the line](https://github.com/mysociety/alaveteli/blob/develop/config/alert-tracks-debian.example#L29) that adds the `rbenv` shims to each script's `PATH` 

#### Mail handling scripts

Create a mail script wrapper in your deployment user's home directory 

bundle exec rake config_files:convert_wrapper DEPLOY_USER=alaveteli SCRIPT_FILE=config/run-with-rbenv-path.example > /home/alaveteli/run-with-rbenv-path

Make it executable 

chmod a+x /home/alaveteli/run-with-rbenv-path


##### Postfix

Edit the line 

#alaveteli unix - n n - 50 pipe flags=R user=alaveteli argv=/var/www/alaveteli/script/mailin


to read 

#alaveteli unix - n n - 50 pipe flags=R user=alaveteli argv=/home/alaveteli/run-with-rbenv-path /var/www/alaveteli/script/mailin

If you have alaveteli set up to filter messages to site admin addresses, edit the commented-out line

#alaveteli_replies unix - n n - 50 pipe flags=R user=alaveteli argv=/var/www/alaveteli/script/handle-mail-replies


to read 

#alaveteli_replies unix - n n - 50 pipe flags=R user=alaveteli argv=/home/alaveteli/run-with-rbenv-path /var/www/alaveteli/script/handle-mail-replies


Reload postfix

postfix reload


##### Exim

Edit the commented out pipe line in the file `/var/www/alaveteli/config/aliases`

#^foi\+.*: "|/home/vagrant/alaveteli/script/mailin", backupfoi


 to read

#^foi\+.*: "|/home/alaveteli/run-with-rbenv-path /var/www/alaveteli/script/mailin", backupfoi



If you have Alaveteli set up to filter messages to site admin addresses, edit the commented-out line

#user-support: |/var/www/alaveteli/script/handle-mail-replies


to read

#user-support: "|/home/alaveteli/run-with-rbenv-path /var/www/alaveteli/script/handle-mail-replies"


Reload exim

update-exim4.conf service exim4 restart




## 7.  Install ruby dependencies under the new ruby version

`script/rails-post-deploy`

## 8.  Clean up any badly-encoded data

One of the big differences between ruby 1.8.7 and later versions of ruby is in the handling of character encodings. You can read more about this in http://yehudakatz.com/2010/05/05/ruby-1-9-encodings-a-primer-and-the-solution-for-rails/.

We've added a temporary task to Alaveteli in Release 0.22 to help clean up any badly encoded data already in the database. 

Note: if you first installed Alaveteli at version 0.19 or later, your database should be `UTF-8` by default, so you shouldn't have any badly encoded existing data in the database.

To run the task in `dryrun` mode to get a preview of the data to be fixed:

bundle exec rake temp:fix_invalid_utf8


To fix the data:

bundle exec rake temp:fix_invalid_utf8 DRYRUN=0


## 9. Bring your site back up

Remove `public/down.html` 

Restart your app server 

```service alaveteli restart```


## 10. Restore incoming email handling

Allow mail to be piped to the incoming mail handler scripts.


### Exim



In `/var/www/alaveteli/config/aliases`, uncomment the line 

#^foi\+.*: "|/home/vagrant/alaveteli/script/mailin", backupfoi

if you're switching ruby system wide or  

#^foi\+.*: "|/home/alaveteli/run-with-rbenv-path /var/www/alaveteli/script/mailin", backupfoi

if you're switching using rbenv. 
Remove the line 

^foi\+.*: "|/home/vagrant/alaveteli/script/defer"


If you have Alaveteli set up to filter messages to site admin addresses, uncomment the line

#user-support: |/var/www/alaveteli/script/handle-mail-replies


for site-wide ruby or 

#user-support: "|/home/alaveteli/run-with-rbenv-path /var/www/alaveteli/script/handle-mail-replies"

for `rbenv` ruby. Remove the line 

user-support: |/var/www/alaveteli/script/defer



### Postfix

In /etc/postfix/master.cf, uncomment the line

#alaveteli unix - n n - 50 pipe flags=R user=alaveteli argv=/var/www/alaveteli/script/mailin


for site-wide ruby, or 

#alaveteli unix - n n - 50 pipe flags=R user=alaveteli argv=/home/alaveteli/run-with-rbenv-path /var/www/alaveteli/script/mailin


for per-user ruby with `rbenv`. Remove the line 

alaveteli unix - n n - 50 pipe flags=R user=alaveteli argv=/var/www/alaveteli/script/defer



If you have Alaveteli set up to filter messages to site admin addresses, restore that pipe too. Uncomment the line

#alaveteli_replies unix - n n - 50 pipe flags=R user=alaveteli argv=/var/www/alaveteli/script/handle-mail-replies

for site-wide ruby, or 

#alaveteli_replies unix - n n - 50 pipe flags=R user=alaveteli argv=/home/alaveteli/run-with-rbenv-path /var/www/alaveteli/script/handle-mail-replies


for per-user ruby with `rbenv`. Remove the line 

alaveteli_replies unix - n n - 50 pipe flags=R user=alaveteli argv=/var/www/alaveteli/script/defer


Reload postfix:

postfix reload