Late 2019 tree deployment steps

Deploying a new tree

We need to get the deployment of a new synth tree more streamlined using ansible (see this card), but the sake of making that easier, this page memorializes the process for building and deploying 12.3 in December, 2019.

These notes use a variable tree_version for the major.minor numbers for the tree to deploy. Note that tree_version=12.3. You'll need to replace ${tree_version} with the appropriate #.# form in each step (until we get this in a script)

Build the tree, and serve it on nexttree for some period of time for review

cd $OPENTREE_ROOT/ot-ansible/
git pull origin

Hand edit next_synth_version in group_vars/all.yml and (if needed propinquity_branch in roles/fetch-propinquity/vars/main.yml)

git commit -m "updated next_synth_version" -a
git push origin master
ansible-playbook --limit=nexttree playbk-build-synth-pipeline.yml
ansible-playbook --limit=nexttree playbk-rebuild-synth-tree.yml
ansible-playbook --limit=nexttree playbk-serve-next-tree.yml

Copy it to files

The next bounce off your local machine is done because the different deployed machines (nexttree and files in this case) don't have keys to allow them each to log in to the opentree user on each other.

Connect to nexttree from your local machine:

ssh opentree@nexttree.opentreeoflife.org
cd synth/tree_builds/
bash ~/synth/propinquity/bin/make_tree_tarball.sh ./opentree"${tree_version}" opentree"${tree_version}" "${tree_version}"
tar cfvz opentree"${tree_version}".tgz opentree"${tree_version}"
exit

from your local machine:

scp opentree@nexttree.opentreeoflife.org:~/synth/tree_builds/opentree"${tree_version}".tgz .
scp opentree"${tree_version}".tgz opentree@files.opentreeoflife.org:~/
scp opentree@nexttree.opentreeoflife.org:~/synth/tree_builds/opentree"${tree_version}_tree".tgz .
scp opentree"${tree_version}_tree".tgz opentree@files.opentreeoflife.org:~/

you can then remove your local copies, if you don't need them.

As opentree@files.opentreeoflife.org:

ssh opentree@files.opentreeoflife.org
# Create the appropriate destination directory:
cd /var/www/html/synthesis
mkdir "opentree${tree_version}"
cd "opentree${tree_version}"
# now we can move the tar archives
mv ~/"opentree${tree_version}.tgz" .
mv ~/"opentree${tree_version}_tree.tgz" .
# unarchive the files, to allow browsing the data artifacts on `files`
tar xfvz "opentree${tree_version}.tgz"
tar xfvz "opentree${tree_version}_tree.tgz"

Copy it to opentreeoflifebackup

From an account on files.opentreeoflife.org that is in the filesreader group and has environment configured to push to the opentreeoflifebackup.org S3 backup:

source awsenv/bin/activate  # puts the aws command on PATH
cd /var/www/html
aws s3 sync --exact-timestamps "synthesis/${tree_version}" "s3://opentreeoflifebackup.org/synthesis/${tree_version}"

Update statistics and log files

In order for the 'About' links for the synth tree and the taxonomy to work correctly, you must update various files.

• opentree/webapp/static/statistics/ott.json
• opentree/webapp/static/statistics/synthesis.json
• germinator/doc/ot-synthesis-v${SYNTH_VERSION}.md
• reference-taxonomy/doc/ott${OTT_VERSION}.md

I think that the version number for the latest synth on the about page is determined by the last entry in synthesis.json. Therefore, you might want to wait to redeploy the opentree application until after the webservices are running for the new synth tree.

Deploy to production

Put tree into maintenance mode

See https://github.com/OpenTreeOfLife/germinator/tree/master/deploy#notifying-users-of-scheduled-downtime

Deploy the new tree with germinator scripts

On a local machine:

cd "$OPENTREE_ROOT/deployed-systems"
git pull origin

hand-edit SYNTH_URL in production/api.config to be SYNTH_URL=http://files.opentreeoflife.org/synthesis/opentree${tree_version}/opentree${tree_version}.tgz

git commit -m "updated SYNTH_URL" production/api.config
git push origin master
cd ../germinator/deploy
bash ./push.sh -c "$OPENTREE_ROOT/deployed-systems/production/api.config" otcetera

Take tree out of maintenance mode

You didn't close the tab with https://github.com/OpenTreeOfLife/germinator/tree/master/deploy#notifying-users-of-scheduled-downtime did you? If you did re-open it...