Skip to content

Latest commit

 

History

History
200 lines (156 loc) · 6.27 KB

deployment.org

File metadata and controls

200 lines (156 loc) · 6.27 KB
Raw

Example deployment

All packages for this deployment will be installed into a separate profile. We will refer to the profile as $GUIX_PROFILE.

export GUIX_PROFILE=/var/guix/profiles/custom/rcas/.guix-profile

We will use Nginx as a reverse proxy that forwards requests to http://localhost:3000 to one of 20 rcas-web application servers. For the job queue we will need a Redis server.

Installing software

Let’s first install the software packages and the locale files for the glibc to avoid any locale warnings that usually arise when using Guix software on a GNU distribution other than GuixSD.

guix package -p $GUIX_PROFILE -i rcas-web nginx redis glibc-locales

Once Guix has finished installing things we can activate the profile:

bash
source $GUIX_PROFILE/etc/profile

Preparing directory structure

Next, we’ll prepare a directory structure for Nginx configuration files, Nginx working directory, log and pid directories, GTF files used by RCAS, and directories for storing uploaded files and generated reports.

mkdir -p /srv/rcas-web/nginx/logs
mkdir -p /srv/rcas-web/rcas/{gtf,uploads,reports,logs,pids}

Nginx

The goal in using Nginx is only to expose a single port and dispatch to one of a whole range of application web servers.

The following configuration file suffices for our purposes. Save it as /srv/rcas-web/nginx/nginx.conf.

worker_processes 5;
error_log /dev/null emerg;
pid /srv/rcas-web/nginx/nginx.pid;

events {}

http {
  upstream rcas-web {
    # Forward request to least connected application server.
    least_conn;
    server localhost:3001;
    server localhost:3002;
    server localhost:3003;
    server localhost:3004;
    server localhost:3005;
    server localhost:3006;
    server localhost:3007;
    server localhost:3008;
    server localhost:3009;
    server localhost:3010;
    server localhost:3011;
    server localhost:3012;
    server localhost:3013;
    server localhost:3014;
    server localhost:3015;
    server localhost:3016;
    server localhost:3017;
    server localhost:3018;
    server localhost:3019;
    server localhost:3020;
  }

  error_log /dev/null emerg;
  access_log /srv/rcas-web/nginx/logs/access.log;

  # Forward all requests on port 3000 to one of the application servers.
  server {
    listen 3000;
    location / {
      proxy_pass http://rcas-web;
    }
  }
}

Let’s start Nginx with this configuration file and set its working directory to /srv/rcas-web/nginx.

nginx -c /srv/rcas-web/nginx/nginx.conf -p /srv/rcas-web/nginx

Redis

Redis doesn’t need any configuration. We only need to start it:

redis-server > /srv/rcas-web/redis.log &

RCAS web

RCAS needs GTF files for four genome assemblies (hg19, mm9, ce10, and dm3) in order to produce meaningful reports. They can be downloaded from the Ensembl database.

Here are direct links to the needed files:

  • hg19
  • mm9
  • ce10
  • dm3

Unpack them to /srv/rcas-web/rcas/gtf and rename them to “hg19.gtf”, “mm9.gtf”, “ce10.gtf”, and “dm3.gtf”, respectively.

This can be achieved with this shell snippet:

cd gtf
wget -O hg19.gtf.gz ftp://ftp.ensembl.org/pub/release-75/gtf/homo_sapiens/Homo_sapiens.GRCh37.75.gtf.gz
wget -O mm9.gtf.gz  ftp://ftp.ensembl.org/pub/release-67/gtf/mus_musculus/Mus_musculus.NCBIM37.67.gtf.gz
wget -O ce10.gtf.gz ftp://ftp.ensembl.org/pub/release-67/gtf/caenorhabditis_elegans/Caenorhabditis_elegans.WBcel215.67.gtf.gz
wget -O dm3.gtf.gz  ftp://ftp.ensembl.org/pub/release-78/gtf/drosophila_melanogaster/Drosophila_melanogaster.BDGP5.78.gtf.gz

gunzip *.gtf.gz
cd -

RCAS can use gene sets from the Molecular SignaturesDatabase (MSigDB). Since it is not clear whether collections may be redistributed legally, we opted not to package them with RCAS. To use gene sets, dump them in a directory and add a record for each gene set to the configuration file. Each record is a list with an internally used key followed by a label and the path to the file.

Next, we need to tell RCAS web where to find these files and the directories for uploaded BED files and generated reports. Save the following configuration file as /srv/rcas-web/rcas/rcas.conf.

((upload-dir  . "/srv/rcas-web/rcas/uploads")
 (results-dir . "/srv/rcas-web/rcas/reports")
 (gtf-files   . ((hg19 . "/srv/rcas-web/rcas/gtf/hg19.gtf")
                 (mm9  . "/srv/rcas-web/rcas/gtf/mm9.gtf")
                 (ce10 . "/srv/rcas-web/rcas/gtf/ce10.gtf")
                 (dm3  . "/srv/rcas-web/rcas/gtf/dm3.gtf")))
 (msigdb      . ((c2.cp "Canonical pathways"
                        "/srv/rcas-web/rcas/msigdb/c2.cp.v5.0.entrez.gmt"))))

Now all that’s left is to start the application web servers and the background workers with a script like this:

#!/bin/bash

# This script can be used on a computer where Guix is available, but
# where GuixSD is not used.

set -e
set -u

# This is needed to suppress locale warnings.  The "glibc-locales"
# package should be installed to this profile.
GUIX_PROFILE=/var/guix/profiles/custom/rcas/.guix-profile
export GUIX_LOCPATH=${GUIX_PROFILE}/lib/locale

# This must match the installation prefix.
ROOT=/srv/rcas-web/rcas

# Prepare directories
mkdir -p ${ROOT}/logs
mkdir -p ${ROOT}/pids

# Spawn web processes
for port in $(seq 3001 3020); do
    rcas-web --config=${ROOT}/rcas.conf server ${port} \
             > ${ROOT}/logs/rcas-web-${port}.log &
    echo $! > ${ROOT}/pids/rcas-web-${port}.pid
done

# Spawn background workers
for id in $(seq 20); do
    rcas-web --config=${ROOT}/rcas.conf worker \
             > ${ROOT}/logs/rcas-web-worker-${id}.log &
    echo $! > ${ROOT}/pids/rcas-web-worker-${id}.pid
done