Skip to content

Website setup and administration

Jump to bottom Edit New page
Jim Allman edited this page Jul 31, 2014 · 17 revisions

NOTE: This page is superceded by the (much more concise) Installation page in this wiki, and will be removed shortly.

Here are general instructions for deploying the Fossil Calibrations website and database, with brief command-line tests to validate the results of many steps. Note that the tests below use fossils.mysite.com for the public domain name, and 167.88.120.104 for the server's IP address. Please substitute your own values here!

System requirements for the server

The FCD website is a small system, so we use a single server for the website files and database. It's not expected to see heavy traffic, but some of the database operations (esp. taxonomic search queries) are demanding. The current development server has these specs, which should serve as a minimum for system requirements:

  • 1024MB RAM + 1024MB VSwap
$ free -m
             total       used       free     shared    buffers     cached
Mem:          1024         87        936          0          0         21
-/+ buffers/cache:         66        957
Swap:         1024          0       1024
  • 4 CPU Core Access (may not be fully utilized)
  • 150GB SSD-Cached HDD Space (or a fast hard drive)
  • 1Gbps Port
  • 3000GB Bandwidth
  • OS/version: CentOS 5 32-bit
$ cat /etc/*release*
CentOS release 5.10 (Final)

$ uname -a
Linux fossils.mysite.com 2.6.32-042stab090.5 #1 SMP Sat Jun 21 00:15:09 MSK 2014 i686 i686 i386 GNU/Linux

In our development machines, we've used XAMPP to quickly install Apache + MySQL + PHP, but the components can also be installed piecemeal using the preferred package-management system for your server's OS. Note: If you do opt to use XAMPP, please follow their recommendations to lock it down, as its default settings are highly insecure!

Apache webserver

  • version 2.2.3 (this need not be exact)
$ apachectl -v
Server version: Apache/2.2.3
  • add VirtualHost directive for normal traffic
<VirtualHost *:80> 
    ServerAdmin admin@mysite.com 
    DocumentRoot /opt/lampp/htdocs/fossil-calibration 
    # Note that location is typical for a XAMPP setup
    ServerName fossils.mysite.com  
</VirtualHost>
  • un-comment the Include statement to use vhosts (if needed) in httpd.conf
  • add VirtualHost directive for secure (HTTPS) traffic
<VirtualHost _default_:443>
   # General setup for the virtual host
   DocumentRoot "/opt/lampp/htdocs/fossil-calibration"
   # Note that location is typical for a XAMPP setup
   ServerName fossils.mysite.com:443
   ServerAdmin admin@mysite.com
   ErrorLog /opt/lampp/logs/error_log
   # ... etc. ...
  • installed modules on dev server (far more than we need but possibly useful for troubleshooting):
$ apachectl -M
Loaded Modules:
 core_module (static)
  mpm_prefork_module (static)
  http_module (static)
  so_module (static)
  auth_basic_module (shared)
  auth_digest_module (shared)
  authn_file_module (shared)
  authn_alias_module (shared)
  authn_anon_module (shared)
  authn_dbm_module (shared)
  authn_default_module (shared)
  authz_host_module (shared)
  authz_user_module (shared)
  authz_owner_module (shared)
  authz_groupfile_module (shared)
  authz_dbm_module (shared)
  authz_default_module (shared)
  ldap_module (shared)
  authnz_ldap_module (shared)
  include_module (shared)
  log_config_module (shared)
  logio_module (shared)
  env_module (shared)
  ext_filter_module (shared)
  mime_magic_module (shared)
  expires_module (shared)
  deflate_module (shared)
  headers_module (shared)
  usertrack_module (shared)
  setenvif_module (shared)
  mime_module (shared)
  dav_module (shared)
  status_module (shared)
  autoindex_module (shared)
  info_module (shared)
  dav_fs_module (shared)
  vhost_alias_module (shared)
  negotiation_module (shared)
  dir_module (shared)
  actions_module (shared)
  speling_module (shared)
  userdir_module (shared)
  alias_module (shared)
  rewrite_module (shared)
  proxy_module (shared)
  proxy_balancer_module (shared)
  proxy_ftp_module (shared)
  proxy_http_module (shared)
  proxy_connect_module (shared)
  cache_module (shared)
  suexec_module (shared)
  disk_cache_module (shared)
  file_cache_module (shared)
  mem_cache_module (shared)
  cgi_module (shared)
  version_module (shared)
  proxy_ajp_module (shared)
  • ports and SSL (typical webserver, firewall must allow outside access to port :80 and :443)
# Use telnet from an *external* machine to verify that firewall and DNS are correct. 
# Test for normal web traffic on port :80

$ telnet fossils.mysite.com 80
Trying 167.88.120.104...
Connected to fossils.mysite.com.

# Test again for secure (HTTPS) requests on port :443

$ telnet fossils.mysite.com 443
Trying 167.88.120.104...
Connected to fossils.mysite.com.

Support for HTTPS and secure logins

  • secure a domain name and set up DNS
  • create and install SSL certificates (possibly self-signed, if only admin users use HTTPS)
  • configure apache to support SSL on (standard) port 443; see Apache webserver above

MySQL database

  • version: MySQL v 5.5
# NOTE that this checks the version inside a XAMPP installation
$ /opt/lampp/bin/mysql -u mysql -V
...
Server version: 5.5.8 Source distribution
  • configuration tweaks, e.g. in mysql/my.cnf
    • thread_stack to support heavy recursion in FCD's stored procedures
   [mysqld]
    ...
   # DEFAULT: thread_stack = 192K
   ## jimA: boosting this to support recursion in FCD!
   thread_stack = 1920K

Note that MySQL v5.6 introduced full-text search for InnoDB tables, which would allow us to search all text in calibration records. Currently we search a wide variety of metadata, which works well enough.

# TODO: Quick tests using the mysql command-line client?

PHP

  • version: 5.3 (or newer is probably fine)
# NOTE that this checks the version inside a XAMPP installation
$ /opt/lampp/bin/php -v
PHP 5.3.5 (cli) (built: Jan  7 2011 10:03:34)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies
  • enable automatic session (re)init for each request in php.ini: 
    session.auto_start = 1

Setting up the FCD website

  • install git client on the new server, if it's not there already
  • install files (clone FCD repo) into a sensible location
    • typically, this is in htdocs or wherever your apache server expects to find site docs
    • for example, in our XAMPP setup this was /opt/lampp/htdocs/fossil-calibration
  • create Site.conf with sensitive information (adapt Site.conf.EXAMPLE in the site root)
    • add real username and password
    • replace any dummy values from `Site.conf.EXAMPLE' that include 'REPLACE' or 'EXAMPLE'
    • see latest dev server (or contact developers) for current FCD usernames and passwords
  • adjust paths or credentials to db
  • configure apache to recognize and serve the site

    • modify iptables (in webmin control panel: Networking > Linux Firewall)

      ... in the main Packet Filtering table:
Accept :: If protocol is TCP and input interface is eth0 and destination port is 643 :: secure browsing to VE 106 (fossils.mysite.com)
... in the NAT table:
Destination NAT :: If protocol is TCP and source and destination ports are 643 :: forward HTTPS on port 643 to VE106:443

    • Note that we force some pages to HTTPS (this should be handled automatically in PHP): See these URLs to experience the bounce to HTTPS:

Quick test

  • Test all site features (with old data, if necessary)
    • browsing/search studies
    • login as admin
    • login as reviewer
    • editing an existing study
    • creating a study
    • deleting a study
    • admin tools (all)
      • rebuild all calibration trees This seemed to work pretty quickly, monitoring with 
        $ top -u nobody
      • update searchable multitrees, ~3 min
      • update calibrations-by-clade table, ~9 min
      • update auto-complete lists, ~8 min

Site monitoring and administration

  • start and stop the servers
  • start automatically on (re)boot
  • include phpmyadmin?
  • monitoring tools?
  • where to find error logs (OLD locations in Xampp installation):
    • /opt/lampp/var/mysql/fossils.mysite.com.err
    • /opt/lampp/var/mysql/fossils.log
    • /opt/lampp/var/mysql/fossils-slow.log
  • responding to email and contact forms (new/shared mailbox?)
  • using the site's "broadcast message" feature
  • periodic updates to NCBI taxonomy
Add a custom footer
Add a custom sidebar
Clone this wiki locally