Skip to content

Setup a development environment on mountain lion

Jump to bottom Edit New page
talonsensei edited this page Sep 16, 2013 · 1 revision

Where Things Go

  • /usr/local: This should contain your entire development environment
  • /usr/local/bin: Most of the binary files
  • /usr/local/Cellar/httpd/2.2.25/share/apache2: Contains htdocs and cgi-bin directories
  • /usr/local/lib/python2.7/site-packages: All of the shared python modules
  • /usr/local/etc: configuration files, including httpd.conf
  • /usr/local/var: Apache log files, Postgres config
  • /usr/local/defaultdb: Postgres database
  • /usr/local/Cellar: Where Homebrew installs all applications
  • /Library/Caches/Homebrew: All download files pulled by Homebrew are stored here

Installation

To get up and running you will have to install various tools. This guide was written under the assumptions that:

  • You are using Mac OS X 10.8: Mountain Lion
  • You are not afraid of the Terminal
  • You are smart enough not to run and hide if something goes wrong, and instead use Google to help you figure things out.

We use Homebrew as a package manager and we will use VirtualEnv to set up a virtual environment for the server to run in. We believe that this provides the greatest flexibility and ease of use at the present time. Please follow this guide closely and make sure that things are done in order, as occasionally that matters.

  1. Prerequisites =============
  • Software Update: The first thing to do on your shiny new machine is to install all available system updates
  • XCode: Obtain XCode from the App Store and install it. Then run it and open Preferences -> Downloads and install the Command Line Tools. It is possible to just install the command line tools without XCode, but this is simplest.
  • Chrome: Install the latest version of Google Chrome from http://chrome.google.com
  • Sublime: We recommend installing the Sublime text editor. (Not required but very nice to have) You can make Sublime a commandline editor with this command: ln -s "/Applications/Sublime Text 2.app/Contents/SharedSupport/bin/subl" /usr/local/bin/subl
  • Homebrew: Install Homebrew using all defaults
    • Homebrew homepage is here
      To install Homebrew, copy and paste this into Terminal (DO NOT USE SUDO):

ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"

  • Run brew update and then brew doctor. At this point you should be good to go! Homebrew updates very often, so it is always a good idea to run brew update as the first thing on any day you decide to work with brew

  • Fix some OS-specific things Turns out that the upgrade from Lion to Mountain Lion broke a few things, so we need to fix them so that they behave the way we (and the software) expect them to.

  • Fix broken Toolchain: Run the following in Terminal
    [ "$(sw_vers -productVersion | sed 's/^\(10\.[0-9]\).*/\1/')" = "10.8" ] && bash -c "[ -d /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain ] && sudo bash -c 'cd /Applications/Xcode.app/Contents/Developer/Toolchains/ && ln -vs XcodeDefault.xctoolchain OSX10.8.xctoolchain' || sudo bash -c 'mkdir -vp /Applications/Xcode.app/Contents/Developer/Toolchains/OSX10.8.xctoolchain/usr && cd /Applications/Xcode.app/Contents/Developer/Toolchains/OSX10.8.xctoolchain/usr && ln -vs /usr/bin'"

  • Fix locate. Later we will install ack which can be used to find things, but it is good to get the system tool working

    sudo dscl . -create /Users/nobody
sudo dscl . -create /Users/nobody UserShell /usr/bin/false
sudo dscl . -create /Users/nobody Realname "Unprivileged User"
sudo dscl . -create /Users/nobody UniqueID "13"
sudo dscl . -create /Users/nobody PrimaryGroupID: "-2"
sudo dscl . -create /Users/nobody NFSHomeDirectory "/var/empty"

    This creates a "nobody" user that is used to maintain the locate database. Enable the root user from System Preferences. Then log into Terminal as root and execute this command once as root.

    /usr/libexec/locatedb.updatedb

    This will create the database for the first time. After this it is no longer necessary to run as root. If desired, this command can be aliased in the .bash_profile

  • Install Python (upgrade to 2.7.5+)
    Use Homebrew to install the latest 2.7.x version of Python. We are not using Python 3 because many libraries / packages we use are not available for that version yet.

    brew install python

  • Install GitHub for Mac: Download from http://mac.github.com/

  • Install Git: brew install git This installs pip as a dependency, which is good, we will use it

  1. Set up Virtual Environment ============================= Next we are going to install virtualenv and virtualenvwrapper which will allow us to create a unique virtual environment to work in.

    pip install virtualenv
pip install virtualenwrapper

    Add the virtual environment root location to the .bash_profile with the following:

    export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh

    Create a new virtual environment. For example on the machine Nauru we said:

    mkvirtualenv nauru_dev

    Which creates a new environment called nauru_dev. We can switch to the environment by saying workon nauru_dev. A nice tutorial of virtualwrapper commands can be found here

  2. Install Major Apps ===================== Next we will install Postgres, Django, Apache. Configuration will be saved for later.

  • Install Postgres 
    brew install postgresql
  • Setup the postgres user Find a unique ID for a new user and new group using the following commands. Look at the output for a unused UniqueID and a unused PrimaryGroupID. It doesn't need to be sequential: 
    dscl . -list /Users UniqueID
dscl . -list /Groups PrimaryGroupID
    Use the unique ID (183 in our case) to create the user and group: 
    sudo dscl . -create /Groups/postgres 
sudo dscl . -create /Groups/postgres RealName 'PostgreSQL Server'
sudo dscl . -create /Groups/postgres PrimaryGroupID 183          
sudo dscl . -create /Groups/postgres UserShell /usr/bin/false
sudo dscl . -create /Groups/postgres Password 'postgres'
sudo dscl . -create /Groups/postgres NFSHomeDirectory /var/empty

sudo dscl . -create /Users/postgres   
sudo dscl . -create /Users/postgres Password 'postgres'
sudo dscl . -create /Users/postgres RealName 'PostgreSQL Server'
sudo dscl . -create /Users/postgres PrimaryGroupID 183
sudo dscl . -create /Users/postgres UniqueID 183      
sudo dscl . -create /Users/postgres UserShell /bin/sh
sudo dscl . -create /Users/postgres NFSHomeDirectory /usr/local/var/postgres
  • Install Django and Apache Install Django and Apache into the virtual environment 
    workon nauru_dev
pip install django
brew tap homebrew/apache
brew install zlib       # version 1.2.8+
brew link zlib --force
brew install httpd      # version 2.2.25+
brew install mod_wsgi   # version 3.4+
    I contributed a fix to the brew Formula to install mod_wsgi, so this should work without a problem. However if you get an error after the mod_wsgi install, you may need to compile it from source. An explanation of how to do this can be found here.

  1. Install minor apps and libraries =================================== Install utility apps, 3rd party binaries and python libraries that will be needed for our applications

    pip install fabric
pip install south                      # version 0.8.2+
pip install biopython                  # version 1.61+

brew install R                         # version 3.0.1+
sudo ln -s /usr/local/opt/r/R.framework /Library/Frameworks

pip install rpy2                       # version 2.3.6+
sudo pip install numpy                 # version 1.7.1+
sudo pip install scipy                 # version 0.12.0+
pip install ipython                    # version 1.0.0+
sudo pip install matplotlib            # version 1.3.0+
pip install pandas                     # version 0.12.0+
sudo pip install python-ldap           # version 2.4.13+
pip install ipdb                       # version 0.7+
pip install pil                        # version 1.1.7+
pip install xlwt                       # version 0.7.5+
pip install python-leveshtein          # version 0.10.2+

brew tap homebrew/science 
brew install mcl                       # version 12.135+
brew install emboss                    # version 6.6.0+
brew install clustal-omega             # version 1.1.0+
brew install clustal-w                 # version 2.1+
brew install hmmr                      # version 3.0+
brew install muscle                    # version 3.8.31+
brew install blast                     # version 2.2.28+
pip install psycopg2                   # version 2.5.1+
brew install ack                       # version 2.04+

  2. Configure Everything =======================

  • Configure Postgres
    Have already created the postgres user and group. So need to create the database and get it going.

    • Create the database
      To create and initialize the database:
    sudo mkdir -p /usr/local/var/postgres/data
sudo chown -R postgres:postgres /usr/local/var/postgres
sudo chown -R postgres:postgres /usr/local/var/postgres/data
sudo su postgres -c '/usr/local/Cellar/postgresql/9.2.4/bin/initdb -D /usr/local/var/postgres/data'

    To start the postgres and create the superuser:

    sudo su postgres
sh-3.2$ /usr/local/Cellar/postgresql/9.2.4/bin/pg_ctl -D /usr/local/var/postgres/data -l logfile start
server starting
sh-3.2$ psql postgres postgres
psql (9.2.4)
Type "help" for help.

postgres=# CREATE ROLE bfxadmin CREATEDB CREATEUSER LOGIN PASSWORD 'bfx_admin' CREATEROLE;
CREATE ROLE
postgres=# CREATE DATABASE bfxdb  WITH OWNER "bfxadmin" ENCODING 'UTF8';
CREATE DATABASE
postgres=#\q

exit

    Now you need to modify the plist so that Apple's launchctl will properly start the database whenever the machine is restarted. Take the plist created for you by Homebrew and modify it to match what is here

    sudo cp /usr/local/Cellar/postgresql/9.2.4/homebrew.mxcl.postgresql.plist /Library/LaunchDaemons/.
sudo nano /Library/LaunchDaemons/homebrew.mxcl.postgresql.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple/DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>KeepAlive</key><true/>
   <key>Label</key><string>homebrew.mxcl.postgresql</string>
   <key>UserName</key><string>postgres</string>
   <key>ProgramArguments</key>
   <array>
         <string>/usr/local/opt/postgres/bin/postgres</string>
         <string>-D</string>
         <string>/usr/local/var/postgres/data</string>
         <string>-r</string>
         <string>/usr/local/var/postgres/server.log</string>
   </array>
   <key>RunAtLoad</key><true/>
   <key>WorkingDirectory</key><string>/usr/local</string>
   <key>StandardErrorPath</key><string>/usr/local/var/postgres/server.log</string>
</dict>
</plist>

    Now load the daemon. The next time the machine is restarted, the database should start automatically

    sudo launchctl load -w /Library/LaunchDaemons/homebrew.mxcl.postgresql.plist

  • Configure Apache
    Make sure to add /usr/local/Cellar/httpd/2.2.25/sbin to the .bash_profile PATH so that it is the first thing loaded, like this

    export PATH=/usr/local/Cellar/httpd/2.2.25/sbin:$PATH

    Edit /usr/local/etc/apache2/httpd.conf and add the following:

    • add LoadModule wsgi_module libexec/mod_wsgi.so
    • Since mod_wsgi has already been installed under httpd, the full path to the .so is not needed
    • Uncomment the line Include /usr/local/etc/apache2/extra/httpd-vhosts.conf
    • Uncomment the line Include /usr/local/etc/apache2/extra/httpd-default.conf
    • Edit /usr/local/etc/apache2/extra/httpd-vhosts.conf to comment out the default stuff and to add this
    <VirtualHost *:80>
   WSGIDaemonProcess / processes=2 threads=15
   WSGIApplicationGroup %{GLOBAL}
   WSGIScriptAlias / /usr/local/var/django/code/ri/wsgi.py
   
   ServerAdmin evans@xoma.com
   DocumentRoot "/usr/local/var/django/code"
   ServerName nauru.xoma.com
   ErrorLog "/usr/local/var/apache2/log/nauru-error_log"
   CustomLog "/usr/local/var/apache2/log/nauru-access_log" common
   
   <Directory "/usr/local/var/django/code/ri">
   <Files wsgi.py>
       Options FollowSymLinks
       Order deny,allow
       Allow from all
   </Files>
   </Directory>
   
   <Directory "/usr/local/var/django/code">
      Options FollowSymLinks
      Order allow,deny
      Allow from all
   </Directory>
   
   Alias "/static/admin" "/usr/local/var/django/code/static/admin"
   
   <Directory "/usr/local/var/django/code/static/admin">
      Options FollowSymLinks
      Order allow,deny
      Allow from all
   </Directory>
   
   Alias "/static" "/usr/local/var/django/code/static">
   <Directory "/usr/local/var/django/code/static">
      Options FollowSymLinks
      Order allow,deny
      Allow from all
   </Directory>
   
   Alias "/media" "/usr/local/var/django/code/media"
   <Directory "/usr/local/var/django/code/media">
      Options FollowSymLinks
      Order allow,deny
      Allow from all
   </Directory>
  
</VirtualHost>

    Finally move the plist for Apache into the LaunchDaemons folder so that the server will be automatically started

    sudo cp /usr/local/Cellar/httpd/2.2.25/homebrew.mxcl.httpd.plist /Library/LaunchDaemons/.

  • Setting up Django
    Find a place to run Django from. For the Nauru server, we will create /usr/local/var/django/code as the place to start out the new project. Remember to be working in the right virtual env before setting up the Django project

    workon nauru_dev
cd /usr/local/var/django/code
python /Users/kaiju/.virtualenvs/nauru_dev/bin/django-admin.py startproject ri
    • When I did this it created a nested path/ri/ri structure. Move the files out of the inner ri directory into the outer one and delete the inner one.
    • edit settings.py
    DATABASES = {'default': {'ENGINE': 'django.db.backends.postgresql_psycopg2',
                         'NAME': 'bfxdb',
                         'USER': 'bfxadmin',
                         'PASSWORD': 'bfx_admin',
                         'HOST': '', 
                         'PORT': '',
                         },
               'celldb': {'ENGINE': 'django.db.backends.postgresql_psycopg2',
                          'NAME': 'celldb',
                          'USER': 'mark',
                          'PASSWORD': 'mark',
                          'HOST': 'tonga.xoma.com',
                          'PORT': '',
                         }
             }
ADMINS = (('Mark Evans', 'evans@xoma.com'),
          ('Eugene Chuba', 'chuba@xoma.com'),)
TIME_ZONE = 'America/Los_Angeles'
ROOT_URLCONF = 'ri.urls'
SETTINGS_FILE_FOLDER = '/usr/local/var/django/code/ri'
MEDIA_ROOT = '/usr/local/var/django/code/media'
MEDIA_URL = '/media/'
STATIC_ROOT = 'usr/local/var/django/code/static'
STATIC_URL = '/static/'
STATICFILES_DIRS = ( os.path.join(SETTINGS_FILE_FOLDER,'static'), )
STATICFILES_FINDERS = ( 'django.contrib.staticfiles.finders.FileSystemFinder',
                        'django.contrib.staticfiles.finders.AppDirectoriesFinder',
                        'django.contrib.staticfiles.finders.DefaultStorageFinder',)
ADMIN_MEDIA_PREFIX = '/static/admin/'
WSGI_APPLICATION = 'ri.wsgi.application'
TEMPLATE_DIRS = ('/usr/local/var/django/code/ri/templates' )
# Make sure to uncomment all INSTALLED_APPS, MIDDLEWARE_CLASSES, TEMPLATE_CONTEXT_PROCESSORS, TEMPLATE_LOADERS

    Next, from where manage.py is located, initialize the project and Django database tables. Follow the directions to create the Django superuser if it ask you to.

    python manage.py syncdb
python manage.py collectstatic
    • Edit the wsgi.py file that was created by startproject so that it looks like below. Since we will use R and potentially other applications installed on the server, we must add their paths to the django environment when it is activated.
    import os,sys,site
os.environ["R_HOME"]='/usr/local/Cellar/r/3.0.1/R.framework/Resources'
site.addsitedir("/Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages")

# Activate the virtual environment
# Taken from http://thecodeship.com/deployment/deploy-django-apache-virtualenv-and-mod_wsgi/
activate_env = os.path.expanduser("/Users/kaiju/.virtualenvs/nauru_dev/bin/activate_this.py")
execfile(activate_env, dict(__file__=activate_env))

sys.path.insert(0,'/usr/local/var/django/code')
sys.path.insert(0,'/usr/local/var/django/code/ri')

# We defer to a DJANGO_SETTINGS_MODULE already in the environment. This breaks
# if running multiple sites in the same mod_wsgi process. To fix this, use
# mod_wsgi daemon mode with each site in its own daemon process, or use
# os.environ["DJANGO_SETTINGS_MODULE"] = "ri.settings"
os.environ.setdefault("DJANGO_SETTINGS_MODULE","ri.settings")

# This application object is used by any WSGI server configured to use this
# file. This includes Django's development server, if the WSGI_APPLICATION
# setting points here.
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
    • test Django by running python manage.py runserver 0.0.0.0:8000 and connecting to localhost:8000/admin with a browser
    • test Django is running via Apache by doing sudo apachectl start or sudo apachectl restart then going to localhost/admin in a browser.
    • If there are any problems with these tests, it is likely due to mod_wsgi not being properly configured in Apache.
    • Restart your server. At this point, everything should be working (Apache, Postgres, Django) and you should be able to connect to http://servername.com/admin (e.g. http://nauru.xoma.com/admin) using the superuser account you created

  • Install required R packages

    • The first time you run install, you will be asked to choose a repository. Pick one close to your location. After that, it will no longer ask you during that same session.
    • Launch R from the terminal window by just typing R, then run the following in R:
    install.packages("drc",dependencies=TRUE)
install.packages("ggplot2",dependencies=TRUE)
install.packages("lattice",dependencies=TRUE)
install.packages("Rserve",dependencies=TRUE)
install.packages("pheatmap",dependencies=TRUE)
install.packages("RColorBrewer",dependencies=TRUE)
    • You can verify that a packge has been properly installed by typing library(ggplot2), for example, where the name of the package without quotes is inside the ()
    • Exit R
    q()
Save workspace image? [y/n/c]: n

  • Set up new GitHub repository for the new codebase that will be RI-codebase v.4 and deployed on Nauru

    • in /usr/local/var/django/code run git init
    • nano .gitignore
    • Add the following to .gitignore
    *.log
*.pot
*.pyc
*.swp
local_settings.py
media/facstool/
.DS_Store
    • Continue setting up repository
    touch README.md
git add .
git commit -a -u "initial commit of RI codebase on Nauru"
git remote add origin https://github.com/xoma-ri/code2.git
git push -u origin master
    • login to GitHub and configure the settings for the new repo, add contributors, etc.
Add a custom footer
Add a custom sidebar
Clone this wiki locally