Mac OS X Installation
The pre-installation section targets users who may be unfamiliar with the terminal, homebrew, python environments, etc. If you already have a mac set up for developing/hacking (and you have homebrew, git, the github client, your own thoughts on how python should be installed, mysql or postgresql, an IDE and your own opinions) you may wish to note that I'm recommending using ~/dev as a working directory for installing mdid3 into and just skip on ahead to #installing-mdid3.
Terminal can be found at /Applications/Utilities/Terminal.app
If you knew that already and are rolling your eyes, skip to the next step.
There's no way to explain terminal in this document, but the link above is to a guide that's pretty good. In general, just type what your told to type and things should work out (The $ sign is the beginning of a new line)
When you open it for the first time, type the following (don't type the # or anything after it):
pwd # Tells you the current directory
# should make the terminal print /Users/_yourusername
mkdir dev # creates a folder (directory) called dev
cd dev # changes the current directory to dev
mkdir mdid-data # a place for mdid-data to go
cd .. # change directory one level up (back to dev)
Note: your home directory (/Users/_yourusername) is often referred to with the shorthand ~/, which is a working shortcut in the terminal - you can type cd ~/dev from any directory to change back to the dev directory in your home account. more...
Leave the terminal window open, we'll be coming back...
To get very far at all on a mac, you have to make sure the basic tools are installed, but luckily that's as easy as installing Xcode. You can check if you have it already with the terminal command xcode-select -p
If you get a response back like /Applications/Xcode.app/Contents/Developer then you're good.
But if you don't (e.g. xcode-select: error: unable to get active developer directory... )
then you'll have to install it. You should be able to use this command:
xcode-select --install
which should "open a dialog for installation of the command line developer tools" and allow you to install the developer tools you'll need to get MDID working.
- How to Install Command Line Tools in OS X Mavericks (Without Xcode)
- Building from the Command Line with Xcode FAQ
Mac OS X lacks a "standard" package manager like Ubuntu (apt-get) or Redhat/Centos (rpm/yum) operating systems, but luckily there's homebrew. Anything installed by Homebrew will be symlikned to /usr/local
# install homebrew package manager
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
note: the above url changed at least once during the writing of this guide, so if you get an error like
curl: (22) The requested URL returned error: 400 Bad Requestplease check the homepage of the homebrew project for the correct url
That might take a few minutes, but when it's done, run brew doctor to ensure that everything is ok (doctor should return Your system is ready to brew.).
To try it out, install wget, a useful shell command by typing:
brew install wget
If you previously installed home-brew, it's a good idea to type the following three commands before continuing:
brew update
brew doctor
brew upgrade
Or if you know what you're doing, you've probably skimmed to here so let's start setting up MDID3... (after the homebrew info)
Optional but after I worked with MDID3 for a month or two on my mac I really wished I had not messed up the python distribution the mac ships with
Unless you know better, just follow the Hitchhiker's Guide To Installing Python on Mac OS X
Summary of the guide, all commands in Terminal.app:
# add to your .bash_profile or .bash_rc file
export PATH=/usr/local/bin:$PATH
# install a more developer friendly python
brew install python --framework
# again with the .bash_profile
export PATH=/usr/local/share/python:$PATH
So we've installed a better python for running MDID3, but we can do a little more to give ourselves flexibility in the future by installing virtualenv & virtualenvwrapper - these will allow you to have as many separate python installations as you need, which is useful if you'd like to see what happens if you upgrade to newer version of python or django or anything else.
pip install virtualenv
pip install virtualenvwrapper
Then add an environment variable to allow the virtualenvwrapper to make things easy for you:
export WORKON_HOME=~/.virtualenvs
export PROJECT_HOME=$HOME/Dev
source /usr/local/bin/virtualenvwrapper.sh
mkdir ~/.virtualenvs
This command will append the WORKON_HOME to your profile so it still works next time you login
echo 'export WORKON_HOME=~/.virtualenvs' >> ~/.bash_profile
echo 'export source /usr/local/bin/virtualenvwrapper.sh' >> ~/.bash_profile
or edit your ~/.bash_profile file (nano ~/.bash_profile to edit) manually:
# brews before ...uh... the apple-distributed binaries
export PATH=/usr/local/bin:$PATH
# use custom python
export PATH=/usr/local/share/python:$PATH
# virtualenv settings
export WORKON_HOME=~/.virtualenvs
# add commands and autocomplete
source /usr/local/bin/virtualenvwrapper.sh
Save the .bash_profile (if using nano it's ctrl-x to exit and then y for "yes, save") and then back in the terminal type:
mkvirtualenv mdid3
and after seeing the informational text of the virtualenv creation process, your shell prompt should be something like this:
(mdid3)MDIDs-Mac:~ mdid$:
The leading bit in parentheses tells you which python virtualenv is currently active. You can get out of that virtualenv by either typing deactivate or closing the terminal window.
With many python projects, you're ready to go -- but MDID3 is particularly sophisticated, and like a sophisticated person it has sophisticated needs. Some of those needs (like the mysql connector, PIL, etc.) need to be installed with Homebrew not in a virtualenv so it's very important to issue this command now:
deactivate
virtualenv docs virtualenvwrapper docs
Why? Because there's really no way around engaging with the source code of MDID3 when you are setting it up. Integrated Development Environments (IDEs) do things like display source code with special formatting to make it more understandable (many text editors also do this), and an IDE with Python/Django support will actually evaluate the code and offer (sometimes) helpful hints when things don't work.
- I make no secret of thinking Pycharm is the best commercial editor
- XCode is free and has some python support
- If you just want a text editor, at least get TextWrangler
- Lots of people like Sublime
- Other thoughts from someone else
Ok, back to it... Make sure that you are not in a virtualenv by typing:
deactivate
If you're starting here and already have your development thing going, consider typing the following before continuing, just to make sure you're up to date and everything is good:
brew update
brew upgrade
brew doctor
You may also wish to use brew list to see what's already installed
brew install memcached
Post-install you will get instructions about configuring launchd to start memcached at login, and be told that you can "just run" /usr/local/opt/memcached/bin/memcached - but you should be able to just type
memcached -h
to see the help screen for memcached. We're going to almost forget about memcached now, all you need to know is you start it with -d (for daemon, i.e. "in the background") like
memcached -d
and quit it inelegantly with the command
pkill memcached
Download couchbase community edition and use the installer to install it. Couchbase Community Server 3.0.1
Follow the instructions to set up as a memcached server (not couchbase).
It's probably best to install via home-brew
brew install mysql
Then start the server like this:
mysql.server restart
Stop it like this:
mysql.server stop
Check on it with:
mysql.server status
IMPORTANT SECURITY NOTE homebrew installs mysql with no root password, which is technically dangerous, and could possibly put your mac at risk. You should really run the
mysql_secure_installationcommand and set the root password to whatever your password is at least (answer Y to all the other questions)
mysql_secure_installation
- Install MySQL on Mac OSX using Homebrew
- Download mysql community edition
- Installing MySQL on Mac OS X (official docs)
- http://www.djangoapp.com/blog/2011/07/25/django-installation-on-mac-os-x-lion/'
Note: You can use PostgreSQL if you prefer, but if you do you probably already have it installed
RabbitMQ is the (new) way that MDID3 handles server-side jobs like data import.
In terminal:
brew install rabbitmq
echo 'export PATH=$PATH:/usr/local/sbin' >> ~/.bash_profile
Alternatively to add the command
rabbitmq-servermanually to your system,nano ~/.bash_profileand add the linePATH=$PATH:/usr/local/sbinat the bottom
Start rabbitmq-server (in the background)
rabbitmq-server -detached
Check that rabbitmq is running
rabbitmqctl status
Stop/Quit rabbitmq
rabbitmqctl stop
Since support has been added for solr4 we might as well install that...
First things first, you're going to need java 8, and when I ran through this documentation on a fresh install of Yosemite the JRE didn't include command line support, so download and install JDK 8
When that's installed, check that your terminal shell can see java with java -version and assuming that returns something like java version "1.8.0_40" we can install solr:
brew install solr
Don't start solr yet, all the config files are in the MDID3 repo which we haven't downloaded yet
There are some things that don't work as well as everyone would like.
pip install mysql-python doesn't seem to always work.
Try this instead:
pip install -Iv http://sourceforge.net/projects/mysql-python/files/latest/download?source=files
You may have to add
export PATH=$PATH:/usr/local/bin/mysql
to your ~/.bash_profile
Additional notes:
Follow this guide - I'm pretty sure it's correct (only if you're going to need ldap support, of course)
http://projects.skurfer.com/posts/2011/python_ldap_lion/
The image library MDID3 uses (Pil or more recently Pillow) is a bit problematic on the mac due to some inconsistencies with where jpeg-8d is installed, or something. But homebrew does it right (note: the following command installs without JPEG2000 support due to problems with openjpeg) - deactivate first to make sure you're not in the mdid3 virtualenv we created earlier.
deactivate # if you've installed a virtualenv
brew install Homebrew/python/pillow
Without these a required library will fail to install later.
brew install freetds
brew install unixodbc
Git is a system for source code management. MDID3 is distributed via GitHub, which is sort of like youtube for source code. In order to install MDID, you need to at least install git, but if you aren't comfortable with a command line you'll probably be better off using GitHub's app.
-
Download Github for Mac and install it
- Consider doing this free tutorial to get a sense of what git is (optional)
A: Yes. Right Now, in fact.
-
Go to the official repo in a browser and click the Clone in Desktop link
-
Save the repo in the
~/devdirectory you made in the last step
cd ~/dev
git clone https://github.com/cit-jmu/rooibos.git mdid3
Ok, now we're on the road to starting up.
To install the rest of the libraries, do this command in the root of the MDID repo:
workon mdid3 # if you installed virtualenvwrapper and created mdid3 as a v-env
pip install -r requirements.txt
Terminal:
nano mdid.sql
and copy/paste this text in
CREATE DATABASE rooibos CHARACTER SET utf8;
GRANT ALL PRIVILEGES ON rooibos.* TO rooibos@localhost IDENTIFIED BY 'rooibos';
UPDATE mysql.user SET Select_priv='Y',Insert_priv='Y',
Update_priv='Y',Delete_priv='Y',Create_priv='Y',
Drop_priv='Y',Index_priv='Y',Alter_priv='Y'
WHERE Host='localhost' AND User='rooibos';
FLUSH PRIVILEGES;
\qhit ctrl-o and hit enter to save, and then on the command line type:
mysql -u 'root' < mdid.sql
and since we're in the terminal, lets go for broke and start solr...
cd ~/dev/rooibos/solr
nohup java -server -Djava.headless=True -jar start.jar
... then finish setting up the database
cd ../rooibos
python manage.py syncdb --noinput
pip install -f requirements.txt
./manage.py collectstatic
./manage.py migrate --fake-initial
You may get a message that says
Your models have changes that are not yet reflected in a migration, and so won't be applied. Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
in this case, try:
./manage.py makemigrations
./manage.py migrate
and then make the cache table
python manage.py createcachetable cache
... start the workers
python manage.py runworkers --server &
and then
... start the mdid development server
python manage.py runserver
If everything went correctly, you should be able to navigate to http://127.0.0.1:8000 and start using MDID.
tbd