Skip to content

Configuring an OS X Development Environment With Docker Containers

Jump to bottom
Adam J. Arling edited this page Oct 20, 2017 · 14 revisions

Overview

This page covers how to set up an Avalon development environment using docker containers to provide Solr, Fedora, Matterhorn (transcoding), Nginx (streaming), and Redis (message broker). Your Avalon app itself will continue to function as a regular rails application.

This was last tested on OS X 10.12.3 using Docker 1.13.

Installing Dependencies

The dependencies for this setup are mediainfo, ffmpeg, Docker, git, and Ruby. At the time of writing this document the Avalon dev team was using ffmpeg 3.2.3, mediainfo 0.7.92.1, Docker 1.13, and Ruby 2.3.1. If you are unfamiliar with installing this dependencies, please follow the suggested methods below:

Docker

Visit the Docker for Mac website and download the Stable channel dmg. Then install the dmg. Make sure you start Docker at least once after installing it. It runs a few more post install tasks and you can't continue with the rest of the instructions until this has been done.

ffmpeg, mediainfo, and git

  1. If you do not have homebrew installed on your machine, install homebrew via the directions on the homebrew website.
  2. Open a terminal and run:
  3. brew install ffmpeg
  4. brew install mediainfo
  5. brew install git

If needed you can install specific versions in homebrew, to do this:

  1. Search for versions of a homebrew tap via: brew search ffmpeg
  2. You will see a results list that will include versions in the form of homebrew/versions/ffmpeg28 (this would be for ffmpeg 2.8)
  3. Running brew install home/versions/ffmpeg28 would then install ffmpeg 2.8 if desired.

Ruby

You can use either RVM or RBENV to manage versions of ruby and to switch between them.

RVM

  1. Install RVM
  2. Install Ruby 2.3.1 by running rvm install 2.3.1
  3. Set Ruby 2.3.1 to your default ruby via rvm use 2.3.1 --default

RBENV

  1. Install RBENV via homebrew with brew install rbenv
  2. Install Ruby 2.3.1 by running rbenv install 2.3.1
  3. Set Ruby 2.3.1 to your default ruby via rbenv use 2.3.1 --default

Configure your Masterfiles directory:

This is needed for transcoding to work properly.

  1. Open a new terminal window and log in as root via sudo su - root
  2. mkdir /masterfiles && chmod 0777 /masterfiles
  3. Configure Docker for Mac to allow you to mount /masterfiles within a container.
  4. Click the Docker for Mac menu bar item and select Preferences....
  5. On the File Sharing tab, click + and add /masterfiles to the list of exported directories.
  6. Click Apply & Restart and wait for Docker to restart.

Setting up the Docker Containers

  1. Git clone the avalon-docker repo:
  • via https: git clone https://github.com/avalonmediasystem/avalon-docker/
  • via ssh: git clone git@github.com:avalonmediasystem/avalon-docker.git
  1. Enter the repo via cd avalon-docker and select the develop branch git checkout development
  2. Follow the instructions in networking.md.
  3. Compose your docker containers: docker-compose pull
  4. Start your docker containers: docker-compose up
  5. To start the containers in test mode (using different persistent volumes, so as not to overwrite your development data), use docker-compose -f test.yml up
  6. Leave this terminal open

Once docker has been started you should be able to access the following:

  • http://solr:8983 and see solrAdmin with an avalon core
  • http://fedora:8984/fedora/ and access fedora's rest endpoint
  • http://matterhorn:8080 and see the matterhorn landing page
  • http://streaming:8880 and see the nginx landing page

Configuring Avalon

  1. Open a new terminal (you do not need to be root in this terminal) and git clone the avalon repo:
  • via https: git clone https://github.com/avalonmediasystem/avalon/
  • via ssh: git clone git@github.com:avalonmediasystem/avalon.git
  1. Enter the repo via cd avalon and select the develop branch git checkout develop
  2. In the root of your avalon directly make a file called .env.development and place the content from the avalon-docker .env.development into it. Do the same for .env.test.
  3. Follow the installation directions in readme.md. However, do not the final bundle exec rake server: command. Skip this step and proceed to the next section.

Starting Avalon

  1. If your docker containers are not up, open a terminal window, cd to your your avalon-docker directory and run docker-compose up, leave this window open
  2. Open a new terminal window and cd into your avalon repo, then run bundle exec rake resque:scheduler, leave this window open.
  3. Open a new terminal window and cd into your avalon repo, then run bundle exec rake resque:workers COUNT=1 QUEUE=*, leave this window open
  4. Open a new terminal window and cd into your avalon repo, then run bundle exec rails s, leave this window open
  5. Visit localhost:3000, you should see the Avalon landing page
  6. In your browser create a new user by going to http://localhost:3000/users/auth/identity/register, use the following:
  • Name: archivist1
  • Email: archivist1@example.com
  • Password: your choice (we recommend using archivist1 as the password as well to keep things simple)
  1. You can now log into your avalon as archivist1 and visit http://localhost:3000/about to see your status. solr and fedora will probably be red since they are currently empty, but all other services should read as green.
  2. You can begin using Avalon, example videos can be found here. Note that via the web interface you can only upload files under 250 mb, so if you attempt to use the 1080p videos for testing you'll need to use Avalon's dropbox feature
  3. For development you can also launch rails console via rails c from the root of your avalon directory.

Some additional notes

  1. If you want to update your containers you can delete containers via:
  2. docker ps -a and note the ID of the container you wish to remove
  3. docker stop ID
  4. docker rm ID
  5. To delete ALL docker containers (not just the Avalon ones, all of them) on your system docker stop $(docker ps -a -q) and `docker rm $(docker ps -a -q).

There is also currently a docker bug where even if you remove the containers following the directions above your qcow2 file (located at: /Users/YOUR USER NAME/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/) will never shrink. As such you may need to occasionally delete this file and then rerun docker-compose pull and docker-compose up to trigger a fresh build of your containers and free up disk space.

You can open the docker app on Mac and in Preferences > Advanced configure how much RAM and CPU your containers use (this will be split across all containers you have started, this is not per container).

Clone this wiki locally