Add Docker documentation and update local development setup documentaion

DOCKER.md

@@ -0,0 +1,55 @@
+Instructions for Setting Up Docker
+-----------------------------------
+
+1. Install the stable community edition of Docker. Install the version that
+corresponds to your operating system from this [page](https://docs.docker.com/install/).
+Make sure the docker app is running. You should see a whale icon in your
+toolbar. If you are not on a Mac, see the [docker-compose installation page](https://docs.docker.com/compose/install/)
+for information on how to set this up on your operating system.
+2. Get the docker image:
+  - Pull them from docker hub
+    1. The docker image contains sensitive data and thus, it cannot be
+    pulled freely. Contact me (henry.doupe@aei.org) if you think that
+    you need this image. If approved, I will grant your docker hub account
+    access to this repository.
+    2. Next, login to docker at the command line with the command
+    `docker login`, and enter your information.
+    3. You can start the service by running:
+    `cd PolicyBrain/distributed && docker-compose up -d`
+    This will pull the required images, configure them, and run them in a
+    detached state. Remove the `-d` flag to run them in the foreground.
+  - Build them locally
+    This may be useful if you do not have access to the sensitive data. With
+    this process, you can run PolicyBrain using only non-sensitive data.
+    1. go to the `distributed` directory.
+    2. run
+    ```
+    export PB_VERSION='type current PolicyBrain version here'
+    docker build -t distributed:$PB_VERSION ./
+    docker build -t celery:$PB_VERSION --file Dockerfile.celery ./
+    docker build -t flask:$PB_VERSION --file Dockerfile.flask
+    ```
+3. Depending on your system you may have to tweak the Docker memory and CPU
+usage limits. I find that I can do most runs with RAM set around 8 GB and
+CPU at 1 or 2. Your mileage may vary. Some of the more memory intensive runs
+such as Tax-Calculator simulations with the CPS file require RAM up to 12 or 13
+GB when using docker. You may have problems doing those runs locally.
+Adjust these parameters by clicking the docker icon in the toolbar,
+selecting preferences, and click the "Advanced" icon. Make sure to click
+"Apply & Restart" so that the adjustments will go into effect.
+
+Other useful commands
+-------------------------
+- View logs
+Get Container ID:
+`docker ps`
+Follow log of service with that ID:
+`docker logs -f CONTAINER_ID`
+
+- View CPU and memory usage:
+`docker stats`
+
+- Experiment within the `distributed` container
+`docker run -it opensourcepolicycenter/distributed`
+
+- More to come...

README.md

@@ -29,44 +29,32 @@ window and run the following commands using bash:
 git clone https://github.com/YOURUSERNAME/PolicyBrain.git
 cd PolicyBrain
 git remote add upstream https://github.com/OpenSourcePolicyCenter/PolicyBrain
-pushd deploy
-./install_taxbrain_server.sh
-popd
+./python_env_build.sh
 export DATABASE_USER=YOUR_POSTGRES_USERNAME DATABASE_PW=YOUR_POSTGRES_PASSWORD
-source activate aei_dropq && source webapp_env.sh
+source activate pb_env && source webapp_env.sh
 python manage.py collectstatic
 python manage.py migrate
 python manage.py runserver
 ```
 Now, the Django app should be up and running.  You can access the local instance of https://www.ospc.org/ at http://localhost:8000.  Next, set up Redis, Flask, and Celery.  This step allows you to submit and run jobs.
-In another terminal, run the following commands using bash:
+These services are configured via Docker in a docker-compose file. You need access to these images for the following commands to work. See DOCKER.md for more information.
 ```
-# Go to the PolicyBrain directory
-cd PolicyBrain/
-source activate aei_dropq && source webapp_env.sh
-cd deploy/taxbrain_server
-
-# ignore the following block if you do not have access to the taxpuf package
-conda config --add channels 'https://conda.anaconda.org/t/YOUR_TOKEN_HERE/opensourcepolicycenter'
-conda install taxpuf
-write-latest-taxpuf
-gunzip -c puf.csv.gz > puf.csv
-
-supervisord -c supervisord_local.conf
-
+# Login to docker with `docker login`
+# Go to the PolicyBrain/distributed directory
+cd PolicyBrain/distributed
+docker-compose up -d
 ```
 
 Now, that the server has been installed, you can start it up simply by running:
 
 ```
 export DATABASE_USER=YOUR_POSTGRES_USERNAME DATABASE_PW=YOUR_POSTGRES_PASSWORD
-source activate aei_dropq && source webapp_env.sh
+source activate pb_env && source webapp_env.sh
 python manage.py runserver
 ```
 
 and in another terminal window, run:
 
 ```
-source activate aei_dropq && source webapp_env.sh
-cd deploy/taxbrain_server && supervisord -c supervisord_local.conf
+docker-compose up -d
 ```