ISB-CGC UI prototyping
This project uses Google App Engine, Python 2.7, Django 1.7.1, and MySQL 5.6
This app is set up to run and deploy on various Google Cloud Projects.
Installation Instructions For Local Development
The system uses Vagrant to setup a consistent, platform independent development environment. To setup your development environment to run locally, you will need to install the following:
- Oracle VirtualBox
If you are on Windows 8 or above, you will need to make sure you have version 5.0.9 or above to support the network interface. This may involve downloading a test build
- PyCharm Pro (Recommended)
From there simply perform these steps.
- Copy the
sample.envfile to a file named
.envin the root directory
- Fill out the
.envfile with the proper values
- For most development environments,
DATABASE_PASSWORDcan be the same, and
GCLOUD_PROJECT_IDis available after creating a project in the Google Cloud Dashboard
GOOGLE_CLIENT_SECRETcan also be obtained in the Google Cloud Dashboard by going to API & Auth > Credentials > Add New > OAuth 2.0 Client > Web Application
- Be sure when developing locally that you have 127.0.0.1 in the list of allowed domains for the OAuth 2.0 key
PyCharm Pro can be used to run your server through Vagrant and the Google App Engine.
- Go to your PyCharm Settings (On Mac, Go to Preferences;
- Select Project: ISB-CGC-Webapp > Project Interpreter
- Click the icon next to the Project Interpreter drop down at the top of the main area
- Click Add Remote
- Select Vagrant (if it asks to start the machine, say yes)
- Set the Python interpreter path to
/home/vagrant/www/shell/python-su.shand click Ok
- Select Languages & Frameworks > Google App Engine
- Change the SDK directory to
- Click Ok to save
- Go to Run > Edit Configurations
- If there is not an App Engine server Configuration, add one
- Set the host to
- Set Additional Options to
--skip_sdk_update_check --use_mtime_file_watcher=True --admin_host 0.0.0.0 /home/vagrant/www
- Optionally set PyCharm to run a browser at url
- Set the Python Interpreter to the Vagrant Machine (if it is not set to that already)
- Set the working directory to
- Click ok to save
- Update app.yaml at root from "runtime:custom" to 'runtime: python27' this is due to a google bug identified by CGC
You will also need to set the shell/python-su.sh file to be executable. You can do this in the vagrant machines command line with the command
chmod +x /home/vagrant/www/shell/python-su.sh
To run your server in PyCharm:
- Make sure your Vagrant machine is running by going to Tools > Vagrant > Up
- Click on the Run or Debug icons in the toolbar
- Click Run or Debug button on the configuration dialog
- Click Continue Anyway to run the machine
Your server will start and the PyCharm console should show all the logs and output from the system. If you are running in debug, you can also use breakpoints to stop the execution and examine variables and code as it runs.
Adding Python Dependencies
To add Python Libraries or Dependencies, you should add them to the requirements.txt file and they will automatically be pulled down when a new developer starts the system.
To update your existing python dependencies because of a change or to pull down additional libraries you need, SSH into the virtual machine and run
pip install. Through PyCharm, you can take the following steps.
- Click Tools > Start SSH session...
- Select the Vagrant VM Connection you set up
cd www; pip install -r requirements.txt --upgrade -t lib/
Or from the command line, you can do this by doing the following
- Open a terminal in the project directory
vagrant sshto login to the virtual machine
- Change directory to the
/home/vagrant/www/is the full path)
pip install -r requirements.txt --upgrade -t lib/
Deploying to a new Google Cloud Project
NOTE: These are only instructions for setting up the AppEngine application and does not include Computational or BigQuery instructions.
- Under APIs & Auth --> APIs
- Enable Identity Toolkit API
- Under Apis & Auth --> Credentials
- Create a new Client ID
- Select Web Application
- Redirect URIs
- Once created, you should Download JSON and place it in the root of your code directory.
- Create another Client ID
- Select Service Account
- Select P12 Key
- Once created, it should automatically download a .p12 file. You will need to convert this into a PEM key and place it in the root of your code directory.
- Also Generate a new JSON key and place it in the root of your code directory.
- Create A new Public API Access
- Create a Server Key and A Browser Key
- Under Storage --> Cloud SQL
- Create a new instance
- Open the advanced options.
- Provide it a name.
- Select the region and at least a Tier of D2 for better performance.
- Use MySQL Database version 5.6.
- Set billing to Package.
- Activation policy should be Always On. (reduces spin up time on each read)
- Assign an IPv4 address to the instance.
- Add an allowed network. It is useful to add the network of your local development computer.
- By default, your App Engine Application ID should already be allowed.
- Update your settings file to point to the correct private key files.
- In the settings file, update the server database settings to use your Cloud SQL settings.
- In the settings file, make sure that it checks the environment variable for SETTINGS_MODE == 'prod'. Check to see that the database settings used in this case are correct for your Cloud SQL instance. This will be used to run migrations on the Cloud SQL instance from your local development machine.
- In your terminal, run
SETTINGS_MODE='prod' ./manage.py syncdb. This should create the necessary Django tables in your Cloud SQL instance. It will also prompt you to create a super user.
- Import the necessary metadata tables and feature_defs table.
- For the cron job to work, you need to get the file NIH_FTP.txt, currently not stored in Drive. See @medullaskyline. Also see special instructions for the paramiko library in appengine_config.py.
- The Cloud Project should now be ready for you to deploy. Either deploy from the Google App Engine Launcher, or through the terminal
appcfg.py updatein the root directory of your project.
- Go to
http://<YOUR-APP-ID>.appspot.com/and hope for the best!
- If the site works, go to
http://<YOUR-APP-ID>.appspot.com/adminand enter the superuser name and password you created.
- Open the Social Applications table in admin to add a new Social Application. Make the provider Google, name it whatever you want ('Google' is fine), copy and paste the client_id and client_secret from our client_secrets.json file into the Social Application's Client id and Secret key fields. Leave the Key field blank.
- Then select isb-cgc.appspot.com, localhost:8000, and localhost:8080 in the Available sites field and move them to the Chosen sites field.