#README

How to [synchronize a forked repo] sync with its upstream parent:

git checkout master
git pull https://github.com/ORIGINAL_OWNER/ORIGINAL_REPO.git BRANCH_NAME
# merge if necessary
git push origin master

##Goals

0. Learn some Python machine learning algorithms
1. Learn the Quantopian API
2. Acquire and examine external data to develop algorithms
3. Demonstrate learning via a Heroku webpage

##Getting started

###Quantopian [API Overview] overview Symbols:

    symbol('goog')  # single
    symbols('goog', 'fb')  # multiple
    sid(24)  # unique ID to Quantopian -- 24 is for aapl

• [Fundamentals] fundamentals_reference:

  • Available for 8000 companies, with over 670 metrics
  • Accessed using get_fundamentals with the same syntax as SQLAlchemy; returns a pandas dataframe
  • Not available during live trading, only in before_trading_start (once per day) to be stored in the context and used in the function handle_data

• Ordering: market, limit, stop, stop limit

• Scheduling: frequency in days, weeks, months, plus order time of day in minutes

• Allowed modules

• Example algorithms

The API is thin -- it's really just about trading. They provide zipline zipline, their backtest functions, as open-source code (github repo / pypi page) to test outside of the quantopian environment. [Here is a how-to] zipline-howto.

###Heroku

Virtual environment To be sure of the environment we can use virtualenv in our repository directory

virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
## to deactivate the virtual environment,
## just type 'deactivate' (without quotes)

Account

First get a heroku account, install the toolbelt, and then click on the Python icon to make a Python app.

git clone https://github.com/tanyaschlusser/try_quantopian.git
cd try_quantopian
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt  # so you can test locally
heroku login
heroku create
heroku apps:create tanya-try-quantopian

These instructions help to deploy in Git. And we are going to copy the [Flask microblog tutorial] flaskr for the main code, except use PyMongo to conncet to a MongoLab MongoDB database instead of SQLite3 and a SQL database.

To look at the logs while the app is running:

    heroku logs --tail

Heroku database and Mongo

The Heroku [Postgres Hobby Tier] heroku_hobby has a 10,000 row limit and the Quantcast data alone are up to 3925 rows each, so we have to be creative and have quite a few columns if we're loading data on heroku. The [MongoLabs developer tier] mongolab gives 500 free megabyes, so we are using them.

Example URLs:

• Postgres postgres://:@:/
• Mongo mongodb://:@:/
  Where the user name and password are not for the MongoLab account, but whatever was set for that specific database

heroku addons:add heroku-postgresql
heroku config  # to get the database URL
# And you need to set the remote Heroku
# environment variable DATABASE_URL
heroku config:set DATABASE_URL=<the database url>
heroku config:set MONGO_URI=<the mongoDB url>

And then restart:

heroku restart

To connect to the database:

heroku pg:psql  # launch a local db connection

The version of Postgres on Heroku right now is 9.3. The [Postgres 9.3 manual] pgmanual.

##Collaboration

• Quantopian: sharing a link on Sean's account
• Github repo: [Our github repo] repo
• Collaboration: [we have a shared IPython notebook on Sagemath] sage

###Sagemath cloud By default network conenctions are disabled on the Sagemath cloud, so you must go the 'settings' page and email the help contact to enable network access.

Pip is not installed on the sagemath cloud, and the user accounts do not have root privilege, so to install libraries you should download the actual package source code, and install from source. You need to navigate to the 'Files' section of the page, then type a command in the small dialog box that says Terminal command ...:

        curl -o pg8000.tar.gz https://pypi.python.org/packages/source/p/pg8000/pg8000-1.10.2.tar.gz#md5=7f7cfa3b4c4b103999f584ad3d813ded
        tar -xzf pg8000
        cd pg8000
        sage --python setup.py install --user

Or else you can use easy_install:

    wget https://bootstrap.pypa.io/ez_setup.py -O
    sage --python ez_setup.py --user
    easy_install --user pg8000
    easy_install --user zipline

##Data

• [Quantquote] qq has a historical set with no headers. The [description for the second-resolution data] qq_sec says the column order is date, open, high, low, close, volume. All price values are adjusted for dividend and splits.

• The [Quantquote symbols list] qq_symbols

• The [Fama French] ff dataset hosted by Kevin French is already accessible via Pandas Remote Data Access

  • [Variable names from Pandas] ff are exactly what their names are in French's page
  • [Research variables definitions] ff__definitions for the breakpoints data

##Presentation

This repo is for the Heroku content that will show off algorithms and data analysis. We will adapt the [Flask microblog tutorial] flaskr to make a blog using Flask and MongoLab via pymongo. This is [a link to mongolab] mongolab; chosen because there is a free (500MB) tier for development. We've used all of the free Heroku Postgres database for the S&P500 data.