ML_Yelp_BusFeatures/Yelp Star Rating Predictions From Attributes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Machine Learning to Predict Yelp Ratings from Attributes\n",
    "\n",
    "The goal of this project is to explore the power of Yelp metadata attributes to predict the rating of a venue. Yelp collects a lot of data on businesses (See [Yelp Developer Docs](https://www.yelp.com/developers/documentation/v2/business)). I will focus on:\n",
    "\n",
    "* city - city in which the business resides\n",
    "* longitude & latitude - coordinates of business\n",
    "* categories - provides a list of catogories the business is associated with\n",
    "* attributes - a list of various features (Take Out, Waiter Service, Alcohol, etc.)\n",
    "\n",
    "How predictive are the 'characteristics' of the restaurant for star ratings?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/matplotlib/__init__.py:872: UserWarning: axes.color_cycle is deprecated and replaced with axes.prop_cycle; please use the latter.\n",
      "  warnings.warn(self.msg_depr % (key, alt_key))\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import gzip\n",
    "import simplejson\n",
    "import re\n",
    "from sklearn.cross_validation import train_test_split\n",
    "import matplotlib\n",
    "import seaborn as sns\n",
    "from sklearn import metrics\n",
    "import matplotlib.pylab as plt\n",
    "import sklearn\n",
    "import numpy as np\n",
    "from sklearn import base\n",
    "from sklearn.externals import joblib\n",
    "from sklearn import neighbors, cross_validation, grid_search\n",
    "import matplotlib.pylab as plt\n",
    "from sklearn.feature_extraction import DictVectorizer\n",
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "from sklearn import linear_model\n",
    "\n",
    "plt.style.use('ggplot')\n",
    "%matplotlib inline\n",
    "pd.options.display.max_columns=25"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = gzip.open('yelp_train_academic_dataset_business.json.gz')\n",
    "data_content = data.read()\n",
    "data.close()\n",
    "lines= re.split('\\n',data_content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>attributes</th>\n",
       "      <th>business_id</th>\n",
       "      <th>categories</th>\n",
       "      <th>city</th>\n",
       "      <th>full_address</th>\n",
       "      <th>hours</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>name</th>\n",
       "      <th>neighborhoods</th>\n",
       "      <th>open</th>\n",
       "      <th>review_count</th>\n",
       "      <th>stars</th>\n",
       "      <th>state</th>\n",
       "      <th>type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>{u'By Appointment Only': True}</td>\n",
       "      <td>vcNAWiLM4dR7D2nwwJ7nCA</td>\n",
       "      <td>[Doctors, Health &amp; Medical]</td>\n",
       "      <td>Phoenix</td>\n",
       "      <td>4840 E Indian School Rd\\nSte 101\\nPhoenix, AZ ...</td>\n",
       "      <td>{u'Thursday': {u'close': u'17:00', u'open': u'...</td>\n",
       "      <td>33.499313</td>\n",
       "      <td>-111.983758</td>\n",
       "      <td>Eric Goldberg, MD</td>\n",
       "      <td>[]</td>\n",
       "      <td>True</td>\n",
       "      <td>7</td>\n",
       "      <td>3.5</td>\n",
       "      <td>AZ</td>\n",
       "      <td>business</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>{u'Take-out': True, u'Price Range': 1, u'Outdo...</td>\n",
       "      <td>JwUE5GmEO-sH1FuwJgKBlQ</td>\n",
       "      <td>[Restaurants]</td>\n",
       "      <td>De Forest</td>\n",
       "      <td>6162 US Highway 51\\nDe Forest, WI 53532</td>\n",
       "      <td>{}</td>\n",
       "      <td>43.238893</td>\n",
       "      <td>-89.335844</td>\n",
       "      <td>Pine Cone Restaurant</td>\n",
       "      <td>[]</td>\n",
       "      <td>True</td>\n",
       "      <td>26</td>\n",
       "      <td>4.0</td>\n",
       "      <td>WI</td>\n",
       "      <td>business</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>{u'Take-out': True, u'Outdoor Seating': False,...</td>\n",
       "      <td>uGykseHzyS5xAMWoN6YUqA</td>\n",
       "      <td>[American (Traditional), Restaurants]</td>\n",
       "      <td>De Forest</td>\n",
       "      <td>505 W North St\\nDe Forest, WI 53532</td>\n",
       "      <td>{u'Monday': {u'close': u'22:00', u'open': u'06...</td>\n",
       "      <td>43.252267</td>\n",
       "      <td>-89.353437</td>\n",
       "      <td>Deforest Family Restaurant</td>\n",
       "      <td>[]</td>\n",
       "      <td>True</td>\n",
       "      <td>16</td>\n",
       "      <td>4.0</td>\n",
       "      <td>WI</td>\n",
       "      <td>business</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>{u'Take-out': True, u'Accepts Credit Cards': T...</td>\n",
       "      <td>LRKJF43s9-3jG9Lgx4zODg</td>\n",
       "      <td>[Food, Ice Cream &amp; Frozen Yogurt, Fast Food, R...</td>\n",
       "      <td>De Forest</td>\n",
       "      <td>4910 County Rd V\\nDe Forest, WI 53532</td>\n",
       "      <td>{u'Monday': {u'close': u'22:00', u'open': u'10...</td>\n",
       "      <td>43.251045</td>\n",
       "      <td>-89.374983</td>\n",
       "      <td>Culver's</td>\n",
       "      <td>[]</td>\n",
       "      <td>True</td>\n",
       "      <td>7</td>\n",
       "      <td>4.5</td>\n",
       "      <td>WI</td>\n",
       "      <td>business</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>{u'Take-out': True, u'Has TV': False, u'Outdoo...</td>\n",
       "      <td>RgDg-k9S5YD_BaxMckifkg</td>\n",
       "      <td>[Chinese, Restaurants]</td>\n",
       "      <td>De Forest</td>\n",
       "      <td>631 S Main St\\nDe Forest, WI 53532</td>\n",
       "      <td>{u'Monday': {u'close': u'22:00', u'open': u'11...</td>\n",
       "      <td>43.240875</td>\n",
       "      <td>-89.343722</td>\n",
       "      <td>Chang Jiang Chinese Kitchen</td>\n",
       "      <td>[]</td>\n",
       "      <td>True</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>WI</td>\n",
       "      <td>business</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          attributes             business_id  \\\n",
       "0                     {u'By Appointment Only': True}  vcNAWiLM4dR7D2nwwJ7nCA   \n",
       "1  {u'Take-out': True, u'Price Range': 1, u'Outdo...  JwUE5GmEO-sH1FuwJgKBlQ   \n",
       "2  {u'Take-out': True, u'Outdoor Seating': False,...  uGykseHzyS5xAMWoN6YUqA   \n",
       "3  {u'Take-out': True, u'Accepts Credit Cards': T...  LRKJF43s9-3jG9Lgx4zODg   \n",
       "4  {u'Take-out': True, u'Has TV': False, u'Outdoo...  RgDg-k9S5YD_BaxMckifkg   \n",
       "\n",
       "                                          categories       city  \\\n",
       "0                        [Doctors, Health & Medical]    Phoenix   \n",
       "1                                      [Restaurants]  De Forest   \n",
       "2              [American (Traditional), Restaurants]  De Forest   \n",
       "3  [Food, Ice Cream & Frozen Yogurt, Fast Food, R...  De Forest   \n",
       "4                             [Chinese, Restaurants]  De Forest   \n",
       "\n",
       "                                        full_address  \\\n",
       "0  4840 E Indian School Rd\\nSte 101\\nPhoenix, AZ ...   \n",
       "1            6162 US Highway 51\\nDe Forest, WI 53532   \n",
       "2                505 W North St\\nDe Forest, WI 53532   \n",
       "3              4910 County Rd V\\nDe Forest, WI 53532   \n",
       "4                 631 S Main St\\nDe Forest, WI 53532   \n",
       "\n",
       "                                               hours   latitude   longitude  \\\n",
       "0  {u'Thursday': {u'close': u'17:00', u'open': u'...  33.499313 -111.983758   \n",
       "1                                                 {}  43.238893  -89.335844   \n",
       "2  {u'Monday': {u'close': u'22:00', u'open': u'06...  43.252267  -89.353437   \n",
       "3  {u'Monday': {u'close': u'22:00', u'open': u'10...  43.251045  -89.374983   \n",
       "4  {u'Monday': {u'close': u'22:00', u'open': u'11...  43.240875  -89.343722   \n",
       "\n",
       "                          name neighborhoods  open  review_count  stars state  \\\n",
       "0            Eric Goldberg, MD            []  True             7    3.5    AZ   \n",
       "1         Pine Cone Restaurant            []  True            26    4.0    WI   \n",
       "2   Deforest Family Restaurant            []  True            16    4.0    WI   \n",
       "3                     Culver's            []  True             7    4.5    WI   \n",
       "4  Chang Jiang Chinese Kitchen            []  True             3    4.0    WI   \n",
       "\n",
       "       type  \n",
       "0  business  \n",
       "1  business  \n",
       "2  business  \n",
       "3  business  \n",
       "4  business  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json_data = [simplejson.loads(line) for line in lines[:-1]]\n",
    "df = pd.DataFrame(json_data)\n",
    "data_as_dict = [dict(df.iloc[i]) for i in xrange(len(df))]\n",
    "df[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(37938, 15)\n"
     ]
    }
   ],
   "source": [
    "print df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, we can see the structure of the data. The 5 categories I will use are attributes, categories, city, latitude, longitude.  Notice that both attribues and categories have different lengths and may be nested (attributes in particular).\n",
    "\n",
    "### Transformers\n",
    "To make this data place nice in sklearn, I have constructed the following generic column transformer, category transformer, and attribute transformer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[33.499313000000001, -111.98375799999999]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.base import TransformerMixin\n",
    "\n",
    "class ColumnSelectTransformer(TransformerMixin):\n",
    "    '''\n",
    "    ColumnSelectTransformer provides a generic transformer to select columns from our input list/dict.\n",
    "    '''\n",
    "    def __init__(self, columns=None):\n",
    "        if columns:\n",
    "            self.columns = columns\n",
    "        else:\n",
    "            self.columns = ['business_id']\n",
    "    \n",
    "    def transform(self, record, *args, **transform_params):\n",
    "        if isinstance(record,dict):\n",
    "            #print 'received dict'\n",
    "            return [record[arg] for arg in self.columns]\n",
    "        elif isinstance(record,list):\n",
    "            #print 'received list'\n",
    "            return [[rec[arg] for arg in self.columns] for rec in record]\n",
    "        \n",
    "    def fit(self, X, y=None, **fit_params):\n",
    "        \n",
    "        return self\n",
    "        \n",
    "class CategoryTransformer(TransformerMixin):\n",
    "    '''\n",
    "    CategorySelectTransformer provides a transformer to select handle the category field\n",
    "    of our input data. There may be multiple, nested categories, this class builds an array of dictionaries \n",
    "    pointing to each category\n",
    "    '''    \n",
    "    def transform(self, record, *args, **transform_params):\n",
    "        if isinstance(record,dict):     \n",
    "            return dict([[cat, 1] for cat in record])\n",
    "        elif isinstance(record,list):\n",
    "            if len(record)==1:\n",
    "                return [dict([[cat, 1] for cat in rec]) for rec in record]\n",
    "            else:\n",
    "                return [dict([[cat, 1] for cat in rec]) for inner in record for rec in inner]\n",
    "                \n",
    "    def fit(self, X, y=None, **fit_params):        \n",
    "        return self\n",
    "        \n",
    "class ModelTransformer(TransformerMixin):\n",
    "    '''\n",
    "    ModelTransformer is a wrapper class to make an estimator act like a transformer\n",
    "    This 'transforms' the data by using predict from the model to transform the data\n",
    "    '''\n",
    "\n",
    "    def __init__(self, model, name):\n",
    "        self.model = model\n",
    "        self.name = name\n",
    "\n",
    "    def fit(self, *args, **kwargs):\n",
    "        #print 'FITTING!'\n",
    "        self.model.fit(*args, **kwargs)\n",
    "        return self\n",
    "\n",
    "    def transform(self, X, **transform_params):\n",
    "        return [self.model.predict(x) for x in X]\n",
    "        \n",
    "\n",
    "def unnest_attributes(att,outer_key=None):\n",
    "    '''\n",
    "    This recursive function seeks to unnest the attributes within the attribute meta data of yelp.\n",
    "    '''\n",
    "    mydict = {}\n",
    "    if not outer_key:\n",
    "        outer_key = ''\n",
    "    else:\n",
    "        outer_key += '_'\n",
    "        \n",
    "    for key in att.keys():\n",
    "        if isinstance(att[key],dict):\n",
    "            inner_dict = unnest_attributes(att[key],key)\n",
    "            for key in inner_dict.keys():\n",
    "                mydict[outer_key+key] = inner_dict[key]\n",
    "        else:\n",
    "            mydict[outer_key+key] = att[key]\n",
    "    \n",
    "    return mydict\n",
    "\n",
    "class AttributesTransformer(TransformerMixin):\n",
    "    '''\n",
    "    AttributesTransformer is a class to handle the attributes field.\n",
    "    \n",
    "    Attributes can be nested and there are often many for each venue.  This works with the unnest_attributes\n",
    "    function to build an array of dictionaries of attributes within the yelp dataset.\n",
    "    '''\n",
    "    def transform(self, record, *args, **transform_params):\n",
    "        if isinstance(record,dict):     \n",
    "            return unnest_attributes(record)\n",
    "        elif isinstance(record,list):\n",
    "            if len(record)==1:\n",
    "                return [unnest_attributes(rec) for rec in record]\n",
    "            else:\n",
    "                return [unnest_attributes(rec) for inner in record for rec in inner]\n",
    "                \n",
    "        \n",
    "    def fit(self, X, y=None, **fit_params):\n",
    "        \n",
    "        return self\n",
    "\n",
    "cst = ColumnSelectTransformer(columns=['latitude','longitude'])\n",
    "print cst.transform(data_as_dict[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### City Predictor\n",
    "The first model I will try is the city predictor. It's a very simple model, where I return the mean value of stars in a given city.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.base import BaseEstimator, RegressorMixin\n",
    "\n",
    "class CityMeanEstimator(BaseEstimator, RegressorMixin):\n",
    "    def __init__(self):\n",
    "        pass\n",
    "    \n",
    "    def fit(self,X,y):\n",
    "        X=[v[0] for v in X]\n",
    "        df=pd.DataFrame({'city': X, 'stars': y},columns=['city','stars'])\n",
    "        df['stars']=df['stars'].astype(float)\n",
    "        self.f = df.groupby('city').mean()\n",
    "        return self\n",
    "    \n",
    "    def predict(self, X):\n",
    "        X=[v[0] for v in X]\n",
    "        val=[]\n",
    "        for x in X:\n",
    "            if x in self.f.index:\n",
    "                val.append(x)\n",
    "            else:\n",
    "                val.append('Chandler')\n",
    "        return [x[0] for x in self.f.loc[val].values]\n",
    "\n",
    "from sklearn.pipeline import FeatureUnion, Pipeline        \n",
    "pipeline = Pipeline([\n",
    "  ('cst', ColumnSelectTransformer(columns=['city'])),\n",
    "  ('classifier', CityMeanEstimator())\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's my model above. I use the Pipeline feature of sklearn to transform my data for each model I create. This way, I can always input my full data, and have the pipeline select the features for me.\n",
    "\n",
    "Okay, I will split up the data into train and test set using `train_test_split`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data_train, data_test, stars_train, stars_test = cross_validation.train_test_split(data_as_dict, df['stars'], random_state=23)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fc8ae505790>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEDCAYAAADQunSaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFtZJREFUeJzt3X+Q1PV9x/Hn3h4cEG4XPRCyRjMqVRR1piRVhuj5g2pw\nimON8T3VNKF1cNQSUkL/aMMkjTqpbaMhaJwmVZMYYtOZNyaS2mjGNBmjMUxaaDUYe1JiDIZD5RB3\nzzs45W77x+3pcnK7X3b3u/vlw+vx1+338939vvycvvbrd7/32VSxWERERMLU1uoAIiISH5W8iEjA\nVPIiIgFTyYuIBEwlLyISMJW8iEjA2qPsZGZrgYXACLDK3TeXjb0P+FdgEvDf7v4XcQQVEZHDV/VM\n3sy6gbnuvghYDtw1bpcvAbe7+0JguFT6IiKSAFEu1ywGNgK4ew8ww8ymA5hZCjgPeLg0vtLdfxdT\nVhEROUxRSn4OsLvscV9pG8As4A1gnZk9aWa3NTifiIjUoZYPXlPjfj4e+DJwAfD7ZnZZI4KJiEj9\nonzw2ss7Z+4AOWBX6ec+4EV3fxHAzH4MzAcerfB6WixHRKQ2qeq7HCxKyT8G3Azca2YLgJ3uPgDg\n7sNm9oKZneLuvwY+AHyn2gv29vYebs6my+VyytlAytk4R0JGUM5Gy+VyNT2v6uUad98EbDGzp4B1\nwAozW2ZmV5R2+TRwv5n9DHjd3R+uKYmIiDRcpPvk3X3NuE1by8Z+DZzfyFAiItIY+otXEZGAqeRF\nRAKmkhcRCZhKXkQkYCp5EZGARbq7RkRE4Ktf/Srbtm3jtddeY//+/Rx//PF0dnZyyy23VHzeD3/4\nQ6ZPn855553XpKTvUMmLyBGpLZ8nlc/X/TrDg4Ok+/ooZrOMZLMV973pppuA0dJ+8cUXufHGGyMd\nY8mSJXXnrJVKXkSOSKl8nvaenrpfJ93VRfuePRyYNw+qlPyhPP3002zYsIF9+/Zxww038Mwzz/D4\n44+TSqU499xz+cQnPsG3vvUtMpkMJ510Ehs3biSVSrFjxw66u7tZtmxZ3f8MlajkRUTq9MILL/DA\nAw+QTqfZunUrd999N21tbVx77bVcffXVAKRSo8vO9PT0sH79eoaHh7nmmmtU8iIiSXfKKaeQTqcB\nmDx5MqtWrSKdTlMoFCgUCgfte+qppzJ58mTgneKPk0peRKRO7e2jVfrKK6+wYcMG7rvvPjo6Orju\nuuvetW9bW3NvatQtlCIiDZLP5znmmGPo6Ohg27ZtvPrqq7z11lsT7l8sxr/yus7kReSQ6r17Zeyu\nlXpEueMlSebOncuUKVNYuXIl8+fPZ+nSpdx5552ceeaZh9y/GZdrUs14JxmneKSs3aycjaOcjdOs\njOkdO+q6e6Wrq4s9e/bUleHAvHkMn3jiIccadQvlzJkz6Yt4C2UrldaTj+VLQ0SkifL5NvL5if9b\nHhwcpq8vHXuOY/dPoSv2o9RuJJut6ZbH8dK5HMPTpjUgUTKp5EUSJp9P0dMz8X+aXV1p9uyJ/z/d\nk7PT6B+cXfPz97ZnyA/Wl7Nz/xSm1/UKopIXkUN6YyDNS9sm1fz8TOckCv21Px/g1BPaVfJ1UsmL\nJEzb4CDp3UMTjg8PDZEed+91HFIzYz+ENIFKXiRhUgMDtO18ecLxdCFDW3/8JU+29ks1khy6T15E\nJGA6kxcRiajWpYbHvPzyy+TzeU477bSYk75DJS8iR6Rqt5pGNXZLajZbJJsdqbhvrUsNj9myZQvD\nw8MqeRGRaqrdahrV2C2p8+YdqPm2+3vuuYfnnnuOkZERrrrqKi644AJ+8YtfcP/999PR0UFXVxc3\n3ngj3/72t5k8eTKzZ8/m3HPPrTt7FCp5EZE6PP300+zdu5d169bx5ptvcsMNN7Bo0SIeeughVq5c\nyRlnnMETTzzB5MmTufTSS5k1a1bTCh5U8iIidXn22Wf51a9+xerVqykWixSLRfbu3cuFF17IHXfc\nwSWXXMLFF19MtkVLJqjkRUTqMGnSJJYuXYqZHbR9yZIlLFy4kCeffJI1a9Zw6623tiSfbqEUEanD\n6aefzs9//nOKxSL79+/n7rvvBmD9+vVMmjSJyy+/nO7ubnbs2EEqleLAgQNNzaczeRGROpx99tnM\nnz+fFStWAHDllVcCMGvWLFavXk1nZyfZbJZrrrmGdDrN7bffzowZM7jooouaki9SyZvZWmAhMAKs\ncvfNZWO/AXaUxorAx9x9VwxZRUTels0WmTev/rPimTOH6es7QDYbfdn1JUuWHPT4+uuvf9c+l112\nGZdddtlB28455xw2bNhQW9AaVS15M+sG5rr7IjObB3wDWFS2SxFY4u77YsooIvIu2exII1YaJpdL\nM23acP0vlFBRrskvBjYCuHsPMMPMyheGS1HDQvYiIhK/KCU/B9hd9rivtK3c18zsSTO7rWHJRESk\nbrXcXTP+rP1zwGrgAuAsM/tI3alERKQhonzw2svBZ+454O0PVt39gbGfzewR4Czge5VesPRdhYmn\nnI2lnNH0bx8m0zlYcZ9MZyb2HB0dHXUfp+7nZ7NN+X20+ncepygl/xhwM3CvmS0Adrr7AICZZYCH\ngQ+7+36gG3iw2gsm/YuS4cj4QmdQzkZLQs5CPk+hwnrxmc5MxfFGaR+aWtdxGpGzkJ9Gb2+832eb\nhN95FLW+EVW9XOPum4AtZvYUsA5YYWbLzOwKdy8wWuqbzOwJYLe7f7emJCIi0nCR7pN39zXjNm0t\nG/sK8JVGhhIRkcbQsgYiIgFTyYuIBEwlLyISMJW8iEjAVPIiIgFTyYuIBEwlLyISMJW8iEjAVPIi\nIgFTyYuIBEwlLyISMJW8iEjAVPIiIgFTyYuIBEwlLyISMJW8iEjAVPIiIgFTyYuIBEwlLyISMJW8\niEjAVPIiIgFTyYuIBEwlLyISMJW8iEjAVPIiIgFTyYuIBEwlLyISMJW8iEjAVPIiIgFrj7KTma0F\nFgIjwCp333yIff4eWOjuFzU2ooiI1KrqmbyZdQNz3X0RsBy46xD7nA6cDxQbnlBERGoW5XLNYmAj\ngLv3ADPMbPq4fe4APtPgbCIiUqcoJT8H2F32uK+0DQAzWwb8GNjR2GgiIlKvSNfkx0mN/WBmxwAf\nBy4FTiwfqySXy9Vw2OZTzsZSzmj6tw+T6RysuE+mMxN7jo6OjrqPU/fzs9mm/D5a/TuPU5SS76Xs\nzB3IAbtKP18MzAZ+BkwBTjazL7n7X1V8wd7eGqI2Vy6XU84GUs7oCvk8hf7ChOOZzkzF8UZpH5pa\n13EakbOQn0Zvb7qu16gmCb/zKGp9I4pS8o8BNwP3mtkCYKe7DwC4+3eB7wKY2fuBb1YreBERaZ6q\n1+TdfROwxcyeAtYBK8xsmZldEXs6ERGpS6Rr8u6+ZtymrYfY57eMXr4REZGE0F+8iogErJa7a0RE\nmmLfgcns2BHvB6+Dg8P09VU+RjZbJJsdiTVHXFTyIpJYbwykebUn3prq6kqzZ0/lY8ybd4BsNtYY\nsVHJi0hyHXiL9O7d1ferw/DQEOlC5Vs9207sADpizREXlbyIJFbqzbdo2/lKrMdIFzK0VbmfPzVv\nDkdqyeuDVxGRgKnkRUQCppIXEQmYSl5EJGAqeRGRgKnkRUQCppIXEQmYSl5EJGAqeRGRgKnkRUQC\npmUNRMrs2TMc+6qH1ex/c3JLjy9hUcmLlNm7F3piXvWwmuPaW/smI2HR5RoRkYDpTF6kzHC+EPvS\nttWkZrb08BIYlbxIuYE3aNv5u9ZmyM5u7fElKLpcIyISMJW8iEjAVPIiIgFTyYuIBEwlLyISMJW8\niEjAVPIiIgFTyYuIBCzSH0OZ2VpgITACrHL3zWVj1wPXAQeAZ9z9k3EEFRGRw1f1TN7MuoG57r4I\nWA7cVTY2FTDgQ+5+PnC6mS2MK6yIiByeKGfyi4GNAO7eY2YzzGy6u7/h7vuASwDMbBqQAV6OLa2I\nSAvsOzC55UtQ53K1PS9Kyc8BNpc97itt2z62wcz+GvgUsM7dX6wtiohIMr0xkObVFi9BvbDGayS1\npE6N3+Du/2hm64BHzexn7r6p0gvkan1LajLlbKwjIefz218i05lpaYaOjo6qGZqRMUqOaup9fiMy\nRFHtGFPb25kxMBR7jjhEKfleRs/cx+SAXQBmdixwlrv/1N2HzOxR4ENAxZLv7e2tMW7z5HI55Wyg\nIyUnQKG/0NLjtw9NrZgh05lpSsZqOappRM56M0QRJWd7YSoD//tKrDmqq+0kKcotlI8BHwUwswXA\nTncfKI21A98oXY8HOAd4vqYkIiLScFVLvnTpZYuZPQWsA1aY2TIzu8LdXwVuAR4vje9294fjjSwi\nIlFFuibv7mvGbdpaNrYeWN/IUCIi0hj6i1cRkYCp5EVEAqaSFxEJmEpeRCRgKnkRkYCp5EVEAqaS\nFxEJmEpeRCRgKnkRkYCp5EVEAqaSFxEJmEpeRCRgKnkRkYCp5EVEAqaSFxEJmEpeRCRgKnkRkYCp\n5EVEAqaSFxEJmEpeRCRgKnkRkYCp5EVEAqaSFxEJmEpeRCRgKnkRkYCp5EVEAtbe6gDSem35PKl8\nPtZjDA8Oku7rm3C8mM0yks3GmkHkaKSSF1L5PO09PbEeI93VRfuePROOH5g3D1TyIg0XqeTNbC2w\nEBgBVrn75rKxi4DbgAPA8+6+PI6gIiJy+KpekzezbmCuuy8ClgN3jdvln4Gr3P18IGNmSxofU0RE\nahHlTH4xsBHA3XvMbIaZTXf3N0rjH3T3Qunn3UBXDDklRvn9U+gfnB3rMfa2Z8gPTvyvW+f+KUyP\nNYHI0SlKyc8BNpc97itt2w4wVvBm9l7gEuCzDc4oMcv3t7Nt26RYj5HpnEShf+JjnHpCu0peJAa1\nfPCaGr/BzI4D/g24yd33VnuBXC5Xw2Gb72jJ2b99mEznYIPSTCzTmZl4LJtNxHw/v/2lijmboaOj\no2qGZmSMkqOaep/fiAxRVDtGs3LEIUrJ9zJ65j4mB+wae2BmncAjwGfc/cdRDtrb23s4GVsil8sd\nNTkL+TyF/kL1HeuQ6cxUPEYhP43e3nSsGaKKey6qaR+aWjFDtblsVo5qGpGz3gxRRMnZjBxxifLH\nUI8BHwUwswXATncfKBtfC6x19x/FkE9EROpQ9Uze3TeZ2RYzewoYBlaY2TLgdUbfAP4UOMXMrgeK\nwHfc/b44Q4uISDSRrsm7+5pxm7aW/Ty1cXFERKSRtHaNiEjAVPIiIgFTyYuIBEwlLyISMJW8iEjA\nVPIiIgFTyYuIBEwlLyISMJW8iEjAVPIiIgFTyYuIBEwlLyISMJW8iEjAVPIiIgFTyYuIBEwlLyIS\nMJW8iEjAVPIiIgFTyYuIBEwlLyISMJW8iEjAVPIiIgFTyYuIBEwlLyISMJW8iEjAVPIiIgFTyYuI\nBEwlLyISsPYoO5nZWmAhMAKscvfNZWMdwD3A6e5+TiwpRUSkJlXP5M2sG5jr7ouA5cBd43a5HfjP\nGLKJiEidolyuWQxsBHD3HmCGmU0vG/8b4N9jyCYiInWKUvJzgN1lj/tK2wBw98FGhxIRkcao5YPX\nVMNTiIhILKJ88NpL2Zk7kAN21XPQXC5Xz9Ob5mjJ2b99mExn/P9DlunMTDyWzSZivp/f/lLFnM3Q\n0dFRNUMzMkbJUU29z29EhiiqHaNZOeIQpeQfA24G7jWzBcBOdx8Yt0+KwzjD7+3tjRywVXK53FGT\ns5DPU+gvNCjRoWU6MxWPUchPo7c3HWuGqOKei2rah6ZWzFBtLpuVo5pG5Kw3QxRRcjYjR1yqlry7\nbzKzLWb2FDAMrDCzZcDr7v59M/sR8D7gRDP7JfBld/9mvLHD0JbPk8rn63qN4cFB0n199eUYmlTX\n80UkuSLdJ+/ua8Zt2lo2dklDEx1FCq8M0f/c3rpeoz8zTL5Q3xnGwHuOq+v5IpJckUpe4pHvb2fb\ntvrOojOdkyj01/cax54xua7ni0hyaVkDEZGAqeRFRAKmkhcRCZhKXkQkYCp5EZGAqeRFRAKmkhcR\nCZhKXkQkYCp5EZGAqeRFRAKmkhcRCZhKXkQkYCp5EZGAqeRFRAKmkhcRCdhRu558tW9lasQ3LlXN\noG9kEpGYHbUlX+1bmRrxjUvV6BuZRCRuR23JV/tWpkZ841I1+kYmEYmbrsmLiARMJS8iEjCVvIhI\nwFpyTX5oqBVHfUc63drji4g0S0tKftPGeO9aqeb4kycxTf8PIyJHgZaUfPG3v2vFYd+WmjkTZhy1\nNxaJyFFE57MiIgFTyYuIBEwlLyISsEgXps1sLbAQGAFWufvmsrE/BP4OOAA86u5fiCOoiIgcvqpn\n8mbWDcx190XAcuCucbvcCVwJnAdcambzGp5SRERqEuVyzWJgI4C79wAzzGw6gJmdBOxx9153LwKP\nlPYXEZEEiFLyc4DdZY/7StsONfYq8N7GRBMRkXrVcrN4qsaxt806bUYNh22caV0dFIeHW5pBRKQZ\nopR8L++cuQPkgF1lY+Vn7seXtlV06U0fjJovVn9w+ZmtjgCc1eoAJUnJ0WI5OK37hFanIDm/jyTk\nSEIGSE6OwxOl5B8DbgbuNbMFwE53HwBw99+aWaeZnchouS8Frq3yepHO9kVEpH6pYrFYdSczuw24\nABgGVgALgNfd/ftmdh7wRaAIPOjuX44xr4iIHIZIJS8iIkcm/cWriEjAVPIiIgFTyYuIBCzWRdXN\n7Gzge8Bad/+ncWOJWfOmSs7fADsYXbenCHzM3Xe9+1Viz/hFRpeOSAP/4O4PlY0laS4r5UzKXE4F\n7gdmAx3AF9z9B2XjiZjPCDkTMZ9leaYAzwK3uvv6su2JmM9SlokyJmIuzewCYEMpYwr4pbv/Zdn4\nYc9lbCVvZtOALzF6C+ah3Alcwug99z81swdLyyY0VYScRWCJu+9rXqqDmdmFwHx3X2RmxwL/AzxU\ntktS5rJazpbPZcnlwH+5+x2l239/BPygbDwR8xkhZ1Lmc8zngD2H2J6U+YSJMyZpLh93d5tg7LDn\nMs7LNfuBPwJeGT+QsDVvJsxZkqL19/Y/AVxd+vl1YJqZpSBxczlhzpIkzCU+6o7SwxOBl8bGkjSf\nlXKWJGI+AczsNOA0Dn4TStR8TpSxJDFzyQQ5ap3L2M7k3X0EeNPskG9Ih1rz5uS4slRSJeeYr5Um\n+El3X9OcZO8oZRwsPVwOPFL6JUPy5nKinGNaOpflzOwpRv9Ke2nZ5sTM55gJco5Jynzewejf0Pz5\nuO1Jms+JMo5JylyeYWYbgWMZvaz0H6XtNc1lUj54Tco76KF8DljN6B+DnWVmH2lVEDO7gtF/QT9Z\nYbeWz2WFnImZSwB3/xBwBfAvFXZr+XxWyJmI+TSzjwM/dfcdpU11r2/VaBEyJmIugf8Dbnb3Pwb+\nDPi6mU10Mh5pLltV8jWtedMK7v6Au/eVzlIfoUULWJjZh4HPMHrdsL9sKFFzWSFnkubyA2Z2QinT\nM0C7mc0sDSdmPqvkTMx8Mnq582oz28To/8F91swuLo0lZT4rZUzMXJYuxWwo/fwC8DKjcwY1zmWs\nd9eUOegdp8Y1b5rhoJxmlgEeBj7s7vuBbuDBZocq5fgisNjd8+VjSZrLSjmTMpcl5wPvBz5tZrOB\n97h7HyRrPivlTNJ8uvufjP1sZp8HfuPuPymNJWI+K2VM0lya2bXA77n7LWZ2HDAL2Am1z2VsyxqY\n2bnAfaWQB4DXgG8CLyRpzZsIOVcC1wH9wNPu/qkWZLwe+DywjdE3oiLwE2BrwuayWs6Wz2Up5xTg\n68AJwBTgFmAmCVuPKULORMxnOTP7W+BFRn//iZrPMRNkTMRclr6Q6TuMXo9vA25l9BbamudSa9eI\niAQsKR+8iohIDFTyIiIBU8mLiARMJS8iEjCVvIhIwFTyIiIBU8mLiARMJS8iErD/B7pdBwsaQ9Pe\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc8ed641750>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#check our distributions of stars between train and test\n",
    "stars_train.hist( alpha=0.3 ,stacked=False,normed=True,color='red',bins=9)\n",
    "stars_test.hist( alpha=0.3 ,stacked=False,normed=True,color='blue',bins=9)\n",
    "plt.legend(['Train','Test'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Now, fit and score the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3.6499816872176778, 3.585046066619419, 3.6655675459632295, 3.6361185983827493, 3.6499816872176778, 3.8244243421052633, 3.6655675459632295, 3.8080519480519479, 3.4166666666666665, 3.6499816872176778]\n",
      "[ 5.   4.   4.5  4.5  3.5  5.   4.   3.5  4.   5. ]\n",
      "0.00132602324228\n",
      "City Model RMSE:  0.885138625141\n",
      "City Model MAE:  0.711788533327\n",
      "City Model R^2:  0.00132602324228\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error\n",
    "pipeline.fit(data_train, stars_train)\n",
    "\n",
    "print pipeline.predict(data_test[:10])\n",
    "print stars_test[:10].values\n",
    "score = pipeline.score(data_test,stars_test)\n",
    "print score\n",
    "preds = pipeline.predict(data_test)\n",
    "print 'City Model RMSE: ', mean_squared_error(stars_test,preds)**0.5\n",
    "print 'City Model MAE: ', mean_absolute_error(stars_test,preds)\n",
    "print 'City Model R^2: ', r2_score(stars_test,preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bunk! It has about zero predictive value. The score that is displayed is the $R^2$ coefficient of determination. The best possible score is 1.0, and lower values are worse. So we can see that this model is not very predictive. But did we really expect the city a business is in to be a factor? I know the old trope that the 3 most important things for a business are: \n",
    "1. location\n",
    "2. location\n",
    "3. location\n",
    "\n",
    "Well, city must not be specific enough. I will try latitude and longitude next. Perhaps it will be beter.\n",
    "\n",
    "### Longitude and Latitude K-Nearest Neighbors Regression Model\n",
    "The target is predicted by local interpolation of the values of its k nearest neighbors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 4.1  3.8  4.   3.8  3.5  4.5  3.4  3.6  3.1  4.5]\n",
      "[ 5.   4.   4.5  4.5  3.5  5.   4.   3.5  4.   5. ]\n",
      "-0.0896369900153\n",
      "LatLong Model RMSE:  0.924571165323\n",
      "LatLong Model MAE:  0.733726937269\n",
      "LatLong Model R^2:  -0.0896369900153\n"
     ]
    }
   ],
   "source": [
    "#kn = neighbors.KNeighborsRegressor()\n",
    "pipeline = Pipeline([\n",
    "  ('cst', ColumnSelectTransformer(columns=['latitude','longitude'])),\n",
    "  ('classifier', neighbors.KNeighborsRegressor())\n",
    "])\n",
    "#latlon_train = cst.transform(data_train)\n",
    "#kn.fit(latlon_train,stars_train)\n",
    "#latlon_test = cst.transform(data_test)\n",
    "#score = kn.score(latlon_test,stars_test)\n",
    "pipeline.fit(data_train, stars_train)\n",
    "\n",
    "print pipeline.predict(data_test[:10])\n",
    "print stars_test[:10].values\n",
    "score = pipeline.score(data_test,stars_test)\n",
    "print score\n",
    "preds = pipeline.predict(data_test)\n",
    "print 'LatLong Model RMSE: ', mean_squared_error(stars_test,preds)**0.5\n",
    "print 'LatLong Model MAE: ', mean_absolute_error(stars_test,preds)\n",
    "print 'LatLong Model R^2: ', r2_score(stars_test,preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I need to tune this model to improve the score. I will use `GridSearchCV` with a `ShuffleSplit` cross validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsRegressor(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "          metric_params=None, n_jobs=1, n_neighbors=85, p=2,\n",
       "          weights='uniform')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "latlon = df.loc[:,['latitude','longitude']]\n",
    "cv = cross_validation.ShuffleSplit(len(df['stars']), n_iter=20, test_size=0.2, random_state=23)\n",
    "param_grid = { \"n_neighbors\": range(80, 100, 5)}\n",
    "nearest_neighbors_cv = grid_search.GridSearchCV(neighbors.KNeighborsRegressor(), param_grid=param_grid, cv=cv)\n",
    "nearest_neighbors_cv.fit(latlon,df['stars'])\n",
    "nearest_neighbors_cv.best_estimator_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best estimator uses 85 neighbors by default for k_neighbors queries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0248118636604\n",
      "LatLong GridCV Model RMSE:  0.874668790703\n",
      "LatLong GridCV Model MAE:  0.699430680021\n",
      "LatLong GridCV Model R^2:  0.0248118636604\n"
     ]
    }
   ],
   "source": [
    "pipeline = Pipeline([\n",
    "  ('cst', ColumnSelectTransformer(columns=['latitude','longitude'])),\n",
    "  ('classifier', nearest_neighbors_cv.best_estimator_)\n",
    "])\n",
    "\n",
    "pipeline.fit(data_train, stars_train)\n",
    "score = pipeline.score(data_test,stars_test)\n",
    "print score\n",
    "preds = pipeline.predict(data_test)\n",
    "print 'LatLong GridCV Model RMSE: ', mean_squared_error(stars_test,preds)**0.5\n",
    "print 'LatLong GridCV Model MAE: ', mean_absolute_error(stars_test,preds)\n",
    "print 'LatLong GridCV Model R^2: ', r2_score(stars_test,preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The predictive power of latitude and longitude alone is fairly poor. So much for location.\n",
    "\n",
    "### Categories TF-IDF Linear Regression\n",
    "Do categories have predicitive value? Might a dentist office score lower than a spa?\n",
    "\n",
    "I will make a pipeline of the following transformations and estimation:\n",
    "\n",
    "1. `ColumnSelectTransformer` for 'categories'\n",
    "\n",
    "2. `CategoryTransformer` to parse the yelp category data\n",
    "\n",
    "3. `DictVectorizer` - Convert categories to numbers for use in sklearn estimators\n",
    "\n",
    "4. `TfidfTransformer` - Normalize categories based upon term frequency - inverse document frequency (TF-IDF) which lessens the influence of common words (high document frequency) and increases local frequency usage (term frequency)\n",
    "\n",
    "5. `LinearRegression` - Predict stars based on the output of TF-IDF\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pipeline = Pipeline([\n",
    "  ('cst', ColumnSelectTransformer(columns=['categories'])),\n",
    "  ('cat_transformer', CategoryTransformer()),\n",
    "  ('vectorizer', DictVectorizer()),\n",
    "  ('tfidf', TfidfTransformer()),\n",
    "  ('classifier', linear_model.LinearRegression())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 4.23965915  4.04571874  4.30456556  3.82425019  3.87279651  3.71963255\n",
      "  3.06553658  3.30915369  3.42135172  4.37127433]\n",
      "[ 5.   4.   4.5  4.5  3.5  5.   4.   3.5  4.   5. ]\n",
      "0.170443165363\n",
      "Category Model RMSE:  0.806719396118\n",
      "Category Model MAE:  0.628842356365\n",
      "Category Model R^2:  0.170443165363\n"
     ]
    }
   ],
   "source": [
    "pipeline.fit(data_train, stars_train)\n",
    "print pipeline.predict(data_test[:10])\n",
    "print stars_test[:10].values\n",
    "score = pipeline.score(data_test,stars_test)\n",
    "print score\n",
    "preds = pipeline.predict(data_test)\n",
    "print 'Category Model RMSE: ', mean_squared_error(stars_test,preds)**0.5\n",
    "print 'Category Model MAE: ', mean_absolute_error(stars_test,preds)\n",
    "print 'Category Model R^2: ', r2_score(stars_test,preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Attributes field\n",
    "\n",
    "Similar to categories (restaurant, etc) Yelp collects attributes of venues (TVs in bar, wifi, outdoor patio, etc.) I will build a model to use the attributes to predict Yelp ratings. This will be similar to the categories model, but with a particular transformer for the attribues field.\n",
    "\n",
    "I will make a pipeline of the following transformations and estimation:\n",
    "\n",
    "1. `ColumnSelectTransformer` for 'attributes'\n",
    "\n",
    "2. `AttributeTransformer` to parse the yelp category data\n",
    "\n",
    "3. `DictVectorizer` - Convert attributes to numbers for use in sklearn estimators\n",
    "\n",
    "4. `TfidfTransformer` - Normalize attributes based upon term frequency - inverse document frequency (TF-IDF)\n",
    "\n",
    "5. `LinearRegression` - Predict stars based on the output of TF-IDF\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pipeline = Pipeline([\n",
    "  ('cst', ColumnSelectTransformer(columns=['attributes'])),\n",
    "  ('att_transformer', AttributesTransformer()),\n",
    "  ('vectorizer', DictVectorizer()),\n",
    "  ('tfidf',TfidfTransformer()),\n",
    "  ('classifier', linear_model.LinearRegression())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 3.6706147   3.99940153  3.9727453   3.83213542  3.79369163  3.6706147\n",
      "  3.31230563  3.41024663  3.58801175  3.72347863]\n",
      "[ 5.   4.   4.5  4.5  3.5  5.   4.   3.5  4.   5. ]\n",
      "0.0789392754202\n",
      "Attribute Model RMSE:  0.850048211357\n",
      "Attribute Model MAE:  0.675276746795\n",
      "Attribute Model R^2:  0.0789392754202\n"
     ]
    }
   ],
   "source": [
    "pipeline.fit(data_train, stars_train)\n",
    "print pipeline.predict(data_test[:10])\n",
    "print stars_test[:10].values\n",
    "score = pipeline.score(data_test,stars_test)\n",
    "print score\n",
    "preds = pipeline.predict(data_test)\n",
    "print 'Attribute Model RMSE: ', mean_squared_error(stars_test,preds)**0.5\n",
    "print 'Attribute Model MAE: ', mean_absolute_error(stars_test,preds)\n",
    "print 'Attribute Model R^2: ', r2_score(stars_test,preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Attributes are another very weak predictor of Yelp star rating\n",
    "\n",
    "### Combined model\n",
    "\n",
    "Three weak predictors (city, lat/long, attributes) and one fair predictor (categories). I will now build an overall model to combine these four predictors into an overall predictor.\n",
    "\n",
    "The key at this stage is to use a `FeatureUnion` to combine the predictors. `FeatureUnion` assumes that only transformations are made to the data, so I have to 'fool' it by using the `ModelTransformer` class to create a \"transformation\" of that returns the prediction for each model. Then, I combine these features in a `LinearRegression` model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "pipeline = Pipeline([\n",
    "  ('features', FeatureUnion([\n",
    "    ('city', Pipeline([\n",
    "      ('cst', ColumnSelectTransformer(columns=['city'])),\n",
    "      ('classifier', ModelTransformer(CityMeanEstimator(),'city class'))\n",
    "    ])),\n",
    "    ('neighborhod', Pipeline([\n",
    "      ('lat lon cst', ColumnSelectTransformer(columns=['latitude','longitude'])),\n",
    "      ('nearest neighbors', ModelTransformer(neighbors.KNeighborsRegressor(n_neighbors=94),'neigh class'))\n",
    "    ])),\n",
    "    ('category', Pipeline([\n",
    "      ('cat cst', ColumnSelectTransformer(columns=['categories'])),\n",
    "      ('cat_transformer', CategoryTransformer()),\n",
    "      ('cat vectorizer', DictVectorizer()),\n",
    "      ('cat tfidf', TfidfTransformer()),\n",
    "      ('classifier', ModelTransformer(linear_model.LinearRegression(),'cat class'))\n",
    "    ])),\n",
    "    ('attributes', Pipeline([\n",
    "      ('att cst', ColumnSelectTransformer(columns=['attributes'])),\n",
    "      ('att_transformer', AttributesTransformer()),\n",
    "      ('att vectorizer', DictVectorizer()),\n",
    "      ('att tfidf',TfidfTransformer()),\n",
    "      ('classifier', ModelTransformer(linear_model.LinearRegression(),'att class'))\n",
    "    ]))\n",
    "  ])),\n",
    "  ('final classifier', linear_model.LinearRegression())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 3.6706147   3.99940255  3.97274505  3.83213406  3.79369298  3.6706147\n",
      "  3.31230485  3.41025357  3.58801357  3.72347899]\n",
      "[ 5.   4.   4.5  4.5  3.5  5.   4.   3.5  4.   5. ]\n",
      "0.0789390332998\n",
      "Ensemble Model RMSE:  0.850048323084\n",
      "Ensemble Model MAE:  0.675276799799\n",
      "Ensemble Model R^2:  0.0789390332998\n"
     ]
    }
   ],
   "source": [
    "pipeline.fit(data_train,stars_train)\n",
    "print pipeline.predict(data_test[:10])\n",
    "print stars_test[:10].values\n",
    "score = pipeline.score(data_test,stars_test)\n",
    "print score\n",
    "preds = pipeline.predict(data_test)\n",
    "print 'Ensemble Model RMSE: ', mean_squared_error(stars_test,preds)**0.5\n",
    "print 'Ensemble Model MAE: ', mean_absolute_error(stars_test,preds)\n",
    "print 'Ensemble Model R^2: ', r2_score(stars_test,preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Many business data fields have predictive power for Yelp star rating \n",
    "\n",
    "Individually, a data field had little predictive power, with the exception of the categories field. \n",
    "\n",
    "A new business owner could use this analysis to determine which category of business to start, and maybe where to start it. But who starts a business this way? And why would someone think business success is had by high Yelp star rating?\n",
    "\n",
    "Anyhow, it is interesting to know what kind of baseline one can expect given this information from yelp metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAECCAYAAAARlssoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGMVJREFUeJzt3X20XXV95/F3grmpkIcSGKyn9QFGVFph6BBrTCoPahSm\nZoyzWF8nVUcDAQ3jiGUxgEWpa3zo0DSMnUIYViAObZnSb0ZwhGqU1TJFLjU6rnaCqyRZJNaRdakQ\nHhKCcG9I7vyx9905HHLvDeT87r0nvF9/3b1/53fO9/72OeezH87ee9rw8DCSJAFMn+wCJElTh6Eg\nSWoYCpKkhqEgSWoYCpKkhqEgSWq8otQTR8R0YA1wCjAIrMjMbR2PORK4CzgvM7fUfW4E3gjsAy7I\nzC2lapQkPV/JLYWlQF9mLgSuAFa3N0bEfOAe4Hhg5GSJ9wBHZeZvAv8J+FLB+iRJHUqGwiJgA0Bm\nbgTmd7T3UQVH+5bAM8DciJgGzAWGCtYnSepQMhTmALvapvfWu4cAyMz7MvOhjj79wC8Am4EbgD8u\nWJ8kqUPJUNgFzG5/rczcN06fy4D+zHwTcCpwc0T0lSpQkvR8xQ40U631LwHWR8QCYNNB9DmK/VsX\nTwAzgCPG6ePFmyTppZnWOaNkKNwOLI6I/np6eUQsA2Zl5tpR+qwCvhoR36UKhM9k5jPjvdDAwEBX\nCi6p1Wr1RJ29wvHsHseyu3plPFut1gHnTzsMrpI63CsLoBfq7BWOZ/c4lt3VK+NZh8ILthQ8eU2S\n1DAUJEkNQ0GS1DAUJEkNQ0GS1DAUJEkNQ0GS1DAUJEmNkmc0S1LXPD44zCO790x2GePavnuAocGy\nF3g+btYM5s18wXlnXWEoSOoJj+zew+Ubtk92GVPC1WefwLyZZa4V6u4jSVLDUJAkNQwFSVLDUJAk\nNQwFSVLDUJAkNQwFSVKj2HkKETEdWAOcAgwCKzJzW8djjgTuAs7LzC31vM9Q3dt5BnBtZt5cqkZJ\n0vOV3FJYCvRl5kLgCmB1e2NEzAfuAY4Hhut5ZwJvr/ucCZxQsD5JUoeSobAI2ACQmRuB+R3tfVTB\nsaVt3nuB+yPi68AdwDcK1idJ6lAyFOYAu9qm99a7lADIzPsy86GOPscCpwHnAp8AbilYnySpQ8lr\nH+0CZrdNT8/MfeP02QE8kJnPAVsj4tmIODYzd4zVqdVqHWKpE6NX6uwVjmf39MJYbt89MNklTBl9\nM/uKLbOSodBPdcB4fUQsADYdRJ97gYuBayKiBRwFPDZep4GBqf9mabVaPVFnr3A8u6dXxrL0lUd7\nydDg0CEvs9FCpeTuo9uBZyOin+og8+9ExLKIuGC0Dpn5l8DfRcT3qY4nXJSZwwVrlCS1KbalUH+Z\nr+yYvfUAjzurY/ryUjVJksbmyWuSpIahIElqGAqSpIahIElqGAqSpIahIElqGAqSpIahIElqGAqS\npIahIElqGAqSpIahIElqGAqSpIahIElqGAqSpIahIElqGAqSpEaxO69FxHRgDXAKMAisyMxtHY85\nErgLOC8zt7TNPw74IfCuzHzB3dokSWWU3FJYCvRl5kLgCqr7NDciYj5wD3A8MNw2fwZwA/B0wdok\nSQdQMhQWARsAMnMjML+jvY8qOLZ0zF8FXA88XLA2SdIBlAyFOcCutum99S4lADLzvsx8qL1DRHwM\neDQzv1PPmlawPklSh2LHFKgCYXbb9PTM3DdOn+XAcES8GzgVuDki3p+ZPxurU6vVOrRKJ0iv1Nkr\nHM/u6YWx3L57YLJLmDL6ZvYVW2YlQ6EfWAKsj4gFwKbxOmTmGSN/R8TdwMfHCwSAgYGp/2ZptVo9\nUWevcDy7p1fGcmhwaLJLmDKGBocOeZmNFiolQ+F2YHFE9NfTyyNiGTArM9cWfF1J0ktULBQycxhY\n2TH7BT8vzcyzRul/wPmSpHI8eU2S1DAUJEkNQ0GS1DAUJEkNQ0GS1DAUJEkNQ0GS1DAUJEkNQ0GS\n1DAUJEkNQ0GS1DAUJEkNQ0GS1DAUJEkNQ0GS1DAUJEkNQ0GS1Ch257WImA6sAU4BBoEVmbmt4zFH\nAncB52XmloiYAawDXgfMBL6YmXeUqlGS9HwltxSWAn2ZuRC4Aljd3hgR84F7gOOB4Xr2h4BHM/N0\n4Gzg2oL1SZI6lAyFRcAGgMzcCMzvaO+jCo4tbfPWA1e11fZcwfokSR2K7T4C5gC72qb3RsT0zNwH\nkJn3AURE84DMfLqeN5sqIK4sWJ8kqUPJUNgFzG6bbgJhLBHxGuA24LrMvPVgXqjVar20CidYr9TZ\nKxzP7umFsdy+e2CyS5gy+mb2FVtmJUOhH1gCrI+IBcCm8TpExKuA7wAXZebdB/tCAwNT/83SarV6\nos5e4Xh2T6+M5dDg0GSXMGUMDQ4d8jIbLVRKhsLtwOKI6K+nl0fEMmBWZq4dpc/vAnOBqyJi5NjC\nOZn5bME6JUm1YqGQmcPAyo7ZWw/wuLPa/r4YuLhUTZKksXnymiSpYShIkhqGgiSpYShIkhqGgiSp\nYShIkhqGgiSpYShIkhqGgiSpYShIkhqGgiSpYShIkhqGgiSpYShIkhqGgiSpYShIkhrFbrITEdOB\nNcApwCCwIjO3dTzmSOAu4LzM3HIwfSRJ5ZTcUlgK9GXmQuAKYHV7Y0TMB+4BjgeGD6aPJKmskqGw\nCNgAkJkbgfkd7X1UIbDlRfSRJBVUMhTmALvapvfWu4cAyMz7MvOhF9NHklRWyS/cXcDs9tfKzH0F\n+kiSuqTYgWagH1gCrI+IBcCmQn1otVovuciJ1Ct19grHs3t6YSy37x6Y7BKmjL6ZfcWWWclQuB1Y\nHBH99fTyiFgGzMrMtQfb52BeaGBg6r9ZWq1WT9TZKxzP7umVsRwaHJrsEqaMocGhQ15mo4VKsVDI\nzGFgZcfsrQd43Fnj9JEkTZAxjylExFkR8dGIeFXH/I+WLUuSNBlGDYWIuAT4b0AAmyPinW3Nny5d\nmCRp4o21pXA+8NbM/C1gGfAXEXHKxJQlSZoMY4XCUGbuAsjMDcClwP+KiGMnpDJJ0oQbKxR2RMR5\nEfFKgMy8GbgN+CbVSWaSpMPMWKGwkuonodE271Lge8DrC9YkSZoko/4kNTMfBN7RMW8Y+FRErCpd\nmCRp4o17nkJEvIFqC2F7Zv4BQGb+tHRhkqSJN2ooRMRbgcuBXwWuB746UUVJkibHWFsK/cAdwG9k\n5u4JqkeSNInGOtB8AvAT4IGI+EpEvG6CapIkTZJRQyEzH8rMS4CTgUeAcyasKknSpBj3QHNmPgl8\nuX1eRCzOzLuKVSVJmhRjHWg+Dfhj4DFgeWbuqHchfYVqq+EXJqZESdJEGeuYwvXA14AfA5+NiA8C\nPwKOArwGkiQdhsbafTQnM1dHxCuALcAHgQsy89aJKU2SNNHG2lL4OUBmPke1q+gcA0GSDm9jbSlM\na/v70cz8+xfzxBExHVhDtatpEFiRmdva2pcAnwOeA9Zl5o11nxuBNwL7qLZMtryY15UkvXRjhcIR\nETGPKhym1383MvPxcZ57KdCXmQsj4m3A6noeETEDuAaYT7VF0h8R3wD+JXBUZv5mRLwb+BJw7kv4\nvyRJL8FYu4/eAuwAHm37e0fbvPEsAjYAZOZGqgAYcRLwYGbuzMw9wL3A6cAzwNyImAbMBbxTtyRN\noLGukjrm/ZsPwhxgV9v03oiYnpn76radbW1PUYXA7VTHLzYDxwBLDrEGSdKLcKhf/GPZBcxuf606\nEKAKhPa22cCTVBfg68/MNwGnAjdHRF/BGiVJbcY9o/kQ9FOt6a+PiAXApra2zcCJEXE08DTVrqM/\npDqmMLJ18QQwAzhivBdqtVpdLLucXqmzVzie3dMLY7l998BklzBl9M3sK7bMpg0PDxd54vq4wMiv\nj6C6i9tpwKzMXBsR7wOuotpauSkzr4+IX6S6RPexVIHwlYP4GezwwMDUf7O0Wi16oc5e0Qvj+fjg\nMI/s3jPZZYyrb2YfQ4NlD98dN2sG82ZOG/+BY9j82BCXb9jepYp629Vnn8Cbjzm0nSh1qLxgoRQL\nhQlkKLwM9cJ4+iW2Xze+xBzP/UqGQsljCpKkHmMoSJIahoIkqWEoSJIahoIkqWEoSJIahoIkqWEo\nSJIahoIkqWEoSJIahoIkqWEoSJIahoIkqWEoSJIahoIkqWEoSJIahoIkqVHsHs0RMZ39t+McBFZk\n5ra29iXA54DngHWZeWM9/zNU93aeAVybmTeXqlGS9HwltxSWAn2ZuRC4Alg90hARM4BrgMXAGcCF\nEXFcRJwJvL3ucyZwQsH6JEkdSobCImADQGZuBOa3tZ0EPJiZOzNzD3AvcDrwHuD+iPg6cAfwjYL1\nSZI6lAyFOcCutum99S6lkbadbW1PAXOBY6nC41zgE8AtBeuTJHUodkyBKhBmt01Pz8x99d87O9pm\nA08CjwGbM/M5YGtEPBsRx2bmjrFeqNVqdbHscnqlzl4x1cdz++6ByS5hyuib2XfIy8vx3K8b4zma\nkqHQT3XAeH1ELAA2tbVtBk6MiKOBp6l2Ha0CngUuBq6JiBZwFFVQjGlgYOq/WVqtVk/U2St6YTyH\nBocmu4QpY2hw6JCXl+O5XzfGc7RQKRkKtwOLI6K/nl4eEcuAWZm5NiIuAb5NtQvrpsx8GPjLiDg9\nIr5fz78oM4cL1ihJalMsFOov85Uds7e2td8J3HmAfpeXqkmSNDZPXpMkNQwFSVLDUJAkNQwFSVLD\nUJAkNQwFSVLDUJAkNQwFSVLDUJAkNQwFSVLDUJAkNQwFSVLDUJAkNQwFSVLDUJAkNQwFSVLDUJAk\nNYrdeS0ipgNrgFOAQWBFZm5ra18CfA54DliXmTe2tR0H/BB4V2ZuRZI0IUpuKSwF+jJzIXAFsHqk\nISJmANcAi4EzgAvrIBhpuwF4umBtkqQDKBkKi4ANAJm5EZjf1nYS8GBm7szMPcC9wOl12yrgeuDh\ngrVJkg6gZCjMAXa1Te+tdymNtO1sa3sKmBsRHwMezczv1POnFaxPktSh2DEFqkCY3TY9PTP31X/v\n7GibDTwJfAoYjoh3A6cCN0fE+zPzZ2O9UKvV6l7VBfVKnb1iqo/n9t0Dk13ClNE3s++Ql5fjuV83\nxnM0JUOhH1gCrI+IBcCmtrbNwIkRcTTVsYPTgVWZ+bWRB0TE3cDHxwsEgIGBqf9mabVaPVFnr+iF\n8RwaHJrsEqaMocGhQ15ejud+3RjP0UKlZCjcDiyOiP56enlELANmZebaiLgE+DbVLqybMtNjCJI0\nyYqFQmYOAys7Zm9ta78TuHOM/mcVKk2SNApPXpMkNQwFSVLDUJAkNQwFSVLDUJAkNQwFSVLDUJAk\nNQwFSVLDUJAkNQwFSVLDUJAkNQwFSVLDUJAkNQwFSVLDUJAkNUreZEc96PHBYR7ZvWeyyxjX9t0D\nxe/EddysGcyb6W3C9fJiKOh5Htm9h8s3bJ/sMqaEq88+gXkz+ya7DGlCFQuFiJgOrAFOAQaBFZm5\nra19CfA54DlgXWbeGBEzgHXA64CZwBcz845SNUqSnq/kMYWlQF9mLgSuAFaPNNRf/tcAi4EzgAsj\n4jjgQ8CjmXk6cDZwbcH6JEkdSobCImADQGZuBOa3tZ0EPJiZOzNzD3AvcDqwHriqrbbnCtYnSepQ\n8pjCHGBX2/TeiJiemfvqtp1tbU8BczPzaYCImE0VEFcWrE+S1KFkKOwCZrdNjwQCVIHQ3jYbeAIg\nIl4D3AZcl5m3HswLtVqtQ692AvRCndt3D0x2CVNG38y+Q1pmjuV+hzqW4Hi268Z4jqZkKPQDS4D1\nEbEA2NTWthk4MSKOBp6m2nW0KiJeBXwHuCgz7z7YFxoYmPpvllar1RN1lv6ZZy8ZGhw6pGXmWO53\nqGM58hyqdGM8RwuVkqFwO7A4Ivrr6eURsQyYlZlrI+IS4NtUxw5uysyHI+KPgLnAVRExcmzhnMx8\ntmCdkqRasVDIzGFgZcfsrW3tdwJ3dvS5GLi4VE2SpLF5mQtJUsNQkCQ1DAVJUsNQkCQ1DAVJUsNQ\nkCQ1DAVJUsNQkCQ1DAVJUsNQkCQ1DAVJUsNQkCQ1DAVJUsNQkCQ1DAVJUsNQkCQ1it1kJyKmA2uA\nU4BBYEVmbmtrXwJ8DngOWJeZN47XR5JUVskthaVAX2YuBK4AVo80RMQM4BpgMXAGcGFEHFf3mXmg\nPpKk8kqGwiJgA0BmbgTmt7WdBDyYmTszcw9wL3B63edbo/SRJBVWbPcRMAfY1Ta9NyKmZ+a+um1n\nW9tTwNxx+oxq82NDXSq5nO27BxgaLFvncbNmMG/mtKKvIenwVjIUdgGz26bbv9x3drTNBp4cp8+o\nLt+w/RBLPTxcffYJzJvZN9llSOphJUOhH1gCrI+IBcCmtrbNwIkRcTTwNNWuo1XA8Bh9RvWD//jO\nbtb9stZqwQ9Ofv1kl3FYcCy7y/GcGNOGh4eLPHFETGP/L4kAlgOnAbMyc21EvA+4iuq4xk2Zef2B\n+mTm1iIFSpJeoFgoSJJ6jyevSZIahoIkqWEoSJIahoIkqVHyJ6kvKxHxa8DVwJHALOCbwNeBJZn5\nhYj4APC9zHx4EsucdBFxJvDxzFx2EI/9ZGZe+2L6vFxFxGXAp4HjM3MwIt4CHJ2Z342I04EnMvP+\njj6XA38N/CpwbGYe1GVlRpZLl/+FSXWgz29mfn6Ux74G+BeZeefEVThx3FLogoj4ReDPgYsz853A\nAuBk4G2Z+YX6YZ+iOmP75e7F/NztymJVHH4+TPUe/Lf19LlUX/YA5wGtzg6ZeXVm/uAlvNZhtVxG\n+/xGxIWjdHkX1SV5Dkv+JLULIuKjwKmZ+Ttt844CFlJ9IP8UuAXYCtwInJiZl0XEEcDfAfMzc+pf\nq6MLDrTWHxHnAhcBM6hC4wPAJ6jOY1kLrAc+0bmlEBGLgS8AzwKPUY31rwOXU11l9wTg1sz8ckS8\nAfjvwBDwE+D1mXlWsX90AtVj+h+AzwB/RnVhyfuoxuXfAd8A/onqxNB7gQeAfwCOBm4Ffgn4EHAE\n1Vry5zPzWxHxj8AbM3MoIv5z3e+Xgd+jWi6fBm4A3kC1gvnZzPyb8v9xd43y+T2S6grOa4BfAV5N\nNY6/RzV2rwT+PdV76Y+Aaex/Dz4FXEd1XtY/AcdTjf00YB3VOA8Dn8rMTRHxE/YvkyXAb2TmExGx\nkuq8rlVFB6CDWwrd8Wrgx+0zMvNpYE/99zeBvwc+QrVGsrS+TPjZwF+/XAJhDCcCv5WZ76D6YLw3\nM78EPJ6Zn6T6MD1PfaLjDcAHMvNM4G+Az1J92F4L/BuqNb7L6i6rgC/Wa4L9Zf+dCbeC6gTQrVRh\n+CvAV4FrMvP7VBeZvCwzf1q3LcvMSzqe45HMfBfVl9J19fi2rzEOA8OZ+WX2L5cLgEcz8wyqILqu\n3L9Y1IE+vz+v5/9tZp4NvI1qxWQf8PvALfXuo7XARfUKxjep3m//GpiXmW8DzgdeUz/tHwL/pR6v\ni4Gb6vnty+QW9m/tfYhqRWZCGQrd8RP2L3gAIuJ4qst3PG9TLDN3U32BvRf4GNWWw8vdo8DNEbGO\n6mz2gznWdSywq+0YzXeBX6v/vj8z99Uf7GfqeW+mWnuGam35sFBfKuYc4OKI+BbVhSU/WTcf6OqI\nOzLziQPMvwcgMx+hugbZMR3tB3quk4F/FRF3A/8TOCIi5r34/2LSjfb5fS3w1oj4M6pL/c+sm6ex\nfzxOAq6vx+A8qi2pNwN/C5CZO6gu60M9f2Sc/2/ba7Yvk3XAR+pjHD/LzEe7+H8eFEOhO+4Ezo6I\nE6C5X8Rqqi+7kTfPPqrNRqjWLi4A/llm/miCa51SImIu8Hngg1Rj8gz7x2ysS77uAOZExC/V02cA\nW+q/D7RP9EdUu/Og2oI4XHwYuDEz35uZ51Ct0b6HagxGPt/t773RLjC5ACAifhl4Zf1l9izQqrca\nfr3tsSPL5QHgz+u15PcDCRwocKa60T6/pwJPZuaHqULhyPrxe9k/tpuBj9Rj8LtUu5h+BLy9fq6j\ngTfWj32AakWRiDgVGFmhaZZJZv4/qouDXskkrTD666MuyMyn6v2Sa+vdQrOp3hwPAO+oH3Yf8CcR\nsTgzvx8R/xw4rH7BcZCGgfdExMgBzmnA96jWrB6hOu4yclD0HyLiT6g2s9v7APw2VYjcFhH7gMep\ntrxO5oW7PaA6zrAuIi6lukrvni7/X5PlfKpgACAzn4mIr1HtD/9kRDwAbAR+PyJ+zOgH+o+JiL8C\njqIaV4A/oNol8o9U+8tHjCyXFVTv+f9N9SOK6zKz5w5SjvH5/Svgf0TEaVRbE/8nIl4N3A9cGRE/\nBFYCfxoRr6Aa2/My88GIOCci+qmOKfyc6ljWpfVrXEp1/Oz8uoTOMVsL/Feq3UcTzgPNk6B+432X\nat/57smu5+UgIn4b2JiZ2yJiBbAgM1dMdl06/ETEm6gOXP9FRBxDteXw2vqGYgfT/1zgLaP9JLY0\ntxQmWL2v8jaq+1IbCBPnp8CtEfFzqrXo88d5vPRS/RS4OiI+TbXb7rIXEQhfptoV+r6C9Y3JLQVJ\nUsMDzZKkhqEgSWoYCpKkhqEgSWoYCpKkhqEgSWr8f56hxb6YTHiGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x390791d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.bar([1,2,3,4],[0.00132602324228, 0.0248118636604, 0.0789390332998, 0.170443168974])\n",
    "plt.xticks([1.5,2.5,3.5,4.5], ['City','LatLong','Attribute','Category'])\n",
    "#plt.gca().set_xticklabel(['City','LatLong','Attribute','Category'])\n",
    "#plt.g\n",
    "#plt.xticks(['City','LatLong','Attribute','Category'])\n",
    "f = plt.gcf()\n",
    "f.set_size_inches(6,4)\n",
    "plt.ylabel(\"R^2\")\n",
    "plt.savefig('ML_Yelp_Bus.png',dpi=300)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}