Assignment 3 - Richard.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# White House Posts' Effect on Stock Market\n",
    "\n",
    "### Author: Richard Doan\n",
    "### Partner: Jon Khaykin\n",
    "\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Given recent events, there seems to be correlations between text data sources and stock market motions. For example, there are a few [bots that use Twitter data to make stock transactions](http://www.npr.org/2017/04/14/523890750/twitter-bot-botus-will-buy-and-sell-stock-based-on-trumps-tweets). \n",
    "This inspired us to consider stock market motions given announcements made by the White House itself, president and staff included. \n",
    "\n",
    "To run through the examples interactively, please run the code in the **Appendix** at the bottom of this notebook after reading this.\n",
    "\n",
    "## Data Schema and Collection Method\n",
    "\n",
    "Because we are looking into how White House announcements affect the stock market, our dataset will be composed of raw White House blog posts and a few index fund prices. Major index fund prices are used as indicators as to how strong a country's economy is, which is what inspired the use of them in our analysis. The index funds we will be focusing are Dow Jones, S&P 500, and NASDAQ.\n",
    "\n",
    "For each blog post, we get the closing prices of our three index funds on the day that blog post is posted. We then grab the price of the same funds, but for the next day. From there we take the price of the index funds on the next day, subtract that from the current day to get the price change, and then average over the three index fund price changes. This gave us numeric data to work with. We decided to average the fund prices instead of taking the max or the median. \n",
    "\n",
    "We noticed that absolute price changes may not give us as much information as relative price changes, so we repeated the above, but in addition to the subtraction, we divided the change in price with the price of the blog postdate and then averaged. Labels were then generated from the average proportion change of the three index fund prices. If the proportion change was positive, then we labeled the corresponding blog post as 1, otherwise, we labeled the blog post as -1. \n",
    "\n",
    "There were a few nuances that we had to overcome, which have added biases to our data collection methods. We weren't sure which blog posts would correspond to market motions, so we took every post that was on the White House blog page. This meant that there are cases where there are many posts to one day, so there are many posts to one closing price. To overcome this problem, we concatenated the blog posts that occur on the same day in this analysis. We also remove NaNs from our dataframe. These NaN value come from the fact that some posts are published on a day where the stock market isn't open for trading.\n",
    "\n",
    "Below is the resulting dataframe from our collection method.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Body</th>\n",
       "      <th>Title</th>\n",
       "      <th>Dow Jones Delta</th>\n",
       "      <th>Dow Jones Proportion</th>\n",
       "      <th>Nasdaq Delta</th>\n",
       "      <th>Nasdaq Proportion</th>\n",
       "      <th>S&amp;P 500 Delta</th>\n",
       "      <th>S&amp;P 500 Proportion</th>\n",
       "      <th>Mean Delta</th>\n",
       "      <th>Mean Proportion</th>\n",
       "      <th>Label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017-03-13</td>\n",
       "      <td>On Monday President Donald J Trump welcomed to...</td>\n",
       "      <td>Readout of President Donald J Trumps Meeting w...</td>\n",
       "      <td>-44.111328</td>\n",
       "      <td>-0.002112</td>\n",
       "      <td>-18.959961</td>\n",
       "      <td>-0.003227</td>\n",
       "      <td>-8.020020</td>\n",
       "      <td>-0.003379</td>\n",
       "      <td>-23.697104</td>\n",
       "      <td>-0.002906</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017-03-06</td>\n",
       "      <td>President Donald J Trump spoke separately toda...</td>\n",
       "      <td>Readout of the Presidents Calls with Prime Min...</td>\n",
       "      <td>-29.580078</td>\n",
       "      <td>-0.001412</td>\n",
       "      <td>-15.250000</td>\n",
       "      <td>-0.002607</td>\n",
       "      <td>-6.920166</td>\n",
       "      <td>-0.002913</td>\n",
       "      <td>-17.250082</td>\n",
       "      <td>-0.002311</td>\n",
       "      <td>-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017-03-27</td>\n",
       "      <td>James S Brady Press Briefing Room PM EDTSENIOR...</td>\n",
       "      <td>Background Briefing on the Presidents Energy I...</td>\n",
       "      <td>150.519531</td>\n",
       "      <td>0.007324</td>\n",
       "      <td>34.770020</td>\n",
       "      <td>0.005953</td>\n",
       "      <td>16.979980</td>\n",
       "      <td>0.007251</td>\n",
       "      <td>67.423180</td>\n",
       "      <td>0.006843</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date                                               Body  \\\n",
       "0  2017-03-13  On Monday President Donald J Trump welcomed to...   \n",
       "1  2017-03-06  President Donald J Trump spoke separately toda...   \n",
       "2  2017-03-27  James S Brady Press Briefing Room PM EDTSENIOR...   \n",
       "\n",
       "                                               Title  Dow Jones Delta  \\\n",
       "0  Readout of President Donald J Trumps Meeting w...       -44.111328   \n",
       "1  Readout of the Presidents Calls with Prime Min...       -29.580078   \n",
       "2  Background Briefing on the Presidents Energy I...       150.519531   \n",
       "\n",
       "   Dow Jones Proportion  Nasdaq Delta  Nasdaq Proportion  S&P 500 Delta  \\\n",
       "0             -0.002112    -18.959961          -0.003227      -8.020020   \n",
       "1             -0.001412    -15.250000          -0.002607      -6.920166   \n",
       "2              0.007324     34.770020           0.005953      16.979980   \n",
       "\n",
       "   S&P 500 Proportion  Mean Delta  Mean Proportion  Label  \n",
       "0           -0.003379  -23.697104        -0.002906     -1  \n",
       "1           -0.002913  -17.250082        -0.002311     -1  \n",
       "2            0.007251   67.423180         0.006843      1  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/rykard95/wh_stocks/master/data/dataset.csv', dtype=DATATYPES).dropna()\n",
    "dataset = join_blog_posts_on_date(df)\n",
    "dataset['Title'] = dataset['Title'].apply(lambda title: title +' ')\n",
    "dataset['Body'] = dataset['Body'].apply(lambda body: body + ' ')\n",
    "dataset[:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Word Analysis\n",
    "\n",
    "Once we have our data scheme, I focused on the word level of our project. The hope was to gain insight as to what featurization method would be best to create a predictive model. Text analysis is trickier than numeric problems in machine learning. There are many ways to featurize unstructured text data, and the choice of featurization often introduces latent biases into machine learning models. \n",
    "\n",
    "Below I grouped the blog posts according to their label, concatenated the body of the blog posts together, and then tokenized (removing stop words) the larger text. I used the difference of proportions method to see if there were any major usage differences of certain words between the different labels. It seems as though the largest difference seen across the labels was when 'spicer' was mentioned, and the usage difference was only 0.002 more in the negative label.\n",
    "\n",
    "I then repeated the same process, but with the titles of the blog posts instead. The results of the difference of proportions method seem more promising for the titles than those of the bodies. Here we see the largest difference of 0.014 more in the negative label. The word associated with this difference was 'meeting'. From these results, it seems as though the title might give us more information about stock market motions. A reason for this might be that because titles need to be concise, so each word must carry more distinctive information about the theme. In which case, more distinctive themes could correlate in some way to stock market motions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The total number of positive words is 141560\n",
      "The total number of negative words is 140548\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Word</th>\n",
       "      <th>Positive</th>\n",
       "      <th>Negative</th>\n",
       "      <th>Positive Proportion</th>\n",
       "      <th>Negative Proportion</th>\n",
       "      <th>diff</th>\n",
       "      <th>diff proportion</th>\n",
       "      <th>abs diff proportion</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>spicer</td>\n",
       "      <td>1044</td>\n",
       "      <td>1252</td>\n",
       "      <td>0.007</td>\n",
       "      <td>0.009</td>\n",
       "      <td>-208</td>\n",
       "      <td>-0.002</td>\n",
       "      <td>0.002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>president</td>\n",
       "      <td>2628</td>\n",
       "      <td>2501</td>\n",
       "      <td>0.019</td>\n",
       "      <td>0.018</td>\n",
       "      <td>127</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>applause</td>\n",
       "      <td>259</td>\n",
       "      <td>180</td>\n",
       "      <td>0.002</td>\n",
       "      <td>0.001</td>\n",
       "      <td>79</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>128</th>\n",
       "      <td>healthcare</td>\n",
       "      <td>178</td>\n",
       "      <td>295</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.002</td>\n",
       "      <td>-117</td>\n",
       "      <td>-0.001</td>\n",
       "      <td>0.001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110</th>\n",
       "      <td>eo</td>\n",
       "      <td>193</td>\n",
       "      <td>34</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.000</td>\n",
       "      <td>159</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>actions</td>\n",
       "      <td>201</td>\n",
       "      <td>78</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>123</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>regulations</td>\n",
       "      <td>202</td>\n",
       "      <td>88</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>114</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>plan</td>\n",
       "      <td>205</td>\n",
       "      <td>287</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.002</td>\n",
       "      <td>-82</td>\n",
       "      <td>-0.001</td>\n",
       "      <td>0.001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>women</td>\n",
       "      <td>209</td>\n",
       "      <td>90</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>119</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>process</td>\n",
       "      <td>224</td>\n",
       "      <td>297</td>\n",
       "      <td>0.002</td>\n",
       "      <td>0.002</td>\n",
       "      <td>-73</td>\n",
       "      <td>-0.001</td>\n",
       "      <td>0.001</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Word  Positive  Negative  Positive Proportion  \\\n",
       "3         spicer      1044      1252                0.007   \n",
       "0      president      2628      2501                0.019   \n",
       "73      applause       259       180                0.002   \n",
       "128   healthcare       178       295                0.001   \n",
       "110           eo       193        34                0.001   \n",
       "103      actions       201        78                0.001   \n",
       "101  regulations       202        88                0.001   \n",
       "99          plan       205       287                0.001   \n",
       "97         women       209        90                0.001   \n",
       "88       process       224       297                0.002   \n",
       "\n",
       "     Negative Proportion  diff  diff proportion  abs diff proportion  \n",
       "3                  0.009  -208           -0.002                0.002  \n",
       "0                  0.018   127            0.001                0.001  \n",
       "73                 0.001    79            0.001                0.001  \n",
       "128                0.002  -117           -0.001                0.001  \n",
       "110                0.000   159            0.001                0.001  \n",
       "103                0.001   123            0.001                0.001  \n",
       "101                0.001   114            0.001                0.001  \n",
       "99                 0.002   -82           -0.001                0.001  \n",
       "97                 0.001   119            0.001                0.001  \n",
       "88                 0.002   -73           -0.001                0.001  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "body_df = generate_word_df(dataset, text='Body')\n",
    "body_df[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The total number of positive words is 1509\n",
      "The total number of negative words is 1340\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Word</th>\n",
       "      <th>Positive</th>\n",
       "      <th>Negative</th>\n",
       "      <th>Positive Proportion</th>\n",
       "      <th>Negative Proportion</th>\n",
       "      <th>diff</th>\n",
       "      <th>diff proportion</th>\n",
       "      <th>abs diff proportion</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>meeting</td>\n",
       "      <td>11</td>\n",
       "      <td>29</td>\n",
       "      <td>0.007</td>\n",
       "      <td>0.022</td>\n",
       "      <td>-18</td>\n",
       "      <td>-0.014</td>\n",
       "      <td>0.014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>march</td>\n",
       "      <td>6</td>\n",
       "      <td>21</td>\n",
       "      <td>0.004</td>\n",
       "      <td>0.016</td>\n",
       "      <td>-15</td>\n",
       "      <td>-0.012</td>\n",
       "      <td>0.012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>call</td>\n",
       "      <td>24</td>\n",
       "      <td>7</td>\n",
       "      <td>0.016</td>\n",
       "      <td>0.005</td>\n",
       "      <td>17</td>\n",
       "      <td>0.011</td>\n",
       "      <td>0.011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>president</td>\n",
       "      <td>145</td>\n",
       "      <td>116</td>\n",
       "      <td>0.096</td>\n",
       "      <td>0.087</td>\n",
       "      <td>29</td>\n",
       "      <td>0.010</td>\n",
       "      <td>0.010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>house</td>\n",
       "      <td>21</td>\n",
       "      <td>32</td>\n",
       "      <td>0.014</td>\n",
       "      <td>0.024</td>\n",
       "      <td>-11</td>\n",
       "      <td>-0.010</td>\n",
       "      <td>0.010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>donald</td>\n",
       "      <td>41</td>\n",
       "      <td>47</td>\n",
       "      <td>0.027</td>\n",
       "      <td>0.035</td>\n",
       "      <td>-6</td>\n",
       "      <td>-0.008</td>\n",
       "      <td>0.008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>j</td>\n",
       "      <td>41</td>\n",
       "      <td>47</td>\n",
       "      <td>0.027</td>\n",
       "      <td>0.035</td>\n",
       "      <td>-6</td>\n",
       "      <td>-0.008</td>\n",
       "      <td>0.008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173</th>\n",
       "      <td>proclaims</td>\n",
       "      <td>2</td>\n",
       "      <td>12</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.009</td>\n",
       "      <td>-10</td>\n",
       "      <td>-0.008</td>\n",
       "      <td>0.008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>trumps</td>\n",
       "      <td>26</td>\n",
       "      <td>12</td>\n",
       "      <td>0.017</td>\n",
       "      <td>0.009</td>\n",
       "      <td>14</td>\n",
       "      <td>0.008</td>\n",
       "      <td>0.008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>order</td>\n",
       "      <td>25</td>\n",
       "      <td>12</td>\n",
       "      <td>0.017</td>\n",
       "      <td>0.009</td>\n",
       "      <td>13</td>\n",
       "      <td>0.008</td>\n",
       "      <td>0.008</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Word  Positive  Negative  Positive Proportion  Negative Proportion  \\\n",
       "29     meeting        11        29                0.007                0.022   \n",
       "53       march         6        21                0.004                0.016   \n",
       "12        call        24         7                0.016                0.005   \n",
       "0    president       145       116                0.096                0.087   \n",
       "14       house        21        32                0.014                0.024   \n",
       "4       donald        41        47                0.027                0.035   \n",
       "5            j        41        47                0.027                0.035   \n",
       "173  proclaims         2        12                0.001                0.009   \n",
       "9       trumps        26        12                0.017                0.009   \n",
       "10       order        25        12                0.017                0.009   \n",
       "\n",
       "     diff  diff proportion  abs diff proportion  \n",
       "29    -18           -0.014                0.014  \n",
       "53    -15           -0.012                0.012  \n",
       "12     17            0.011                0.011  \n",
       "0      29            0.010                0.010  \n",
       "14    -11           -0.010                0.010  \n",
       "4      -6           -0.008                0.008  \n",
       "5      -6           -0.008                0.008  \n",
       "173   -10           -0.008                0.008  \n",
       "9      14            0.008                0.008  \n",
       "10     13            0.008                0.008  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "title_df = generate_word_df(dataset, 'Title')\n",
    "title_df[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Most Distinctive Words\n",
    "\n",
    "Below is a different analysis of the most distinctive words. Instead of the difference of proportions, I used a chi-squared test to determine what words were statistically different in frequency across our two labels. I first do this analysis using the blog post body, and then I repeat the process using the blog post titles.\n",
    "\n",
    "When analyzing words in the body, there doesn't seem to be any overlap between the words found to be significantly different using the difference of proportions and the chi-squared test. The same holds for words in the title. \n",
    "\n",
    "This discrepancy is worrisome because it means that a bag of words method may not give us as meaningful of a representation of our blog posts as we would like. We will want to perform operations such as stemming in order to tease out the topic of that post.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "These are the words given by chi-squared...\n",
      "\n",
      "['publish' 'portion' 'proper' 'rightmr' 'soldiers' 'informed' 'honestly'\n",
      " 'area' 'dollars' 'free' 'addition' 'having' 'sell' 'readout' 'parties'\n",
      " 'range' 'kids' 'learned' 'change' 'mike' 'wrote' 'subsequent'\n",
      " 'generations' 'break' 'challenge' 'marketplace' 'lack' 'litigation' 'gave'\n",
      " 'policy' 'rights' 'peaceful' 'hearings' 'largely' 'russians' 'function'\n",
      " 'directing' 'uniform' 'sides' 'took' 'nominee' 'broken' 'list' 'pence'\n",
      " 'right' 'direct' 'problems' 'aspect' 'developing' 'chemical'\n",
      " 'professional' 'forgotten' 'rolling' 'private' 'hope' 'friend' 'allies'\n",
      " 'ending' 'empower' 'door' 'creators' 'additionally' 'troops' 'alex'\n",
      " 'keeps' 'reset' 'negotiations' 'hoping' 'launched' 'german' 'extremely'\n",
      " 'taxpayer' 'presented' 'talks' 'district' 'arent' 'specifically'\n",
      " 'extraordinary' 'car' 'named' 'podium' 'prepare' 'revenues' 'grant'\n",
      " 'component' 'service' 'course' 'international' 'yeah' 'simple' 'bless'\n",
      " 'particular' 'reinvest' 'meetingsq' 'implosion' 'conviction' 'propel'\n",
      " 'systemthe' 'andsjres' 'distressed']\n",
      "\n",
      "\n",
      "These are the words given by the difference of proportions...\n",
      "\n",
      "['spicer' 'president' 'applause' 'healthcare' 'eo' 'actions' 'regulations'\n",
      " 'plan' 'women' 'process' 'general' 'rule' 'regulatory' 'secretary' 'see'\n",
      " 'jobs' 'good' 'today' 'action' 'information' 'hes' 'mulvaney' 'agenda'\n",
      " 'percent' 'trade' 'foreign' 'drug' 'insurance' 'coal' 'guidance'\n",
      " 'intelligence' 'director' 'ireland' 'money' 'energy' 'budget' 'unified'\n",
      " 'federal' 'obamacare' 'states' 'united' 'order' 'trump' 'think' 'get'\n",
      " 'thank' 'said' 'one' 'mr' 'going' 'ed' 'inc' 'bullet' 'helpfully' 'todayq'\n",
      " 'proclaim' 'genuine' 'hotel' 'seriousness' 'obtaining' 'slowly' 'discount'\n",
      " 'reassuring' 'ablebodied' 'persecuted' 'unveiled' 'manages' 'deployment'\n",
      " 'finished' 'transaction' 'discoveries' 'accomplishing' 'attribution'\n",
      " 'urging' 'evaluated' 'spurred' 'unelected' 'disabled' 'possibilities'\n",
      " 'andv' 'movements' 'targeting' 'elwood' 'elaine' 'relatively' 'embodied'\n",
      " 'swiftly' 'wrongdoing' 'inspires' 'parade' 'revised' 'tomorrows' 'devoted'\n",
      " 'linguistic' 'blueprint' 'reams' 'scoring' 'fulfills' 'ceciliaq' 'signal']\n",
      "\n",
      "\n",
      "This is this the list of words from the difference of proportion list that are in the chi-squared list...\n",
      "\n",
      "[False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False]\n"
     ]
    }
   ],
   "source": [
    "vectorizer = CountVectorizer(stop_words='english')\n",
    "body_data = vectorizer.fit_transform(dataset['Body'])\n",
    "\n",
    "indexes = np.argsort(chi2(body_data, dataset['Label'])[0])[:100]\n",
    "top_body_words = np.array(vectorizer.get_feature_names())[indexes]\n",
    "print(\"These are the words given by chi-squared...\")\n",
    "print()\n",
    "print(top_body_words)\n",
    "print()\n",
    "print()\n",
    "print(\"These are the words given by the difference of proportions...\")\n",
    "print()\n",
    "print(np.array(body_df['Word'][:100].tolist()))\n",
    "print()\n",
    "print()\n",
    "print(\"This is this the list of words from the difference of proportion list that are in the chi-squared list...\")\n",
    "print()\n",
    "print(np.array([word in top_body_words for word in body_df['Word'][:100].tolist()]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "These are the words given by chi-squared...\n",
      "\n",
      "['statement' 'discussion' 'guests' 'travel' 'discuss' 'committee' 'majesty'\n",
      " 'insurance' 'dan' 'treasury' 'initiative' 'alsisi' 'promote' 'make'\n",
      " 'federalism' 'sanders' 'angela' 'investment' 'bipartisan' 'excellence'\n",
      " 'bin' 'second' 'support' 'prolife' 'photos' 'regulations' 'contracting'\n",
      " 'empowerment' 'karen' 'charter' 'authority' 'attorney' 'florida'\n",
      " 'upcoming' 'medal' 'acting' 'officials' 'comprehensive' 'reopening'\n",
      " 'reorganizing' 'abuse' 'gerald' 'winter' 'gala' 'new' 'cabinet'\n",
      " 'recipients' 'administrator' 'parentteacher' 'revocation' 'right' 'ford'\n",
      " 'african' 'verma' 'minister' 'address' 'delivers' 'japan' 'establishing'\n",
      " 'orders' 'route' 'week' 'en' 'message' 'care' 'al' 'womens' 'general'\n",
      " 'department' 'chancellor' 'jordan' 'alabadi' 'legislative' 'memorandum'\n",
      " 'weekly' 'nominate' 'intent' 'respect' 'gorsuch' 'visit' 'supreme'\n",
      " 'budgetary' 'entitled' 'impact' 'congress' 'analysis' 'announces'\n",
      " 'secretary' 'press' 'independence' 'court' 'joint' 'neil' 'sean' 'spicer'\n",
      " 'jobs' 'haider' 'iraq' 'germany' 'session']\n",
      "\n",
      "\n",
      "These are the words given by the difference of proportions...\n",
      "\n",
      "['meeting' 'march' 'call' 'president' 'house' 'donald' 'j' 'proclaims'\n",
      " 'trumps' 'order' 'white' 'act' 'united' 'daily' 'executive' 'hr' 'day'\n",
      " 'everything' 'energy' 'signs' 'signing' 'melania' 'hjres' 'lady' 'black'\n",
      " 'swearingin' 'first' 'national' 'presidents' 'law' 'economic' 'leaders'\n",
      " 'declaration' 'approves' 'notice' 'ceremony' 'april' 'readout' 'people'\n",
      " 'disapproving' 'regarding' 'announce' 'key' 'budget' 'regulatory'\n",
      " 'commission' 'director' 'r' 'egg' 'red' 'colleges' 'business' 'obamacare'\n",
      " 'deputy' 'states' 'healthcare' 'continuation' 'staff' 'trump' 'israeli'\n",
      " 'universities' 'posts' 'affairs' 'historically' 'industry' 'appointments'\n",
      " 'enforcement' 'roll' 'meets' 'easter' 'disaster' 'rule' 'public'\n",
      " 'conference' 'vice' 'remarks' 'briefing' 'federal' 'american' 'photo'\n",
      " 'presidential' 'prime' 'host' 'shinzo' 'confidence' 'listening' 'waters'\n",
      " 'council' 'netanyahu' 'justice' 'succession' 'high' 'congressman' 'health'\n",
      " 'bilateral' 'authorization' 'joint' 'labor' 'within' 'service']\n",
      "\n",
      "\n",
      "This is this the list of words from the difference of proportion list that are in the chi-squared list...\n",
      "\n",
      "[False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      " False False False False False False False False False False False False\n",
      "  True False False False]\n"
     ]
    }
   ],
   "source": [
    "vectorizer = CountVectorizer(stop_words='english')\n",
    "title_data = vectorizer.fit_transform(dataset['Title'])\n",
    "\n",
    "indexes = np.argsort(chi2(title_data, dataset['Label'])[0])[:100]\n",
    "top_title_words = np.array(vectorizer.get_feature_names())[indexes]\n",
    "print(\"These are the words given by chi-squared...\")\n",
    "print()\n",
    "print(top_title_words)\n",
    "print()\n",
    "print()\n",
    "print(\"These are the words given by the difference of proportions...\")\n",
    "print()\n",
    "print(np.array(title_df['Word'][:100].tolist()))\n",
    "print()\n",
    "print()\n",
    "print(\"This is this the list of words from the difference of proportion list that are in the chi-squared list...\")\n",
    "print()\n",
    "print(np.array([word in top_title_words for word in title_df['Word'][:100].tolist()]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clustering \n",
    "\n",
    "After noticing that inconclusiveness of the above methods, we approached our problem from a clustering standpoint to see why it was so difficult finding distinct words. We used the cosine distance metric to determine the similarity of two blog posts. This metric was used so that our plots would be agnostic of how long the text documents were. We also used MDS, which we learned about in class.\n",
    "\n",
    "Below we see that the plot of the dataset created from the blog titles seems to have a few weak clusters based on label, while the dataset created by the blog post bodies is more mixed. The clusters in title date plot could indicate that the blog titles form a nice feature set for our labels, at least better for us than the data generated from the blog post bodies.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "    <div class=\"bk-root\">\n",
       "        <div class=\"bk-plotdiv\" id=\"41017ada-fba1-484e-8035-cd4c4bd70848\"></div>\n",
       "    </div>\n",
       "<script type=\"text/javascript\">\n",
       "  \n",
       "  (function(global) {\n",
       "    function now() {\n",
       "      return new Date();\n",
       "    }\n",
       "  \n",
       "    var force = false;\n",
       "  \n",
       "    if (typeof (window._bokeh_onload_callbacks) === \"undefined\" || force === true) {\n",
       "      window._bokeh_onload_callbacks = [];\n",
       "      window._bokeh_is_loading = undefined;\n",
       "    }\n",
       "  \n",
       "  \n",
       "    \n",
       "    if (typeof (window._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "      window._bokeh_timeout = Date.now() + 0;\n",
       "      window._bokeh_failed_load = false;\n",
       "    }\n",
       "  \n",
       "    var NB_LOAD_WARNING = {'data': {'text/html':\n",
       "       \"<div style='background-color: #fdd'>\\n\"+\n",
       "       \"<p>\\n\"+\n",
       "       \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "       \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "       \"</p>\\n\"+\n",
       "       \"<ul>\\n\"+\n",
       "       \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "       \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "       \"</ul>\\n\"+\n",
       "       \"<code>\\n\"+\n",
       "       \"from bokeh.resources import INLINE\\n\"+\n",
       "       \"output_notebook(resources=INLINE)\\n\"+\n",
       "       \"</code>\\n\"+\n",
       "       \"</div>\"}};\n",
       "  \n",
       "    function display_loaded() {\n",
       "      if (window.Bokeh !== undefined) {\n",
       "        var el = document.getElementById(\"41017ada-fba1-484e-8035-cd4c4bd70848\");\n",
       "        el.textContent = \"BokehJS \" + Bokeh.version + \" successfully loaded.\";\n",
       "      } else if (Date.now() < window._bokeh_timeout) {\n",
       "        setTimeout(display_loaded, 100)\n",
       "      }\n",
       "    }\n",
       "  \n",
       "    function run_callbacks() {\n",
       "      window._bokeh_onload_callbacks.forEach(function(callback) { callback() });\n",
       "      delete window._bokeh_onload_callbacks\n",
       "      console.info(\"Bokeh: all callbacks have finished\");\n",
       "    }\n",
       "  \n",
       "    function load_libs(js_urls, callback) {\n",
       "      window._bokeh_onload_callbacks.push(callback);\n",
       "      if (window._bokeh_is_loading > 0) {\n",
       "        console.log(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "        return null;\n",
       "      }\n",
       "      if (js_urls == null || js_urls.length === 0) {\n",
       "        run_callbacks();\n",
       "        return null;\n",
       "      }\n",
       "      console.log(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "      window._bokeh_is_loading = js_urls.length;\n",
       "      for (var i = 0; i < js_urls.length; i++) {\n",
       "        var url = js_urls[i];\n",
       "        var s = document.createElement('script');\n",
       "        s.src = url;\n",
       "        s.async = false;\n",
       "        s.onreadystatechange = s.onload = function() {\n",
       "          window._bokeh_is_loading--;\n",
       "          if (window._bokeh_is_loading === 0) {\n",
       "            console.log(\"Bokeh: all BokehJS libraries loaded\");\n",
       "            run_callbacks()\n",
       "          }\n",
       "        };\n",
       "        s.onerror = function() {\n",
       "          console.warn(\"failed to load library \" + url);\n",
       "        };\n",
       "        console.log(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "        document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "      }\n",
       "    };var element = document.getElementById(\"41017ada-fba1-484e-8035-cd4c4bd70848\");\n",
       "    if (element == null) {\n",
       "      console.log(\"Bokeh: ERROR: autoload.js configured with elementid '41017ada-fba1-484e-8035-cd4c4bd70848' but no matching script tag was found. \")\n",
       "      return false;\n",
       "    }\n",
       "  \n",
       "    var js_urls = [];\n",
       "  \n",
       "    var inline_js = [\n",
       "      function(Bokeh) {\n",
       "        (function() {\n",
       "          var fn = function() {\n",
       "            var docs_json = {\"6c0d4d95-6409-46e8-9382-ea1dc49e8fb9\":{\"roots\":{\"references\":[{\"attributes\":{\"plot\":null,\"text\":\"MDS: White House Post Bodies\"},\"id\":\"a4c4c436-b821-4ced-986a-fe5c1d66b7bc\",\"type\":\"Title\"},{\"attributes\":{\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"223c3e4c-107b-46ec-bd0e-5b5004c427eb\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"cd378e89-5c23-4c97-b0d1-e4125c6e2965\",\"type\":\"BasicTicker\"},{\"attributes\":{\"axis_label\":\"y\",\"formatter\":{\"id\":\"bddd3fc2-bc1c-4e34-91b6-05f7ba33f6f6\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"cd378e89-5c23-4c97-b0d1-e4125c6e2965\",\"type\":\"BasicTicker\"}},\"id\":\"df9ecb4b-7429-4d8e-9e2f-caa872f8d3f2\",\"type\":\"LinearAxis\"},{\"attributes\":{\"overlay\":{\"id\":\"57745f1f-e745-4995-aa9a-08fc17fb2892\",\"type\":\"BoxAnnotation\"},\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"a4ef6852-8681-43ed-b28b-8815c6f60e20\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"axis_label\":\"x\",\"formatter\":{\"id\":\"04c0ab92-66d0-4f17-940e-21ac144fb74d\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"300dd613-fcd5-468f-98db-78fc31aa3615\",\"type\":\"BasicTicker\"}},\"id\":\"311c658f-b2ff-4503-b449-da9f05bb9e41\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x_values\",\"y_values\"],\"data\":{\"chart_index\":[{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1}],\"label\":[-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],\"x_values\":{\"__ndarray__\":\"EFn1V1zjsz+T4iZSyoy7v3KoM6371sS/NcnDDPPQvr8nOh2MJjuxPznkQ2G3rcs/qXHUaVxs1D+3aFb3jDa0v4Qib8Y445q/6stLGjyoxT8HoULXab/Iv7JULHusw4u/HhXg2kd8yb8zGOJCJqPGPwkjqw8c/Ik/Qb1IqVa7zb9AWQl8r8rQPwHrx15xp7q/qYwbdaBhuz+brxPQtcyWPw==\",\"dtype\":\"float64\",\"shape\":[20]},\"y_values\":{\"__ndarray__\":\"nzQ6NY2csb9dBlRO+CzIv0bf/Nct07K/WKMDZt/Uiz9hDLZlJD2sPxXePrb2CKm/+0OS/1itvD9TkOOFjGGBP3jKKpiyv7u/hBleZVNSub9PwmCkUPXEv83Lh8OyQrK/CZfaDKuJkr/JyWwy5GXdP1BtuK0ZU8C/PZvJJbEZsj929V+qdkmZP52l6O3eeKG/gNTiJ7nF1b/Uwq5/kmiQPw==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"d167ab69-e329-4cc7-bc82-83cb4bed85e8\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"below\":[{\"id\":\"311c658f-b2ff-4503-b449-da9f05bb9e41\",\"type\":\"LinearAxis\"}],\"css_classes\":null,\"left\":[{\"id\":\"df9ecb4b-7429-4d8e-9e2f-caa872f8d3f2\",\"type\":\"LinearAxis\"}],\"renderers\":[{\"id\":\"57745f1f-e745-4995-aa9a-08fc17fb2892\",\"type\":\"BoxAnnotation\"},{\"id\":\"61390e73-9469-4031-ad5d-75bd7b5095df\",\"type\":\"GlyphRenderer\"},{\"id\":\"6485e97f-7ca7-45e7-8ae0-4e220b81428f\",\"type\":\"GlyphRenderer\"},{\"id\":\"a658039d-3bf6-412b-8fef-4dfc79305af2\",\"type\":\"Legend\"},{\"id\":\"311c658f-b2ff-4503-b449-da9f05bb9e41\",\"type\":\"LinearAxis\"},{\"id\":\"df9ecb4b-7429-4d8e-9e2f-caa872f8d3f2\",\"type\":\"LinearAxis\"},{\"id\":\"dd45bb19-736f-4d70-8f4b-46139e944c45\",\"type\":\"Grid\"},{\"id\":\"30f96faa-2794-445a-a872-e394b953310d\",\"type\":\"Grid\"}],\"title\":{\"id\":\"a4c4c436-b821-4ced-986a-fe5c1d66b7bc\",\"type\":\"Title\"},\"tool_events\":{\"id\":\"0bf89257-c455-4474-90ac-a06ef3cde431\",\"type\":\"ToolEvents\"},\"toolbar\":{\"id\":\"93daf0e2-b5be-4c58-b40a-0223865369cc\",\"type\":\"Toolbar\"},\"x_mapper_type\":\"auto\",\"x_range\":{\"id\":\"75e003ab-d87e-4cfa-9ef4-c8aa66d28eb0\",\"type\":\"Range1d\"},\"y_mapper_type\":\"auto\",\"y_range\":{\"id\":\"caa44eac-e873-48ba-b89d-5c202cfff1f8\",\"type\":\"Range1d\"}},\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"},{\"attributes\":{},\"id\":\"300dd613-fcd5-468f-98db-78fc31aa3615\",\"type\":\"BasicTicker\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"822f1f16-41e5-42bc-a5b6-266a6c4fff07\",\"type\":\"PanTool\"},{\"id\":\"2eabba23-dae1-44b1-a6d6-4c443d9b7c2d\",\"type\":\"WheelZoomTool\"},{\"id\":\"a4ef6852-8681-43ed-b28b-8815c6f60e20\",\"type\":\"BoxZoomTool\"},{\"id\":\"5cc25313-960c-4d00-a5e4-9804ae6ab438\",\"type\":\"SaveTool\"},{\"id\":\"223c3e4c-107b-46ec-bd0e-5b5004c427eb\",\"type\":\"ResetTool\"},{\"id\":\"a32bbcd7-a2a2-484d-aa49-24d4db603925\",\"type\":\"HelpTool\"}]},\"id\":\"93daf0e2-b5be-4c58-b40a-0223865369cc\",\"type\":\"Toolbar\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"value\":\"#5ab738\"},\"line_color\":{\"value\":\"#5ab738\"},\"size\":{\"units\":\"screen\",\"value\":8},\"x\":{\"field\":\"x_values\"},\"y\":{\"field\":\"y_values\"}},\"id\":\"6b20fc5b-7a38-4d10-84fe-ba2723cded54\",\"type\":\"Circle\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"cd378e89-5c23-4c97-b0d1-e4125c6e2965\",\"type\":\"BasicTicker\"}},\"id\":\"30f96faa-2794-445a-a872-e394b953310d\",\"type\":\"Grid\"},{\"attributes\":{\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"2eabba23-dae1-44b1-a6d6-4c443d9b7c2d\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"data_source\":{\"id\":\"d167ab69-e329-4cc7-bc82-83cb4bed85e8\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"7aff8156-1ecf-4c41-bc2a-28db02dca48e\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null},\"id\":\"6485e97f-7ca7-45e7-8ae0-4e220b81428f\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x_values\",\"y_values\"],\"data\":{\"chart_index\":[{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1}],\"label\":[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],\"x_values\":{\"__ndarray__\":\"hnf13dmLo7/ysxlkgGGvP7i+5FxEf82/6DVkOPFWwb83LRsasTHRP42uftJeWWu/ARUPQOWdxb/ieiSQvazhv+lkmxMoApO/1BunQgQGwD8QrXwO9emZv3M+irlrDJ4/Q/plPeHqk787sy4krZe5vwEDmMv/cL+/Y4Vk1tzz0j8bcBbpSr2eP5IhqCoWMKq/mqqG4FdC2T9nhLS7GgK7vwpce1RAOKY/9TO+XyeywD8=\",\"dtype\":\"float64\",\"shape\":[22]},\"y_values\":{\"__ndarray__\":\"AbvoibCVzL81Pg1bkgnQP7mkJo/SEb+/mgv+KMSbt79Tp7bDDIrMP7MGH8uhAtW/JADNGpJQ5D9VmOBOGazIPyCqqvEzYJq/MXydSU3bxb+qZG0kP+ShP7pVWvMETIm/u9ztnNw6vz/9qc3OSdzGP0ysQclZyby/LWfUCiSbxL/5fPiNugDEv2ie9ACRpKa/vkjT1XG7yz/QT5h7Z/qyP9iew/b3dLg/5uOFlTpQfj8=\",\"dtype\":\"float64\",\"shape\":[22]}}},\"id\":\"4509e26a-f8c6-4d42-bd25-a7a0d1cbb61c\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"bddd3fc2-bc1c-4e34-91b6-05f7ba33f6f6\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"822f1f16-41e5-42bc-a5b6-266a6c4fff07\",\"type\":\"PanTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"value\":\"#f22c40\"},\"line_color\":{\"value\":\"#f22c40\"},\"size\":{\"units\":\"screen\",\"value\":8},\"x\":{\"field\":\"x_values\"},\"y\":{\"field\":\"y_values\"}},\"id\":\"7aff8156-1ecf-4c41-bc2a-28db02dca48e\",\"type\":\"Circle\"},{\"attributes\":{\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"a32bbcd7-a2a2-484d-aa49-24d4db603925\",\"type\":\"HelpTool\"},{\"attributes\":{\"label\":{\"value\":\"1\"},\"renderers\":[{\"id\":\"61390e73-9469-4031-ad5d-75bd7b5095df\",\"type\":\"GlyphRenderer\"}]},\"id\":\"c8b2f469-91a5-4e2f-9c6a-5b40709a2531\",\"type\":\"LegendItem\"},{\"attributes\":{\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"5cc25313-960c-4d00-a5e4-9804ae6ab438\",\"type\":\"SaveTool\"},{\"attributes\":{\"items\":[{\"id\":\"c8b2f469-91a5-4e2f-9c6a-5b40709a2531\",\"type\":\"LegendItem\"},{\"id\":\"f362f200-3793-4270-b972-c3e77be47215\",\"type\":\"LegendItem\"}],\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"a658039d-3bf6-412b-8fef-4dfc79305af2\",\"type\":\"Legend\"},{\"attributes\":{\"label\":{\"value\":\"-1\"},\"renderers\":[{\"id\":\"6485e97f-7ca7-45e7-8ae0-4e220b81428f\",\"type\":\"GlyphRenderer\"}]},\"id\":\"f362f200-3793-4270-b972-c3e77be47215\",\"type\":\"LegendItem\"},{\"attributes\":{\"plot\":{\"id\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\",\"subtype\":\"Chart\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"300dd613-fcd5-468f-98db-78fc31aa3615\",\"type\":\"BasicTicker\"}},\"id\":\"dd45bb19-736f-4d70-8f4b-46139e944c45\",\"type\":\"Grid\"},{\"attributes\":{\"callback\":null,\"end\":0.7323381294965082,\"start\":-0.4376959067162286},\"id\":\"caa44eac-e873-48ba-b89d-5c202cfff1f8\",\"type\":\"Range1d\"},{\"attributes\":{},\"id\":\"0bf89257-c455-4474-90ac-a06ef3cde431\",\"type\":\"ToolEvents\"},{\"attributes\":{\"callback\":null,\"end\":0.4893753474644802,\"start\":-0.647037560088657},\"id\":\"75e003ab-d87e-4cfa-9ef4-c8aa66d28eb0\",\"type\":\"Range1d\"},{\"attributes\":{},\"id\":\"04c0ab92-66d0-4f17-940e-21ac144fb74d\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"data_source\":{\"id\":\"4509e26a-f8c6-4d42-bd25-a7a0d1cbb61c\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"6b20fc5b-7a38-4d10-84fe-ba2723cded54\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null},\"id\":\"61390e73-9469-4031-ad5d-75bd7b5095df\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"plot\":null,\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"57745f1f-e745-4995-aa9a-08fc17fb2892\",\"type\":\"BoxAnnotation\"}],\"root_ids\":[\"cec918c1-0aef-4386-a4ec-c51ee9dce278\"]},\"title\":\"Bokeh Application\",\"version\":\"0.12.5\"}};\n",
       "            var render_items = [{\"docid\":\"6c0d4d95-6409-46e8-9382-ea1dc49e8fb9\",\"elementid\":\"41017ada-fba1-484e-8035-cd4c4bd70848\",\"modelid\":\"cec918c1-0aef-4386-a4ec-c51ee9dce278\"}];\n",
       "            \n",
       "            Bokeh.embed.embed_items(docs_json, render_items);\n",
       "          };\n",
       "          if (document.readyState != \"loading\") fn();\n",
       "          else document.addEventListener(\"DOMContentLoaded\", fn);\n",
       "        })();\n",
       "      },\n",
       "      function(Bokeh) {\n",
       "      }\n",
       "    ];\n",
       "  \n",
       "    function run_inline_js() {\n",
       "      \n",
       "      if ((window.Bokeh !== undefined) || (force === true)) {\n",
       "        for (var i = 0; i < inline_js.length; i++) {\n",
       "          inline_js[i](window.Bokeh);\n",
       "        }if (force === true) {\n",
       "          display_loaded();\n",
       "        }} else if (Date.now() < window._bokeh_timeout) {\n",
       "        setTimeout(run_inline_js, 100);\n",
       "      } else if (!window._bokeh_failed_load) {\n",
       "        console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "        window._bokeh_failed_load = true;\n",
       "      } else if (force !== true) {\n",
       "        var cell = $(document.getElementById(\"41017ada-fba1-484e-8035-cd4c4bd70848\")).parents('.cell').data().cell;\n",
       "        cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "      }\n",
       "  \n",
       "    }\n",
       "  \n",
       "    if (window._bokeh_is_loading === 0) {\n",
       "      console.log(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "      run_inline_js();\n",
       "    } else {\n",
       "      load_libs(js_urls, function() {\n",
       "        console.log(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "        run_inline_js();\n",
       "      });\n",
       "    }\n",
       "  }(this));\n",
       "</script>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "body_cos_dist = 1 - cosine_similarity(body_data)\n",
    "\n",
    "mds = MDS(n_components=2, dissimilarity=\"precomputed\", random_state=1)\n",
    "\n",
    "pos = mds.fit_transform(body_cos_dist)# shape (n_components, n_samples)\n",
    "df = pd.DataFrame()\n",
    "df['x'] = pos[:, 0]\n",
    "df['y'] = pos[:, 1]\n",
    "df['label'] = dataset['Label']\n",
    "\n",
    "p = Scatter(df, x='x', y='y', title=\"MDS: White House Post Bodies\", color='label',\n",
    "           legend=\"top_right\")\n",
    "\n",
    "show(p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "    <div class=\"bk-root\">\n",
       "        <div class=\"bk-plotdiv\" id=\"38821ab0-a249-4641-a363-f1c2b715a011\"></div>\n",
       "    </div>\n",
       "<script type=\"text/javascript\">\n",
       "  \n",
       "  (function(global) {\n",
       "    function now() {\n",
       "      return new Date();\n",
       "    }\n",
       "  \n",
       "    var force = false;\n",
       "  \n",
       "    if (typeof (window._bokeh_onload_callbacks) === \"undefined\" || force === true) {\n",
       "      window._bokeh_onload_callbacks = [];\n",
       "      window._bokeh_is_loading = undefined;\n",
       "    }\n",
       "  \n",
       "  \n",
       "    \n",
       "    if (typeof (window._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "      window._bokeh_timeout = Date.now() + 0;\n",
       "      window._bokeh_failed_load = false;\n",
       "    }\n",
       "  \n",
       "    var NB_LOAD_WARNING = {'data': {'text/html':\n",
       "       \"<div style='background-color: #fdd'>\\n\"+\n",
       "       \"<p>\\n\"+\n",
       "       \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "       \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "       \"</p>\\n\"+\n",
       "       \"<ul>\\n\"+\n",
       "       \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "       \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "       \"</ul>\\n\"+\n",
       "       \"<code>\\n\"+\n",
       "       \"from bokeh.resources import INLINE\\n\"+\n",
       "       \"output_notebook(resources=INLINE)\\n\"+\n",
       "       \"</code>\\n\"+\n",
       "       \"</div>\"}};\n",
       "  \n",
       "    function display_loaded() {\n",
       "      if (window.Bokeh !== undefined) {\n",
       "        var el = document.getElementById(\"38821ab0-a249-4641-a363-f1c2b715a011\");\n",
       "        el.textContent = \"BokehJS \" + Bokeh.version + \" successfully loaded.\";\n",
       "      } else if (Date.now() < window._bokeh_timeout) {\n",
       "        setTimeout(display_loaded, 100)\n",
       "      }\n",
       "    }\n",
       "  \n",
       "    function run_callbacks() {\n",
       "      window._bokeh_onload_callbacks.forEach(function(callback) { callback() });\n",
       "      delete window._bokeh_onload_callbacks\n",
       "      console.info(\"Bokeh: all callbacks have finished\");\n",
       "    }\n",
       "  \n",
       "    function load_libs(js_urls, callback) {\n",
       "      window._bokeh_onload_callbacks.push(callback);\n",
       "      if (window._bokeh_is_loading > 0) {\n",
       "        console.log(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "        return null;\n",
       "      }\n",
       "      if (js_urls == null || js_urls.length === 0) {\n",
       "        run_callbacks();\n",
       "        return null;\n",
       "      }\n",
       "      console.log(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "      window._bokeh_is_loading = js_urls.length;\n",
       "      for (var i = 0; i < js_urls.length; i++) {\n",
       "        var url = js_urls[i];\n",
       "        var s = document.createElement('script');\n",
       "        s.src = url;\n",
       "        s.async = false;\n",
       "        s.onreadystatechange = s.onload = function() {\n",
       "          window._bokeh_is_loading--;\n",
       "          if (window._bokeh_is_loading === 0) {\n",
       "            console.log(\"Bokeh: all BokehJS libraries loaded\");\n",
       "            run_callbacks()\n",
       "          }\n",
       "        };\n",
       "        s.onerror = function() {\n",
       "          console.warn(\"failed to load library \" + url);\n",
       "        };\n",
       "        console.log(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "        document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "      }\n",
       "    };var element = document.getElementById(\"38821ab0-a249-4641-a363-f1c2b715a011\");\n",
       "    if (element == null) {\n",
       "      console.log(\"Bokeh: ERROR: autoload.js configured with elementid '38821ab0-a249-4641-a363-f1c2b715a011' but no matching script tag was found. \")\n",
       "      return false;\n",
       "    }\n",
       "  \n",
       "    var js_urls = [];\n",
       "  \n",
       "    var inline_js = [\n",
       "      function(Bokeh) {\n",
       "        (function() {\n",
       "          var fn = function() {\n",
       "            var docs_json = {\"1d620358-b6c4-451e-bf5a-3e6205877d3f\":{\"roots\":{\"references\":[{\"attributes\":{\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"0ecf71ec-a0ad-4219-b849-e84ac5b0bc88\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"a6ee46f9-cff1-4619-8906-d32aa88eb986\",\"type\":\"BasicTicker\"},{\"attributes\":{\"callback\":null,\"end\":0.6080082050994976,\"start\":-0.8005368289138298},\"id\":\"3f0b8019-50e3-4175-9bd3-41628d4ea501\",\"type\":\"Range1d\"},{\"attributes\":{\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"e7889707-3e6b-4cf3-a6e9-a9b4f034ac9d\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"fa800347-5a89-4a2a-9fa5-acf060ccdae1\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"axis_label\":\"y\",\"formatter\":{\"id\":\"e9e1f1d1-8e6b-4264-87d2-3f18c6b7d70c\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"13196520-e5bc-4cc4-8c6a-359fa7835778\",\"type\":\"BasicTicker\"}},\"id\":\"73f07536-9f9c-4205-bdcb-e793ff95e57a\",\"type\":\"LinearAxis\"},{\"attributes\":{\"overlay\":{\"id\":\"1c500e2d-062d-4c09-b0d3-0002aa2ab805\",\"type\":\"BoxAnnotation\"},\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"6e618b2b-281e-4b98-918e-df00ce10dc81\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"dc5091b1-1e5f-41e6-b429-2e6f1a2d6ce7\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"callback\":null,\"end\":0.5329114600609397,\"start\":-0.707600905654598},\"id\":\"a086c0a0-bc4d-4461-aa67-24d9263fe560\",\"type\":\"Range1d\"},{\"attributes\":{\"plot\":null,\"text\":\"MDS: White House Post Titles\"},\"id\":\"8e9a111d-252d-415d-a925-4e5384fcd311\",\"type\":\"Title\"},{\"attributes\":{\"data_source\":{\"id\":\"223d2848-3d04-413a-85f6-0a7996b0c036\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"9488ef1a-ff33-4368-960b-fa0e5efd5584\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null},\"id\":\"20b4a739-9ae4-4ced-9be4-695a6e694658\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x_values\",\"y_values\"],\"data\":{\"chart_index\":[{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1},{\"label\":1}],\"label\":[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],\"x_values\":{\"__ndarray__\":\"ZxRJDn01tL/1d88D4RrFvzfDKyZ+O9C/gb7aRh6qzz86lmBGEiXWv0AAgX1Bx8w/lQePzqFTyL/kX3LQ6x/DPwriskX4Gdq/BKdQi+W2vb/17+wkcPvLP7oK+q0kg8c/XYfRlUdFqT+YktMhviS+v70Wdu5jvdI/WtnwsXc9xb835eWXkauSP762vMOSesa//LOe8OED2D/p5ltbvQ26P6xlMsL6Q9S/lk2AIoDqob8=\",\"dtype\":\"float64\",\"shape\":[22]},\"y_values\":{\"__ndarray__\":\"hF+J5AhlzT/qa6N0QdzIP2pSwf0yy9C/NolPAj+L3b9w6fUELxPOvzWHglwAn9o/tktYU27c5b9MeKoGFwTWv2no9z2jfsC/IM41A+86dr9c+MYB09CuP3bTToKmxcE/4Z7Sny7ErD8rgVrSnbbTP1Boxs3I/MG/IfhV/THM2T/5svsSeWbfP/7sXR37MWY/zfz2vBiFwL/TvVEoTOG4v5G8TJTnd4e/lKx1fOeNo78=\",\"dtype\":\"float64\",\"shape\":[22]}}},\"id\":\"82c897a8-cb45-47eb-a69e-8a269dfdade7\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"items\":[{\"id\":\"23058ac2-a86b-4f70-9487-7d738a529941\",\"type\":\"LegendItem\"},{\"id\":\"9ca86661-ca22-4c29-ae60-066af0bc877e\",\"type\":\"LegendItem\"}],\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"e31f7964-910f-4fbb-b6f0-760312742e3c\",\"type\":\"Legend\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x_values\",\"y_values\"],\"data\":{\"chart_index\":[{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1},{\"label\":-1}],\"label\":[-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],\"x_values\":{\"__ndarray__\":\"pA4j5hn8tT8zqlGW6u/WP/3ZJCtzFJu/D1FO2L4brb91Nk6VcU6/v/VOGffbtdm/pTbKUpjytj9zk78YasDZPyp8V0LJT9E/44SiK4J92z8/w/B5YZ+Fv0qd+EqnltC/wUAA8thwwD99Qcpnz1Xjv0QyxzMsRac/r78vlf4UsL9nOyBzYpnOvxeWSLg+2tM/Dat0W/RAzD8EoeVdmhKqvw==\",\"dtype\":\"float64\",\"shape\":[20]},\"y_values\":{\"__ndarray__\":\"/hf5N9PZxj/1RWakQgLJP7BGKlZgv9a/WjpnEbA4pj/w3xHDBb69vw1JaVRLMM4/fxcjHx5G2D+kgFpGntCxP1pQWOrVp9E/Zxngx/AVab84mitOIVbPv5lL9y6XE7w/oDENjVjasr/KhJBQairAvyzGEdGClco/FDxFg9E20L+a0lGliODGPzM57tZaFdS/gCdqcxLr0L8EM5BtPUS9Pw==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"223d2848-3d04-413a-85f6-0a7996b0c036\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"label\":{\"value\":\"-1\"},\"renderers\":[{\"id\":\"20b4a739-9ae4-4ced-9be4-695a6e694658\",\"type\":\"GlyphRenderer\"}]},\"id\":\"9ca86661-ca22-4c29-ae60-066af0bc877e\",\"type\":\"LegendItem\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"value\":\"#f22c40\"},\"line_color\":{\"value\":\"#f22c40\"},\"size\":{\"units\":\"screen\",\"value\":8},\"x\":{\"field\":\"x_values\"},\"y\":{\"field\":\"y_values\"}},\"id\":\"9488ef1a-ff33-4368-960b-fa0e5efd5584\",\"type\":\"Circle\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"plot\":null,\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"1c500e2d-062d-4c09-b0d3-0002aa2ab805\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"13196520-e5bc-4cc4-8c6a-359fa7835778\",\"type\":\"BasicTicker\"},{\"attributes\":{\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"b4e23b53-c9ce-45cf-ad3c-47464fc0dc9d\",\"type\":\"HelpTool\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"value\":\"#5ab738\"},\"line_color\":{\"value\":\"#5ab738\"},\"size\":{\"units\":\"screen\",\"value\":8},\"x\":{\"field\":\"x_values\"},\"y\":{\"field\":\"y_values\"}},\"id\":\"d93e7260-f9fc-41f1-856a-1d66c1a65e06\",\"type\":\"Circle\"},{\"attributes\":{\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"}},\"id\":\"89300795-bcd4-4e6b-84fe-ab610e517062\",\"type\":\"SaveTool\"},{\"attributes\":{\"below\":[{\"id\":\"b5b9ec6b-50c7-4e6e-962f-b176ac4127b5\",\"type\":\"LinearAxis\"}],\"css_classes\":null,\"left\":[{\"id\":\"73f07536-9f9c-4205-bdcb-e793ff95e57a\",\"type\":\"LinearAxis\"}],\"renderers\":[{\"id\":\"1c500e2d-062d-4c09-b0d3-0002aa2ab805\",\"type\":\"BoxAnnotation\"},{\"id\":\"1546b4e2-a4ed-4e0d-8f56-d8ad836661cd\",\"type\":\"GlyphRenderer\"},{\"id\":\"20b4a739-9ae4-4ced-9be4-695a6e694658\",\"type\":\"GlyphRenderer\"},{\"id\":\"e31f7964-910f-4fbb-b6f0-760312742e3c\",\"type\":\"Legend\"},{\"id\":\"b5b9ec6b-50c7-4e6e-962f-b176ac4127b5\",\"type\":\"LinearAxis\"},{\"id\":\"73f07536-9f9c-4205-bdcb-e793ff95e57a\",\"type\":\"LinearAxis\"},{\"id\":\"4c43031e-83ef-45cc-8195-803859244cee\",\"type\":\"Grid\"},{\"id\":\"a4c31520-933b-477d-8e17-45b205a73ff4\",\"type\":\"Grid\"}],\"title\":{\"id\":\"8e9a111d-252d-415d-a925-4e5384fcd311\",\"type\":\"Title\"},\"tool_events\":{\"id\":\"bdd0ee5f-1e03-4ae4-aa76-cf4ab2c1489d\",\"type\":\"ToolEvents\"},\"toolbar\":{\"id\":\"0fb7a6af-5e27-424b-818b-24e43b23629f\",\"type\":\"Toolbar\"},\"x_mapper_type\":\"auto\",\"x_range\":{\"id\":\"a086c0a0-bc4d-4461-aa67-24d9263fe560\",\"type\":\"Range1d\"},\"y_mapper_type\":\"auto\",\"y_range\":{\"id\":\"3f0b8019-50e3-4175-9bd3-41628d4ea501\",\"type\":\"Range1d\"}},\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"},{\"attributes\":{\"label\":{\"value\":\"1\"},\"renderers\":[{\"id\":\"1546b4e2-a4ed-4e0d-8f56-d8ad836661cd\",\"type\":\"GlyphRenderer\"}]},\"id\":\"23058ac2-a86b-4f70-9487-7d738a529941\",\"type\":\"LegendItem\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"13196520-e5bc-4cc4-8c6a-359fa7835778\",\"type\":\"BasicTicker\"}},\"id\":\"a4c31520-933b-477d-8e17-45b205a73ff4\",\"type\":\"Grid\"},{\"attributes\":{\"data_source\":{\"id\":\"82c897a8-cb45-47eb-a69e-8a269dfdade7\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"d93e7260-f9fc-41f1-856a-1d66c1a65e06\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null},\"id\":\"1546b4e2-a4ed-4e0d-8f56-d8ad836661cd\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"0ecf71ec-a0ad-4219-b849-e84ac5b0bc88\",\"type\":\"PanTool\"},{\"id\":\"dc5091b1-1e5f-41e6-b429-2e6f1a2d6ce7\",\"type\":\"WheelZoomTool\"},{\"id\":\"6e618b2b-281e-4b98-918e-df00ce10dc81\",\"type\":\"BoxZoomTool\"},{\"id\":\"89300795-bcd4-4e6b-84fe-ab610e517062\",\"type\":\"SaveTool\"},{\"id\":\"e7889707-3e6b-4cf3-a6e9-a9b4f034ac9d\",\"type\":\"ResetTool\"},{\"id\":\"b4e23b53-c9ce-45cf-ad3c-47464fc0dc9d\",\"type\":\"HelpTool\"}]},\"id\":\"0fb7a6af-5e27-424b-818b-24e43b23629f\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"bdd0ee5f-1e03-4ae4-aa76-cf4ab2c1489d\",\"type\":\"ToolEvents\"},{\"attributes\":{\"axis_label\":\"x\",\"formatter\":{\"id\":\"fa800347-5a89-4a2a-9fa5-acf060ccdae1\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"a6ee46f9-cff1-4619-8906-d32aa88eb986\",\"type\":\"BasicTicker\"}},\"id\":\"b5b9ec6b-50c7-4e6e-962f-b176ac4127b5\",\"type\":\"LinearAxis\"},{\"attributes\":{\"plot\":{\"id\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\",\"subtype\":\"Chart\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"a6ee46f9-cff1-4619-8906-d32aa88eb986\",\"type\":\"BasicTicker\"}},\"id\":\"4c43031e-83ef-45cc-8195-803859244cee\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"e9e1f1d1-8e6b-4264-87d2-3f18c6b7d70c\",\"type\":\"BasicTickFormatter\"}],\"root_ids\":[\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\"]},\"title\":\"Bokeh Application\",\"version\":\"0.12.5\"}};\n",
       "            var render_items = [{\"docid\":\"1d620358-b6c4-451e-bf5a-3e6205877d3f\",\"elementid\":\"38821ab0-a249-4641-a363-f1c2b715a011\",\"modelid\":\"c90bbf19-8a0d-4705-9c2b-a5d8ea33ca6c\"}];\n",
       "            \n",
       "            Bokeh.embed.embed_items(docs_json, render_items);\n",
       "          };\n",
       "          if (document.readyState != \"loading\") fn();\n",
       "          else document.addEventListener(\"DOMContentLoaded\", fn);\n",
       "        })();\n",
       "      },\n",
       "      function(Bokeh) {\n",
       "      }\n",
       "    ];\n",
       "  \n",
       "    function run_inline_js() {\n",
       "      \n",
       "      if ((window.Bokeh !== undefined) || (force === true)) {\n",
       "        for (var i = 0; i < inline_js.length; i++) {\n",
       "          inline_js[i](window.Bokeh);\n",
       "        }if (force === true) {\n",
       "          display_loaded();\n",
       "        }} else if (Date.now() < window._bokeh_timeout) {\n",
       "        setTimeout(run_inline_js, 100);\n",
       "      } else if (!window._bokeh_failed_load) {\n",
       "        console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "        window._bokeh_failed_load = true;\n",
       "      } else if (force !== true) {\n",
       "        var cell = $(document.getElementById(\"38821ab0-a249-4641-a363-f1c2b715a011\")).parents('.cell').data().cell;\n",
       "        cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "      }\n",
       "  \n",
       "    }\n",
       "  \n",
       "    if (window._bokeh_is_loading === 0) {\n",
       "      console.log(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "      run_inline_js();\n",
       "    } else {\n",
       "      load_libs(js_urls, function() {\n",
       "        console.log(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "        run_inline_js();\n",
       "      });\n",
       "    }\n",
       "  }(this));\n",
       "</script>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "title_cos_dist = 1 - cosine_similarity(title_data)\n",
    "\n",
    "mds = MDS(n_components=2, dissimilarity=\"precomputed\", random_state=1)\n",
    "pos = mds.fit_transform(title_cos_dist)# shape (n_components, n_samples)\n",
    "df = pd.DataFrame()\n",
    "df['x'] = pos[:, 0]\n",
    "df['y'] = pos[:, 1]\n",
    "df['label'] = dataset['Label']\n",
    "\n",
    "p = Scatter(df, x='x', y='y', title=\"MDS: White House Post Titles\", color='label',\n",
    "           legend=\"top_right\")\n",
    "\n",
    "show(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discussion\n",
    "\n",
    "From our exploration, it seems as though a simple bag of words model over our text data is not enough to create a sufficient vector space given our labels. Ideally, our featurization would place the labeled data in a space where a statistical model can be trained to discriminate the labels with high accuracy. As seen in the plots above, our data points are only semi-clustered when looking at the blog post titles. However, from Jon's topic models, it does seem as though there is a relationship between posts about 'Regulation' and stock market movements. From his findings, it should be possible to find a vector space where we can easily train a predictive model to estimate stock market movement.\n",
    "\n",
    "Another problem that hinders us is the level of noise in the stock market itself. We would want to make sure that the movements that we look at are meaningful and not due to chance. That way when there is a significant movement after a blog post, we can more accurately say that there is a correlation between the type of content in the blog post and the movement the stock market makes. This assignment helped us stop and think more critically about our data collection process and analysis methods.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "\n",
    "- Label our data after denoising our stock data\n",
    "- Stem our text data\n",
    "- Look into combining the title and body of a posts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Appendix: Imports, Globals, and Helper Functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div class=\"bk-root\">\n",
       "        <a href=\"http://bokeh.pydata.org\" target=\"_blank\" class=\"bk-logo bk-logo-small bk-logo-notebook\"></a>\n",
       "        <span id=\"d281e57b-e53b-4cc1-8f39-d103f4ba055d\">Loading BokehJS ...</span>\n",
       "    </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(global) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  var force = true;\n",
       "\n",
       "  if (typeof (window._bokeh_onload_callbacks) === \"undefined\" || force === true) {\n",
       "    window._bokeh_onload_callbacks = [];\n",
       "    window._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "\n",
       "  \n",
       "  if (typeof (window._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "    window._bokeh_timeout = Date.now() + 5000;\n",
       "    window._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  var NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    if (window.Bokeh !== undefined) {\n",
       "      var el = document.getElementById(\"d281e57b-e53b-4cc1-8f39-d103f4ba055d\");\n",
       "      el.textContent = \"BokehJS \" + Bokeh.version + \" successfully loaded.\";\n",
       "    } else if (Date.now() < window._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function run_callbacks() {\n",
       "    window._bokeh_onload_callbacks.forEach(function(callback) { callback() });\n",
       "    delete window._bokeh_onload_callbacks\n",
       "    console.info(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(js_urls, callback) {\n",
       "    window._bokeh_onload_callbacks.push(callback);\n",
       "    if (window._bokeh_is_loading > 0) {\n",
       "      console.log(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.log(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    window._bokeh_is_loading = js_urls.length;\n",
       "    for (var i = 0; i < js_urls.length; i++) {\n",
       "      var url = js_urls[i];\n",
       "      var s = document.createElement('script');\n",
       "      s.src = url;\n",
       "      s.async = false;\n",
       "      s.onreadystatechange = s.onload = function() {\n",
       "        window._bokeh_is_loading--;\n",
       "        if (window._bokeh_is_loading === 0) {\n",
       "          console.log(\"Bokeh: all BokehJS libraries loaded\");\n",
       "          run_callbacks()\n",
       "        }\n",
       "      };\n",
       "      s.onerror = function() {\n",
       "        console.warn(\"failed to load library \" + url);\n",
       "      };\n",
       "      console.log(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "    }\n",
       "  };var element = document.getElementById(\"d281e57b-e53b-4cc1-8f39-d103f4ba055d\");\n",
       "  if (element == null) {\n",
       "    console.log(\"Bokeh: ERROR: autoload.js configured with elementid 'd281e57b-e53b-4cc1-8f39-d103f4ba055d' but no matching script tag was found. \")\n",
       "    return false;\n",
       "  }\n",
       "\n",
       "  var js_urls = [\"https://cdn.pydata.org/bokeh/release/bokeh-0.12.5.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.12.5.min.js\"];\n",
       "\n",
       "  var inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    \n",
       "    function(Bokeh) {\n",
       "      \n",
       "    },\n",
       "    \n",
       "    function(Bokeh) {\n",
       "      \n",
       "      document.getElementById(\"d281e57b-e53b-4cc1-8f39-d103f4ba055d\").textContent = \"BokehJS is loading...\";\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "      console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-0.12.5.min.css\");\n",
       "      Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-0.12.5.min.css\");\n",
       "      console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.12.5.min.css\");\n",
       "      Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.12.5.min.css\");\n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if ((window.Bokeh !== undefined) || (force === true)) {\n",
       "      for (var i = 0; i < inline_js.length; i++) {\n",
       "        inline_js[i](window.Bokeh);\n",
       "      }if (force === true) {\n",
       "        display_loaded();\n",
       "      }} else if (Date.now() < window._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!window._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      window._bokeh_failed_load = true;\n",
       "    } else if (force !== true) {\n",
       "      var cell = $(document.getElementById(\"d281e57b-e53b-4cc1-8f39-d103f4ba055d\")).parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (window._bokeh_is_loading === 0) {\n",
       "    console.log(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(js_urls, function() {\n",
       "      console.log(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(this));"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from re import sub\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "from nltk.corpus import stopwords\n",
    "\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.manifold import MDS\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "from sklearn.feature_selection import chi2\n",
    "\n",
    "from bokeh.io import output_notebook, show\n",
    "from bokeh.charts import Scatter\n",
    "\n",
    "output_notebook()\n",
    "\n",
    "\n",
    "stopwords = set(stopwords.words('english'))\n",
    "\n",
    "DATATYPES = {'Dow Jones Value': np.float32,\n",
    "            'Dow Jones Delta': np.float32,\n",
    "            'Dow Jones Proportion': np.float32,\n",
    "            'Nasdaq Value': np.float32,\n",
    "            'Nasdaq Delta': np.float32,\n",
    "            'Nasdaq Proportion': np.float32,\n",
    "            'S&P 500 Value': np.float32,\n",
    "            'S&P 500 Delta': np.float32,\n",
    "            'S&P 500 Proportion': np.float32 \n",
    "            }\n",
    "\n",
    "def join_blog_posts_on_date(df):\n",
    "    posts = {}\n",
    "    titles = {}\n",
    "    for index,row in df.iterrows():\n",
    "        date = row['Date']\n",
    "        if date not in posts:\n",
    "            posts[date] = []\n",
    "            titles[date] = []\n",
    "        posts[date].append(row['Body'])\n",
    "        titles[date].append(row['Title'])\n",
    "\n",
    "    posts = {date:' '.join(posts[date]) for date in posts}\n",
    "    titles = {date: ' '.join(titles[date]) for date in titles}\n",
    "\n",
    "    posts = pd.DataFrame(list(posts.items()))\n",
    "    posts.columns = ['Date', 'Body']\n",
    "\n",
    "    titles = pd.DataFrame(list(titles.items()))\n",
    "    titles.columns = ['Date', 'Title']\n",
    "\n",
    "    dj_deltas = df[['Date', 'Dow Jones Delta']].drop_duplicates()\n",
    "    nd_deltas = df[['Date', 'Nasdaq Delta']].drop_duplicates()\n",
    "    sp_deltas = df[['Date', 'S&P 500 Delta']].drop_duplicates()\n",
    "\n",
    "    dj_delta_prop = (df['Dow Jones Delta'] / df['Dow Jones Value']).drop_duplicates()\n",
    "    nd_delta_prop = (df['Nasdaq Delta'] / df['Nasdaq Value']).drop_duplicates()\n",
    "    sp_delta_prop = (df['S&P 500 Delta'] / df['S&P 500 Value']).drop_duplicates()\n",
    "\n",
    "    dj_deltas['Dow Jones Proportion'] = dj_delta_prop\n",
    "    nd_deltas['Nasdaq Proportion'] = nd_delta_prop\n",
    "    sp_deltas['S&P 500 Proportion'] = sp_delta_prop\n",
    "    \n",
    "    dataset = pd.merge(posts, titles, how='inner', on=['Date'])\n",
    "    dataset = pd.merge(dataset, dj_deltas, how='inner', on=['Date'])\n",
    "    dataset = pd.merge(dataset, nd_deltas, how='inner', on=['Date'])\n",
    "    dataset = pd.merge(dataset, sp_deltas, how='inner', on=['Date'])\n",
    "\n",
    "    dataset['Body'] = dataset['Body'].str.replace('\\d+', '').str.replace('[^a-zA-Z ]', '')\n",
    "    dataset['Title'] = dataset['Title'].str.replace('\\d+', '').str.replace('[^a-zA-Z ]', '')\n",
    "    dataset['Mean Delta'] = (dataset['Dow Jones Delta'] + dataset['Nasdaq Delta'] + dataset['S&P 500 Delta']) / 3\n",
    "    dataset['Mean Proportion'] = (dataset['Dow Jones Proportion'] + \n",
    "                                  dataset['Nasdaq Proportion'] + \n",
    "                                  dataset['S&P 500 Proportion']) / 3\n",
    "    dataset['Label'] = (dataset['Mean Proportion'] >= 0).apply(int)\n",
    "    dataset['Label'] = 2 * dataset['Label'] - 1\n",
    "    return dataset\n",
    "\n",
    "def generate_word_df(df, text='Body'):\n",
    "    grouped = df.groupby('Label')\n",
    "    joined_text = grouped[text].sum()\n",
    "    negative_text = joined_text[-1]\n",
    "    positive_text = joined_text[1]\n",
    "    \n",
    "    positive_text_words = count_words(positive_text)\n",
    "    negative_text_words = count_words(negative_text)\n",
    "    \n",
    "    positive_words = pd.DataFrame(list(positive_text_words.items()), \n",
    "                              columns=['Word', 'Positive']).sort_values(by='Positive',\n",
    "                                                                     ascending=False)\n",
    "    negative_words = pd.DataFrame(list(negative_text_words.items()), \n",
    "                              columns=['Word', 'Negative']).sort_values(by='Negative',\n",
    "                                                                     ascending=False)\n",
    "    \n",
    "    word_df = pd.merge(positive_words, negative_words, how='inner', on=['Word'])\n",
    "\n",
    "    total_positive = word_df['Positive'].sum()\n",
    "    total_negative = word_df['Negative'].sum()\n",
    "    word_df['Positive Proportion'] = word_df['Positive'] / total_positive\n",
    "    word_df['Negative Proportion'] = word_df['Negative'] / total_negative\n",
    "    word_df['diff'] = word_df['Positive'] - word_df['Negative']\n",
    "    word_df['diff proportion'] = word_df['Positive Proportion'] - word_df['Negative Proportion']\n",
    "    print(\"The total number of positive words is %d\" % word_df['Positive'].sum())\n",
    "    print(\"The total number of negative words is %d\" % word_df['Negative'].sum())\n",
    "    \n",
    "    word_df = word_df.round({'Positive Proportion':3, 'Negative Proportion':3, 'diff proportion':3})\n",
    "    word_df['abs diff proportion'] = word_df['diff proportion'].abs()\n",
    "    word_df = word_df.sort_values(by='abs diff proportion', ascending=False)\n",
    "    return word_df\n",
    "\n",
    "def count_words(text):\n",
    "    text = sub('[^a-zA-Z ]', '', text)\n",
    "    counts = {}\n",
    "    for word in text.lower().split():\n",
    "        if word in stopwords:\n",
    "            continue\n",
    "        if word not in counts:\n",
    "            counts[word] = 0\n",
    "        counts[word] += 1\n",
    "    return counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}