6. AutoEncoders and Embeddings/6.2 NLP and Deep Learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Natural Language Processing using Artificial Neural Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> “In God we trust. All others must bring data.” – W. Edwards Deming, statistician"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Word Embeddings\n",
    "\n",
    "### What?\n",
    "Convert words to vectors in a high dimensional space. Each dimension denotes an aspect like gender, type of object / word.\n",
    "\n",
    "\"Word embeddings\" are a family of natural language processing techniques aiming at mapping semantic meaning into a geometric space. This is done by associating a numeric vector to every word in a dictionary, such that the distance (e.g. L2 distance or more commonly cosine distance) between any two vectors would capture part of the semantic relationship between the two associated words. The geometric space formed by these vectors is called an embedding space.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why?\n",
    "By converting words to vectors we build relations between words. More similar the words in a dimension, more closer their scores are.\n",
    "\n",
    "### Example\n",
    "_W(green) = (1.2, 0.98, 0.05, ...)_\n",
    "\n",
    "_W(red) = (1.1, 0.2, 0.5, ...)_\n",
    "\n",
    "Here the vector values of _green_ and _red_ are very similar in one dimension because they both are colours. The value for second dimension is very different because red might be depicting something negative in the training data while green is used for positiveness.\n",
    "\n",
    "By vectorizing we are indirectly building different kind of relations between words."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example of `word2vec` using gensim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using gpu device 0: GeForce GTX 760 (CNMeM is enabled with initial size: 90.0% of memory, cuDNN 4007)\n"
     ]
    }
   ],
   "source": [
    "from gensim.models import word2vec\n",
    "from gensim.models.word2vec import Word2Vec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reading blog post from data directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "DATA_DIRECTORY = os.path.join(os.path.abspath(os.path.curdir), '..', \n",
    "                              'data', 'word_embeddings')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "male_posts = []\n",
    "female_post = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with open(os.path.join(DATA_DIRECTORY,\"male_blog_list.txt\"),\"rb\") as male_file:\n",
    "    male_posts= pickle.load(male_file)\n",
    "    \n",
    "with open(os.path.join(DATA_DIRECTORY,\"female_blog_list.txt\"),\"rb\") as female_file:\n",
    "    female_posts = pickle.load(female_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2252\n",
      "2611\n"
     ]
    }
   ],
   "source": [
    "print(len(female_posts))\n",
    "print(len(male_posts))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "filtered_male_posts = list(filter(lambda p: len(p) > 0, male_posts))\n",
    "filtered_female_posts = list(filter(lambda p: len(p) > 0, female_posts))\n",
    "posts = filtered_female_posts + filtered_male_posts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2247 2595 4842\n"
     ]
    }
   ],
   "source": [
    "print(len(filtered_female_posts), len(filtered_male_posts), len(posts))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Word2Vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "w2v = Word2Vec(size=200, min_count=1)\n",
    "w2v.build_vocab(map(lambda x: x.split(), posts[:100]), )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'see.': <gensim.models.word2vec.Vocab at 0x7f61aa4f1908>,\n",
       " 'never.': <gensim.models.word2vec.Vocab at 0x7f61aa4f1dd8>,\n",
       " 'driving': <gensim.models.word2vec.Vocab at 0x7f61aa4f1e48>,\n",
       " 'buddy': <gensim.models.word2vec.Vocab at 0x7f61aa4f0240>,\n",
       " 'DEFENSE': <gensim.models.word2vec.Vocab at 0x7f61aa4f0438>,\n",
       " 'interval': <gensim.models.word2vec.Vocab at 0x7f61aa4f04e0>,\n",
       " 'Right': <gensim.models.word2vec.Vocab at 0x7f61aa4f06a0>,\n",
       " 'minds,': <gensim.models.word2vec.Vocab at 0x7f61aa4f06d8>,\n",
       " 'earth.': <gensim.models.word2vec.Vocab at 0x7f61aa4f0710>,\n",
       " 'pleasure': <gensim.models.word2vec.Vocab at 0x7f61aa4f08d0>,\n",
       " 'school,': <gensim.models.word2vec.Vocab at 0x7f61aa4f0cc0>,\n",
       " 'someone': <gensim.models.word2vec.Vocab at 0x7f61aa4f0ef0>,\n",
       " 'dangit...': <gensim.models.word2vec.Vocab at 0x7f61aa4f23c8>,\n",
       " 'one!': <gensim.models.word2vec.Vocab at 0x7f61aa4f2c88>,\n",
       " 'hard.': <gensim.models.word2vec.Vocab at 0x7f61aa4e25c0>,\n",
       " 'programs,': <gensim.models.word2vec.Vocab at 0x7f61aa4e27b8>,\n",
       " 'SEEEENNNIIIOOORS!!!': <gensim.models.word2vec.Vocab at 0x7f61aa4e27f0>,\n",
       " 'two)': <gensim.models.word2vec.Vocab at 0x7f61aa4e2828>,\n",
       " \"o'\": <gensim.models.word2vec.Vocab at 0x7f61aa4e28d0>,\n",
       " '--': <gensim.models.word2vec.Vocab at 0x7f61aa4e2a58>,\n",
       " 'this-actually': <gensim.models.word2vec.Vocab at 0x7f61aa4e2b70>,\n",
       " 'swimming.': <gensim.models.word2vec.Vocab at 0x7f61aa4e2c50>,\n",
       " 'people.': <gensim.models.word2vec.Vocab at 0x7f61aa4e2cc0>,\n",
       " 'turn': <gensim.models.word2vec.Vocab at 0x7f61aa4e2e48>,\n",
       " 'happened': <gensim.models.word2vec.Vocab at 0x7f61aa4e2fd0>,\n",
       " 'clothing:': <gensim.models.word2vec.Vocab at 0x7f61aa4e22e8>,\n",
       " 'it!': <gensim.models.word2vec.Vocab at 0x7f61aa4e2048>,\n",
       " 'church': <gensim.models.word2vec.Vocab at 0x7f61aa4e21d0>,\n",
       " 'boring.': <gensim.models.word2vec.Vocab at 0x7f61aa4e2240>,\n",
       " 'freaky': <gensim.models.word2vec.Vocab at 0x7f61aa4ea278>,\n",
       " 'Democrats,': <gensim.models.word2vec.Vocab at 0x7f61aa4ea320>,\n",
       " '*kick': <gensim.models.word2vec.Vocab at 0x7f61aa4ea358>,\n",
       " '\"It': <gensim.models.word2vec.Vocab at 0x7f61aa4ea550>,\n",
       " 'wet': <gensim.models.word2vec.Vocab at 0x7f61aa4ea6d8>,\n",
       " 'snooze': <gensim.models.word2vec.Vocab at 0x7f61aa4ea7b8>,\n",
       " 'points': <gensim.models.word2vec.Vocab at 0x7f61aa4ea978>,\n",
       " 'Sen.': <gensim.models.word2vec.Vocab at 0x7f61aa4ea9b0>,\n",
       " 'although': <gensim.models.word2vec.Vocab at 0x7f61aa4eaac8>,\n",
       " 'Charlotte': <gensim.models.word2vec.Vocab at 0x7f61aa4eab00>,\n",
       " 'lil...but': <gensim.models.word2vec.Vocab at 0x7f61aa4eab38>,\n",
       " 'oneo': <gensim.models.word2vec.Vocab at 0x7f61aa4eac50>,\n",
       " 'course;': <gensim.models.word2vec.Vocab at 0x7f61aa4eada0>,\n",
       " 'Bring': <gensim.models.word2vec.Vocab at 0x7f61aa4eadd8>,\n",
       " '(compared': <gensim.models.word2vec.Vocab at 0x7f61aa4eae48>,\n",
       " 'ugh.': <gensim.models.word2vec.Vocab at 0x7f61aa4eaef0>,\n",
       " 'sit': <gensim.models.word2vec.Vocab at 0x7f61aa553a20>,\n",
       " 'dipped?': <gensim.models.word2vec.Vocab at 0x7f61aa4eafd0>,\n",
       " 'based': <gensim.models.word2vec.Vocab at 0x7f61aa4ec978>,\n",
       " 'A.I.': <gensim.models.word2vec.Vocab at 0x7f61aa4ec080>,\n",
       " 'breathing.': <gensim.models.word2vec.Vocab at 0x7f61aa4ec128>,\n",
       " 'multi-millionaire': <gensim.models.word2vec.Vocab at 0x7f61aa4ec208>,\n",
       " 'groups': <gensim.models.word2vec.Vocab at 0x7f61aa4ec278>,\n",
       " 'on': <gensim.models.word2vec.Vocab at 0x7f61aa4ec2b0>,\n",
       " 'animals),': <gensim.models.word2vec.Vocab at 0x7f61aa4d8630>,\n",
       " 'Manners?': <gensim.models.word2vec.Vocab at 0x7f61aa4ec320>,\n",
       " 'you?]:': <gensim.models.word2vec.Vocab at 0x7f61aa445f60>,\n",
       " 'redistribute': <gensim.models.word2vec.Vocab at 0x7f61aa4dbba8>,\n",
       " 'omg.': <gensim.models.word2vec.Vocab at 0x7f61aa4ec470>,\n",
       " 'dance?:': <gensim.models.word2vec.Vocab at 0x7f61aa4ec4a8>,\n",
       " 'Canada)': <gensim.models.word2vec.Vocab at 0x7f61aa553b00>,\n",
       " 'came': <gensim.models.word2vec.Vocab at 0x7f61aa4ec550>,\n",
       " 'poof': <gensim.models.word2vec.Vocab at 0x7f61aa4ec588>,\n",
       " 'brownies.': <gensim.models.word2vec.Vocab at 0x7f61aa4ec630>,\n",
       " 'Not': <gensim.models.word2vec.Vocab at 0x7f61aa4ec710>,\n",
       " 'spaces': <gensim.models.word2vec.Vocab at 0x7f61aa4ec780>,\n",
       " 'destroy': <gensim.models.word2vec.Vocab at 0x7f61aa4ec860>,\n",
       " 'maybe.': <gensim.models.word2vec.Vocab at 0x7f61aa4ec898>,\n",
       " 'Industrial': <gensim.models.word2vec.Vocab at 0x7f61aa4ec9e8>,\n",
       " 'boring': <gensim.models.word2vec.Vocab at 0x7f61aa4ecb00>,\n",
       " 'is:': <gensim.models.word2vec.Vocab at 0x7f61aa4ecd30>,\n",
       " 'question.': <gensim.models.word2vec.Vocab at 0x7f61aa4ecd68>,\n",
       " 'long-lasting': <gensim.models.word2vec.Vocab at 0x7f61aa4ecda0>,\n",
       " 'sun': <gensim.models.word2vec.Vocab at 0x7f61aa5dc1d0>,\n",
       " 'CrAp*': <gensim.models.word2vec.Vocab at 0x7f61aa4ed080>,\n",
       " 'irresistable': <gensim.models.word2vec.Vocab at 0x7f61aa4ed0f0>,\n",
       " 'dont...i': <gensim.models.word2vec.Vocab at 0x7f61aa4ed128>,\n",
       " 'loss.': <gensim.models.word2vec.Vocab at 0x7f61aa4ed160>,\n",
       " 'easy': <gensim.models.word2vec.Vocab at 0x7f61aa4ed2b0>,\n",
       " 'wanna': <gensim.models.word2vec.Vocab at 0x7f61aa4635c0>,\n",
       " 'Gaviota': <gensim.models.word2vec.Vocab at 0x7f61aa4ed4a8>,\n",
       " 'nose': <gensim.models.word2vec.Vocab at 0x7f61aa4ed518>,\n",
       " 'slept': <gensim.models.word2vec.Vocab at 0x7f61aa4ed5c0>,\n",
       " 'hahahahah': <gensim.models.word2vec.Vocab at 0x7f61aa4ed5f8>,\n",
       " 'halloween': <gensim.models.word2vec.Vocab at 0x7f61aa4ed630>,\n",
       " 'shes': <gensim.models.word2vec.Vocab at 0x7f61aa553c50>,\n",
       " 'realize': <gensim.models.word2vec.Vocab at 0x7f61aa4ed860>,\n",
       " 'twice': <gensim.models.word2vec.Vocab at 0x7f61aa4ed908>,\n",
       " 'lift': <gensim.models.word2vec.Vocab at 0x7f61aa4eda90>,\n",
       " 'china,': <gensim.models.word2vec.Vocab at 0x7f61aa4edc88>,\n",
       " 'Standard.)': <gensim.models.word2vec.Vocab at 0x7f61aa4edcc0>,\n",
       " 'worried': <gensim.models.word2vec.Vocab at 0x7f61aa4edda0>,\n",
       " 'Opposite': <gensim.models.word2vec.Vocab at 0x7f61aa4eddd8>,\n",
       " 'chin.': <gensim.models.word2vec.Vocab at 0x7f61aa4edef0>,\n",
       " 'Garden': <gensim.models.word2vec.Vocab at 0x7f61aa4ebcc0>,\n",
       " 'guy': <gensim.models.word2vec.Vocab at 0x7f61aa4ebd68>,\n",
       " 'remmeber': <gensim.models.word2vec.Vocab at 0x7f61aa4ebef0>,\n",
       " 'fence,': <gensim.models.word2vec.Vocab at 0x7f61aa4eb128>,\n",
       " 'apologizing': <gensim.models.word2vec.Vocab at 0x7f61aa4eb160>,\n",
       " 'next.': <gensim.models.word2vec.Vocab at 0x7f61aa4eb2b0>,\n",
       " 'MATTERS': <gensim.models.word2vec.Vocab at 0x7f61aa4eb2e8>,\n",
       " 'rugs': <gensim.models.word2vec.Vocab at 0x7f61aa4eb320>,\n",
       " 'her...': <gensim.models.word2vec.Vocab at 0x7f61aa4eb438>,\n",
       " 'energy,': <gensim.models.word2vec.Vocab at 0x7f61aa4eb4a8>,\n",
       " 'recorded,': <gensim.models.word2vec.Vocab at 0x7f61aa4eb588>,\n",
       " 'pepsi.': <gensim.models.word2vec.Vocab at 0x7f61aa4eb710>,\n",
       " 'r': <gensim.models.word2vec.Vocab at 0x7f61aa4eb860>,\n",
       " '13': <gensim.models.word2vec.Vocab at 0x7f61aa4eb898>,\n",
       " 'at:': <gensim.models.word2vec.Vocab at 0x7f61aa5dc390>,\n",
       " 'cheaper': <gensim.models.word2vec.Vocab at 0x7f61aa4ee9b0>,\n",
       " 'children!': <gensim.models.word2vec.Vocab at 0x7f61aa5b6c88>,\n",
       " 'tree': <gensim.models.word2vec.Vocab at 0x7f61aa4eecc0>,\n",
       " 'met': <gensim.models.word2vec.Vocab at 0x7f61aa4eecf8>,\n",
       " 'one,': <gensim.models.word2vec.Vocab at 0x7f61aa4eeda0>,\n",
       " 'rejected?': <gensim.models.word2vec.Vocab at 0x7f61aa4eee48>,\n",
       " 'Marianne’s': <gensim.models.word2vec.Vocab at 0x7f61aa4eee80>,\n",
       " 'Icenhower': <gensim.models.word2vec.Vocab at 0x7f61aa4ee978>,\n",
       " 'day!': <gensim.models.word2vec.Vocab at 0x7f61aa4ee1d0>,\n",
       " 'leaving': <gensim.models.word2vec.Vocab at 0x7f61aa4ee240>,\n",
       " '2110': <gensim.models.word2vec.Vocab at 0x7f61aa4ee2b0>,\n",
       " 'kiss:': <gensim.models.word2vec.Vocab at 0x7f61aa4ee748>,\n",
       " 'nearest': <gensim.models.word2vec.Vocab at 0x7f61aa4ee780>,\n",
       " 'aimlessly': <gensim.models.word2vec.Vocab at 0x7f61aa4ee7b8>,\n",
       " 'sprint': <gensim.models.word2vec.Vocab at 0x7f61aa4ee898>,\n",
       " 'kids!)': <gensim.models.word2vec.Vocab at 0x7f61aa536048>,\n",
       " 'canteen': <gensim.models.word2vec.Vocab at 0x7f61aa5360f0>,\n",
       " 'weekend!': <gensim.models.word2vec.Vocab at 0x7f61aa536160>,\n",
       " 'him': <gensim.models.word2vec.Vocab at 0x7f61aa536198>,\n",
       " 'scariest': <gensim.models.word2vec.Vocab at 0x7f61aa5361d0>,\n",
       " 'this?': <gensim.models.word2vec.Vocab at 0x7f61aa536208>,\n",
       " '\"choosing': <gensim.models.word2vec.Vocab at 0x7f61aa536240>,\n",
       " 'Talk': <gensim.models.word2vec.Vocab at 0x7f61aa5362b0>,\n",
       " 'weeks': <gensim.models.word2vec.Vocab at 0x7f61aa5362e8>,\n",
       " \"You'll\": <gensim.models.word2vec.Vocab at 0x7f61aa536390>,\n",
       " 'goodnight': <gensim.models.word2vec.Vocab at 0x7f61aa5363c8>,\n",
       " 'skiing.': <gensim.models.word2vec.Vocab at 0x7f61aa536438>,\n",
       " 'KeEp': <gensim.models.word2vec.Vocab at 0x7f61aa5535f8>,\n",
       " 'week': <gensim.models.word2vec.Vocab at 0x7f61aa536550>,\n",
       " 'norwegian': <gensim.models.word2vec.Vocab at 0x7f61aa5366a0>,\n",
       " 'HAND:': <gensim.models.word2vec.Vocab at 0x7f61aa553780>,\n",
       " 'fact,': <gensim.models.word2vec.Vocab at 0x7f61aa5367f0>,\n",
       " 'thanksgiving': <gensim.models.word2vec.Vocab at 0x7f61aa536828>,\n",
       " 'me..argh...': <gensim.models.word2vec.Vocab at 0x7f61aa536860>,\n",
       " 'she': <gensim.models.word2vec.Vocab at 0x7f61aa536898>,\n",
       " 'Tree': <gensim.models.word2vec.Vocab at 0x7f61aa536908>,\n",
       " 'combat.': <gensim.models.word2vec.Vocab at 0x7f61aa536940>,\n",
       " 'mitosis': <gensim.models.word2vec.Vocab at 0x7f61aa536978>,\n",
       " 'offered': <gensim.models.word2vec.Vocab at 0x7f61aa5369e8>,\n",
       " 'no..': <gensim.models.word2vec.Vocab at 0x7f61aa536a20>,\n",
       " '(there': <gensim.models.word2vec.Vocab at 0x7f61aa536a58>,\n",
       " 'aspirations': <gensim.models.word2vec.Vocab at 0x7f61aa536a90>,\n",
       " 'page': <gensim.models.word2vec.Vocab at 0x7f61aa536ac8>,\n",
       " 'Least': <gensim.models.word2vec.Vocab at 0x7f61aa536b38>,\n",
       " 'each': <gensim.models.word2vec.Vocab at 0x7f61aa536b70>,\n",
       " 'ride...': <gensim.models.word2vec.Vocab at 0x7f61aa536ba8>,\n",
       " 'doesn’t': <gensim.models.word2vec.Vocab at 0x7f61aa536c18>,\n",
       " 'FUCK': <gensim.models.word2vec.Vocab at 0x7f61aa536c50>,\n",
       " 'gona': <gensim.models.word2vec.Vocab at 0x7f61aa536dd8>,\n",
       " 'window': <gensim.models.word2vec.Vocab at 0x7f61aa536e10>,\n",
       " 'end': <gensim.models.word2vec.Vocab at 0x7f61aa536e48>,\n",
       " 'expected': <gensim.models.word2vec.Vocab at 0x7f61aa536eb8>,\n",
       " 'well.': <gensim.models.word2vec.Vocab at 0x7f61aa536ef0>,\n",
       " 'called': <gensim.models.word2vec.Vocab at 0x7f61aa460748>,\n",
       " \"needn't\": <gensim.models.word2vec.Vocab at 0x7f61aa536f28>,\n",
       " 'doesnt': <gensim.models.word2vec.Vocab at 0x7f61aa536f60>,\n",
       " 'venturing': <gensim.models.word2vec.Vocab at 0x7f61aa440a90>,\n",
       " 'alex': <gensim.models.word2vec.Vocab at 0x7f61aa536fd0>,\n",
       " 'here:': <gensim.models.word2vec.Vocab at 0x7f61aa53b048>,\n",
       " 'ewWw': <gensim.models.word2vec.Vocab at 0x7f61aa53b0b8>,\n",
       " 'pole?': <gensim.models.word2vec.Vocab at 0x7f61aa53b0f0>,\n",
       " 'melody,': <gensim.models.word2vec.Vocab at 0x7f61aa5b6eb8>,\n",
       " 'motivated': <gensim.models.word2vec.Vocab at 0x7f61aa53b128>,\n",
       " 'Well,': <gensim.models.word2vec.Vocab at 0x7f61aa53b160>,\n",
       " 'says:': <gensim.models.word2vec.Vocab at 0x7f61aa53b198>,\n",
       " 'worm': <gensim.models.word2vec.Vocab at 0x7f61aa53b1d0>,\n",
       " '[some': <gensim.models.word2vec.Vocab at 0x7f61aa553f98>,\n",
       " 'name': <gensim.models.word2vec.Vocab at 0x7f61aa53b320>,\n",
       " 'Leave\"': <gensim.models.word2vec.Vocab at 0x7f61aa53b358>,\n",
       " '4th': <gensim.models.word2vec.Vocab at 0x7f61aa404ba8>,\n",
       " \"It's...\": <gensim.models.word2vec.Vocab at 0x7f61aa53b390>,\n",
       " 'problem??': <gensim.models.word2vec.Vocab at 0x7f61aa553fd0>,\n",
       " 'remember': <gensim.models.word2vec.Vocab at 0x7f61aa53b470>,\n",
       " 'o': <gensim.models.word2vec.Vocab at 0x7f61aa4e32b0>,\n",
       " 'letters.': <gensim.models.word2vec.Vocab at 0x7f61aa53b4a8>,\n",
       " 'jean': <gensim.models.word2vec.Vocab at 0x7f61aa53b4e0>,\n",
       " 'thing.': <gensim.models.word2vec.Vocab at 0x7f61aa53b518>,\n",
       " 'friend?]:': <gensim.models.word2vec.Vocab at 0x7f61aa53b588>,\n",
       " 'am!': <gensim.models.word2vec.Vocab at 0x7f61aa53b5c0>,\n",
       " 'side...': <gensim.models.word2vec.Vocab at 0x7f61aa53b6a0>,\n",
       " 'Yet': <gensim.models.word2vec.Vocab at 0x7f61aa53b6d8>,\n",
       " 'easier': <gensim.models.word2vec.Vocab at 0x7f61aa53b828>,\n",
       " 'babies': <gensim.models.word2vec.Vocab at 0x7f61aa53b860>,\n",
       " 'You?': <gensim.models.word2vec.Vocab at 0x7f61aa53b898>,\n",
       " 'wedding:': <gensim.models.word2vec.Vocab at 0x7f61aa53b8d0>,\n",
       " '2.)': <gensim.models.word2vec.Vocab at 0x7f61aa53b908>,\n",
       " 'first...then': <gensim.models.word2vec.Vocab at 0x7f61aa53b940>,\n",
       " 'LA:': <gensim.models.word2vec.Vocab at 0x7f61aa53b978>,\n",
       " 'but,)': <gensim.models.word2vec.Vocab at 0x7f61aa53b9b0>,\n",
       " 'not,': <gensim.models.word2vec.Vocab at 0x7f61aa53ba20>,\n",
       " 'possession': <gensim.models.word2vec.Vocab at 0x7f61aa53ba58>,\n",
       " 'its': <gensim.models.word2vec.Vocab at 0x7f61aa53ba90>,\n",
       " 'stop': <gensim.models.word2vec.Vocab at 0x7f61aa53bac8>,\n",
       " 'Thanks': <gensim.models.word2vec.Vocab at 0x7f61aa53bb00>,\n",
       " 'durin': <gensim.models.word2vec.Vocab at 0x7f61aa53bb38>,\n",
       " 'rings': <gensim.models.word2vec.Vocab at 0x7f61aa53bb70>,\n",
       " 'Specifics': <gensim.models.word2vec.Vocab at 0x7f61aa53bba8>,\n",
       " 'http://www.kingsofchaos.com/recruit.php?uniqid=jm8bja2z': <gensim.models.word2vec.Vocab at 0x7f61aa53bbe0>,\n",
       " 'lace': <gensim.models.word2vec.Vocab at 0x7f61aa53bc18>,\n",
       " 'pretended': <gensim.models.word2vec.Vocab at 0x7f61aa53bc50>,\n",
       " 'clothes': <gensim.models.word2vec.Vocab at 0x7f61aa53bd30>,\n",
       " 'wong': <gensim.models.word2vec.Vocab at 0x7f61aa53bd68>,\n",
       " '38': <gensim.models.word2vec.Vocab at 0x7f61aa5ce390>,\n",
       " 'country.': <gensim.models.word2vec.Vocab at 0x7f61aa53bda0>,\n",
       " 'criticism': <gensim.models.word2vec.Vocab at 0x7f61aa53bdd8>,\n",
       " 'NATIONAL': <gensim.models.word2vec.Vocab at 0x7f61aa53be48>,\n",
       " \"that's\": <gensim.models.word2vec.Vocab at 0x7f61aa53beb8>,\n",
       " 'conclusively': <gensim.models.word2vec.Vocab at 0x7f61aa53bef0>,\n",
       " 'cartoons,': <gensim.models.word2vec.Vocab at 0x7f61aa53bf28>,\n",
       " 'chest/lungs': <gensim.models.word2vec.Vocab at 0x7f61aa53bf60>,\n",
       " 'whilst': <gensim.models.word2vec.Vocab at 0x7f61aa5dc7b8>,\n",
       " \"I'm,\": <gensim.models.word2vec.Vocab at 0x7f61aa3feb38>,\n",
       " 'Tata.': <gensim.models.word2vec.Vocab at 0x7f61aa53bfd0>,\n",
       " 'mix': <gensim.models.word2vec.Vocab at 0x7f61aa533160>,\n",
       " 'popularity': <gensim.models.word2vec.Vocab at 0x7f61aa533390>,\n",
       " 'park)': <gensim.models.word2vec.Vocab at 0x7f61aa5333c8>,\n",
       " '(trampled': <gensim.models.word2vec.Vocab at 0x7f61aa5336a0>,\n",
       " 'reminded': <gensim.models.word2vec.Vocab at 0x7f61aa5339b0>,\n",
       " 'says.': <gensim.models.word2vec.Vocab at 0x7f61aa533a58>,\n",
       " 'repetition,': <gensim.models.word2vec.Vocab at 0x7f61aa533ac8>,\n",
       " 'Size?': <gensim.models.word2vec.Vocab at 0x7f61aa533c18>,\n",
       " \"hm...i'm\": <gensim.models.word2vec.Vocab at 0x7f61aa533e10>,\n",
       " 'interesting,': <gensim.models.word2vec.Vocab at 0x7f61aa560160>,\n",
       " 'exams': <gensim.models.word2vec.Vocab at 0x7f61aa533f28>,\n",
       " 'crusts.': <gensim.models.word2vec.Vocab at 0x7f61aa533f60>,\n",
       " 'filling': <gensim.models.word2vec.Vocab at 0x7f61aa533fd0>,\n",
       " 'gets': <gensim.models.word2vec.Vocab at 0x7f61aa4e51d0>,\n",
       " 'his': <gensim.models.word2vec.Vocab at 0x7f61aa4e5208>,\n",
       " 'Friday,': <gensim.models.word2vec.Vocab at 0x7f61aa4e5240>,\n",
       " 'f': <gensim.models.word2vec.Vocab at 0x7f61aa4e5278>,\n",
       " 'too!': <gensim.models.word2vec.Vocab at 0x7f61aa4e52e8>,\n",
       " 'Made': <gensim.models.word2vec.Vocab at 0x7f61aa4e5400>,\n",
       " 'accidentally': <gensim.models.word2vec.Vocab at 0x7f61aa4e5438>,\n",
       " '\"New': <gensim.models.word2vec.Vocab at 0x7f61aa4e5470>,\n",
       " 'COURSE.': <gensim.models.word2vec.Vocab at 0x7f61aa4e54a8>,\n",
       " '[please': <gensim.models.word2vec.Vocab at 0x7f61aa572240>,\n",
       " 'this...': <gensim.models.word2vec.Vocab at 0x7f61aa4e5630>,\n",
       " 'soon': <gensim.models.word2vec.Vocab at 0x7f61aa4e5710>,\n",
       " 'worry': <gensim.models.word2vec.Vocab at 0x7f61aa4e57b8>,\n",
       " 'Job]:': <gensim.models.word2vec.Vocab at 0x7f61aa4e58d0>,\n",
       " 'deal': <gensim.models.word2vec.Vocab at 0x7f61aa4e59b0>,\n",
       " 'pounding': <gensim.models.word2vec.Vocab at 0x7f61aa4e59e8>,\n",
       " '[Are': <gensim.models.word2vec.Vocab at 0x7f61aa4e5a90>,\n",
       " 'begin': <gensim.models.word2vec.Vocab at 0x7f61aa4e5b00>,\n",
       " 'isolated': <gensim.models.word2vec.Vocab at 0x7f61aa4e5c18>,\n",
       " 'anyways': <gensim.models.word2vec.Vocab at 0x7f61aa4e5c50>,\n",
       " 'garbage': <gensim.models.word2vec.Vocab at 0x7f61aa4e5c88>,\n",
       " 'awww': <gensim.models.word2vec.Vocab at 0x7f61aa4e5cf8>,\n",
       " 'intelligence': <gensim.models.word2vec.Vocab at 0x7f61aa4e5d68>,\n",
       " 'being': <gensim.models.word2vec.Vocab at 0x7f61aa4e5e48>,\n",
       " 'married?]:': <gensim.models.word2vec.Vocab at 0x7f61aa4e5eb8>,\n",
       " 'omg': <gensim.models.word2vec.Vocab at 0x7f61aa440dd8>,\n",
       " '...': <gensim.models.word2vec.Vocab at 0x7f61aa4e5f28>,\n",
       " 'highlight': <gensim.models.word2vec.Vocab at 0x7f61aa4e5fd0>,\n",
       " 'to': <gensim.models.word2vec.Vocab at 0x7f61aa4e8978>,\n",
       " 'AHH': <gensim.models.word2vec.Vocab at 0x7f61aa4e8b38>,\n",
       " 'OVER!!!!!!!!!': <gensim.models.word2vec.Vocab at 0x7f61aa4e8b70>,\n",
       " 'Cried': <gensim.models.word2vec.Vocab at 0x7f61aa4e8c18>,\n",
       " 'SAYING?!?!?': <gensim.models.word2vec.Vocab at 0x7f61aa4e8c50>,\n",
       " 'olivia.': <gensim.models.word2vec.Vocab at 0x7f61aa4e8da0>,\n",
       " \"she'll\": <gensim.models.word2vec.Vocab at 0x7f61aa4e8f60>,\n",
       " 'community,': <gensim.models.word2vec.Vocab at 0x7f61aa4e8f98>,\n",
       " 'cold.': <gensim.models.word2vec.Vocab at 0x7f61aa5dc978>,\n",
       " 'not': <gensim.models.word2vec.Vocab at 0x7f61aa4e8898>,\n",
       " 'transcripts': <gensim.models.word2vec.Vocab at 0x7f61aa4e8160>,\n",
       " 'promises...i': <gensim.models.word2vec.Vocab at 0x7f61aa5c7ba8>,\n",
       " 'totem': <gensim.models.word2vec.Vocab at 0x7f61aa4e82e8>,\n",
       " 'naked,': <gensim.models.word2vec.Vocab at 0x7f61aa554320>,\n",
       " 'hate': <gensim.models.word2vec.Vocab at 0x7f61aa4e8358>,\n",
       " 'gas': <gensim.models.word2vec.Vocab at 0x7f61aa4e85c0>,\n",
       " 'beat': <gensim.models.word2vec.Vocab at 0x7f61aa4e85f8>,\n",
       " 'Jungle': <gensim.models.word2vec.Vocab at 0x7f61aa4e8748>,\n",
       " 'band': <gensim.models.word2vec.Vocab at 0x7f61aa5697b8>,\n",
       " 'ought': <gensim.models.word2vec.Vocab at 0x7f61aa4e8828>,\n",
       " 'ishouldnt': <gensim.models.word2vec.Vocab at 0x7f61aa4e7128>,\n",
       " 'funni': <gensim.models.word2vec.Vocab at 0x7f61aa4e7208>,\n",
       " 'camera': <gensim.models.word2vec.Vocab at 0x7f61aa4e7278>,\n",
       " \"Mom's\": <gensim.models.word2vec.Vocab at 0x7f61aa4e7400>,\n",
       " 'invitations': <gensim.models.word2vec.Vocab at 0x7f61aa4e7438>,\n",
       " 'sheets,': <gensim.models.word2vec.Vocab at 0x7f61aa4e7470>,\n",
       " 'sony': <gensim.models.word2vec.Vocab at 0x7f61aa4e74a8>,\n",
       " 'Could': <gensim.models.word2vec.Vocab at 0x7f61aa4e7588>,\n",
       " '\"goodness\"': <gensim.models.word2vec.Vocab at 0x7f61aa4e75c0>,\n",
       " 'commentators': <gensim.models.word2vec.Vocab at 0x7f61aa4e7668>,\n",
       " 'learned': <gensim.models.word2vec.Vocab at 0x7f61aa4e7710>,\n",
       " 'quit': <gensim.models.word2vec.Vocab at 0x7f61aa4e7748>,\n",
       " \"mother's\": <gensim.models.word2vec.Vocab at 0x7f61aa5dc9e8>,\n",
       " 'Hussein,': <gensim.models.word2vec.Vocab at 0x7f61aa5b9320>,\n",
       " 'Funny,': <gensim.models.word2vec.Vocab at 0x7f61aa4e7860>,\n",
       " 'Actually': <gensim.models.word2vec.Vocab at 0x7f61aa4e7898>,\n",
       " 'upsetting.': <gensim.models.word2vec.Vocab at 0x7f61aa4e7a90>,\n",
       " 'ring!)': <gensim.models.word2vec.Vocab at 0x7f61aa4e7b00>,\n",
       " 'material': <gensim.models.word2vec.Vocab at 0x7f61aa4e7b38>,\n",
       " '…': <gensim.models.word2vec.Vocab at 0x7f61aa4e7be0>,\n",
       " 'kind': <gensim.models.word2vec.Vocab at 0x7f61aa4e7c18>,\n",
       " 'Moon\"': <gensim.models.word2vec.Vocab at 0x7f61aa4e7cc0>,\n",
       " 'james,': <gensim.models.word2vec.Vocab at 0x7f61aa4e7d30>,\n",
       " 'regardless': <gensim.models.word2vec.Vocab at 0x7f61aa4e7d68>,\n",
       " 'WATCHED': <gensim.models.word2vec.Vocab at 0x7f61aa4e7da0>,\n",
       " 'possibly': <gensim.models.word2vec.Vocab at 0x7f61aa5bbe10>,\n",
       " 'Make': <gensim.models.word2vec.Vocab at 0x7f61aa4e7ef0>,\n",
       " 'airplanes,': <gensim.models.word2vec.Vocab at 0x7f61aa463f60>,\n",
       " 'Exaggerated,': <gensim.models.word2vec.Vocab at 0x7f61aa4e7f98>,\n",
       " 'head,': <gensim.models.word2vec.Vocab at 0x7f61aa4e7fd0>,\n",
       " 'graceful': <gensim.models.word2vec.Vocab at 0x7f61aa4d9128>,\n",
       " 'but': <gensim.models.word2vec.Vocab at 0x7f61aa4d9550>,\n",
       " 'low': <gensim.models.word2vec.Vocab at 0x7f61aa4d95f8>,\n",
       " 'it!!!': <gensim.models.word2vec.Vocab at 0x7f61aa4d9710>,\n",
       " 'usual)': <gensim.models.word2vec.Vocab at 0x7f61aa4d97f0>,\n",
       " 'doing?:': <gensim.models.word2vec.Vocab at 0x7f61aa4d9908>,\n",
       " \"wat's\": <gensim.models.word2vec.Vocab at 0x7f61aa4d99b0>,\n",
       " 'disadvantages': <gensim.models.word2vec.Vocab at 0x7f61aa5dcb00>,\n",
       " 'breaks': <gensim.models.word2vec.Vocab at 0x7f61aa4d9cf8>,\n",
       " 'partner,': <gensim.models.word2vec.Vocab at 0x7f61aa4d8048>,\n",
       " 'totally': <gensim.models.word2vec.Vocab at 0x7f61aa4d80f0>,\n",
       " 'break?!': <gensim.models.word2vec.Vocab at 0x7f61aa4d81d0>,\n",
       " 'remember,': <gensim.models.word2vec.Vocab at 0x7f61aa3fedd8>,\n",
       " 'nose.': <gensim.models.word2vec.Vocab at 0x7f61aa4d82b0>,\n",
       " '...gets': <gensim.models.word2vec.Vocab at 0x7f61aa4d82e8>,\n",
       " 'circles': <gensim.models.word2vec.Vocab at 0x7f61aa4d8320>,\n",
       " 'list?': <gensim.models.word2vec.Vocab at 0x7f61aa4d84a8>,\n",
       " 'babble.': <gensim.models.word2vec.Vocab at 0x7f61aa4ec2e8>,\n",
       " 'Those': <gensim.models.word2vec.Vocab at 0x7f61aa5c19b0>,\n",
       " 'hers,': <gensim.models.word2vec.Vocab at 0x7f61aa554518>,\n",
       " 'Kucinich).': <gensim.models.word2vec.Vocab at 0x7f61aa4d8a90>,\n",
       " 'toxic,': <gensim.models.word2vec.Vocab at 0x7f61aa4d8ac8>,\n",
       " 'mates.': <gensim.models.word2vec.Vocab at 0x7f61aa4d8be0>,\n",
       " 'rock!': <gensim.models.word2vec.Vocab at 0x7f61aa4d8d68>,\n",
       " 'birthday': <gensim.models.word2vec.Vocab at 0x7f61aa4d8e48>,\n",
       " 'okay-': <gensim.models.word2vec.Vocab at 0x7f61aa4d8ef0>,\n",
       " 'Twenty-six': <gensim.models.word2vec.Vocab at 0x7f61aa4d8f60>,\n",
       " 'Molly': <gensim.models.word2vec.Vocab at 0x7f61aa4d8f98>,\n",
       " 'everyone.i': <gensim.models.word2vec.Vocab at 0x7f61aa4d8fd0>,\n",
       " 'brought': <gensim.models.word2vec.Vocab at 0x7f61aa4db320>,\n",
       " 'rusty.': <gensim.models.word2vec.Vocab at 0x7f61aa4db358>,\n",
       " \"Let's\": <gensim.models.word2vec.Vocab at 0x7f61aa4db390>,\n",
       " 'soon?': <gensim.models.word2vec.Vocab at 0x7f61aa4db400>,\n",
       " '19.': <gensim.models.word2vec.Vocab at 0x7f61aa4db4e0>,\n",
       " 'shuffle': <gensim.models.word2vec.Vocab at 0x7f61aa4db8d0>,\n",
       " \"you're\": <gensim.models.word2vec.Vocab at 0x7f61aa4dbac8>,\n",
       " 'somehow?': <gensim.models.word2vec.Vocab at 0x7f61aa4ec400>,\n",
       " 'naked?]:': <gensim.models.word2vec.Vocab at 0x7f61aa5d4c18>,\n",
       " '...i': <gensim.models.word2vec.Vocab at 0x7f61aa4dbd68>,\n",
       " 'friend': <gensim.models.word2vec.Vocab at 0x7f61aa4da048>,\n",
       " 'away;': <gensim.models.word2vec.Vocab at 0x7f61aa4da320>,\n",
       " 'tending': <gensim.models.word2vec.Vocab at 0x7f61aa4da358>,\n",
       " 'creates': <gensim.models.word2vec.Vocab at 0x7f61aa4da5f8>,\n",
       " 'certitude,': <gensim.models.word2vec.Vocab at 0x7f61aa4da668>,\n",
       " 'job...some': <gensim.models.word2vec.Vocab at 0x7f61aa4da6a0>,\n",
       " 'room.': <gensim.models.word2vec.Vocab at 0x7f61aa4da748>,\n",
       " '...will': <gensim.models.word2vec.Vocab at 0x7f61aa4da7b8>,\n",
       " 'mincing': <gensim.models.word2vec.Vocab at 0x7f61aa4da8d0>,\n",
       " 'dog/cat/bird/fish,': <gensim.models.word2vec.Vocab at 0x7f61aa4da908>,\n",
       " 'way,': <gensim.models.word2vec.Vocab at 0x7f61aa5bf0b8>,\n",
       " 'nvm...': <gensim.models.word2vec.Vocab at 0x7f61aa4daba8>,\n",
       " 'illness,': <gensim.models.word2vec.Vocab at 0x7f61aa4dac18>,\n",
       " 'good.': <gensim.models.word2vec.Vocab at 0x7f61aa4dacc0>,\n",
       " 'bother??': <gensim.models.word2vec.Vocab at 0x7f61aa4dada0>,\n",
       " 'curse': <gensim.models.word2vec.Vocab at 0x7f61aa4dadd8>,\n",
       " \"daughter's\": <gensim.models.word2vec.Vocab at 0x7f61aa4daf60>,\n",
       " '(albeit,': <gensim.models.word2vec.Vocab at 0x7f61aa4dc3c8>,\n",
       " 'okay.': <gensim.models.word2vec.Vocab at 0x7f61aa4ede48>,\n",
       " 'boxers': <gensim.models.word2vec.Vocab at 0x7f61aa4dc588>,\n",
       " 'Calculus,': <gensim.models.word2vec.Vocab at 0x7f61aa4dc6a0>,\n",
       " 'MEAN': <gensim.models.word2vec.Vocab at 0x7f61aa4dc7b8>,\n",
       " 'rosie.': <gensim.models.word2vec.Vocab at 0x7f61aa4dc9e8>,\n",
       " 'hard': <gensim.models.word2vec.Vocab at 0x7f61aa4dca90>,\n",
       " 'life...think': <gensim.models.word2vec.Vocab at 0x7f61aa4dcba8>,\n",
       " 'takes': <gensim.models.word2vec.Vocab at 0x7f61aa4dce48>,\n",
       " 'pretty.': <gensim.models.word2vec.Vocab at 0x7f61aa5c1c88>,\n",
       " 'award': <gensim.models.word2vec.Vocab at 0x7f61aa4dceb8>,\n",
       " 'their': <gensim.models.word2vec.Vocab at 0x7f61aa4dcf28>,\n",
       " 'plainly.': <gensim.models.word2vec.Vocab at 0x7f61aa4dcfd0>,\n",
       " 'noone': <gensim.models.word2vec.Vocab at 0x7f61aa4ca1d0>,\n",
       " 'say...no': <gensim.models.word2vec.Vocab at 0x7f61aa438978>,\n",
       " 'thats': <gensim.models.word2vec.Vocab at 0x7f61aa4ca2e8>,\n",
       " 'learning': <gensim.models.word2vec.Vocab at 0x7f61aa5dcd30>,\n",
       " 'sleep': <gensim.models.word2vec.Vocab at 0x7f61aa4ca4a8>,\n",
       " 'against': <gensim.models.word2vec.Vocab at 0x7f61aa4ca5c0>,\n",
       " 'rubbish': <gensim.models.word2vec.Vocab at 0x7f61aa565358>,\n",
       " 'years,': <gensim.models.word2vec.Vocab at 0x7f61aa4ca9b0>,\n",
       " 'theatre)': <gensim.models.word2vec.Vocab at 0x7f61aa4caa20>,\n",
       " '[Kissed': <gensim.models.word2vec.Vocab at 0x7f61aa4caa90>,\n",
       " 'love?': <gensim.models.word2vec.Vocab at 0x7f61aa4caba8>,\n",
       " 'Forgetting': <gensim.models.word2vec.Vocab at 0x7f61aa4cada0>,\n",
       " 'Whoever': <gensim.models.word2vec.Vocab at 0x7f61aa4cb358>,\n",
       " 'bacon': <gensim.models.word2vec.Vocab at 0x7f61aa4cb438>,\n",
       " 'wishing': <gensim.models.word2vec.Vocab at 0x7f61aa5ce940>,\n",
       " 'fantastic.': <gensim.models.word2vec.Vocab at 0x7f61aa4cb898>,\n",
       " 'rosalie...': <gensim.models.word2vec.Vocab at 0x7f61aa4cbc18>,\n",
       " 'souned': <gensim.models.word2vec.Vocab at 0x7f61aa5dce48>,\n",
       " 'bulbous': <gensim.models.word2vec.Vocab at 0x7f61aa4cbeb8>,\n",
       " 'in-depth': <gensim.models.word2vec.Vocab at 0x7f61aa4cbef0>,\n",
       " 'proof': <gensim.models.word2vec.Vocab at 0x7f61aa4cd240>,\n",
       " 'however,': <gensim.models.word2vec.Vocab at 0x7f61aa4cd278>,\n",
       " 'at': <gensim.models.word2vec.Vocab at 0x7f61aa4cd390>,\n",
       " \"you'll\": <gensim.models.word2vec.Vocab at 0x7f61aa4cd438>,\n",
       " 'Will': <gensim.models.word2vec.Vocab at 0x7f61aa4cd780>,\n",
       " 'Chotky': <gensim.models.word2vec.Vocab at 0x7f61aa4cda90>,\n",
       " 'o0o!': <gensim.models.word2vec.Vocab at 0x7f61aa4cf2b0>,\n",
       " 'overnight,': <gensim.models.word2vec.Vocab at 0x7f61aa442208>,\n",
       " '6.': <gensim.models.word2vec.Vocab at 0x7f61aa4cf400>,\n",
       " 'expensive': <gensim.models.word2vec.Vocab at 0x7f61aa4cf518>,\n",
       " 'employers': <gensim.models.word2vec.Vocab at 0x7f61aa4cf550>,\n",
       " 'especially': <gensim.models.word2vec.Vocab at 0x7f61aa4cf828>,\n",
       " 'lives,': <gensim.models.word2vec.Vocab at 0x7f61aa4cf860>,\n",
       " 'dumb': <gensim.models.word2vec.Vocab at 0x7f61aa4cf898>,\n",
       " 'EVERYONE!!!': <gensim.models.word2vec.Vocab at 0x7f61aa4cfc88>,\n",
       " 'mind,': <gensim.models.word2vec.Vocab at 0x7f61aa4cfd68>,\n",
       " 'terms': <gensim.models.word2vec.Vocab at 0x7f61aa4cffd0>,\n",
       " 'deception': <gensim.models.word2vec.Vocab at 0x7f61aa4ce390>,\n",
       " 'glad.': <gensim.models.word2vec.Vocab at 0x7f61aa4ce5c0>,\n",
       " '20:': <gensim.models.word2vec.Vocab at 0x7f61aa453e48>,\n",
       " 'disappeared!!!!!!!!': <gensim.models.word2vec.Vocab at 0x7f61aa4ce978>,\n",
       " 'candy:': <gensim.models.word2vec.Vocab at 0x7f61aa4ceba8>,\n",
       " 'PRODUCTIVE!!': <gensim.models.word2vec.Vocab at 0x7f61aa5d7048>,\n",
       " 'Goals': <gensim.models.word2vec.Vocab at 0x7f61aa4cec18>,\n",
       " 'like,': <gensim.models.word2vec.Vocab at 0x7f61aa4cecf8>,\n",
       " 'Carter': <gensim.models.word2vec.Vocab at 0x7f61aa4cedd8>,\n",
       " 'So': <gensim.models.word2vec.Vocab at 0x7f61aa4cef60>,\n",
       " '5:': <gensim.models.word2vec.Vocab at 0x7f61aa4d2048>,\n",
       " 'stalled.': <gensim.models.word2vec.Vocab at 0x7f61aa4d2208>,\n",
       " 'fewer': <gensim.models.word2vec.Vocab at 0x7f61aa4d26d8>,\n",
       " 'lies': <gensim.models.word2vec.Vocab at 0x7f61aa4d27b8>,\n",
       " 'faces': <gensim.models.word2vec.Vocab at 0x7f61aa5b9828>,\n",
       " 'im': <gensim.models.word2vec.Vocab at 0x7f61aa4d2898>,\n",
       " 'kina': <gensim.models.word2vec.Vocab at 0x7f61aa4d2ba8>,\n",
       " 'Each': <gensim.models.word2vec.Vocab at 0x7f61aa4d2e10>,\n",
       " 'know...even': <gensim.models.word2vec.Vocab at 0x7f61aa4d2e48>,\n",
       " 'thrown': <gensim.models.word2vec.Vocab at 0x7f61aa4d2eb8>,\n",
       " \"can't\": <gensim.models.word2vec.Vocab at 0x7f61aa4d3128>,\n",
       " 'close-minded.': <gensim.models.word2vec.Vocab at 0x7f61aa4d31d0>,\n",
       " 'aint': <gensim.models.word2vec.Vocab at 0x7f61aa4d3240>,\n",
       " 'the': <gensim.models.word2vec.Vocab at 0x7f61aa4d36d8>,\n",
       " 'Ikea': <gensim.models.word2vec.Vocab at 0x7f61aa4d37b8>,\n",
       " 'trying': <gensim.models.word2vec.Vocab at 0x7f61aa4d38d0>,\n",
       " 'Coulter': <gensim.models.word2vec.Vocab at 0x7f61aa4d3940>,\n",
       " 'cleaner,': <gensim.models.word2vec.Vocab at 0x7f61aa4d3b00>,\n",
       " 'Mix]\"': <gensim.models.word2vec.Vocab at 0x7f61aa4d3ba8>,\n",
       " 'surface,': <gensim.models.word2vec.Vocab at 0x7f61aa4d3c50>,\n",
       " 'mean,': <gensim.models.word2vec.Vocab at 0x7f61aa4d50b8>,\n",
       " 'Graham),': <gensim.models.word2vec.Vocab at 0x7f61aa4d5160>,\n",
       " 'Congress,': <gensim.models.word2vec.Vocab at 0x7f61aa4d5198>,\n",
       " 'animals': <gensim.models.word2vec.Vocab at 0x7f61aa4d5208>,\n",
       " 'small': <gensim.models.word2vec.Vocab at 0x7f61aa4d5278>,\n",
       " 'steps.': <gensim.models.word2vec.Vocab at 0x7f61aa4d5390>,\n",
       " '[relationship]': <gensim.models.word2vec.Vocab at 0x7f61aa4d5438>,\n",
       " '[Wanted': <gensim.models.word2vec.Vocab at 0x7f61aa4d55c0>,\n",
       " 'finals...too': <gensim.models.word2vec.Vocab at 0x7f61aa4d55f8>,\n",
       " 'definitely.': <gensim.models.word2vec.Vocab at 0x7f61aa554eb8>,\n",
       " 'I:': <gensim.models.word2vec.Vocab at 0x7f61aa4d56a0>,\n",
       " 'what...even': <gensim.models.word2vec.Vocab at 0x7f61aa4eef98>,\n",
       " '......': <gensim.models.word2vec.Vocab at 0x7f61aa4d5978>,\n",
       " 'lies).': <gensim.models.word2vec.Vocab at 0x7f61aa4d59e8>,\n",
       " 'longer': <gensim.models.word2vec.Vocab at 0x7f61aa4d5a90>,\n",
       " 'animals.': <gensim.models.word2vec.Vocab at 0x7f61aa5b9908>,\n",
       " 'mindless': <gensim.models.word2vec.Vocab at 0x7f61aa4d5b38>,\n",
       " 'disappear….': <gensim.models.word2vec.Vocab at 0x7f61aa4d5cc0>,\n",
       " 'places': <gensim.models.word2vec.Vocab at 0x7f61aa4d6c88>,\n",
       " 'sheets.': <gensim.models.word2vec.Vocab at 0x7f61aa4d6cf8>,\n",
       " 'here.': <gensim.models.word2vec.Vocab at 0x7f61aa4d6d68>,\n",
       " 'both,': <gensim.models.word2vec.Vocab at 0x7f61aa4d6e10>,\n",
       " 'xela': <gensim.models.word2vec.Vocab at 0x7f61aa4d6e80>,\n",
       " 'creeping': <gensim.models.word2vec.Vocab at 0x7f61aa4d6be0>,\n",
       " 'dressy': <gensim.models.word2vec.Vocab at 0x7f61aa4d6048>,\n",
       " 'melting': <gensim.models.word2vec.Vocab at 0x7f61aa4d6198>,\n",
       " '30': <gensim.models.word2vec.Vocab at 0x7f61aa4d6240>,\n",
       " 'Questions': <gensim.models.word2vec.Vocab at 0x7f61aa5b99e8>,\n",
       " 'indicates': <gensim.models.word2vec.Vocab at 0x7f61aa4d6390>,\n",
       " 'guess': <gensim.models.word2vec.Vocab at 0x7f61aa4d63c8>,\n",
       " '37': <gensim.models.word2vec.Vocab at 0x7f61aa4d6400>,\n",
       " 'strong,': <gensim.models.word2vec.Vocab at 0x7f61aa4d6588>,\n",
       " \"I'd\": <gensim.models.word2vec.Vocab at 0x7f61aa4d6668>,\n",
       " 'Band': <gensim.models.word2vec.Vocab at 0x7f61aa4d6940>,\n",
       " 'portly.': <gensim.models.word2vec.Vocab at 0x7f61aa4d6a20>,\n",
       " 'dere': <gensim.models.word2vec.Vocab at 0x7f61aa4d6ac8>,\n",
       " 'weeee': <gensim.models.word2vec.Vocab at 0x7f61aa5b9ef0>,\n",
       " 'reason': <gensim.models.word2vec.Vocab at 0x7f61aa4d6b00>,\n",
       " 'az': <gensim.models.word2vec.Vocab at 0x7f61aa4d7208>,\n",
       " 'pond..': <gensim.models.word2vec.Vocab at 0x7f61aa5e1ef0>,\n",
       " 'anyway).': <gensim.models.word2vec.Vocab at 0x7f61aa4d77b8>,\n",
       " 'adventurous': <gensim.models.word2vec.Vocab at 0x7f61aa4d7828>,\n",
       " 'supply': <gensim.models.word2vec.Vocab at 0x7f61aa4d70b8>,\n",
       " 'Bored': <gensim.models.word2vec.Vocab at 0x7f61aa4d7240>,\n",
       " 'black': <gensim.models.word2vec.Vocab at 0x7f61aa4d7278>,\n",
       " 'cambridge?': <gensim.models.word2vec.Vocab at 0x7f61aa4d7358>,\n",
       " 'noise': <gensim.models.word2vec.Vocab at 0x7f61aa4d7438>,\n",
       " 'Winnipeg.': <gensim.models.word2vec.Vocab at 0x7f61aa4d7470>,\n",
       " 'There': <gensim.models.word2vec.Vocab at 0x7f61aa4d74e0>,\n",
       " 'chat': <gensim.models.word2vec.Vocab at 0x7f61aa4d7588>,\n",
       " 'HERE': <gensim.models.word2vec.Vocab at 0x7f61aa4d76a0>,\n",
       " 'choose': <gensim.models.word2vec.Vocab at 0x7f61aa4d78d0>,\n",
       " 'morality,': <gensim.models.word2vec.Vocab at 0x7f61aa4d7a90>,\n",
       " 'favors': <gensim.models.word2vec.Vocab at 0x7f61aa4d7b38>,\n",
       " '[If': <gensim.models.word2vec.Vocab at 0x7f61aa4d7c50>,\n",
       " 'nvm,': <gensim.models.word2vec.Vocab at 0x7f61aa4d7cc0>,\n",
       " 'tragedy': <gensim.models.word2vec.Vocab at 0x7f61aa4d7d30>,\n",
       " 'japanese': <gensim.models.word2vec.Vocab at 0x7f61aa4d7da0>,\n",
       " 'invite': <gensim.models.word2vec.Vocab at 0x7f61aa54b780>,\n",
       " 'way.': <gensim.models.word2vec.Vocab at 0x7f61aa4d7e10>,\n",
       " 'HAPPY': <gensim.models.word2vec.Vocab at 0x7f61aa4d7f98>,\n",
       " 'fierce': <gensim.models.word2vec.Vocab at 0x7f61aa5e78d0>,\n",
       " 'fools': <gensim.models.word2vec.Vocab at 0x7f61aa4d13c8>,\n",
       " 'goes': <gensim.models.word2vec.Vocab at 0x7f61aa4d1400>,\n",
       " 'wafers': <gensim.models.word2vec.Vocab at 0x7f61aa4d1470>,\n",
       " ':-D': <gensim.models.word2vec.Vocab at 0x7f61aa5e7c88>,\n",
       " 'feathers': <gensim.models.word2vec.Vocab at 0x7f61aa5e7e10>,\n",
       " 'still...': <gensim.models.word2vec.Vocab at 0x7f61aa4425f8>,\n",
       " 'selene': <gensim.models.word2vec.Vocab at 0x7f61aa4d15c0>,\n",
       " 'dinner\"': <gensim.models.word2vec.Vocab at 0x7f61aa4d16a0>,\n",
       " 'EVERY': <gensim.models.word2vec.Vocab at 0x7f61aa4d16d8>,\n",
       " '(2)': <gensim.models.word2vec.Vocab at 0x7f61aa4d1710>,\n",
       " 'hormones': <gensim.models.word2vec.Vocab at 0x7f61aa4d1860>,\n",
       " 'singing': <gensim.models.word2vec.Vocab at 0x7f61aa4d1898>,\n",
       " 'carry': <gensim.models.word2vec.Vocab at 0x7f61aa4d1a58>,\n",
       " 'bestfriend': <gensim.models.word2vec.Vocab at 0x7f61aa4d1ac8>,\n",
       " 'AmeriCorps': <gensim.models.word2vec.Vocab at 0x7f61aa4d1b00>,\n",
       " 'tuesday': <gensim.models.word2vec.Vocab at 0x7f61aa4d1c50>,\n",
       " 'plants.': <gensim.models.word2vec.Vocab at 0x7f61aa4d1fd0>,\n",
       " 'Presidential': <gensim.models.word2vec.Vocab at 0x7f61aa4d1048>,\n",
       " 'dunno...i': <gensim.models.word2vec.Vocab at 0x7f61aa4d1278>,\n",
       " '[few': <gensim.models.word2vec.Vocab at 0x7f61aa4dd048>,\n",
       " 'exercise.': <gensim.models.word2vec.Vocab at 0x7f61aa4dd080>,\n",
       " 'WITH': <gensim.models.word2vec.Vocab at 0x7f61aa4dd198>,\n",
       " 'Figueroa': <gensim.models.word2vec.Vocab at 0x7f61aa4dd1d0>,\n",
       " 'softens': <gensim.models.word2vec.Vocab at 0x7f61aa4dd320>,\n",
       " 'true.': <gensim.models.word2vec.Vocab at 0x7f61aa466eb8>,\n",
       " 'ballpark': <gensim.models.word2vec.Vocab at 0x7f61aa4dd588>,\n",
       " 'sleep,': <gensim.models.word2vec.Vocab at 0x7f61aa4dd5f8>,\n",
       " 'names.': <gensim.models.word2vec.Vocab at 0x7f61aa4dd6d8>,\n",
       " 'you’re': <gensim.models.word2vec.Vocab at 0x7f61aa4dd710>,\n",
       " 'price': <gensim.models.word2vec.Vocab at 0x7f61aa4dd7f0>,\n",
       " 'pig': <gensim.models.word2vec.Vocab at 0x7f61aa4dd940>,\n",
       " 'time:': <gensim.models.word2vec.Vocab at 0x7f61aa4dda20>,\n",
       " 'Colella': <gensim.models.word2vec.Vocab at 0x7f61aa4dda90>,\n",
       " 'gift': <gensim.models.word2vec.Vocab at 0x7f61aa4ddb70>,\n",
       " 'american': <gensim.models.word2vec.Vocab at 0x7f61aa4ddba8>,\n",
       " 'poopie': <gensim.models.word2vec.Vocab at 0x7f61aa4ddcf8>,\n",
       " 'floor': <gensim.models.word2vec.Vocab at 0x7f61aa4ddd68>,\n",
       " 'talked': <gensim.models.word2vec.Vocab at 0x7f61aa4dddd8>,\n",
       " 'age': <gensim.models.word2vec.Vocab at 0x7f61aa4dde10>,\n",
       " 'sad.': <gensim.models.word2vec.Vocab at 0x7f61aa4dde48>,\n",
       " 'usually': <gensim.models.word2vec.Vocab at 0x7f61aa4dde80>,\n",
       " \"i'd\": <gensim.models.word2vec.Vocab at 0x7f61aa4040f0>,\n",
       " 'New]:': <gensim.models.word2vec.Vocab at 0x7f61aa4ddeb8>,\n",
       " 'out,': <gensim.models.word2vec.Vocab at 0x7f61aa4ddef0>,\n",
       " 'Secondly,': <gensim.models.word2vec.Vocab at 0x7f61aa4ddf28>,\n",
       " 'kicked': <gensim.models.word2vec.Vocab at 0x7f61aa4e0048>,\n",
       " 'stuff': <gensim.models.word2vec.Vocab at 0x7f61aa4e0080>,\n",
       " 'essences': <gensim.models.word2vec.Vocab at 0x7f61aa4e0128>,\n",
       " 'live': <gensim.models.word2vec.Vocab at 0x7f61aa4e0198>,\n",
       " 'aditi.': <gensim.models.word2vec.Vocab at 0x7f61aa4e01d0>,\n",
       " 'prepare,': <gensim.models.word2vec.Vocab at 0x7f61aa4e0320>,\n",
       " 'Ave': <gensim.models.word2vec.Vocab at 0x7f61aa4e0358>,\n",
       " 'Given': <gensim.models.word2vec.Vocab at 0x7f61aa4e0438>,\n",
       " 'C\"': <gensim.models.word2vec.Vocab at 0x7f61aa4e04e0>,\n",
       " 'touching': <gensim.models.word2vec.Vocab at 0x7f61aa4e0588>,\n",
       " 'Jeep),': <gensim.models.word2vec.Vocab at 0x7f61aa5df438>,\n",
       " 'Los': <gensim.models.word2vec.Vocab at 0x7f61aa4e0668>,\n",
       " 'wide.': <gensim.models.word2vec.Vocab at 0x7f61aa4e06d8>,\n",
       " 'though.': <gensim.models.word2vec.Vocab at 0x7f61aa4e0748>,\n",
       " 'sometime,': <gensim.models.word2vec.Vocab at 0x7f61aa554dd8>,\n",
       " 'had.': <gensim.models.word2vec.Vocab at 0x7f61aa4e07f0>,\n",
       " 'dreams': <gensim.models.word2vec.Vocab at 0x7f61aa4e0908>,\n",
       " 'jobs': <gensim.models.word2vec.Vocab at 0x7f61aa4e0940>,\n",
       " 'bike': <gensim.models.word2vec.Vocab at 0x7f61aa4e09b0>,\n",
       " 'waterfall': <gensim.models.word2vec.Vocab at 0x7f61aa4e09e8>,\n",
       " 'uhh....': <gensim.models.word2vec.Vocab at 0x7f61aa4e0a58>,\n",
       " 'strenuous': <gensim.models.word2vec.Vocab at 0x7f61aa4e0a90>,\n",
       " 'overly-perky': <gensim.models.word2vec.Vocab at 0x7f61aa554e48>,\n",
       " '....that': <gensim.models.word2vec.Vocab at 0x7f61aa4e0b70>,\n",
       " 'fraud': <gensim.models.word2vec.Vocab at 0x7f61aa4e0be0>,\n",
       " 'ahaha': <gensim.models.word2vec.Vocab at 0x7f61aa4e0c88>,\n",
       " 'New': <gensim.models.word2vec.Vocab at 0x7f61aa4e0cc0>,\n",
       " 'shopping': <gensim.models.word2vec.Vocab at 0x7f61aa4e0d30>,\n",
       " 'extra': <gensim.models.word2vec.Vocab at 0x7f61aa4e0d68>,\n",
       " 'use.': <gensim.models.word2vec.Vocab at 0x7f61aa4e0e10>,\n",
       " 'running--while': <gensim.models.word2vec.Vocab at 0x7f61aa4e0e80>,\n",
       " \"won't\": <gensim.models.word2vec.Vocab at 0x7f61aa4e0ef0>,\n",
       " 'no:': <gensim.models.word2vec.Vocab at 0x7f61aa4e0f28>,\n",
       " 'verb,': <gensim.models.word2vec.Vocab at 0x7f61aa4e0f60>,\n",
       " 'punch': <gensim.models.word2vec.Vocab at 0x7f61aa4e0f98>,\n",
       " 'tamar.': <gensim.models.word2vec.Vocab at 0x7f61aa4e0fd0>,\n",
       " 'summer': <gensim.models.word2vec.Vocab at 0x7f61aa4e3080>,\n",
       " 'got': <gensim.models.word2vec.Vocab at 0x7f61aa4e30f0>,\n",
       " 'breath,': <gensim.models.word2vec.Vocab at 0x7f61aa4e3240>,\n",
       " 'answer': <gensim.models.word2vec.Vocab at 0x7f61aa4e3278>,\n",
       " 'selves': <gensim.models.word2vec.Vocab at 0x7f61aa5d4eb8>,\n",
       " 'everthing': <gensim.models.word2vec.Vocab at 0x7f61aa4dd908>,\n",
       " 'nap,': <gensim.models.word2vec.Vocab at 0x7f61aa4e3470>,\n",
       " 'CBC': <gensim.models.word2vec.Vocab at 0x7f61aa4e34a8>,\n",
       " 'argument': <gensim.models.word2vec.Vocab at 0x7f61aa4e34e0>,\n",
       " 'if': <gensim.models.word2vec.Vocab at 0x7f61aa4e3550>,\n",
       " 'sorts': <gensim.models.word2vec.Vocab at 0x7f61aa5edd30>,\n",
       " 'fields,': <gensim.models.word2vec.Vocab at 0x7f61aa4e35c0>,\n",
       " 'canning': <gensim.models.word2vec.Vocab at 0x7f61aa4e36a0>,\n",
       " 'worry..': <gensim.models.word2vec.Vocab at 0x7f61aa4e36d8>,\n",
       " 'curtains!': <gensim.models.word2vec.Vocab at 0x7f61aa4e3828>,\n",
       " 'why…': <gensim.models.word2vec.Vocab at 0x7f61aa4e38d0>,\n",
       " 'fainting': <gensim.models.word2vec.Vocab at 0x7f61aa4e3940>,\n",
       " 'ONLY': <gensim.models.word2vec.Vocab at 0x7f61aa4e3978>,\n",
       " 'no-one': <gensim.models.word2vec.Vocab at 0x7f61aa4e3a58>,\n",
       " 'floating': <gensim.models.word2vec.Vocab at 0x7f61aa4e3a90>,\n",
       " 'messy,': <gensim.models.word2vec.Vocab at 0x7f61aa4e3b38>,\n",
       " 'third': <gensim.models.word2vec.Vocab at 0x7f61aa4e3ba8>,\n",
       " 'stood,': <gensim.models.word2vec.Vocab at 0x7f61aa4e3c18>,\n",
       " 'fishing?': <gensim.models.word2vec.Vocab at 0x7f61aa4e3cc0>,\n",
       " 'shall': <gensim.models.word2vec.Vocab at 0x7f61aa4e3d30>,\n",
       " 'everything': <gensim.models.word2vec.Vocab at 0x7f61aa4e3d68>,\n",
       " 'dog': <gensim.models.word2vec.Vocab at 0x7f61aa4e3da0>,\n",
       " 'semester!': <gensim.models.word2vec.Vocab at 0x7f61aa53ca20>,\n",
       " 'hurts': <gensim.models.word2vec.Vocab at 0x7f61aa4e3dd8>,\n",
       " 'blab': <gensim.models.word2vec.Vocab at 0x7f61aa4e3e10>,\n",
       " 'Cyan425:': <gensim.models.word2vec.Vocab at 0x7f61aa4e3e80>,\n",
       " 'kid': <gensim.models.word2vec.Vocab at 0x7f61aa4e3eb8>,\n",
       " 'Rumsfeld': <gensim.models.word2vec.Vocab at 0x7f61aa4e3ef0>,\n",
       " 'be:': <gensim.models.word2vec.Vocab at 0x7f61aa4e3f60>,\n",
       " 'character': <gensim.models.word2vec.Vocab at 0x7f61aa4e3f98>,\n",
       " 'too;': <gensim.models.word2vec.Vocab at 0x7f61aa4e3fd0>,\n",
       " 'cheese.': <gensim.models.word2vec.Vocab at 0x7f61aa4e4048>,\n",
       " 'showin': <gensim.models.word2vec.Vocab at 0x7f61aa4e4080>,\n",
       " 'DiFranco.': <gensim.models.word2vec.Vocab at 0x7f61aa4e40b8>,\n",
       " 'weeks.': <gensim.models.word2vec.Vocab at 0x7f61aa4e40f0>,\n",
       " 'authorized': <gensim.models.word2vec.Vocab at 0x7f61aa4e4128>,\n",
       " 'Or': <gensim.models.word2vec.Vocab at 0x7f61aa4e4208>,\n",
       " 'easier.': <gensim.models.word2vec.Vocab at 0x7f61aa4e4278>,\n",
       " 'deserve': <gensim.models.word2vec.Vocab at 0x7f61aa4e42b0>,\n",
       " 'reads': <gensim.models.word2vec.Vocab at 0x7f61aa4e42e8>,\n",
       " 'beautiful': <gensim.models.word2vec.Vocab at 0x7f61aa4e4390>,\n",
       " 'avril': <gensim.models.word2vec.Vocab at 0x7f61aa4e43c8>,\n",
       " 'days.': <gensim.models.word2vec.Vocab at 0x7f61aa5b9e80>,\n",
       " '\"can': <gensim.models.word2vec.Vocab at 0x7f61aa4e4400>,\n",
       " 'player:': <gensim.models.word2vec.Vocab at 0x7f61aa4e4470>,\n",
       " 'american??': <gensim.models.word2vec.Vocab at 0x7f61aa4e44a8>,\n",
       " 'Michelle': <gensim.models.word2vec.Vocab at 0x7f61aa4e44e0>,\n",
       " 'confusing,': <gensim.models.word2vec.Vocab at 0x7f61aa4e4550>,\n",
       " 'YoUr': <gensim.models.word2vec.Vocab at 0x7f61aa4e4588>,\n",
       " 'away...': <gensim.models.word2vec.Vocab at 0x7f61aa4e5358>,\n",
       " 'handed': <gensim.models.word2vec.Vocab at 0x7f61aa4e45f8>,\n",
       " 'casual': <gensim.models.word2vec.Vocab at 0x7f61aa4e46a0>,\n",
       " 'colorful': <gensim.models.word2vec.Vocab at 0x7f61aa4e4828>,\n",
       " 'lives.': <gensim.models.word2vec.Vocab at 0x7f61aa4e4898>,\n",
       " 'selfishness...busying': <gensim.models.word2vec.Vocab at 0x7f61aa4e4978>,\n",
       " 'shakes': <gensim.models.word2vec.Vocab at 0x7f61aa4e4a20>,\n",
       " 'workouts.': <gensim.models.word2vec.Vocab at 0x7f61aa4e4b70>,\n",
       " 'upon': <gensim.models.word2vec.Vocab at 0x7f61aa4e4cc0>,\n",
       " 'BACK': <gensim.models.word2vec.Vocab at 0x7f61aa4e4d68>,\n",
       " 'Radio': <gensim.models.word2vec.Vocab at 0x7f61aa4e4e48>,\n",
       " '\"Truly,': <gensim.models.word2vec.Vocab at 0x7f61aa4e4e80>,\n",
       " 'lord': <gensim.models.word2vec.Vocab at 0x7f61aa4e4ef0>,\n",
       " 'Opening': <gensim.models.word2vec.Vocab at 0x7f61aa4e4f28>,\n",
       " 'counts?': <gensim.models.word2vec.Vocab at 0x7f61aa4e4f60>,\n",
       " 'sorry?': <gensim.models.word2vec.Vocab at 0x7f61aa5df780>,\n",
       " 'His': <gensim.models.word2vec.Vocab at 0x7f61aa4e1710>,\n",
       " 'article': <gensim.models.word2vec.Vocab at 0x7f61aa4e1860>,\n",
       " '(Dear': <gensim.models.word2vec.Vocab at 0x7f61aa4e1898>,\n",
       " 'FAITH': <gensim.models.word2vec.Vocab at 0x7f61aa4e1b70>,\n",
       " 'Girl**': <gensim.models.word2vec.Vocab at 0x7f61aa4e1c18>,\n",
       " 'school': <gensim.models.word2vec.Vocab at 0x7f61aa5df7b8>,\n",
       " 'hheeh.': <gensim.models.word2vec.Vocab at 0x7f61aa4e1dd8>,\n",
       " 'done,': <gensim.models.word2vec.Vocab at 0x7f61aa4e1e48>,\n",
       " 'foot': <gensim.models.word2vec.Vocab at 0x7f61aa4e1eb8>,\n",
       " 'change...ppl': <gensim.models.word2vec.Vocab at 0x7f61aa4e1f28>,\n",
       " 'lungs': <gensim.models.word2vec.Vocab at 0x7f61aa4e1fd0>,\n",
       " \"didn't\": <gensim.models.word2vec.Vocab at 0x7f61aa4e1630>,\n",
       " ']': <gensim.models.word2vec.Vocab at 0x7f61aa4e1048>,\n",
       " 'summer.': <gensim.models.word2vec.Vocab at 0x7f61aa4e1080>,\n",
       " 'side,': <gensim.models.word2vec.Vocab at 0x7f61aa4e10b8>,\n",
       " 'this': <gensim.models.word2vec.Vocab at 0x7f61aa4e1128>,\n",
       " 'step': <gensim.models.word2vec.Vocab at 0x7f61aa4e1160>,\n",
       " 'sloth': <gensim.models.word2vec.Vocab at 0x7f61aa4e11d0>,\n",
       " 'essences,': <gensim.models.word2vec.Vocab at 0x7f61aa4e1438>,\n",
       " 'spice': <gensim.models.word2vec.Vocab at 0x7f61aa4e14e0>,\n",
       " 'Interesting:': <gensim.models.word2vec.Vocab at 0x7f61aa4e1518>,\n",
       " 'survive': <gensim.models.word2vec.Vocab at 0x7f61aa4e1588>,\n",
       " 'intelligence\"': <gensim.models.word2vec.Vocab at 0x7f61aa4e15c0>,\n",
       " 'cliff': <gensim.models.word2vec.Vocab at 0x7f61aa53c048>,\n",
       " 'dragging': <gensim.models.word2vec.Vocab at 0x7f61aa53c080>,\n",
       " 'Worst': <gensim.models.word2vec.Vocab at 0x7f61aa5c1ba8>,\n",
       " '\"L\"': <gensim.models.word2vec.Vocab at 0x7f61aa53c160>,\n",
       " 'columnists': <gensim.models.word2vec.Vocab at 0x7f61aa53c198>,\n",
       " 'shopping.': <gensim.models.word2vec.Vocab at 0x7f61aa53c1d0>,\n",
       " 'have...satisfied': <gensim.models.word2vec.Vocab at 0x7f61aa53c208>,\n",
       " 'lie.': <gensim.models.word2vec.Vocab at 0x7f61aa53c278>,\n",
       " 'flying': <gensim.models.word2vec.Vocab at 0x7f61aa53c320>,\n",
       " 'perhaps': <gensim.models.word2vec.Vocab at 0x7f61aa53c358>,\n",
       " 'myself..': <gensim.models.word2vec.Vocab at 0x7f61aa53c390>,\n",
       " 'thing.)': <gensim.models.word2vec.Vocab at 0x7f61aa53c3c8>,\n",
       " 'shattered': <gensim.models.word2vec.Vocab at 0x7f61aa53c400>,\n",
       " 'ACL': <gensim.models.word2vec.Vocab at 0x7f61aa53c438>,\n",
       " 'dressed,': <gensim.models.word2vec.Vocab at 0x7f61aa53c4a8>,\n",
       " 'someone...and': <gensim.models.word2vec.Vocab at 0x7f61aa53c588>,\n",
       " 'Random': <gensim.models.word2vec.Vocab at 0x7f61aa558a20>,\n",
       " 'painful': <gensim.models.word2vec.Vocab at 0x7f61aa53c5f8>,\n",
       " 'Florida?]:': <gensim.models.word2vec.Vocab at 0x7f61aa53c630>,\n",
       " 'Gulf': <gensim.models.word2vec.Vocab at 0x7f61aa53c668>,\n",
       " 'stupid': <gensim.models.word2vec.Vocab at 0x7f61aa53c6a0>,\n",
       " 'kneecap': <gensim.models.word2vec.Vocab at 0x7f61aa53c6d8>,\n",
       " '26th': <gensim.models.word2vec.Vocab at 0x7f61aa53c710>,\n",
       " 'recently': <gensim.models.word2vec.Vocab at 0x7f61aa53c748>,\n",
       " 'Eye': <gensim.models.word2vec.Vocab at 0x7f61aa53c780>,\n",
       " 'Insecure:': <gensim.models.word2vec.Vocab at 0x7f61aa53c7b8>,\n",
       " 'Organized:': <gensim.models.word2vec.Vocab at 0x7f61aa53c7f0>,\n",
       " 'school...*sigh*': <gensim.models.word2vec.Vocab at 0x7f61aa53c828>,\n",
       " 'shoulders': <gensim.models.word2vec.Vocab at 0x7f61aa53c860>,\n",
       " 'MoO': <gensim.models.word2vec.Vocab at 0x7f61aa53c898>,\n",
       " 'following': <gensim.models.word2vec.Vocab at 0x7f61aa53c8d0>,\n",
       " 'on,': <gensim.models.word2vec.Vocab at 0x7f61aa53c908>,\n",
       " 'pollution,': <gensim.models.word2vec.Vocab at 0x7f61aa53c940>,\n",
       " 'rosalie': <gensim.models.word2vec.Vocab at 0x7f61aa53c978>,\n",
       " 'law': <gensim.models.word2vec.Vocab at 0x7f61aa53c9b0>,\n",
       " 'norway,': <gensim.models.word2vec.Vocab at 0x7f61aa53c9e8>,\n",
       " 'have]': <gensim.models.word2vec.Vocab at 0x7f61aa5b6438>,\n",
       " '...cheers': <gensim.models.word2vec.Vocab at 0x7f61aa53ca58>,\n",
       " 'DrAmA': <gensim.models.word2vec.Vocab at 0x7f61aa53ca90>,\n",
       " 'searching': <gensim.models.word2vec.Vocab at 0x7f61aa5b4240>,\n",
       " 'people!': <gensim.models.word2vec.Vocab at 0x7f61aa53cb00>,\n",
       " 'fun!': <gensim.models.word2vec.Vocab at 0x7f61aa53cb38>,\n",
       " 'Yellowcard': <gensim.models.word2vec.Vocab at 0x7f61aa53cb70>,\n",
       " 'terminally': <gensim.models.word2vec.Vocab at 0x7f61aa53cba8>,\n",
       " 'right.': <gensim.models.word2vec.Vocab at 0x7f61aa53cbe0>,\n",
       " 'feet': <gensim.models.word2vec.Vocab at 0x7f61aa53cc18>,\n",
       " 'person.': <gensim.models.word2vec.Vocab at 0x7f61aa53cc50>,\n",
       " \"they're\": <gensim.models.word2vec.Vocab at 0x7f61aa53cc88>,\n",
       " 'Opposition': <gensim.models.word2vec.Vocab at 0x7f61aa53ccc0>,\n",
       " \"veterans'\": <gensim.models.word2vec.Vocab at 0x7f61aa53ccf8>,\n",
       " 'Quiz': <gensim.models.word2vec.Vocab at 0x7f61aa53cd30>,\n",
       " 'lying,': <gensim.models.word2vec.Vocab at 0x7f61aa53cd68>,\n",
       " '7.': <gensim.models.word2vec.Vocab at 0x7f61aa53cda0>,\n",
       " 'mention': <gensim.models.word2vec.Vocab at 0x7f61aa53cdd8>,\n",
       " 'weirdest': <gensim.models.word2vec.Vocab at 0x7f61aa53ce10>,\n",
       " '\"Stay': <gensim.models.word2vec.Vocab at 0x7f61aa5d7b00>,\n",
       " 'rear': <gensim.models.word2vec.Vocab at 0x7f61aa53ce48>,\n",
       " 'clairol': <gensim.models.word2vec.Vocab at 0x7f61aa53ce80>,\n",
       " 'nvm': <gensim.models.word2vec.Vocab at 0x7f61aa53ceb8>,\n",
       " 'minute': <gensim.models.word2vec.Vocab at 0x7f61aa558358>,\n",
       " 'getting': <gensim.models.word2vec.Vocab at 0x7f61aa53cf60>,\n",
       " 'prefer': <gensim.models.word2vec.Vocab at 0x7f61aa53cf98>,\n",
       " 'open': <gensim.models.word2vec.Vocab at 0x7f61aa53cfd0>,\n",
       " 'feeble': <gensim.models.word2vec.Vocab at 0x7f61aa54b048>,\n",
       " 'October': <gensim.models.word2vec.Vocab at 0x7f61aa5bb160>,\n",
       " 'LIKE': <gensim.models.word2vec.Vocab at 0x7f61aa54b0b8>,\n",
       " 'do': <gensim.models.word2vec.Vocab at 0x7f61aa54b0f0>,\n",
       " 'amount': <gensim.models.word2vec.Vocab at 0x7f61aa54b128>,\n",
       " 'gerbils': <gensim.models.word2vec.Vocab at 0x7f61aa54b160>,\n",
       " 'nasty': <gensim.models.word2vec.Vocab at 0x7f61aa558400>,\n",
       " 'Responsible:': <gensim.models.word2vec.Vocab at 0x7f61aa54b1d0>,\n",
       " 'America.': <gensim.models.word2vec.Vocab at 0x7f61aa54b208>,\n",
       " '\"I\\'d': <gensim.models.word2vec.Vocab at 0x7f61aa54b240>,\n",
       " 'game': <gensim.models.word2vec.Vocab at 0x7f61aa54b278>,\n",
       " 'behind\"': <gensim.models.word2vec.Vocab at 0x7f61aa54b2b0>,\n",
       " 'Free': <gensim.models.word2vec.Vocab at 0x7f61aa54b2e8>,\n",
       " '6:30.': <gensim.models.word2vec.Vocab at 0x7f61aa54b320>,\n",
       " 'doom,': <gensim.models.word2vec.Vocab at 0x7f61aa54b358>,\n",
       " 'family,': <gensim.models.word2vec.Vocab at 0x7f61aa54b390>,\n",
       " 'odd': <gensim.models.word2vec.Vocab at 0x7f61aa54b3c8>,\n",
       " 'bio': <gensim.models.word2vec.Vocab at 0x7f61aa54b400>,\n",
       " 'going...': <gensim.models.word2vec.Vocab at 0x7f61aa54b438>,\n",
       " 'post-its,': <gensim.models.word2vec.Vocab at 0x7f61aa54b470>,\n",
       " 'teachers': <gensim.models.word2vec.Vocab at 0x7f61aa54b4a8>,\n",
       " 'Time': <gensim.models.word2vec.Vocab at 0x7f61aa54b4e0>,\n",
       " '11:10': <gensim.models.word2vec.Vocab at 0x7f61aa54b518>,\n",
       " 'orchestra...': <gensim.models.word2vec.Vocab at 0x7f61aa569b38>,\n",
       " 'jacket': <gensim.models.word2vec.Vocab at 0x7f61aa54b588>,\n",
       " 'Talkative:': <gensim.models.word2vec.Vocab at 0x7f61aa54b5c0>,\n",
       " 'left-middle': <gensim.models.word2vec.Vocab at 0x7f61aa54b5f8>,\n",
       " 'radical': <gensim.models.word2vec.Vocab at 0x7f61aa54b630>,\n",
       " 'forever.': <gensim.models.word2vec.Vocab at 0x7f61aa54b668>,\n",
       " 'Guess': <gensim.models.word2vec.Vocab at 0x7f61aa560b38>,\n",
       " 'them,': <gensim.models.word2vec.Vocab at 0x7f61aa54b6d8>,\n",
       " 'normal,': <gensim.models.word2vec.Vocab at 0x7f61aa5dfc18>,\n",
       " \"lavigne's\": <gensim.models.word2vec.Vocab at 0x7f61aa54b748>,\n",
       " 'places.': <gensim.models.word2vec.Vocab at 0x7f61aa54b7b8>,\n",
       " 'laugh': <gensim.models.word2vec.Vocab at 0x7f61aa54b7f0>,\n",
       " 'vik': <gensim.models.word2vec.Vocab at 0x7f61aa54b860>,\n",
       " 'yet...or': <gensim.models.word2vec.Vocab at 0x7f61aa5cec50>,\n",
       " 'night..': <gensim.models.word2vec.Vocab at 0x7f61aa54b898>,\n",
       " 'states': <gensim.models.word2vec.Vocab at 0x7f61aa54b8d0>,\n",
       " 'done)': <gensim.models.word2vec.Vocab at 0x7f61aa54b908>,\n",
       " 'excuses': <gensim.models.word2vec.Vocab at 0x7f61aa5dfcc0>,\n",
       " 'treason.': <gensim.models.word2vec.Vocab at 0x7f61aa54b978>,\n",
       " 'Gold': <gensim.models.word2vec.Vocab at 0x7f61aa54b9b0>,\n",
       " 'words?': <gensim.models.word2vec.Vocab at 0x7f61aa54b9e8>,\n",
       " 'fall': <gensim.models.word2vec.Vocab at 0x7f61aa54ba20>,\n",
       " 'online': <gensim.models.word2vec.Vocab at 0x7f61aa54ba58>,\n",
       " 'lips,': <gensim.models.word2vec.Vocab at 0x7f61aa54ba90>,\n",
       " 'PLEAAAASSSSSSEEEEEEE': <gensim.models.word2vec.Vocab at 0x7f61aa54bac8>,\n",
       " 'God': <gensim.models.word2vec.Vocab at 0x7f61aa54bb00>,\n",
       " 'b/c': <gensim.models.word2vec.Vocab at 0x7f61aa54bb38>,\n",
       " 'worst': <gensim.models.word2vec.Vocab at 0x7f61aa54bb70>,\n",
       " 'cancelling': <gensim.models.word2vec.Vocab at 0x7f61aa54bba8>,\n",
       " 'by': <gensim.models.word2vec.Vocab at 0x7f61aa54bbe0>,\n",
       " 'BS': <gensim.models.word2vec.Vocab at 0x7f61aa54bc18>,\n",
       " 'bugs': <gensim.models.word2vec.Vocab at 0x7f61aa54bc50>,\n",
       " 'succumb': <gensim.models.word2vec.Vocab at 0x7f61aa54bc88>,\n",
       " 'baby...': <gensim.models.word2vec.Vocab at 0x7f61aa54bcc0>,\n",
       " 'seems': <gensim.models.word2vec.Vocab at 0x7f61aa54bcf8>,\n",
       " 'color(s):': <gensim.models.word2vec.Vocab at 0x7f61aa54bd30>,\n",
       " 'Washington-based': <gensim.models.word2vec.Vocab at 0x7f61aa54bd68>,\n",
       " 'support': <gensim.models.word2vec.Vocab at 0x7f61aa54bda0>,\n",
       " 'never)': <gensim.models.word2vec.Vocab at 0x7f61aa54bdd8>,\n",
       " 'afternoon': <gensim.models.word2vec.Vocab at 0x7f61aa54be10>,\n",
       " 'sprints.': <gensim.models.word2vec.Vocab at 0x7f61aa54be48>,\n",
       " 'tank': <gensim.models.word2vec.Vocab at 0x7f61aa54be80>,\n",
       " 'center': <gensim.models.word2vec.Vocab at 0x7f61aa445080>,\n",
       " 'repetition': <gensim.models.word2vec.Vocab at 0x7f61aa54bef0>,\n",
       " 'loneliness': <gensim.models.word2vec.Vocab at 0x7f61aa5e5a90>,\n",
       " '\"Fast': <gensim.models.word2vec.Vocab at 0x7f61aa54bf60>,\n",
       " 'UNDERWORLD': <gensim.models.word2vec.Vocab at 0x7f61aa438208>,\n",
       " '(hmm,': <gensim.models.word2vec.Vocab at 0x7f61aa54bfd0>,\n",
       " 'shoes.': <gensim.models.word2vec.Vocab at 0x7f61aa54f048>,\n",
       " '(chocolate': <gensim.models.word2vec.Vocab at 0x7f61aa54f080>,\n",
       " 'THE': <gensim.models.word2vec.Vocab at 0x7f61aa54f0b8>,\n",
       " 'bakin': <gensim.models.word2vec.Vocab at 0x7f61aa54f0f0>,\n",
       " 'those': <gensim.models.word2vec.Vocab at 0x7f61aa54f128>,\n",
       " 'post...my': <gensim.models.word2vec.Vocab at 0x7f61aa5dfe48>,\n",
       " 'about.': <gensim.models.word2vec.Vocab at 0x7f61aa54f198>,\n",
       " 'helped': <gensim.models.word2vec.Vocab at 0x7f61aa54f1d0>,\n",
       " 'hit': <gensim.models.word2vec.Vocab at 0x7f61aa54f240>,\n",
       " 'unlike': <gensim.models.word2vec.Vocab at 0x7f61aa54f278>,\n",
       " 'comments,': <gensim.models.word2vec.Vocab at 0x7f61aa54f2b0>,\n",
       " 'yellow.': <gensim.models.word2vec.Vocab at 0x7f61aa54f2e8>,\n",
       " 'youll': <gensim.models.word2vec.Vocab at 0x7f61aa54f320>,\n",
       " 'Finally': <gensim.models.word2vec.Vocab at 0x7f61aa54f358>,\n",
       " 'David': <gensim.models.word2vec.Vocab at 0x7f61aa54f390>,\n",
       " 'cover': <gensim.models.word2vec.Vocab at 0x7f61aa54f3c8>,\n",
       " 'Colin': <gensim.models.word2vec.Vocab at 0x7f61aa54f400>,\n",
       " 'complain': <gensim.models.word2vec.Vocab at 0x7f61aa54f438>,\n",
       " 'sometime': <gensim.models.word2vec.Vocab at 0x7f61aa54f470>,\n",
       " 'shore,': <gensim.models.word2vec.Vocab at 0x7f61aa54f4a8>,\n",
       " 'be?]:': <gensim.models.word2vec.Vocab at 0x7f61aa4420b8>,\n",
       " 'lee': <gensim.models.word2vec.Vocab at 0x7f61aa54f4e0>,\n",
       " 'Lonely': <gensim.models.word2vec.Vocab at 0x7f61aa54f518>,\n",
       " 'starred': <gensim.models.word2vec.Vocab at 0x7f61aa54f550>,\n",
       " 'sumtin': <gensim.models.word2vec.Vocab at 0x7f61aa54f588>,\n",
       " 'tints?': <gensim.models.word2vec.Vocab at 0x7f61aa54f5c0>,\n",
       " 'homework': <gensim.models.word2vec.Vocab at 0x7f61aa54f5f8>,\n",
       " 'towers': <gensim.models.word2vec.Vocab at 0x7f61aa54f630>,\n",
       " 'saddest': <gensim.models.word2vec.Vocab at 0x7f61aa46bcf8>,\n",
       " 'Garden,': <gensim.models.word2vec.Vocab at 0x7f61aa54f668>,\n",
       " 'green,': <gensim.models.word2vec.Vocab at 0x7f61aa54f6a0>,\n",
       " 'you:': <gensim.models.word2vec.Vocab at 0x7f61aa54f6d8>,\n",
       " 'sex?': <gensim.models.word2vec.Vocab at 0x7f61aa54f710>,\n",
       " 'black,': <gensim.models.word2vec.Vocab at 0x7f61aa54f748>,\n",
       " 'feasible,': <gensim.models.word2vec.Vocab at 0x7f61aa54f780>,\n",
       " 'YOU...': <gensim.models.word2vec.Vocab at 0x7f61aa54f7b8>,\n",
       " 'trouble?': <gensim.models.word2vec.Vocab at 0x7f61aa54f7f0>,\n",
       " 'me...appreciative': <gensim.models.word2vec.Vocab at 0x7f61aa4609e8>,\n",
       " 'learner': <gensim.models.word2vec.Vocab at 0x7f61aa54f860>,\n",
       " 'hours': <gensim.models.word2vec.Vocab at 0x7f61aa54f898>,\n",
       " 'feast': <gensim.models.word2vec.Vocab at 0x7f61aa54f8d0>,\n",
       " 'again!': <gensim.models.word2vec.Vocab at 0x7f61aa54f908>,\n",
       " 'tip': <gensim.models.word2vec.Vocab at 0x7f61aa54f940>,\n",
       " 'You...': <gensim.models.word2vec.Vocab at 0x7f61aa54f978>,\n",
       " 'KNOW': <gensim.models.word2vec.Vocab at 0x7f61aa54f9b0>,\n",
       " 'purple': <gensim.models.word2vec.Vocab at 0x7f61aa54f9e8>,\n",
       " 'Dreams': <gensim.models.word2vec.Vocab at 0x7f61aa54fa20>,\n",
       " 'here': <gensim.models.word2vec.Vocab at 0x7f61aa54fa58>,\n",
       " 'accused': <gensim.models.word2vec.Vocab at 0x7f61aa54fa90>,\n",
       " 'since': <gensim.models.word2vec.Vocab at 0x7f61aa54fac8>,\n",
       " 'HATE': <gensim.models.word2vec.Vocab at 0x7f61aa54fb00>,\n",
       " 'walk': <gensim.models.word2vec.Vocab at 0x7f61aa54fb38>,\n",
       " 'outta': <gensim.models.word2vec.Vocab at 0x7f61aa54fb70>,\n",
       " 'yet,': <gensim.models.word2vec.Vocab at 0x7f61aa54fba8>,\n",
       " \"other...we're\": <gensim.models.word2vec.Vocab at 0x7f61aa54fbe0>,\n",
       " 'look': <gensim.models.word2vec.Vocab at 0x7f61aa54fc18>,\n",
       " ':-/': <gensim.models.word2vec.Vocab at 0x7f61aa54fc50>,\n",
       " 'yet': <gensim.models.word2vec.Vocab at 0x7f61aa54fc88>,\n",
       " 'background': <gensim.models.word2vec.Vocab at 0x7f61aa54fcc0>,\n",
       " 'is.': <gensim.models.word2vec.Vocab at 0x7f61aa54fcf8>,\n",
       " 'now...': <gensim.models.word2vec.Vocab at 0x7f61aa54fd68>,\n",
       " 'grow': <gensim.models.word2vec.Vocab at 0x7f61aa54fda0>,\n",
       " 'dough': <gensim.models.word2vec.Vocab at 0x7f61aa54fdd8>,\n",
       " 'government,': <gensim.models.word2vec.Vocab at 0x7f61aa54fe10>,\n",
       " 'okie...that': <gensim.models.word2vec.Vocab at 0x7f61aa54fe48>,\n",
       " 'plan': <gensim.models.word2vec.Vocab at 0x7f61aa54fe80>,\n",
       " 'ummm...': <gensim.models.word2vec.Vocab at 0x7f61aa54feb8>,\n",
       " 'king....': <gensim.models.word2vec.Vocab at 0x7f61aa54fef0>,\n",
       " 'Marianne': <gensim.models.word2vec.Vocab at 0x7f61aa54ff60>,\n",
       " 'until': <gensim.models.word2vec.Vocab at 0x7f61aa54ff98>,\n",
       " 'mashed': <gensim.models.word2vec.Vocab at 0x7f61aa5e1208>,\n",
       " 'rain': <gensim.models.word2vec.Vocab at 0x7f61aa553048>,\n",
       " 'freshman': <gensim.models.word2vec.Vocab at 0x7f61aa553080>,\n",
       " 'calls': <gensim.models.word2vec.Vocab at 0x7f61aa5530b8>,\n",
       " \"us...we're\": <gensim.models.word2vec.Vocab at 0x7f61aa5530f0>,\n",
       " 'Soviet': <gensim.models.word2vec.Vocab at 0x7f61aa553128>,\n",
       " 'gears,': <gensim.models.word2vec.Vocab at 0x7f61aa553160>,\n",
       " 'knife': <gensim.models.word2vec.Vocab at 0x7f61aa553198>,\n",
       " 'Floods,': <gensim.models.word2vec.Vocab at 0x7f61aa5531d0>,\n",
       " '(and': <gensim.models.word2vec.Vocab at 0x7f61aa553208>,\n",
       " 'America': <gensim.models.word2vec.Vocab at 0x7f61aa553240>,\n",
       " 'shi,': <gensim.models.word2vec.Vocab at 0x7f61aa553278>,\n",
       " 'considering': <gensim.models.word2vec.Vocab at 0x7f61aa5532b0>,\n",
       " 'committed': <gensim.models.word2vec.Vocab at 0x7f61aa5532e8>,\n",
       " 'situation,': <gensim.models.word2vec.Vocab at 0x7f61aa553320>,\n",
       " 'stole': <gensim.models.word2vec.Vocab at 0x7f61aa553358>,\n",
       " 'brushing': <gensim.models.word2vec.Vocab at 0x7f61aa553390>,\n",
       " 'happily': <gensim.models.word2vec.Vocab at 0x7f61aa5533c8>,\n",
       " 'hand': <gensim.models.word2vec.Vocab at 0x7f61aa553400>,\n",
       " 'problem': <gensim.models.word2vec.Vocab at 0x7f61aa553438>,\n",
       " 'us': <gensim.models.word2vec.Vocab at 0x7f61aa553470>,\n",
       " 'color': <gensim.models.word2vec.Vocab at 0x7f61aa5534a8>,\n",
       " 'barely': <gensim.models.word2vec.Vocab at 0x7f61aa558a90>,\n",
       " '2:': <gensim.models.word2vec.Vocab at 0x7f61aa553518>,\n",
       " 'repetition.': <gensim.models.word2vec.Vocab at 0x7f61aa553550>,\n",
       " 'ready': <gensim.models.word2vec.Vocab at 0x7f61aa553588>,\n",
       " 'everynight,': <gensim.models.word2vec.Vocab at 0x7f61aa5535c0>,\n",
       " 'brownies': <gensim.models.word2vec.Vocab at 0x7f61aa5364a8>,\n",
       " 'freaked': <gensim.models.word2vec.Vocab at 0x7f61aa553630>,\n",
       " 'medium.': <gensim.models.word2vec.Vocab at 0x7f61aa447048>,\n",
       " 'IS': <gensim.models.word2vec.Vocab at 0x7f61aa5536a0>,\n",
       " 'helps': <gensim.models.word2vec.Vocab at 0x7f61aa5536d8>,\n",
       " 'sophie?': <gensim.models.word2vec.Vocab at 0x7f61aa553710>,\n",
       " '\"Trust': <gensim.models.word2vec.Vocab at 0x7f61aa553748>,\n",
       " 'Now,': <gensim.models.word2vec.Vocab at 0x7f61aa536748>,\n",
       " 'tact': <gensim.models.word2vec.Vocab at 0x7f61aa5537b8>,\n",
       " 'needs': <gensim.models.word2vec.Vocab at 0x7f61aa5537f0>,\n",
       " 'uniter,': <gensim.models.word2vec.Vocab at 0x7f61aa553828>,\n",
       " 'He': <gensim.models.word2vec.Vocab at 0x7f61aa553860>,\n",
       " 'family)': <gensim.models.word2vec.Vocab at 0x7f61aa553898>,\n",
       " 'again...or': <gensim.models.word2vec.Vocab at 0x7f61aa5538d0>,\n",
       " 'hearts': <gensim.models.word2vec.Vocab at 0x7f61aa553908>,\n",
       " 'react': <gensim.models.word2vec.Vocab at 0x7f61aa5e1400>,\n",
       " 'Flogging': <gensim.models.word2vec.Vocab at 0x7f61aa553978>,\n",
       " 'running,': <gensim.models.word2vec.Vocab at 0x7f61aa5539e8>,\n",
       " 'razors': <gensim.models.word2vec.Vocab at 0x7f61aa4eaf28>,\n",
       " 'rarely': <gensim.models.word2vec.Vocab at 0x7f61aa445630>,\n",
       " 'daunted:': <gensim.models.word2vec.Vocab at 0x7f61aa553a90>,\n",
       " 'very': <gensim.models.word2vec.Vocab at 0x7f61aa553ac8>,\n",
       " 'around': <gensim.models.word2vec.Vocab at 0x7f61aa4ec4e0>,\n",
       " 'except': <gensim.models.word2vec.Vocab at 0x7f61aa553b38>,\n",
       " 'war,\"': <gensim.models.word2vec.Vocab at 0x7f61aa553b70>,\n",
       " 'become': <gensim.models.word2vec.Vocab at 0x7f61aa553ba8>,\n",
       " 'know,,,': <gensim.models.word2vec.Vocab at 0x7f61aa553be0>,\n",
       " 'asleep': <gensim.models.word2vec.Vocab at 0x7f61aa553c18>,\n",
       " 'sad...that': <gensim.models.word2vec.Vocab at 0x7f61aa4ed668>,\n",
       " 'of,': <gensim.models.word2vec.Vocab at 0x7f61aa553c88>,\n",
       " 'week,': <gensim.models.word2vec.Vocab at 0x7f61aa553cc0>,\n",
       " 'SATs...fuun...sux...but': <gensim.models.word2vec.Vocab at 0x7f61aa553cf8>,\n",
       " '...[should': <gensim.models.word2vec.Vocab at 0x7f61aa553d30>,\n",
       " 'dropped': <gensim.models.word2vec.Vocab at 0x7f61aa4ee0b8>,\n",
       " 'sure,': <gensim.models.word2vec.Vocab at 0x7f61aa553da0>,\n",
       " 'cool.': <gensim.models.word2vec.Vocab at 0x7f61aa553dd8>,\n",
       " 'jetlag': <gensim.models.word2vec.Vocab at 0x7f61aa553e10>,\n",
       " 'fit.': <gensim.models.word2vec.Vocab at 0x7f61aa54b828>,\n",
       " 'Arrogant:': <gensim.models.word2vec.Vocab at 0x7f61aa553e80>,\n",
       " 'now?]:': <gensim.models.word2vec.Vocab at 0x7f61aa553eb8>,\n",
       " 'objectives': <gensim.models.word2vec.Vocab at 0x7f61aa553ef0>,\n",
       " 'me...they': <gensim.models.word2vec.Vocab at 0x7f61aa553f28>,\n",
       " 'call': <gensim.models.word2vec.Vocab at 0x7f61aa553f60>,\n",
       " 'Today': <gensim.models.word2vec.Vocab at 0x7f61aa53b240>,\n",
       " 'checking': <gensim.models.word2vec.Vocab at 0x7f61aa53b400>,\n",
       " 'tried': <gensim.models.word2vec.Vocab at 0x7f61aa554048>,\n",
       " 'old,': <gensim.models.word2vec.Vocab at 0x7f61aa554080>,\n",
       " 'glasses': <gensim.models.word2vec.Vocab at 0x7f61aa5540b8>,\n",
       " 'bill': <gensim.models.word2vec.Vocab at 0x7f61aa5540f0>,\n",
       " 'fourth,': <gensim.models.word2vec.Vocab at 0x7f61aa554128>,\n",
       " 'better': <gensim.models.word2vec.Vocab at 0x7f61aa554160>,\n",
       " 'ground': <gensim.models.word2vec.Vocab at 0x7f61aa554198>,\n",
       " 'More': <gensim.models.word2vec.Vocab at 0x7f61aa5541d0>,\n",
       " 'gameroom': <gensim.models.word2vec.Vocab at 0x7f61aa4e5588>,\n",
       " 'above': <gensim.models.word2vec.Vocab at 0x7f61aa554240>,\n",
       " 'eventful.': <gensim.models.word2vec.Vocab at 0x7f61aa554278>,\n",
       " 'happen': <gensim.models.word2vec.Vocab at 0x7f61aa5542b0>,\n",
       " 'Lazy': <gensim.models.word2vec.Vocab at 0x7f61aa5542e8>,\n",
       " 'license': <gensim.models.word2vec.Vocab at 0x7f61aa4e8320>,\n",
       " 'bleating': <gensim.models.word2vec.Vocab at 0x7f61aa554358>,\n",
       " 'start.': <gensim.models.word2vec.Vocab at 0x7f61aa554390>,\n",
       " 'will': <gensim.models.word2vec.Vocab at 0x7f61aa5543c8>,\n",
       " '?': <gensim.models.word2vec.Vocab at 0x7f61aa554400>,\n",
       " 'napping': <gensim.models.word2vec.Vocab at 0x7f61aa554438>,\n",
       " 'Better?': <gensim.models.word2vec.Vocab at 0x7f61aa554470>,\n",
       " 'linoleum': <gensim.models.word2vec.Vocab at 0x7f61aa5544a8>,\n",
       " 'SOMETHING!': <gensim.models.word2vec.Vocab at 0x7f61aa5544e0>,\n",
       " 'sophie': <gensim.models.word2vec.Vocab at 0x7f61aa4d8828>,\n",
       " 'reacts,': <gensim.models.word2vec.Vocab at 0x7f61aa554550>,\n",
       " 'Car\"': <gensim.models.word2vec.Vocab at 0x7f61aa554588>,\n",
       " 'extinct.': <gensim.models.word2vec.Vocab at 0x7f61aa5e1550>,\n",
       " 'knowin': <gensim.models.word2vec.Vocab at 0x7f61aa5545f8>,\n",
       " 'looks': <gensim.models.word2vec.Vocab at 0x7f61aa554630>,\n",
       " 'alex!': <gensim.models.word2vec.Vocab at 0x7f61aa554668>,\n",
       " 'analyze': <gensim.models.word2vec.Vocab at 0x7f61aa5546a0>,\n",
       " 'internet': <gensim.models.word2vec.Vocab at 0x7f61aa5546d8>,\n",
       " 'am,': <gensim.models.word2vec.Vocab at 0x7f61aa554710>,\n",
       " \"I'll\": <gensim.models.word2vec.Vocab at 0x7f61aa554748>,\n",
       " 'go:': <gensim.models.word2vec.Vocab at 0x7f61aa554780>,\n",
       " 'hardest': <gensim.models.word2vec.Vocab at 0x7f61aa5547b8>,\n",
       " 'bed:': <gensim.models.word2vec.Vocab at 0x7f61aa5547f0>,\n",
       " 'tower!!': <gensim.models.word2vec.Vocab at 0x7f61aa554828>,\n",
       " '(analyze': <gensim.models.word2vec.Vocab at 0x7f61aa554860>,\n",
       " 'Rice': <gensim.models.word2vec.Vocab at 0x7f61aa554898>,\n",
       " 'bravest': <gensim.models.word2vec.Vocab at 0x7f61aa5548d0>,\n",
       " ...}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w2v.vocab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.082851942583535218"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w2v.similarity('I', 'My')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I've tried starting blog after blog and it just never feels right.  Then I read today that it feels strange to most people, but the more you do it the better it gets (hmm, sounds suspiciously like something else!) so I decided to give it another try.    My husband bought me a notepad at  urlLink McNally  (the best bookstore in Western Canada) with that title and a picture of a 50s housewife grinning desperately.  Each page has something funny like \"New curtains!  Hurrah!\".  For some reason it struck me as absolutely hilarious and has stuck in my head ever since.  What were those women thinking?\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.037229111896779618"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(posts[5])\n",
    "w2v.similarity('ring', 'husband')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.11547398696865138"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w2v.similarity('ring', 'housewife')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.14627530812290576"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w2v.similarity('women', 'housewife')  # Diversity friendly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Doc2Vec\n",
    "\n",
    "The same technique of word2vec is extrapolated to documents. Here, we do everything done in word2vec + we vectorize the documents too"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 0 for male, 1 for female\n",
    "y_posts = np.concatenate((np.zeros(len(filtered_male_posts)),\n",
    "                          np.ones(len(filtered_female_posts))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4842"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(y_posts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convolutional Neural Networks for Sentence Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train convolutional network for sentiment analysis. \n",
    "\n",
    "Based on\n",
    "\"Convolutional Neural Networks for Sentence Classification\" by Yoon Kim\n",
    "http://arxiv.org/pdf/1408.5882v2.pdf\n",
    "\n",
    "For 'CNN-non-static' gets to 82.1% after 61 epochs with following settings:\n",
    "embedding_dim = 20          \n",
    "filter_sizes = (3, 4)\n",
    "num_filters = 3\n",
    "dropout_prob = (0.7, 0.8)\n",
    "hidden_dims = 100\n",
    "\n",
    "For 'CNN-rand' gets to 78-79% after 7-8 epochs with following settings:\n",
    "embedding_dim = 20          \n",
    "filter_sizes = (3, 4)\n",
    "num_filters = 150\n",
    "dropout_prob = (0.25, 0.5)\n",
    "hidden_dims = 150\n",
    "\n",
    "For 'CNN-static' gets to 75.4% after 7 epochs with following settings:\n",
    "embedding_dim = 100          \n",
    "filter_sizes = (3, 4)\n",
    "num_filters = 150\n",
    "dropout_prob = (0.25, 0.5)\n",
    "hidden_dims = 150\n",
    "\n",
    "* it turns out that such a small data set as \"Movie reviews with one\n",
    "sentence per review\"  (Pang and Lee, 2005) requires much smaller network\n",
    "than the one introduced in the original article:\n",
    "- embedding dimension is only 20 (instead of 300; 'CNN-static' still requires ~100)\n",
    "- 2 filter sizes (instead of 3)\n",
    "- higher dropout probabilities and\n",
    "- 3 filters per filter size is enough for 'CNN-non-static' (instead of 100)\n",
    "- embedding initialization does not require prebuilt Google Word2Vec data.\n",
    "Training Word2Vec on the same \"Movie reviews\" data set is enough to \n",
    "achieve performance reported in the article (81.6%)\n",
    "\n",
    "** Another distinct difference is slidind MaxPooling window of length=2\n",
    "instead of MaxPooling over whole feature map as in the article"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using gpu device 0: GeForce GTX 760 (CNMeM is enabled with initial size: 90.0% of memory, cuDNN 4007)\n",
      "Using Theano backend.\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import word_embedding\n",
    "from word2vec import train_word2vec\n",
    "\n",
    "from keras.models import Sequential, Model\n",
    "from keras.layers import (Activation, Dense, Dropout, Embedding, \n",
    "                          Flatten, Input, \n",
    "                          Conv1D, MaxPooling1D)\n",
    "from keras.layers.merge import Concatenate\n",
    "\n",
    "np.random.seed(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parameters\n",
    "\n",
    "Model Variations. See Kim Yoon's Convolutional Neural Networks for \n",
    "Sentence Classification, Section 3 for detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model variation is CNN-rand\n"
     ]
    }
   ],
   "source": [
    "model_variation = 'CNN-rand'  #  CNN-rand | CNN-non-static | CNN-static\n",
    "print('Model variation is %s' % model_variation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Model Hyperparameters\n",
    "sequence_length = 56\n",
    "embedding_dim = 20          \n",
    "filter_sizes = (3, 4)\n",
    "num_filters = 150\n",
    "dropout_prob = (0.25, 0.5)\n",
    "hidden_dims = 150"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Training parameters\n",
    "batch_size = 32\n",
    "num_epochs = 100\n",
    "val_split = 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Word2Vec parameters, see train_word2vec\n",
    "min_word_count = 1  # Minimum word count                        \n",
    "context = 10        # Context window size    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Preparation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading data...\n"
     ]
    }
   ],
   "source": [
    "# Load data\n",
    "print(\"Loading data...\")\n",
    "x, y, vocabulary, vocabulary_inv = word_embedding.load_data()\n",
    "\n",
    "if model_variation=='CNN-non-static' or model_variation=='CNN-static':\n",
    "    embedding_weights = train_word2vec(x, vocabulary_inv, \n",
    "                                       embedding_dim, min_word_count, \n",
    "                                       context)\n",
    "    if model_variation=='CNN-static':\n",
    "        x = embedding_weights[0][x]\n",
    "elif model_variation=='CNN-rand':\n",
    "    embedding_weights = None\n",
    "else:\n",
    "    raise ValueError('Unknown model variation')    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Shuffle data\n",
    "shuffle_indices = np.random.permutation(np.arange(len(y)))\n",
    "x_shuffled = x[shuffle_indices]\n",
    "y_shuffled = y[shuffle_indices].argmax(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vocabulary Size: 18765\n"
     ]
    }
   ],
   "source": [
    "print(\"Vocabulary Size: {:d}\".format(len(vocabulary)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building CNN Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "graph_in = Input(shape=(sequence_length, embedding_dim))\n",
    "convs = []\n",
    "for fsz in filter_sizes:\n",
    "    conv = Conv1D(filters=num_filters,\n",
    "                  filter_length=fsz,\n",
    "                  padding='valid',\n",
    "                  activation='relu',\n",
    "                  strides=1)(graph_in)\n",
    "    pool = MaxPooling1D(pool_length=2)(conv)\n",
    "    flatten = Flatten()(pool)\n",
    "    convs.append(flatten)\n",
    "    \n",
    "if len(filter_sizes)>1:\n",
    "    out = Concatenate()(convs)\n",
    "else:\n",
    "    out = convs[0]\n",
    "\n",
    "graph = Model(input=graph_in, output=out)\n",
    "\n",
    "# main sequential model\n",
    "model = Sequential()\n",
    "if not model_variation=='CNN-static':\n",
    "    model.add(Embedding(len(vocabulary), embedding_dim, input_length=sequence_length,\n",
    "                        weights=embedding_weights))\n",
    "model.add(Dropout(dropout_prob[0], input_shape=(sequence_length, embedding_dim)))\n",
    "model.add(graph)\n",
    "model.add(Dense(hidden_dims))\n",
    "model.add(Dropout(dropout_prob[1]))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Dense(1))\n",
    "model.add(Activation('sigmoid'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 9595 samples, validate on 1067 samples\n",
      "Epoch 1/100\n",
      "1s - loss: 0.6516 - acc: 0.6005 - val_loss: 0.5692 - val_acc: 0.7151\n",
      "Epoch 2/100\n",
      "1s - loss: 0.4556 - acc: 0.7896 - val_loss: 0.5154 - val_acc: 0.7573\n",
      "Epoch 3/100\n",
      "1s - loss: 0.3556 - acc: 0.8532 - val_loss: 0.5050 - val_acc: 0.7816\n",
      "Epoch 4/100\n",
      "1s - loss: 0.2978 - acc: 0.8779 - val_loss: 0.5335 - val_acc: 0.7901\n",
      "Epoch 5/100\n",
      "1s - loss: 0.2599 - acc: 0.8972 - val_loss: 0.5592 - val_acc: 0.7769\n",
      "Epoch 6/100\n",
      "1s - loss: 0.2248 - acc: 0.9112 - val_loss: 0.5559 - val_acc: 0.7685\n",
      "Epoch 7/100\n",
      "1s - loss: 0.1994 - acc: 0.9219 - val_loss: 0.5760 - val_acc: 0.7704\n",
      "Epoch 8/100\n",
      "1s - loss: 0.1801 - acc: 0.9326 - val_loss: 0.6014 - val_acc: 0.7788\n",
      "Epoch 9/100\n",
      "1s - loss: 0.1472 - acc: 0.9449 - val_loss: 0.6637 - val_acc: 0.7751\n",
      "Epoch 10/100\n",
      "1s - loss: 0.1269 - acc: 0.9537 - val_loss: 0.7281 - val_acc: 0.7563\n",
      "Epoch 11/100\n",
      "1s - loss: 0.1123 - acc: 0.9592 - val_loss: 0.7452 - val_acc: 0.7788\n",
      "Epoch 12/100\n",
      "1s - loss: 0.0897 - acc: 0.9658 - val_loss: 0.8504 - val_acc: 0.7591\n",
      "Epoch 13/100\n",
      "1s - loss: 0.0811 - acc: 0.9723 - val_loss: 0.8935 - val_acc: 0.7573\n",
      "Epoch 14/100\n",
      "1s - loss: 0.0651 - acc: 0.9764 - val_loss: 0.8738 - val_acc: 0.7685\n",
      "Epoch 15/100\n",
      "1s - loss: 0.0540 - acc: 0.9809 - val_loss: 0.9407 - val_acc: 0.7648\n",
      "Epoch 16/100\n",
      "1s - loss: 0.0408 - acc: 0.9857 - val_loss: 1.1880 - val_acc: 0.7638\n",
      "Epoch 17/100\n",
      "1s - loss: 0.0341 - acc: 0.9886 - val_loss: 1.2878 - val_acc: 0.7638\n",
      "Epoch 18/100\n",
      "1s - loss: 0.0306 - acc: 0.9901 - val_loss: 1.4448 - val_acc: 0.7573\n",
      "Epoch 19/100\n",
      "1s - loss: 0.0276 - acc: 0.9917 - val_loss: 1.5300 - val_acc: 0.7591\n",
      "Epoch 20/100\n",
      "1s - loss: 0.0249 - acc: 0.9917 - val_loss: 1.4825 - val_acc: 0.7666\n",
      "Epoch 21/100\n",
      "1s - loss: 0.0220 - acc: 0.9937 - val_loss: 1.4357 - val_acc: 0.7601\n",
      "Epoch 22/100\n",
      "1s - loss: 0.0188 - acc: 0.9945 - val_loss: 1.4081 - val_acc: 0.7657\n",
      "Epoch 23/100\n",
      "1s - loss: 0.0182 - acc: 0.9954 - val_loss: 1.7145 - val_acc: 0.7610\n",
      "Epoch 24/100\n",
      "1s - loss: 0.0129 - acc: 0.9964 - val_loss: 1.7047 - val_acc: 0.7704\n",
      "Epoch 25/100\n",
      "1s - loss: 0.0064 - acc: 0.9981 - val_loss: 1.9119 - val_acc: 0.7629\n",
      "Epoch 26/100\n",
      "1s - loss: 0.0108 - acc: 0.9969 - val_loss: 1.8306 - val_acc: 0.7704\n",
      "Epoch 27/100\n",
      "1s - loss: 0.0105 - acc: 0.9973 - val_loss: 1.9624 - val_acc: 0.7619\n",
      "Epoch 28/100\n",
      "1s - loss: 0.0112 - acc: 0.9973 - val_loss: 1.8552 - val_acc: 0.7694\n",
      "Epoch 29/100\n",
      "1s - loss: 0.0110 - acc: 0.9968 - val_loss: 1.8585 - val_acc: 0.7657\n",
      "Epoch 30/100\n",
      "1s - loss: 0.0071 - acc: 0.9983 - val_loss: 2.0571 - val_acc: 0.7694\n",
      "Epoch 31/100\n",
      "1s - loss: 0.0089 - acc: 0.9975 - val_loss: 2.0361 - val_acc: 0.7629\n",
      "Epoch 32/100\n",
      "1s - loss: 0.0074 - acc: 0.9978 - val_loss: 2.0010 - val_acc: 0.7648\n",
      "Epoch 33/100\n",
      "1s - loss: 0.0074 - acc: 0.9981 - val_loss: 2.0995 - val_acc: 0.7498\n",
      "Epoch 34/100\n",
      "1s - loss: 0.0125 - acc: 0.9971 - val_loss: 2.2003 - val_acc: 0.7610\n",
      "Epoch 35/100\n",
      "1s - loss: 0.0074 - acc: 0.9981 - val_loss: 2.1526 - val_acc: 0.7582\n",
      "Epoch 36/100\n",
      "1s - loss: 0.0068 - acc: 0.9984 - val_loss: 2.1754 - val_acc: 0.7648\n",
      "Epoch 37/100\n",
      "1s - loss: 0.0065 - acc: 0.9979 - val_loss: 2.0810 - val_acc: 0.7498\n",
      "Epoch 38/100\n",
      "1s - loss: 0.0078 - acc: 0.9980 - val_loss: 2.3443 - val_acc: 0.7460\n",
      "Epoch 39/100\n",
      "1s - loss: 0.0038 - acc: 0.9991 - val_loss: 2.1696 - val_acc: 0.7629\n",
      "Epoch 40/100\n",
      "1s - loss: 0.0062 - acc: 0.9985 - val_loss: 2.2752 - val_acc: 0.7545\n",
      "Epoch 41/100\n",
      "1s - loss: 0.0044 - acc: 0.9985 - val_loss: 2.3457 - val_acc: 0.7535\n",
      "Epoch 42/100\n",
      "1s - loss: 0.0066 - acc: 0.9985 - val_loss: 2.1172 - val_acc: 0.7629\n",
      "Epoch 43/100\n",
      "1s - loss: 0.0052 - acc: 0.9987 - val_loss: 2.3550 - val_acc: 0.7619\n",
      "Epoch 44/100\n",
      "1s - loss: 0.0024 - acc: 0.9993 - val_loss: 2.3832 - val_acc: 0.7610\n",
      "Epoch 45/100\n",
      "1s - loss: 0.0042 - acc: 0.9989 - val_loss: 2.4242 - val_acc: 0.7648\n",
      "Epoch 46/100\n",
      "1s - loss: 0.0048 - acc: 0.9990 - val_loss: 2.4529 - val_acc: 0.7563\n",
      "Epoch 47/100\n",
      "1s - loss: 0.0036 - acc: 0.9994 - val_loss: 2.8412 - val_acc: 0.7282\n",
      "Epoch 48/100\n",
      "1s - loss: 0.0037 - acc: 0.9991 - val_loss: 2.4515 - val_acc: 0.7619\n",
      "Epoch 49/100\n",
      "1s - loss: 0.0031 - acc: 0.9991 - val_loss: 2.4849 - val_acc: 0.7676\n",
      "Epoch 50/100\n",
      "1s - loss: 0.0078 - acc: 0.9990 - val_loss: 2.5083 - val_acc: 0.7563\n",
      "Epoch 51/100\n",
      "1s - loss: 0.0105 - acc: 0.9981 - val_loss: 2.3538 - val_acc: 0.7601\n",
      "Epoch 52/100\n",
      "1s - loss: 0.0076 - acc: 0.9986 - val_loss: 2.4405 - val_acc: 0.7685\n",
      "Epoch 53/100\n",
      "1s - loss: 0.0043 - acc: 0.9991 - val_loss: 2.5753 - val_acc: 0.7591\n",
      "Epoch 54/100\n",
      "1s - loss: 0.0044 - acc: 0.9989 - val_loss: 2.5550 - val_acc: 0.7582\n",
      "Epoch 55/100\n",
      "1s - loss: 0.0034 - acc: 0.9994 - val_loss: 2.6361 - val_acc: 0.7591\n",
      "Epoch 56/100\n",
      "1s - loss: 0.0041 - acc: 0.9994 - val_loss: 2.6753 - val_acc: 0.7563\n",
      "Epoch 57/100\n",
      "1s - loss: 0.0042 - acc: 0.9990 - val_loss: 2.6464 - val_acc: 0.7601\n",
      "Epoch 58/100\n",
      "1s - loss: 0.0037 - acc: 0.9992 - val_loss: 2.6616 - val_acc: 0.7582\n",
      "Epoch 59/100\n",
      "1s - loss: 0.0060 - acc: 0.9990 - val_loss: 2.6052 - val_acc: 0.7619\n",
      "Epoch 60/100\n",
      "1s - loss: 0.0051 - acc: 0.9990 - val_loss: 2.7033 - val_acc: 0.7498\n",
      "Epoch 61/100\n",
      "1s - loss: 0.0034 - acc: 0.9994 - val_loss: 2.7142 - val_acc: 0.7526\n",
      "Epoch 62/100\n",
      "1s - loss: 0.0047 - acc: 0.9994 - val_loss: 2.7656 - val_acc: 0.7591\n",
      "Epoch 63/100\n",
      "1s - loss: 0.0083 - acc: 0.9990 - val_loss: 2.7971 - val_acc: 0.7526\n",
      "Epoch 64/100\n",
      "1s - loss: 0.0046 - acc: 0.9992 - val_loss: 2.6585 - val_acc: 0.7545\n",
      "Epoch 65/100\n",
      "1s - loss: 0.0062 - acc: 0.9989 - val_loss: 2.6194 - val_acc: 0.7535\n",
      "Epoch 66/100\n",
      "1s - loss: 0.0062 - acc: 0.9993 - val_loss: 2.6255 - val_acc: 0.7694\n",
      "Epoch 67/100\n",
      "1s - loss: 0.0036 - acc: 0.9990 - val_loss: 2.6384 - val_acc: 0.7582\n",
      "Epoch 68/100\n",
      "1s - loss: 0.0066 - acc: 0.9991 - val_loss: 2.6743 - val_acc: 0.7648\n",
      "Epoch 69/100\n",
      "1s - loss: 0.0030 - acc: 0.9995 - val_loss: 2.8236 - val_acc: 0.7535\n",
      "Epoch 70/100\n",
      "1s - loss: 0.0048 - acc: 0.9993 - val_loss: 2.7829 - val_acc: 0.7610\n",
      "Epoch 71/100\n",
      "1s - loss: 0.0062 - acc: 0.9990 - val_loss: 2.6402 - val_acc: 0.7573\n",
      "Epoch 72/100\n",
      "1s - loss: 0.0037 - acc: 0.9992 - val_loss: 2.9089 - val_acc: 0.7526\n",
      "Epoch 73/100\n",
      "1s - loss: 0.0069 - acc: 0.9985 - val_loss: 2.7071 - val_acc: 0.7535\n",
      "Epoch 74/100\n",
      "1s - loss: 0.0033 - acc: 0.9995 - val_loss: 2.6727 - val_acc: 0.7601\n",
      "Epoch 75/100\n",
      "1s - loss: 0.0069 - acc: 0.9990 - val_loss: 2.6967 - val_acc: 0.7601\n",
      "Epoch 76/100\n",
      "1s - loss: 0.0089 - acc: 0.9989 - val_loss: 2.7479 - val_acc: 0.7666\n",
      "Epoch 77/100\n",
      "1s - loss: 0.0046 - acc: 0.9994 - val_loss: 2.7192 - val_acc: 0.7629\n",
      "Epoch 78/100\n",
      "1s - loss: 0.0069 - acc: 0.9989 - val_loss: 2.7173 - val_acc: 0.7629\n",
      "Epoch 79/100\n",
      "1s - loss: 8.6550e-04 - acc: 0.9998 - val_loss: 2.7283 - val_acc: 0.7601\n",
      "Epoch 80/100\n",
      "1s - loss: 0.0011 - acc: 0.9995 - val_loss: 2.8405 - val_acc: 0.7629\n",
      "Epoch 81/100\n",
      "1s - loss: 0.0040 - acc: 0.9994 - val_loss: 2.8725 - val_acc: 0.7619\n",
      "Epoch 82/100\n",
      "1s - loss: 0.0055 - acc: 0.9992 - val_loss: 2.8490 - val_acc: 0.7601\n",
      "Epoch 83/100\n",
      "1s - loss: 0.0059 - acc: 0.9989 - val_loss: 2.7838 - val_acc: 0.7545\n",
      "Epoch 84/100\n",
      "1s - loss: 0.0054 - acc: 0.9994 - val_loss: 2.8706 - val_acc: 0.7526\n",
      "Epoch 85/100\n",
      "1s - loss: 0.0060 - acc: 0.9992 - val_loss: 2.9374 - val_acc: 0.7516\n",
      "Epoch 86/100\n",
      "1s - loss: 0.0087 - acc: 0.9982 - val_loss: 2.7966 - val_acc: 0.7573\n",
      "Epoch 87/100\n",
      "1s - loss: 0.0084 - acc: 0.9991 - val_loss: 2.8620 - val_acc: 0.7619\n",
      "Epoch 88/100\n",
      "1s - loss: 0.0053 - acc: 0.9990 - val_loss: 2.8450 - val_acc: 0.7601\n",
      "Epoch 89/100\n",
      "1s - loss: 0.0054 - acc: 0.9990 - val_loss: 2.8303 - val_acc: 0.7629\n",
      "Epoch 90/100\n",
      "1s - loss: 0.0073 - acc: 0.9991 - val_loss: 2.8474 - val_acc: 0.7657\n",
      "Epoch 91/100\n",
      "1s - loss: 0.0037 - acc: 0.9994 - val_loss: 3.0151 - val_acc: 0.7432\n",
      "Epoch 92/100\n",
      "1s - loss: 0.0017 - acc: 0.9999 - val_loss: 2.9555 - val_acc: 0.7582\n",
      "Epoch 93/100\n",
      "1s - loss: 0.0080 - acc: 0.9991 - val_loss: 2.9178 - val_acc: 0.7554\n",
      "Epoch 94/100\n",
      "1s - loss: 0.0078 - acc: 0.9991 - val_loss: 2.8724 - val_acc: 0.7582\n",
      "Epoch 95/100\n",
      "1s - loss: 0.0012 - acc: 0.9997 - val_loss: 2.9582 - val_acc: 0.7545\n",
      "Epoch 96/100\n",
      "1s - loss: 0.0058 - acc: 0.9989 - val_loss: 2.8944 - val_acc: 0.7479\n",
      "Epoch 97/100\n",
      "1s - loss: 0.0094 - acc: 0.9985 - val_loss: 2.7146 - val_acc: 0.7516\n",
      "Epoch 98/100\n",
      "1s - loss: 0.0044 - acc: 0.9993 - val_loss: 2.9052 - val_acc: 0.7498\n",
      "Epoch 99/100\n",
      "1s - loss: 0.0030 - acc: 0.9995 - val_loss: 3.1474 - val_acc: 0.7470\n",
      "Epoch 100/100\n",
      "1s - loss: 0.0051 - acc: 0.9990 - val_loss: 3.1746 - val_acc: 0.7451\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f78362ae400>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.compile(loss='binary_crossentropy', optimizer='rmsprop', \n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# Training model\n",
    "# ==================================================\n",
    "model.fit(x_shuffled, y_shuffled, batch_size=batch_size,\n",
    "          nb_epoch=num_epochs, validation_split=val_split, verbose=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Another Example\n",
    "\n",
    "Using Keras + [**GloVe**](http://nlp.stanford.edu/projects/glove/) - **Global Vectors for Word Representation**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}