neural_language_models/01_news_groups_doc2vec.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classification problem on the Twenty News Groups dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, we'll use a [neural language model](http://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf) to represent the emails in the <i>twenty newsgroups</i> dataset and build a classifier to identify emails belonging to <i>computer related topics</i> and those which aren't.\n",
    "\n",
    "We'll use the [Paragraph Vector](https://cs.stanford.edu/~quocle/paragraph_vector.pdf) representation as described by Quoc Le et. al to encode the news groups emails as dense vectors and build our classifiers on these dense vectors. We'll harness the popular NLP library in Python - [gensim](https://radimrehurek.com/gensim/), both as a stand-alone in the notebook, as well as in-database on [Greenplum](http://greenplum.org)/[HAWQ](http://hawq.incubator.apache.org/) via PL/Python. \n",
    "\n",
    "More information on the <i>Twenty Newsgroups</i> dataset can be found on the [UCI website](http://kdd.ics.uci.edu/databases/20newsgroups/20newsgroups.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup database connectivity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll reuse our module from the previous notebook (***`00_database_connectivity_setup.ipynb`***) to establish connectivity to the database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%run '00_database_connectivity_setup.ipynb'\n",
    "%matplotlib inline\n",
    "from IPython.display import display\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your connection object is ***`conn`***:\n",
    "1. Queries: You can run your queries using ***```psql.read_sql(\"\"\"<YOUR SQL>\"\"\", conn)```***.\n",
    "2. Create/Delete/Updates: You can run these statements using ***```psql.execute(\"\"\"<YOUR SQL>\"\"\", conn)```***, followed by a ***```conn.commit()```*** command to ensure your transaction is committed. Otherwise your changes will be rolledback if you terminate your kernel.\n",
    "\n",
    "If you created a new connection object (say to connect to a new cluster) as shown in the last section of `00_database_connectivity_setup.ipynb` notebook, use that connection object where needed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Create your schema"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sql = \"\"\"\n",
    "    create schema YOUR_SCHEMA;\n",
    "\"\"\"\n",
    "psql.execute(sql, conn)\n",
    "conn.commit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load the Twenty News Groups dataset into a database table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%execsql\n",
    "-- Define external table to fetch data from HDFS\n",
    "drop external table if exists YOUR_SCHEMA.twenty_news_groups_ext cascade;\n",
    "create external table YOUR_SCHEMA.twenty_news_groups_ext\n",
    "(\n",
    "    doc_id int,\n",
    "    contents text,\n",
    "    label text\n",
    ") location ('pxf://hdm1.gphd.local:50070/user/vatsan/dstraining/twenty_news_groups_processed.tsv?profile=hdfstextsimple') \n",
    "format 'CSV' (DELIMITER = E'\\t');\n",
    "\n",
    "-- create an internal table\n",
    "drop table if exists YOUR_SCHEMA.twenty_news_groups cascade;\n",
    "create table YOUR_SCHEMA.twenty_news_groups\n",
    "as\n",
    "(\n",
    "    select \n",
    "        *\n",
    "    from\n",
    "        YOUR_SCHEMA.twenty_news_groups_ext\n",
    ") distributed randomly;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc_id</th>\n",
       "      <th>contents</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>58</td>\n",
       "      <td>From: acooper@mac.cc.macalstr.edu (Turin Turambar, ME Department of Utter Misery)\\nSubject: Re: Societally acceptable behavior\\nOrganization: Macalester College\\nLines: 55\\n\\nIn article &lt;C5sA29.14s@news.cso.uiuc.edu&gt;, cobb@alexia.lis.uiuc.edu (Mike Cobb) writes:\\n&gt; I guess I'm delving into a religious language area.  What exactly is morality \\n&gt; or morals?  \\n\\nI hope there is not one- with a subject like this you just have a spiral.  What\\nwould then be a morality of a morality of morals.  Labels don't make arguments. \\nOne really needs a solid measuring stick by which most actions can be\\ninterpreted, even though this would hardly seem moral.  For example \"The best\\nthing for me is to ensure that I will eat and drink enough.  Hence all actions\\nmust be weighed against this one statement.\"  whatever helps this goal is\\n\"moral\", whatever does not is \"immoral\"\\n\\nOf course this leads such a blank space: there are so many different ways to\\nfulfill a goal, one would need a \"hyper-morality\" to apply to just the methods.\\n\\n&gt;I never thought of eating meat to be moral or immoral, but I think\\n&gt; it could be.  How do we differentiate between not doing something because it is\\n&gt; a personal choice or preference and not doing something because we see it as \\n&gt; immoral?  Do we fall to what the basis of these morals are?\\n\\nSeems to me we only consider something moral or immoral if we stop to think\\nabout it long enough  :)  On the other hand, maybe it is our first gut\\nreaction...  Which?  Who knows: perhaps here we have a way to discriminate\\nmorals.  I don't instinctively thing vegetarianism is right (the same way I\\ninstinctively feel torture is wrong), but if I thought about it long enough and\\nlistened to the arguments, I could perhaps reason that it was wrong (is that\\npossible!?  :) )  See the difference?\\n\\n&gt; \\n&gt; Also, consensus positions fall to a might makes right.  Or, as you brought out,\\n&gt; if whatever is right is what is societally mandated then whoever is in control\\n&gt; at the time makes what is right\\n&gt; \\n&gt; MC\\n&gt; MAC\\n&gt; --\\n&gt; ****************************************************************\\n&gt;                                                     Michael A. Cobb\\n&gt;  \"...and I won't raise taxes on the middle     University of Illinois\\n&gt;     class to pay for my programs.\"                 Champaign-Urbana\\n&gt;           -Bill Clinton 3rd Debate             cobb@alexia.lis.uiuc.edu\\n&gt;                                               \\n&gt; Nobody can explain everything to anybody.  G.K.Chesterton\\n-- \\n\\n\\nbest regards,\\n\\n--Adam\\n\\n********************************************************************************\\n* Adam John Cooper\\t\\t\"Verily, often have I laughed at the weaklings *\\n* (612) 696-7521\\t\\t   who thought themselves good simply because  *\\n* acooper@macalstr.edu\\t\\t\\t\\tthey had no claws.\"\\t       *\\n********************************************************************************\\n</td>\n",
       "      <td>alt.atheism</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>154</td>\n",
       "      <td>From: frank@D012S658.uucp (Frank O'Dwyer)\\nSubject: Re: Societal basis for morality\\nOrganization: Siemens-Nixdorf AG\\nLines: 49\\nNNTP-Posting-Host: d012s658.ap.mchp.sni.de\\n\\nIn article &lt;1993Apr20.004119.6119@cnsvax.uwec.edu&gt; nyeda@cnsvax.uwec.edu (David Nye) writes:\\n\\nYou asked me to look over here, but I was on my way back anyway :-)\\n\\n#[reply to cobb@alexia.lis.uiuc.edu (Mike Cobb)]\\n# \\n#&gt;If morals come from what is societally accepted, why follow that? What\\n#&gt;right do we have to expect others to follow our notion of societally\\n#&gt;mandated morality?  Pardon the extremism, but couldn't I murder your\\n#&gt;\"brother\" and say that I was exercising my rights as I saw them, was\\n#&gt;doing what felt good, didn't want anyone forcing their morality on me,\\n#&gt;or I don't follow your \"morality\" ?\\n# \\n#I believe that morality is subjective.  Each person is entitled to his\\n#own moral attitudes.  Mine are not a priori more correct than someone\\n#elses.  This does not mean however that I must judge another on the\\n#basis of his rather than my moral standards.  While he is entitled to\\n#believe what his own moral sense tells him, the rest of society is\\n#entitled to pass laws spelling out punishments for behavior that is\\n#offensive to the majority.\\n\\nWhy?  Your last statement.  Why?  By which authority?  \\n\\n#Most criminals do not see their behavior as moral.  The may realize that\\n#it is immoral and not care.  They are thus not following their own moral\\n#system but being immoral.  For someone to lay claim to an alternative\\n#moral system, he must be sincere in his belief in it and it must be\\n#internally consistent.  \\n\\nWhy?  Your last statement.  Why are these things necessary?  \\n\\nAnd believe me, a belief in terrorism can be both sincere and frighteningly\\nconsistent.\\n\\n#Some sociopaths lack an innate moral sense and\\n#thus may be incapable of behaving morally.  While someone like Hitler\\n#may have believed that his actions were moral, we may judge him immoral\\n#by our standards.  Holding that morality is subjective does not mean\\n#that we must excuse the murderer.\\n\\nTrouble is, this would sound just fine coming from someone like Hitler, too.\\n(I do *not* mean any comparison or offence, David.)   Try substituting \\nthe social minority of your choice for 'sociopath', 'Hitler',  and\\n'murderer'.  No logical difference.  Someone like you, vs. someone like\\nHitler. Zero sum.  \\n\\n-- \\nFrank O'Dwyer                                  'I'm not hatching That'\\nodwyer@sse.ie                                  from \"Hens\",  by Evelyn Conlon\\n</td>\n",
       "      <td>alt.atheism</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>217</td>\n",
       "      <td>From: conor@owlnet.rice.edu (Conor Frederick Prischmann)\\nSubject: Re: Genocide is Caused by Theism : Evidence?\\nOrganization: Rice University\\nLines: 23\\n\\nIn article &lt;C60A0s.DvI@mailer.cc.fsu.edu&gt; dekorte@dirac.scri.fsu.edu (Stephen L. DeKorte) writes:\\n&gt;\\n&gt;I saw a 3 hour show on PBS the other day about the history of the\\n&gt;Jews. Appearently, the Cursades(a religious war agianst the muslilams\\n&gt;in 'the holy land') sparked the widespread persecution of muslilams \\n&gt;and jews in europe. Among the supporters of the persiecution, were none \\n&gt;other than Martin Luther, and the Vatican.\\n&gt;\\n&gt;Later, Hitler would use Luthers writings to justify his own treatment\\n&gt;of the jews.\\n&gt;&gt; Genocide is Caused by Theism : Evidence?\\n\\nHeck, I remember reading a quote of Luther as something like: \"Jews should\\nbe shot like deer.\"  And of course much Catholic doctrine for centuries was \\nextremely anti-Semitic.\\n\\n\\n\\n-- \\n\"Are you so sure that your truth and your justice are worth more than the\\ntruths and justices of other centuries?\" - Simone de Beauvoir\\n\"Where is there a certainty that rises above all doubt and withstands all\\ncritique?\" - Karl Jaspers          Rice University, Will Rice College '96\\n</td>\n",
       "      <td>alt.atheism</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>382</td>\n",
       "      <td>From: jhpark@cs.utexas.edu (Jihun Park)\\nSubject: POVray : tga -&gt; rle\\nOrganization: CS Dept, University of Texas at Austin\\nLines: 25\\nNNTP-Posting-Host: pageboy.cs.utexas.edu\\n\\nHello,\\nI have some problem in converting tga file(generated by POVray) to\\nrle file. When I convert, I do not get any warning message. But\\nif I use xloadimage/getx11, something is wrong.\\n\\nError messages are,\\n% targatorle -o o.rle data.tga\\n% xloadimage o.rle\\no.rle is a 0x0 24 bit RLE image with no map (will dither to 8 bits), with gamma of 1.00\\n  Dithering image...done\\n  Building XImage...done\\nxloadimage: X Error: BadValue (integer parameter out of range for operation) on 0x0\\nxloadimage: X Error: BadWindow (invalid Window parameter) on 0xb00003\\n......\\n\\nI know that I need to install ppmtorle and tgatoppm, but I do not spend\\ntime to install them. Even I do not want to generate .rgb from POVray\\nand then convert them to rle, if possible.(.rgb to rle works, but\\nit will mess up my directory with so many files, and it needs 2 more\\nsteps to finally convert to rle file. say cat | rawtorle | rleflip )\\nDoes any body out there have same experience/problems ?\\n\\nThanks in advance,\\n---\\nJ. Park\\n</td>\n",
       "      <td>comp.graphics</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>547</td>\n",
       "      <td>From: jdryburn@smt_6.b21.ingr.com (Joe Ryburn)\\nSubject: Software Development Libs for Old TARGA-16 Boards\\nOrganization: Intergraph\\nLines: 17\\n\\nI am revamping some computer-aided visual inspection systems which\\nuse the old AT&amp;T Targa-16 Board Set (2 cards) to display images from\\na color CCD camera to a color monitor, providing the option to overlay\\ntext or a crosshair. No image capture or manipulation is performed, \\njust display. I would like to know if there is still a source for \\ndevelopment libraries which would allow me to embed commands in my\\nown software to enable the camera, draw crosshairs, print text, etc.\\nI'll be glad to pay if they are commercially available. E-Mail if\\npossible.\\n\\n-- \\n\\n\\n----------------------------------------------------------------\\nJoe Ryburn |  CIM Manager               | Intergraph Corporation\\n           |  Manufacturing Integration | Huntsville, AL 35894\\n----------------------------------------------------------------\\n</td>\n",
       "      <td>comp.graphics</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>847</td>\n",
       "      <td>From: cui@maccs.mcmaster.ca (Jun Cui)\\nSubject: How to respond to WM_LBUTTONDBLCLK?\\nKeywords: MS-Windows, SDK\\nNntp-Posting-Host: maccs.dcss.mcmaster.ca\\nOrganization: Department of Computer Science, McMaster University\\nDistribution: comp.windows.ms\\nLines: 23\\n\\nI'm using BC++'s ObjectWindows (3.1), trying to resond to the left button\\ndouble click message WM_LBUTTONDBLCLK in a combo box. My codes look like:\\n\\nclass MyComboBox : public TComboBox\\n{\\npublic:\\n\\tMyComboBox(PTWindowsObject AParent, int ResourceId, WORD ATextLen) :\\n\\t\\tTComboBox(AParent, ResourceId, ATextLen) {};\\n\\tvirtual void WMLButtonDblClk(RTMessage Msg) = [WM_FIRST + WM_LBUTTONDBLCLK];\\n};\\n\\nvoid MyComboBox::WMLButtonDblClk(RTMessage Msg)\\n{\\n\\t...\\t// responding to the message (selecting the item in the list box)\\n}\\n\\nAnything wrong with my program? Any help would be appreciated.\\n\\n-- jun\\nTo talk to the Lord with PS/2 through MS-Windows\\n\\n\\n\\n</td>\n",
       "      <td>comp.os.ms-windows.misc</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>788</td>\n",
       "      <td>From: les.tom@idcbbs.com (Les Tom) \\nSubject: Replacement for Program M\\nDistribution: world\\nOrganization: IDC BBS - Alameda, CA - 510-865-7115\\nReply-To: les.tom@idcbbs.com (Les Tom) \\nLines: 33\\n\\n           Maw Ying yuan wrote \u0019\\n .................................\\n...From: yuan@wiliki.eng.hawaii.edu (Maw Ying Yuan)\\n...Subject: Replacement for Program Manager and File Manager?\\n...Message-ID: &lt;C68G1G.JuJ@news.Hawaii.Edu&gt;\\n...Date: Thu, 29 Apr 1993 06:44:04 GMT\\n...\\n...replacements for Win3.1's Program Manager and File Manager?\\n...yuan@wiliki.eng.hawaii.edu :)\\n .................................\\n    Hi,\\n      I've been using Plannet Crafter's \"Plug-in\" for Program Manager.\\n      Its listed in most BBS'es as PLUGIN13.  Its an add-on which gives\\n      some needed features to Progman; such as the ability to better\\n      manage your groups, change your cursor and icons on the fly,\\n      constant status of RAM and resources, \"Quick-menu\" (a drop-down\\n      menu of DOS commands or app launcher), plus some other neat stuff.\\n      Used it, liked it, and even reg'd it &lt;G&gt;.\\n\\n      If you can't find it anywhere, let me know and I'll zip it up and\\n      mail it to you (shareware version, of course &lt;g&gt;...)\\n\\nAloha...\\n---\\n .. .es .. All hope abandon, ye who enter messages here.\\n---\\n . SLMR 2.1a . \\n                                                   \\n----\\n+---------------------------------------------------------------------------+\\n| IDC BBS    510-865-7115 (USR HST 14.4k)  510-814-8097 (USR DS v.32bis)    |\\n| Alameda, California  -   Home of KingMail, KingQWK and QWKMerge           |\\n+---------------------------------------------------------------------------+\\n</td>\n",
       "      <td>comp.os.ms-windows.misc</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1007</td>\n",
       "      <td>From: gel@binkley.cs.mcgill.ca (Gilles KHOUZAM)\\nSubject: Re: MAC DISKS IN WINDOWS?\\nLines: 6\\nNntp-Posting-Host: binkley.cs.mcgill.ca\\nOrganization: SOCS - Mcgill University, Montreal, Canada\\nLines: 6\\n\\n&gt;Macsee.zip on ftp.cica.indiana.edu is supposed to read and write Mac disks.  \\n&gt;I've never tried it, though.  Good luck\\n\\nI just tried it, but I can't say that it works, 'cause it doesn't read 800K\\ndisks, it only reads 1.44Mb. If there a program that does read 800K disks,\\nplease let me know.\\n</td>\n",
       "      <td>comp.os.ms-windows.misc</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>975</td>\n",
       "      <td>From: cs_e373@king.ac.uk (Matthew Simpkin)\\nSubject: Re: Adding Icons to Progman.exe\\nOrganization: Kingston University, Kingston-upon-Thames\\nLines: 22\\nNNTP-Posting-Host: ceres.kingston.ac.uk\\nX-Newsreader: TIN [version 1.1 PL8]\\n\\nFlorian Piekert (floppy@s-link.escape.de) wrote:\\n\\n: 09.05.93, 14:48\\n\\n: Hi out there !\\n\\n: Is there anybody who can write me how to add icons to the iconlist (which\\n: is build-into) of Progman.exe ?\\n\\n: I'm grateful for every clue !\\n\\n: Thanx and have a nice day !\\n\\nOne way is to use a program called Iconmaster. It's shareware and If you can't\\nget it mail me, and I'll post it to you.\\n\\n\\n--\\n&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;\\n| Matthew Simpkin\\t | Kingston University  | \"A Primitive Race, You too  |\\n| cs_e373@kingston.ac.uk |  Computer Science    |   shall be asimilated ...\"  |\\n&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;\\n</td>\n",
       "      <td>comp.os.ms-windows.misc</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>943</td>\n",
       "      <td>From: wiseb@colossus.cs.rpi.edu (Bowden Wise)\\nSubject: Displaying stream output in a Window\\nOrganization: Rensselaer Polytechnic Institute Computer Science, Troy NY\\nLines: 11\\nDistribution: comp\\nNNTP-Posting-Host: colossus.cs.rpi.edu\\n\\nI am using OWL and want to display the output from a stream in a popup\\nwindow.  Is there a way to perhaps redirect cout to a window?   Or\\nalternatively set up a separate stream that supports output and be\\nable to display the stream output in an TEdit control?\\n\\nAnyone ever done something like this?\\n--\\n--------------------------------------------------------------------\\n- Bowden Wise\\n  Computer Science Dept, Rensselaer Polytechnic Inst, Troy, NY 12180\\n  internet: wiseb@cs.rpi.edu\\n</td>\n",
       "      <td>comp.os.ms-windows.misc</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   doc_id  \\\n",
       "0  58       \n",
       "1  154      \n",
       "2  217      \n",
       "3  382      \n",
       "4  547      \n",
       "5  847      \n",
       "6  788      \n",
       "7  1007     \n",
       "8  975      \n",
       "9  943      \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  contents  \\\n",
       "0  From: acooper@mac.cc.macalstr.edu (Turin Turambar, ME Department of Utter Misery)\\nSubject: Re: Societally acceptable behavior\\nOrganization: Macalester College\\nLines: 55\\n\\nIn article <C5sA29.14s@news.cso.uiuc.edu>, cobb@alexia.lis.uiuc.edu (Mike Cobb) writes:\\n> I guess I'm delving into a religious language area.  What exactly is morality \\n> or morals?  \\n\\nI hope there is not one- with a subject like this you just have a spiral.  What\\nwould then be a morality of a morality of morals.  Labels don't make arguments. \\nOne really needs a solid measuring stick by which most actions can be\\ninterpreted, even though this would hardly seem moral.  For example \"The best\\nthing for me is to ensure that I will eat and drink enough.  Hence all actions\\nmust be weighed against this one statement.\"  whatever helps this goal is\\n\"moral\", whatever does not is \"immoral\"\\n\\nOf course this leads such a blank space: there are so many different ways to\\nfulfill a goal, one would need a \"hyper-morality\" to apply to just the methods.\\n\\n>I never thought of eating meat to be moral or immoral, but I think\\n> it could be.  How do we differentiate between not doing something because it is\\n> a personal choice or preference and not doing something because we see it as \\n> immoral?  Do we fall to what the basis of these morals are?\\n\\nSeems to me we only consider something moral or immoral if we stop to think\\nabout it long enough  :)  On the other hand, maybe it is our first gut\\nreaction...  Which?  Who knows: perhaps here we have a way to discriminate\\nmorals.  I don't instinctively thing vegetarianism is right (the same way I\\ninstinctively feel torture is wrong), but if I thought about it long enough and\\nlistened to the arguments, I could perhaps reason that it was wrong (is that\\npossible!?  :) )  See the difference?\\n\\n> \\n> Also, consensus positions fall to a might makes right.  Or, as you brought out,\\n> if whatever is right is what is societally mandated then whoever is in control\\n> at the time makes what is right\\n> \\n> MC\\n> MAC\\n> --\\n> ****************************************************************\\n>                                                     Michael A. Cobb\\n>  \"...and I won't raise taxes on the middle     University of Illinois\\n>     class to pay for my programs.\"                 Champaign-Urbana\\n>           -Bill Clinton 3rd Debate             cobb@alexia.lis.uiuc.edu\\n>                                               \\n> Nobody can explain everything to anybody.  G.K.Chesterton\\n-- \\n\\n\\nbest regards,\\n\\n--Adam\\n\\n********************************************************************************\\n* Adam John Cooper\\t\\t\"Verily, often have I laughed at the weaklings *\\n* (612) 696-7521\\t\\t   who thought themselves good simply because  *\\n* acooper@macalstr.edu\\t\\t\\t\\tthey had no claws.\"\\t       *\\n********************************************************************************\\n   \n",
       "1  From: frank@D012S658.uucp (Frank O'Dwyer)\\nSubject: Re: Societal basis for morality\\nOrganization: Siemens-Nixdorf AG\\nLines: 49\\nNNTP-Posting-Host: d012s658.ap.mchp.sni.de\\n\\nIn article <1993Apr20.004119.6119@cnsvax.uwec.edu> nyeda@cnsvax.uwec.edu (David Nye) writes:\\n\\nYou asked me to look over here, but I was on my way back anyway :-)\\n\\n#[reply to cobb@alexia.lis.uiuc.edu (Mike Cobb)]\\n# \\n#>If morals come from what is societally accepted, why follow that? What\\n#>right do we have to expect others to follow our notion of societally\\n#>mandated morality?  Pardon the extremism, but couldn't I murder your\\n#>\"brother\" and say that I was exercising my rights as I saw them, was\\n#>doing what felt good, didn't want anyone forcing their morality on me,\\n#>or I don't follow your \"morality\" ?\\n# \\n#I believe that morality is subjective.  Each person is entitled to his\\n#own moral attitudes.  Mine are not a priori more correct than someone\\n#elses.  This does not mean however that I must judge another on the\\n#basis of his rather than my moral standards.  While he is entitled to\\n#believe what his own moral sense tells him, the rest of society is\\n#entitled to pass laws spelling out punishments for behavior that is\\n#offensive to the majority.\\n\\nWhy?  Your last statement.  Why?  By which authority?  \\n\\n#Most criminals do not see their behavior as moral.  The may realize that\\n#it is immoral and not care.  They are thus not following their own moral\\n#system but being immoral.  For someone to lay claim to an alternative\\n#moral system, he must be sincere in his belief in it and it must be\\n#internally consistent.  \\n\\nWhy?  Your last statement.  Why are these things necessary?  \\n\\nAnd believe me, a belief in terrorism can be both sincere and frighteningly\\nconsistent.\\n\\n#Some sociopaths lack an innate moral sense and\\n#thus may be incapable of behaving morally.  While someone like Hitler\\n#may have believed that his actions were moral, we may judge him immoral\\n#by our standards.  Holding that morality is subjective does not mean\\n#that we must excuse the murderer.\\n\\nTrouble is, this would sound just fine coming from someone like Hitler, too.\\n(I do *not* mean any comparison or offence, David.)   Try substituting \\nthe social minority of your choice for 'sociopath', 'Hitler',  and\\n'murderer'.  No logical difference.  Someone like you, vs. someone like\\nHitler. Zero sum.  \\n\\n-- \\nFrank O'Dwyer                                  'I'm not hatching That'\\nodwyer@sse.ie                                  from \"Hens\",  by Evelyn Conlon\\n                                                                                                                                                                                                                                                                                                                                                                             \n",
       "2  From: conor@owlnet.rice.edu (Conor Frederick Prischmann)\\nSubject: Re: Genocide is Caused by Theism : Evidence?\\nOrganization: Rice University\\nLines: 23\\n\\nIn article <C60A0s.DvI@mailer.cc.fsu.edu> dekorte@dirac.scri.fsu.edu (Stephen L. DeKorte) writes:\\n>\\n>I saw a 3 hour show on PBS the other day about the history of the\\n>Jews. Appearently, the Cursades(a religious war agianst the muslilams\\n>in 'the holy land') sparked the widespread persecution of muslilams \\n>and jews in europe. Among the supporters of the persiecution, were none \\n>other than Martin Luther, and the Vatican.\\n>\\n>Later, Hitler would use Luthers writings to justify his own treatment\\n>of the jews.\\n>> Genocide is Caused by Theism : Evidence?\\n\\nHeck, I remember reading a quote of Luther as something like: \"Jews should\\nbe shot like deer.\"  And of course much Catholic doctrine for centuries was \\nextremely anti-Semitic.\\n\\n\\n\\n-- \\n\"Are you so sure that your truth and your justice are worth more than the\\ntruths and justices of other centuries?\" - Simone de Beauvoir\\n\"Where is there a certainty that rises above all doubt and withstands all\\ncritique?\" - Karl Jaspers          Rice University, Will Rice College '96\\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     \n",
       "3  From: jhpark@cs.utexas.edu (Jihun Park)\\nSubject: POVray : tga -> rle\\nOrganization: CS Dept, University of Texas at Austin\\nLines: 25\\nNNTP-Posting-Host: pageboy.cs.utexas.edu\\n\\nHello,\\nI have some problem in converting tga file(generated by POVray) to\\nrle file. When I convert, I do not get any warning message. But\\nif I use xloadimage/getx11, something is wrong.\\n\\nError messages are,\\n% targatorle -o o.rle data.tga\\n% xloadimage o.rle\\no.rle is a 0x0 24 bit RLE image with no map (will dither to 8 bits), with gamma of 1.00\\n  Dithering image...done\\n  Building XImage...done\\nxloadimage: X Error: BadValue (integer parameter out of range for operation) on 0x0\\nxloadimage: X Error: BadWindow (invalid Window parameter) on 0xb00003\\n......\\n\\nI know that I need to install ppmtorle and tgatoppm, but I do not spend\\ntime to install them. Even I do not want to generate .rgb from POVray\\nand then convert them to rle, if possible.(.rgb to rle works, but\\nit will mess up my directory with so many files, and it needs 2 more\\nsteps to finally convert to rle file. say cat | rawtorle | rleflip )\\nDoes any body out there have same experience/problems ?\\n\\nThanks in advance,\\n---\\nJ. Park\\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              \n",
       "4  From: jdryburn@smt_6.b21.ingr.com (Joe Ryburn)\\nSubject: Software Development Libs for Old TARGA-16 Boards\\nOrganization: Intergraph\\nLines: 17\\n\\nI am revamping some computer-aided visual inspection systems which\\nuse the old AT&T Targa-16 Board Set (2 cards) to display images from\\na color CCD camera to a color monitor, providing the option to overlay\\ntext or a crosshair. No image capture or manipulation is performed, \\njust display. I would like to know if there is still a source for \\ndevelopment libraries which would allow me to embed commands in my\\nown software to enable the camera, draw crosshairs, print text, etc.\\nI'll be glad to pay if they are commercially available. E-Mail if\\npossible.\\n\\n-- \\n\\n\\n----------------------------------------------------------------\\nJoe Ryburn |  CIM Manager               | Intergraph Corporation\\n           |  Manufacturing Integration | Huntsville, AL 35894\\n----------------------------------------------------------------\\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  \n",
       "5  From: cui@maccs.mcmaster.ca (Jun Cui)\\nSubject: How to respond to WM_LBUTTONDBLCLK?\\nKeywords: MS-Windows, SDK\\nNntp-Posting-Host: maccs.dcss.mcmaster.ca\\nOrganization: Department of Computer Science, McMaster University\\nDistribution: comp.windows.ms\\nLines: 23\\n\\nI'm using BC++'s ObjectWindows (3.1), trying to resond to the left button\\ndouble click message WM_LBUTTONDBLCLK in a combo box. My codes look like:\\n\\nclass MyComboBox : public TComboBox\\n{\\npublic:\\n\\tMyComboBox(PTWindowsObject AParent, int ResourceId, WORD ATextLen) :\\n\\t\\tTComboBox(AParent, ResourceId, ATextLen) {};\\n\\tvirtual void WMLButtonDblClk(RTMessage Msg) = [WM_FIRST + WM_LBUTTONDBLCLK];\\n};\\n\\nvoid MyComboBox::WMLButtonDblClk(RTMessage Msg)\\n{\\n\\t...\\t// responding to the message (selecting the item in the list box)\\n}\\n\\nAnything wrong with my program? Any help would be appreciated.\\n\\n-- jun\\nTo talk to the Lord with PS/2 through MS-Windows\\n\\n\\n\\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  \n",
       "6  From: les.tom@idcbbs.com (Les Tom) \\nSubject: Replacement for Program M\\nDistribution: world\\nOrganization: IDC BBS - Alameda, CA - 510-865-7115\\nReply-To: les.tom@idcbbs.com (Les Tom) \\nLines: 33\\n\\n           Maw Ying yuan wrote \u0019\\n .................................\\n...From: yuan@wiliki.eng.hawaii.edu (Maw Ying Yuan)\\n...Subject: Replacement for Program Manager and File Manager?\\n...Message-ID: <C68G1G.JuJ@news.Hawaii.Edu>\\n...Date: Thu, 29 Apr 1993 06:44:04 GMT\\n...\\n...replacements for Win3.1's Program Manager and File Manager?\\n...yuan@wiliki.eng.hawaii.edu :)\\n .................................\\n    Hi,\\n      I've been using Plannet Crafter's \"Plug-in\" for Program Manager.\\n      Its listed in most BBS'es as PLUGIN13.  Its an add-on which gives\\n      some needed features to Progman; such as the ability to better\\n      manage your groups, change your cursor and icons on the fly,\\n      constant status of RAM and resources, \"Quick-menu\" (a drop-down\\n      menu of DOS commands or app launcher), plus some other neat stuff.\\n      Used it, liked it, and even reg'd it <G>.\\n\\n      If you can't find it anywhere, let me know and I'll zip it up and\\n      mail it to you (shareware version, of course <g>...)\\n\\nAloha...\\n---\\n .. .es .. All hope abandon, ye who enter messages here.\\n---\\n . SLMR 2.1a . \\n                                                   \\n----\\n+---------------------------------------------------------------------------+\\n| IDC BBS    510-865-7115 (USR HST 14.4k)  510-814-8097 (USR DS v.32bis)    |\\n| Alameda, California  -   Home of KingMail, KingQWK and QWKMerge           |\\n+---------------------------------------------------------------------------+\\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   \n",
       "7  From: gel@binkley.cs.mcgill.ca (Gilles KHOUZAM)\\nSubject: Re: MAC DISKS IN WINDOWS?\\nLines: 6\\nNntp-Posting-Host: binkley.cs.mcgill.ca\\nOrganization: SOCS - Mcgill University, Montreal, Canada\\nLines: 6\\n\\n>Macsee.zip on ftp.cica.indiana.edu is supposed to read and write Mac disks.  \\n>I've never tried it, though.  Good luck\\n\\nI just tried it, but I can't say that it works, 'cause it doesn't read 800K\\ndisks, it only reads 1.44Mb. If there a program that does read 800K disks,\\nplease let me know.\\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  \n",
       "8  From: cs_e373@king.ac.uk (Matthew Simpkin)\\nSubject: Re: Adding Icons to Progman.exe\\nOrganization: Kingston University, Kingston-upon-Thames\\nLines: 22\\nNNTP-Posting-Host: ceres.kingston.ac.uk\\nX-Newsreader: TIN [version 1.1 PL8]\\n\\nFlorian Piekert (floppy@s-link.escape.de) wrote:\\n\\n: 09.05.93, 14:48\\n\\n: Hi out there !\\n\\n: Is there anybody who can write me how to add icons to the iconlist (which\\n: is build-into) of Progman.exe ?\\n\\n: I'm grateful for every clue !\\n\\n: Thanx and have a nice day !\\n\\nOne way is to use a program called Iconmaster. It's shareware and If you can't\\nget it mail me, and I'll post it to you.\\n\\n\\n--\\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\\n| Matthew Simpkin\\t | Kingston University  | \"A Primitive Race, You too  |\\n| cs_e373@kingston.ac.uk |  Computer Science    |   shall be asimilated ...\"  |\\n<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            \n",
       "9  From: wiseb@colossus.cs.rpi.edu (Bowden Wise)\\nSubject: Displaying stream output in a Window\\nOrganization: Rensselaer Polytechnic Institute Computer Science, Troy NY\\nLines: 11\\nDistribution: comp\\nNNTP-Posting-Host: colossus.cs.rpi.edu\\n\\nI am using OWL and want to display the output from a stream in a popup\\nwindow.  Is there a way to perhaps redirect cout to a window?   Or\\nalternatively set up a separate stream that supports output and be\\nable to display the stream output in an TEdit control?\\n\\nAnyone ever done something like this?\\n--\\n--------------------------------------------------------------------\\n- Bowden Wise\\n  Computer Science Dept, Rensselaer Polytechnic Inst, Troy, NY 12180\\n  internet: wiseb@cs.rpi.edu\\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           \n",
       "\n",
       "                     label  \n",
       "0  alt.atheism              \n",
       "1  alt.atheism              \n",
       "2  alt.atheism              \n",
       "3  comp.graphics            \n",
       "4  comp.graphics            \n",
       "5  comp.os.ms-windows.misc  \n",
       "6  comp.os.ms-windows.misc  \n",
       "7  comp.os.ms-windows.misc  \n",
       "8  comp.os.ms-windows.misc  \n",
       "9  comp.os.ms-windows.misc  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%showsql\n",
    "select\n",
    "    *\n",
    "from\n",
    "    YOUR_SCHEMA.twenty_news_groups\n",
    "limit 10;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Create document vectors using Paragraph Vectors by Quoc Le, Mikolov et. al (local version) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc_id</th>\n",
       "      <th>contents</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5004</td>\n",
       "      <td>From: josephc@cco.caltech.edu (Joseph Chiu)\\nSubject: Re: ^^ Phosphorescent pads ...\\nOrganization: California Institute of Technology, Pasadena\\nLines: 27\\nNNTP-Posting-Host: sandman.caltech.edu\\n\\nbaden@sys6626.bison.mb.ca (baden de bari) writes:\\n\\n&gt; \\n&gt;         A while ago I saw some translucent pads, about 6\"x8\" or so that \\n&gt;could be plugged into (something) and they would glow.  One was red, \\n&gt;another green, another blue.  Interresting.  I was wondering if anyone \\n&gt;would have a feed on anything of this nature, and of the price.\\n&gt;   oh, the pads were rubber-like and were floppy like, about 1cm thick or \\n&gt;so.\\n\\nThe curiosities that you speak of are EL (Electro-Luminescent) pads.  They\\nare mostly used as backlights for LCD's, and as you pointed out, comes\\nin several different colors.  Many of them emit white-ish colors (\"true\"\\nwhite, and blue-whiteis-blue are the ones that I've personally encountered\\nthe most often.)\\n\\nThey vary in their input requirements, however, they tend to operate at \\nabout 100 Vrms, and at much higher than 60 Hz.  (400 Hz to 1 kHz, if I \\nrecall correctly.)   You can get them to work at 120 Vrms, 60 Hz (i.e.,\\nline) but they don't glow as brightly, and they will degrade much faster.\\n(Oh yes, almost forgot, EL's have this tendency to wear out over time...)\\n\\n-- \\nJoseph Chiu        | josephc@cco.caltech.edu  \"OS/2: You gotta get this thing!\" \\nMSC 380 - Caltech  |        \\nPasadena, CA 91126 | Dear IBM: Hire bunch of technically incompetent people\\n+1 818 449 5457    | (like my brother) to test OS/2 installation procedures.\\n</td>\n",
       "      <td>sci.electronics</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>16326</td>\n",
       "      <td>From: whheydt@pbhya.pacbell.com (Wilson Heydt)\\nSubject: Re: Ancient Books\\nOrganization: Pacific * Bell, San Ramon, CA\\nLines: 21\\n\\nIn article &lt;Apr.9.01.11.35.1993.16957@athos.rutgers.edu&gt;, cobb@alexia.lis.uiuc.edu (Mike Cobb) writes:\\n&gt; But, since the manuscripts are so close to the actual event, especially as \\n&gt; compared with ancient \"non-Christian\" history, could it help show that we have\\n&gt; accurate copies of the original texts?  \\n\\nThat's a very weak argument--due the lack (with regard to critical\\nevents) of independent supporting texts.\\n\\nAs for the dating of the oldest extant texts of the NT....  How would\\nyou feel about the US Civil War in a couple of thousand years if the\\nonly extant text was written about *now*?  Now adjust for a largely\\nilliterate population, and one in which every copy of a manuscript is\\ndone by hand....\\n\\n\\t--Hal\\n\\n-- \\nHal Heydt                    |    \\nAnalyst, Pacific*Bell        |  If you think the system is working,\\n510-823-5447                 |  Ask someone who's waiting for a prompt.\\nwhheydt@pbhya.PacBell.COM    |    \\n</td>\n",
       "      <td>soc.religion.christian</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9176</td>\n",
       "      <td>From: doug@sun.sws.uiuc.edu (Doug Ward)\\nSubject: Stealth 24 Video Drivers\\nOrganization: State Water Survey\\nLines: 14\\n\\n\\nI recently purchased a Diamond Stealth 24 Video card\\nand received the wrong drivers.  Does anyone know where\\nI can ftp the windows video drivers for the Stealth 24.\\nI tried the drivers at cica and they don't work.\\nThe right drivers are on the Diamond BBS but the file is almost\\n1 meg and it would take a while to download them\\nat 2400 baud.  Any information would be appreciated.  Please\\ncontact me at doug@sun.sws.uiuc.edu\\n\\nThank you\\n\\nDoug Ward\\n.\\n</td>\n",
       "      <td>comp.os.ms-windows.misc</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8074</td>\n",
       "      <td>From: trb3@Ra.MsState.Edu (Tony R. Boutwell)\\nSubject: HOT NEW 3D Software\\nKeywords: Imagine,3d\\nNntp-Posting-Host: ra.msstate.edu\\nOrganization: Mississippi State University\\nLines: 20\\n\\nThere is a new product for the (IBM'ers) out there... it is called\\nIMAGINE and it just started shipping yesterday... I can personally attest that it will blow the doors off of 3D-Studio.  It is made by IMPUlSE, and is in its\\n3rd version....(1st) for the IBM.... it can do morphing, your standard key-framming animation, it is a raytracer (reflections &amp; shadows), and can do/apply special FX to objects... (like ripple, explode, bounce) things of that nature.  Also it has algorithmic texture maps....and your standard brushmapping also...\\n\\nyou can have animated brushmaps...(ie. live video mapped on the objs)...\\nalso animated backdrops (ie. live video backgrounds)\\nalso animted reflections maps....\\n\\nyou get the idea.... it will run for about 500$ retail (I think)...\\n\\ndont let the low price fool you.... this product can do it all when it\\ncomes to 3D-animation and Renderering...!\\n\\nalso....does anyone here know how to get in the Imagine mailing list??\\nplease e-mail me if you do or post up here....\\n\\noh...the number for IMPULSE is ---&gt;1 800 328 0184\\n\\ntrb3@ra.msstate.edu\\n\\n</td>\n",
       "      <td>comp.graphics</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>14821</td>\n",
       "      <td>From: cyberman@toz.buffalo.ny.us (Cyberman)\\nSubject: Re: Analog switches/Balan\\nLines: 31\\nX-Maildoor: WaflineMail 1.00r\\n\\n{Michael Fulbright} said\\n   \"Analog switches/Balanced\"\\n      to &lt;All&gt; on 04-15-93  01:08\\n MF&gt; I am trying to build a synchronous demodulator and I've hit a snag.\\n MF&gt; In my application I want to be able to change the gain of an\\n MF&gt; op amp amplifier from 1 to -1, controlable via a digital input.\\n MF&gt; The most obvious way I've come up with is to use analog switches\\n MF&gt; to adjust the gain of the op amp. The only analog switch I have\\n MF&gt; experience with it the 4066. Unfortunately I want to switch an\\n MF&gt; AC signal which goes from about -5V to 5V, and the 4066 is only\\n MF&gt; for positive signals.\\n\\n    How about using a 4053 it has a seperate ground for the\\n    analog outputs.  It would get you 3 bits.\\n\\n MF&gt; Another part which caught my eye was the Analog Devices AD630. This\\n MF&gt; is a balanced demodulator which appears to fill exactly the need I\\n MF&gt; have. The data sheet was somewhat skimpy on application notes. Could\\n MF&gt; someone comment on using this chip for the following application?\\n\\n    Or how about a multiplying D/A convertor?  This is\\n    essentiallty what you are makeing.\\n\\n\\n                Stephen Cyberman@Toz.Buffalo.NY.US\\n             Mangled on Fri  04-16-1993  at 13:36:11\\n\\n... Catch the Blue Wave!\\n---\\n * Blue Wave/QWK v2.12 *\\n                          \\n</td>\n",
       "      <td>sci.electronics</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   doc_id  \\\n",
       "0  5004     \n",
       "1  16326    \n",
       "2  9176     \n",
       "3  8074     \n",
       "4  14821    \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  contents  \\\n",
       "0  From: josephc@cco.caltech.edu (Joseph Chiu)\\nSubject: Re: ^^ Phosphorescent pads ...\\nOrganization: California Institute of Technology, Pasadena\\nLines: 27\\nNNTP-Posting-Host: sandman.caltech.edu\\n\\nbaden@sys6626.bison.mb.ca (baden de bari) writes:\\n\\n> \\n>         A while ago I saw some translucent pads, about 6\"x8\" or so that \\n>could be plugged into (something) and they would glow.  One was red, \\n>another green, another blue.  Interresting.  I was wondering if anyone \\n>would have a feed on anything of this nature, and of the price.\\n>   oh, the pads were rubber-like and were floppy like, about 1cm thick or \\n>so.\\n\\nThe curiosities that you speak of are EL (Electro-Luminescent) pads.  They\\nare mostly used as backlights for LCD's, and as you pointed out, comes\\nin several different colors.  Many of them emit white-ish colors (\"true\"\\nwhite, and blue-whiteis-blue are the ones that I've personally encountered\\nthe most often.)\\n\\nThey vary in their input requirements, however, they tend to operate at \\nabout 100 Vrms, and at much higher than 60 Hz.  (400 Hz to 1 kHz, if I \\nrecall correctly.)   You can get them to work at 120 Vrms, 60 Hz (i.e.,\\nline) but they don't glow as brightly, and they will degrade much faster.\\n(Oh yes, almost forgot, EL's have this tendency to wear out over time...)\\n\\n-- \\nJoseph Chiu        | josephc@cco.caltech.edu  \"OS/2: You gotta get this thing!\" \\nMSC 380 - Caltech  |        \\nPasadena, CA 91126 | Dear IBM: Hire bunch of technically incompetent people\\n+1 818 449 5457    | (like my brother) to test OS/2 installation procedures.\\n   \n",
       "1  From: whheydt@pbhya.pacbell.com (Wilson Heydt)\\nSubject: Re: Ancient Books\\nOrganization: Pacific * Bell, San Ramon, CA\\nLines: 21\\n\\nIn article <Apr.9.01.11.35.1993.16957@athos.rutgers.edu>, cobb@alexia.lis.uiuc.edu (Mike Cobb) writes:\\n> But, since the manuscripts are so close to the actual event, especially as \\n> compared with ancient \"non-Christian\" history, could it help show that we have\\n> accurate copies of the original texts?  \\n\\nThat's a very weak argument--due the lack (with regard to critical\\nevents) of independent supporting texts.\\n\\nAs for the dating of the oldest extant texts of the NT....  How would\\nyou feel about the US Civil War in a couple of thousand years if the\\nonly extant text was written about *now*?  Now adjust for a largely\\nilliterate population, and one in which every copy of a manuscript is\\ndone by hand....\\n\\n\\t--Hal\\n\\n-- \\nHal Heydt                    |    \\nAnalyst, Pacific*Bell        |  If you think the system is working,\\n510-823-5447                 |  Ask someone who's waiting for a prompt.\\nwhheydt@pbhya.PacBell.COM    |    \\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          \n",
       "2  From: doug@sun.sws.uiuc.edu (Doug Ward)\\nSubject: Stealth 24 Video Drivers\\nOrganization: State Water Survey\\nLines: 14\\n\\n\\nI recently purchased a Diamond Stealth 24 Video card\\nand received the wrong drivers.  Does anyone know where\\nI can ftp the windows video drivers for the Stealth 24.\\nI tried the drivers at cica and they don't work.\\nThe right drivers are on the Diamond BBS but the file is almost\\n1 meg and it would take a while to download them\\nat 2400 baud.  Any information would be appreciated.  Please\\ncontact me at doug@sun.sws.uiuc.edu\\n\\nThank you\\n\\nDoug Ward\\n.\\n                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                \n",
       "3  From: trb3@Ra.MsState.Edu (Tony R. Boutwell)\\nSubject: HOT NEW 3D Software\\nKeywords: Imagine,3d\\nNntp-Posting-Host: ra.msstate.edu\\nOrganization: Mississippi State University\\nLines: 20\\n\\nThere is a new product for the (IBM'ers) out there... it is called\\nIMAGINE and it just started shipping yesterday... I can personally attest that it will blow the doors off of 3D-Studio.  It is made by IMPUlSE, and is in its\\n3rd version....(1st) for the IBM.... it can do morphing, your standard key-framming animation, it is a raytracer (reflections & shadows), and can do/apply special FX to objects... (like ripple, explode, bounce) things of that nature.  Also it has algorithmic texture maps....and your standard brushmapping also...\\n\\nyou can have animated brushmaps...(ie. live video mapped on the objs)...\\nalso animated backdrops (ie. live video backgrounds)\\nalso animted reflections maps....\\n\\nyou get the idea.... it will run for about 500$ retail (I think)...\\n\\ndont let the low price fool you.... this product can do it all when it\\ncomes to 3D-animation and Renderering...!\\n\\nalso....does anyone here know how to get in the Imagine mailing list??\\nplease e-mail me if you do or post up here....\\n\\noh...the number for IMPULSE is --->1 800 328 0184\\n\\ntrb3@ra.msstate.edu\\n\\n                                                                                                                                                                                                                                                                                                                     \n",
       "4  From: cyberman@toz.buffalo.ny.us (Cyberman)\\nSubject: Re: Analog switches/Balan\\nLines: 31\\nX-Maildoor: WaflineMail 1.00r\\n\\n{Michael Fulbright} said\\n   \"Analog switches/Balanced\"\\n      to <All> on 04-15-93  01:08\\n MF> I am trying to build a synchronous demodulator and I've hit a snag.\\n MF> In my application I want to be able to change the gain of an\\n MF> op amp amplifier from 1 to -1, controlable via a digital input.\\n MF> The most obvious way I've come up with is to use analog switches\\n MF> to adjust the gain of the op amp. The only analog switch I have\\n MF> experience with it the 4066. Unfortunately I want to switch an\\n MF> AC signal which goes from about -5V to 5V, and the 4066 is only\\n MF> for positive signals.\\n\\n    How about using a 4053 it has a seperate ground for the\\n    analog outputs.  It would get you 3 bits.\\n\\n MF> Another part which caught my eye was the Analog Devices AD630. This\\n MF> is a balanced demodulator which appears to fill exactly the need I\\n MF> have. The data sheet was somewhat skimpy on application notes. Could\\n MF> someone comment on using this chip for the following application?\\n\\n    Or how about a multiplying D/A convertor?  This is\\n    essentiallty what you are makeing.\\n\\n\\n                Stephen Cyberman@Toz.Buffalo.NY.US\\n             Mangled on Fri  04-16-1993  at 13:36:11\\n\\n... Catch the Blue Wave!\\n---\\n * Blue Wave/QWK v2.12 *\\n                          \\n                                                                                                                                                             \n",
       "\n",
       "                     label  \n",
       "0  sci.electronics          \n",
       "1  soc.religion.christian   \n",
       "2  comp.os.ms-windows.misc  \n",
       "3  comp.graphics            \n",
       "4  sci.electronics          "
      ]
     },
     "execution_count": 207,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sql = \"\"\"\n",
    "    select \n",
    "        * \n",
    "    from \n",
    "        YOUR_SCHEMA.twenty_news_groups \n",
    "    order by random()\n",
    "\"\"\"\n",
    "df = psql.read_sql(sql, conn)\n",
    "conn.commit()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def get_paragraph_vectors(df):\n",
    "    \"\"\"\n",
    "        Train paragraph vectors\n",
    "    \"\"\"\n",
    "    from gensim.models.doc2vec import TaggedDocument\n",
    "    from gensim.models import Doc2Vec\n",
    "    import pandas as pd    \n",
    "    #words=tokenize_document(d), tags=label) \n",
    "    docs = [\n",
    "            TaggedDocument(words=d.split(), tags=[label]) \n",
    "                for d, label in zip(df['contents'], df['doc_id'])\n",
    "           ]            \n",
    "    doc_ids = df['doc_id']\n",
    "    #Train doc2vec model in Gensim\n",
    "    model = Doc2Vec(\n",
    "                documents=docs, \n",
    "                min_count=1, \n",
    "                window=10, \n",
    "                size=100, \n",
    "                sample=1e-4, \n",
    "                negative=5, \n",
    "                workers=10\n",
    "            )\n",
    "    return model, docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model, docs = get_paragraph_vectors(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Show word vectors for the words in the document "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Word vectors are shared between documents in the `paragraph vector` representation of documents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>word</th>\n",
       "      <th>word2vec</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;C5IF8u.3Ky@news.cso.uiuc.edu&gt;</td>\n",
       "      <td>[0.00297474488616, 0.00426242640242, 0.00115045718849, -0.00414093211293, 0.00552939111367, -0.00301579455845, 0.000586995563935, -0.000823675596621, -0.001064071781, 0.00162940553855, -0.000371181376977, -0.000716799986549, -0.0046420157887, 0.00312513345852, 0.00311536621302, -0.000428203144111, -0.00342879700474, 0.00328808045015, 0.00400523142889, 0.0024073543027, 0.000903616659343, -0.00427946913987, 0.00456818472594, -0.00157911446877, -0.00132594094612, -0.000256380095379, -0.00484062917531, 0.000475894194096, -2.94143537758e-05, 0.00365562411025, -0.00113884359598, -0.000151738233399, -0.0028858743608, -0.0034737479873, 0.00233575399034, 0.000630486523733, -0.00477981474251, 0.00438628485426, 0.00306561239995, 0.00336440373212, 0.00458638183773, -0.00337051413953, -0.00197737594135, -0.00506750587374, -0.000329419301124, -0.00452499743551, 0.00540990335867, -0.00482728052884, -0.00084013963351, -0.00221424805932, 0.00334301032126, -0.00253241928294, -0.00329208909534, -0.00221632421017, 0.00170465942938, 0.00201489054598, -0.00415604515001, -0.00483640469611, 0.00047066519619, -0.00520296394825, -0.00158452056348, -0.00286255823448, -0.00269009801559, -0.00309886224568, 0.000886526191607, 0.00160996825434, -0.00337053276598, 0.00327526475303, -0.00375494174659, 0.00167199445423, -0.00216847658157, -0.00562074314803, -0.000409512227634, 0.00293384445831, 0.0009924932383, -0.00101980485488, 0.00164066359866, -0.0043929615058, -0.000924961641431, -0.00173757772427, -0.00365786417387, -0.00456798402593, 0.0039864978753, 0.001509857364, 0.000410577224102, 0.000437068432802, -0.00490206247196, 0.00369475618936, 0.00434789480641, 0.00193163030781, -0.000422751443693, 0.00459859333932, 0.00258714542724, 0.00189295667224, -0.0017615261022, 5.79569905312e-06, -0.00311522814445, 0.000751218991354, 0.00392152555287, 0.00542231183499]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>360.0;</td>\n",
       "      <td>[0.00164296617731, 0.00402011163533, -0.00341997249052, 0.00181064545177, 0.00115713756531, 0.00499207852408, -0.00666259275749, 0.000712228647899, -0.00435985904187, -0.00590976094827, -0.00336569431238, -0.00503790937364, 0.00229378440417, 0.0013105799444, 0.000285958230961, -0.00116698583588, 0.0041963942349, 0.00190469715744, 0.000540571112651, -0.00199648598209, -0.00499176187441, 0.00207925657742, -0.000580201915, -0.000933780043852, 0.00508917914703, -0.00340990140103, -0.0032930674497, 0.000925277417991, 0.00305562932044, -0.00270137400366, 0.00351696810685, -0.00135356665123, 0.00447391159832, 0.00296539068222, 0.00233768532053, -0.0077789472416, 0.00147385220043, 0.00316610210575, 0.000607696129009, 0.00236090132967, 0.000550734868739, 0.00231869611889, -0.00172822701279, -0.00122039567214, 0.00328909815289, 0.00183499383274, -0.00341626233421, -0.00344296148978, -0.000843502988573, 0.00513871479779, -0.000518121174537, -0.00267654238269, -0.000263952038949, -0.00547413248569, 0.00269497628324, -0.00162485288456, -0.00127516861539, 0.000230978723266, -0.00148325960618, -0.00205011595972, -0.00100799056236, 2.04232055694e-05, 0.00270132115111, 0.00486238626763, -0.000796347449068, -0.0017056124052, -0.0023364268709, 0.00529055809602, 0.00194682227448, 0.00514988647774, 0.00186218705494, 0.00162255950272, 0.00431430572644, -0.000569750729483, 2.56241910392e-05, 0.00176354323048, -0.00265433103777, 0.00273352349177, 0.000839602609631, 0.00507058156654, 0.00429962342605, 0.00220120023005, 0.000238612366957, -0.000779112917371, 0.00105583469849, -0.00102424819488, -0.00122376577929, -0.00337405619211, -0.00218626833521, -0.00430599413812, 0.00116281560622, -0.00111078168266, -0.00206259498373, -0.00262598437257, 0.00330503028817, -0.000975074071903, -0.00379833392799, 0.00310796359554, -0.000367124855984, 0.00195140182041]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>CHE4$PN!]A\\9=ES\"2-U-&gt;35RY1^O@)%40N/:LNSLQRZ11MYVW\"I</td>\n",
       "      <td>[0.00670566083863, -7.87581484474e-06, -0.00451696896926, 0.00120079063345, 0.00169547088444, 0.00531740160659, 0.00220019253902, 1.4106699382e-06, 0.00217704172246, -0.00511439936236, 0.00152852828614, 0.00279566389509, -0.00258536194451, 0.000796087784693, -0.00348979909904, -0.000213264094782, -0.0035747459624, -0.000598112819716, -0.00465873163193, 0.00213118479587, -0.00278941425495, 0.00292693870142, 0.00174413027707, -0.00807938352227, 0.000390542496461, 0.00691892020404, -0.000803683360573, 0.000463944103103, 0.00213684071787, 0.000811428937595, 0.00449485052377, -0.00482045905665, -0.00166017585434, -0.00246830191463, -0.00283294334076, -0.000551049422938, 0.00328575191088, 0.00395453581586, 0.00302625331096, 0.00458269286901, -0.00129209575243, -0.00101099722087, -0.00513690058142, 0.000347049499396, -0.00330302328803, -0.00129728601314, 0.00420563248917, -0.00365815707482, -0.000240945926635, -0.000858772196807, 0.00307879480533, 0.00231451098807, -0.00140998105053, -0.00444172741845, -0.00425683706999, 0.000417479081079, 0.00563792418689, -0.00260218721814, -0.00378362229094, 0.000407784595154, 0.00128395028878, -0.00579293398187, -0.00520406989381, 0.00216693081893, 0.000826761533972, -0.00233249901794, 0.00215489044785, -0.00355174206197, -0.00180029426701, -0.00519510591403, 0.00284486147575, 0.00328912399709, 0.000737764290534, 0.00025975657627, -0.00146570301149, 0.00656075635925, 0.00366906262934, 0.00112273916602, -0.00497675221413, 0.00326345302165, -0.00326123973355, 0.00322795519605, -0.00230185128748, -0.000227761382121, 0.004898365587, 0.00410693464801, -0.000454528257251, -0.00449569616467, -0.00327586289495, -0.00518970331177, 0.00277398293838, 0.00225683790632, -0.00539715774357, -0.00151153549086, 0.0013411664404, 0.00199825898744, -0.000330908194883, 0.00108697032556, -0.000611252326053, -0.00610497267917]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>woods</td>\n",
       "      <td>[0.00775190023705, 0.00144148024265, -0.00484913308173, 0.00452073151246, -0.0109325610101, -0.00555660855025, 0.000874503049999, -0.00975184049457, 0.00122546509374, -0.00461204675958, -0.00575741939247, -0.00153915304691, 0.00412272522226, -0.00597363570705, 0.00170346000232, 0.00163426541258, -0.000417792820372, -0.00308019691147, 0.00151154363994, -0.00140929431655, -0.00486217392609, -2.40055906033e-05, 0.00150109594688, -0.00307337287813, -0.00300468597561, 0.00695545831695, 0.00274720462039, 0.00355858751573, 0.00745694898069, 0.000630682858173, 0.00271724467166, -0.00654885545373, -0.00104922580067, 0.000944466970395, -0.00199690065347, 0.00687483837828, -0.00248539401218, -0.00616966467351, 0.0123061249033, 0.00695088366047, -0.00304097565822, 0.00104740669485, -0.00500980298966, 0.00259305885993, -0.0056721707806, -0.00593376578763, -0.000614581396803, -0.00227987649851, -0.00481418985873, -0.00384941231459, 0.00238774996251, 0.00219878205098, -0.00136520015076, -0.00175328110345, -0.00573545554653, 0.00268785771914, -0.00228368979879, -0.00276386109181, -0.000144422680023, -0.00185984908603, 0.00647442881018, 0.0024444963783, -0.00397222023457, 0.00424491753802, 0.00652982201427, 0.00199837912805, 0.00505082355812, 0.00875545293093, -0.000722809752915, 0.0017281138571, 0.0111130373552, -0.0123584670946, -0.000117376446724, -0.00230172020383, 0.00122013909277, -0.0005347324186, 0.00778546975926, -0.00308744981885, 0.0111347455531, 0.00368426716886, -0.00532422820106, 0.00903806369752, 0.00295929098502, -0.00493036815897, -0.00224643223919, 0.0078581944108, 0.00488903420046, -0.00863202195615, -0.00127873127349, 0.00833647418767, 0.0091689703986, -0.00032873346936, -0.00225358270109, 0.0051863854751, 0.00786034017801, -0.00461172452196, 0.00929170101881, -4.99468587805e-05, -0.0044270735234, -0.0108339488506]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Morte!</td>\n",
       "      <td>[0.00494207302108, -0.0024438847322, 0.0030088624917, -0.00203303317539, 0.00527333002537, -0.00242527993396, -0.000824165646918, -0.00354013964534, 0.00122730038129, 0.00372130889446, 0.0015355333453, 0.00231205369346, 0.000534532533493, 0.00509732589126, -0.00378835713491, -0.00432973215356, 0.00345020927489, -0.00482956133783, 0.00575210247189, 0.00275641167536, -0.00131944275927, -0.000430752756074, -0.00214626104571, 0.000818708795123, -0.00482920510694, 0.00468720821664, 0.00644132029265, 0.000538371445145, 0.000414245499996, 0.00108160590753, -0.00319398846477, -0.00300756841898, 0.0010656635277, 0.00248127686791, -0.00122872530483, -0.00286133633927, -0.000113207046525, -0.000925299711525, 0.00152162020095, 0.0019544896204, -0.00703291920945, 0.00112676841673, -0.00241509708576, 0.00106176058762, 0.00184989569243, -0.00311265396886, -0.000972050183918, -0.00205844081938, -0.00333906942979, -0.00138157000765, 0.00296801282093, 0.000287400733214, 0.00239301426336, -0.00264780409634, -0.000260469561908, 0.000939870486036, -0.00395710812882, -0.00638897577301, -0.00136778363958, 0.00117559370119, -0.000999728916213, -0.000836948864162, 0.00129985297099, -0.000571440905333, 0.0015166355297, -0.00362882995978, 0.00648830411956, 0.00177705485839, 0.00449680816382, 0.00388811482117, 0.000267830939265, 0.000729529070668, 0.00275647174567, -0.00138186849654, 0.00163900607731, -0.00688320817426, -0.00262319785543, -0.000850002747029, 0.00229256320745, -0.00570944696665, 0.00737624336034, -0.00533430185169, -0.00259562628344, 0.000271944794804, 0.00134880142286, 0.00274287769571, 0.00384506443515, 0.00426079006866, 1.6718840925e-06, 4.71113598906e-05, -0.00177620875183, 0.00233344966546, -0.00354219367728, -0.00042690068949, -0.00211244192906, -0.00238348706625, -0.00200613960624, 0.00582953961566, -0.00223327428102, -0.000229943427257]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  word  \\\n",
       "0  <C5IF8u.3Ky@news.cso.uiuc.edu>                        \n",
       "1  360.0;                                                \n",
       "2  CHE4$PN!]A\\9=ES\"2-U->35RY1^O@)%40N/:LNSLQRZ11MYVW\"I   \n",
       "3  woods                                                 \n",
       "4  Morte!                                                \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     word2vec  \n",
       "0  [0.00297474488616, 0.00426242640242, 0.00115045718849, -0.00414093211293, 0.00552939111367, -0.00301579455845, 0.000586995563935, -0.000823675596621, -0.001064071781, 0.00162940553855, -0.000371181376977, -0.000716799986549, -0.0046420157887, 0.00312513345852, 0.00311536621302, -0.000428203144111, -0.00342879700474, 0.00328808045015, 0.00400523142889, 0.0024073543027, 0.000903616659343, -0.00427946913987, 0.00456818472594, -0.00157911446877, -0.00132594094612, -0.000256380095379, -0.00484062917531, 0.000475894194096, -2.94143537758e-05, 0.00365562411025, -0.00113884359598, -0.000151738233399, -0.0028858743608, -0.0034737479873, 0.00233575399034, 0.000630486523733, -0.00477981474251, 0.00438628485426, 0.00306561239995, 0.00336440373212, 0.00458638183773, -0.00337051413953, -0.00197737594135, -0.00506750587374, -0.000329419301124, -0.00452499743551, 0.00540990335867, -0.00482728052884, -0.00084013963351, -0.00221424805932, 0.00334301032126, -0.00253241928294, -0.00329208909534, -0.00221632421017, 0.00170465942938, 0.00201489054598, -0.00415604515001, -0.00483640469611, 0.00047066519619, -0.00520296394825, -0.00158452056348, -0.00286255823448, -0.00269009801559, -0.00309886224568, 0.000886526191607, 0.00160996825434, -0.00337053276598, 0.00327526475303, -0.00375494174659, 0.00167199445423, -0.00216847658157, -0.00562074314803, -0.000409512227634, 0.00293384445831, 0.0009924932383, -0.00101980485488, 0.00164066359866, -0.0043929615058, -0.000924961641431, -0.00173757772427, -0.00365786417387, -0.00456798402593, 0.0039864978753, 0.001509857364, 0.000410577224102, 0.000437068432802, -0.00490206247196, 0.00369475618936, 0.00434789480641, 0.00193163030781, -0.000422751443693, 0.00459859333932, 0.00258714542724, 0.00189295667224, -0.0017615261022, 5.79569905312e-06, -0.00311522814445, 0.000751218991354, 0.00392152555287, 0.00542231183499]       \n",
       "1  [0.00164296617731, 0.00402011163533, -0.00341997249052, 0.00181064545177, 0.00115713756531, 0.00499207852408, -0.00666259275749, 0.000712228647899, -0.00435985904187, -0.00590976094827, -0.00336569431238, -0.00503790937364, 0.00229378440417, 0.0013105799444, 0.000285958230961, -0.00116698583588, 0.0041963942349, 0.00190469715744, 0.000540571112651, -0.00199648598209, -0.00499176187441, 0.00207925657742, -0.000580201915, -0.000933780043852, 0.00508917914703, -0.00340990140103, -0.0032930674497, 0.000925277417991, 0.00305562932044, -0.00270137400366, 0.00351696810685, -0.00135356665123, 0.00447391159832, 0.00296539068222, 0.00233768532053, -0.0077789472416, 0.00147385220043, 0.00316610210575, 0.000607696129009, 0.00236090132967, 0.000550734868739, 0.00231869611889, -0.00172822701279, -0.00122039567214, 0.00328909815289, 0.00183499383274, -0.00341626233421, -0.00344296148978, -0.000843502988573, 0.00513871479779, -0.000518121174537, -0.00267654238269, -0.000263952038949, -0.00547413248569, 0.00269497628324, -0.00162485288456, -0.00127516861539, 0.000230978723266, -0.00148325960618, -0.00205011595972, -0.00100799056236, 2.04232055694e-05, 0.00270132115111, 0.00486238626763, -0.000796347449068, -0.0017056124052, -0.0023364268709, 0.00529055809602, 0.00194682227448, 0.00514988647774, 0.00186218705494, 0.00162255950272, 0.00431430572644, -0.000569750729483, 2.56241910392e-05, 0.00176354323048, -0.00265433103777, 0.00273352349177, 0.000839602609631, 0.00507058156654, 0.00429962342605, 0.00220120023005, 0.000238612366957, -0.000779112917371, 0.00105583469849, -0.00102424819488, -0.00122376577929, -0.00337405619211, -0.00218626833521, -0.00430599413812, 0.00116281560622, -0.00111078168266, -0.00206259498373, -0.00262598437257, 0.00330503028817, -0.000975074071903, -0.00379833392799, 0.00310796359554, -0.000367124855984, 0.00195140182041]         \n",
       "2  [0.00670566083863, -7.87581484474e-06, -0.00451696896926, 0.00120079063345, 0.00169547088444, 0.00531740160659, 0.00220019253902, 1.4106699382e-06, 0.00217704172246, -0.00511439936236, 0.00152852828614, 0.00279566389509, -0.00258536194451, 0.000796087784693, -0.00348979909904, -0.000213264094782, -0.0035747459624, -0.000598112819716, -0.00465873163193, 0.00213118479587, -0.00278941425495, 0.00292693870142, 0.00174413027707, -0.00807938352227, 0.000390542496461, 0.00691892020404, -0.000803683360573, 0.000463944103103, 0.00213684071787, 0.000811428937595, 0.00449485052377, -0.00482045905665, -0.00166017585434, -0.00246830191463, -0.00283294334076, -0.000551049422938, 0.00328575191088, 0.00395453581586, 0.00302625331096, 0.00458269286901, -0.00129209575243, -0.00101099722087, -0.00513690058142, 0.000347049499396, -0.00330302328803, -0.00129728601314, 0.00420563248917, -0.00365815707482, -0.000240945926635, -0.000858772196807, 0.00307879480533, 0.00231451098807, -0.00140998105053, -0.00444172741845, -0.00425683706999, 0.000417479081079, 0.00563792418689, -0.00260218721814, -0.00378362229094, 0.000407784595154, 0.00128395028878, -0.00579293398187, -0.00520406989381, 0.00216693081893, 0.000826761533972, -0.00233249901794, 0.00215489044785, -0.00355174206197, -0.00180029426701, -0.00519510591403, 0.00284486147575, 0.00328912399709, 0.000737764290534, 0.00025975657627, -0.00146570301149, 0.00656075635925, 0.00366906262934, 0.00112273916602, -0.00497675221413, 0.00326345302165, -0.00326123973355, 0.00322795519605, -0.00230185128748, -0.000227761382121, 0.004898365587, 0.00410693464801, -0.000454528257251, -0.00449569616467, -0.00327586289495, -0.00518970331177, 0.00277398293838, 0.00225683790632, -0.00539715774357, -0.00151153549086, 0.0013411664404, 0.00199825898744, -0.000330908194883, 0.00108697032556, -0.000611252326053, -0.00610497267917]  \n",
       "3  [0.00775190023705, 0.00144148024265, -0.00484913308173, 0.00452073151246, -0.0109325610101, -0.00555660855025, 0.000874503049999, -0.00975184049457, 0.00122546509374, -0.00461204675958, -0.00575741939247, -0.00153915304691, 0.00412272522226, -0.00597363570705, 0.00170346000232, 0.00163426541258, -0.000417792820372, -0.00308019691147, 0.00151154363994, -0.00140929431655, -0.00486217392609, -2.40055906033e-05, 0.00150109594688, -0.00307337287813, -0.00300468597561, 0.00695545831695, 0.00274720462039, 0.00355858751573, 0.00745694898069, 0.000630682858173, 0.00271724467166, -0.00654885545373, -0.00104922580067, 0.000944466970395, -0.00199690065347, 0.00687483837828, -0.00248539401218, -0.00616966467351, 0.0123061249033, 0.00695088366047, -0.00304097565822, 0.00104740669485, -0.00500980298966, 0.00259305885993, -0.0056721707806, -0.00593376578763, -0.000614581396803, -0.00227987649851, -0.00481418985873, -0.00384941231459, 0.00238774996251, 0.00219878205098, -0.00136520015076, -0.00175328110345, -0.00573545554653, 0.00268785771914, -0.00228368979879, -0.00276386109181, -0.000144422680023, -0.00185984908603, 0.00647442881018, 0.0024444963783, -0.00397222023457, 0.00424491753802, 0.00652982201427, 0.00199837912805, 0.00505082355812, 0.00875545293093, -0.000722809752915, 0.0017281138571, 0.0111130373552, -0.0123584670946, -0.000117376446724, -0.00230172020383, 0.00122013909277, -0.0005347324186, 0.00778546975926, -0.00308744981885, 0.0111347455531, 0.00368426716886, -0.00532422820106, 0.00903806369752, 0.00295929098502, -0.00493036815897, -0.00224643223919, 0.0078581944108, 0.00488903420046, -0.00863202195615, -0.00127873127349, 0.00833647418767, 0.0091689703986, -0.00032873346936, -0.00225358270109, 0.0051863854751, 0.00786034017801, -0.00461172452196, 0.00929170101881, -4.99468587805e-05, -0.0044270735234, -0.0108339488506]                   \n",
       "4  [0.00494207302108, -0.0024438847322, 0.0030088624917, -0.00203303317539, 0.00527333002537, -0.00242527993396, -0.000824165646918, -0.00354013964534, 0.00122730038129, 0.00372130889446, 0.0015355333453, 0.00231205369346, 0.000534532533493, 0.00509732589126, -0.00378835713491, -0.00432973215356, 0.00345020927489, -0.00482956133783, 0.00575210247189, 0.00275641167536, -0.00131944275927, -0.000430752756074, -0.00214626104571, 0.000818708795123, -0.00482920510694, 0.00468720821664, 0.00644132029265, 0.000538371445145, 0.000414245499996, 0.00108160590753, -0.00319398846477, -0.00300756841898, 0.0010656635277, 0.00248127686791, -0.00122872530483, -0.00286133633927, -0.000113207046525, -0.000925299711525, 0.00152162020095, 0.0019544896204, -0.00703291920945, 0.00112676841673, -0.00241509708576, 0.00106176058762, 0.00184989569243, -0.00311265396886, -0.000972050183918, -0.00205844081938, -0.00333906942979, -0.00138157000765, 0.00296801282093, 0.000287400733214, 0.00239301426336, -0.00264780409634, -0.000260469561908, 0.000939870486036, -0.00395710812882, -0.00638897577301, -0.00136778363958, 0.00117559370119, -0.000999728916213, -0.000836948864162, 0.00129985297099, -0.000571440905333, 0.0015166355297, -0.00362882995978, 0.00648830411956, 0.00177705485839, 0.00449680816382, 0.00388811482117, 0.000267830939265, 0.000729529070668, 0.00275647174567, -0.00138186849654, 0.00163900607731, -0.00688320817426, -0.00262319785543, -0.000850002747029, 0.00229256320745, -0.00570944696665, 0.00737624336034, -0.00533430185169, -0.00259562628344, 0.000271944794804, 0.00134880142286, 0.00274287769571, 0.00384506443515, 0.00426079006866, 1.6718840925e-06, 4.71113598906e-05, -0.00177620875183, 0.00233344966546, -0.00354219367728, -0.00042690068949, -0.00211244192906, -0.00238348706625, -0.00200613960624, 0.00582953961566, -0.00223327428102, -0.000229943427257]    "
      ]
     },
     "execution_count": 211,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word_vecs = pd.DataFrame(zip(model.vocab.keys(),model.syn0.tolist()), columns=['word','word2vec'])\n",
    "word_vecs.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Demonstrate similarity queries on word vectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Arab', 0.8514078855514526),\n",
       " ('family', 0.8412588834762573),\n",
       " ('German', 0.84117591381073),\n",
       " ('villages', 0.8406447172164917),\n",
       " ('Our', 0.8263046741485596),\n",
       " ('early', 0.8227189779281616),\n",
       " ('killed', 0.8222663402557373),\n",
       " ('Russian', 0.8192578554153442),\n",
       " ('led', 0.818084716796875),\n",
       " ('occupied', 0.8180285096168518)]"
      ]
     },
     "execution_count": 213,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.most_similar(positive=['Kuwait'], negative=[], topn=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Which words in a list don't match"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Diamond'"
      ]
     },
     "execution_count": 216,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.doesnt_match(['Pasadena', 'California', 'Austin', 'Diamond'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Demonstrate similarity between words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.58601824772483946"
      ]
     },
     "execution_count": 214,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.similarity('Iraq', 'Muslim')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.47740710585707935"
      ]
     },
     "execution_count": 215,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.similarity('Iraq', 'Christian')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Obtain doc2vec representation of a new document"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.00239663, -0.01788618,  0.05665193, -0.0151288 , -0.00252076,\n",
       "        0.03537806,  0.03646427,  0.01192429,  0.02087626,  0.0387573 ], dtype=float32)"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.infer_vector(['hello', 'world', 'bye', 'world'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Define PL/Python UDF to train doc2vec models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%execsql\n",
    "--1) SFUNC: State transition function, part of a User-Defined-Aggregate definition\n",
    "-- This function will merely stack every row of input, into the GD variable\n",
    "drop function if exists YOUR_SCHEMA.stack_rows(\n",
    "    text,\n",
    "    text[],\n",
    "    text,\n",
    "    text,\n",
    "    text\n",
    ") cascade;\n",
    "create or replace function YOUR_SCHEMA.stack_rows(\n",
    "    key text,\n",
    "    header text[], -- name of the features column and the dependent variable column\n",
    "    doc_id text, -- id field for each document\n",
    "    content text, -- content of the document\n",
    "    label text -- dependent variable column\n",
    ")\n",
    "returns text\n",
    "as\n",
    "$$\n",
    "    if 'header' not in GD:\n",
    "        GD['header'] = header\n",
    "    if not key:\n",
    "        gd_key = 'stack_rows'\n",
    "        GD[gd_key] = [[doc_id, content, label]]\n",
    "        return gd_key\n",
    "    else:\n",
    "        GD[key].append([doc_id, content, label])\n",
    "        return key\n",
    "$$language plpythonu;\n",
    "\n",
    "--2) Define the User-Defined Aggregate (UDA) consisting of a state-transition function (SFUNC), \n",
    "-- a state variable and a FINALFUNC (optional)\n",
    "drop aggregate if exists YOUR_SCHEMA.stack_rows( \n",
    "    text[], -- header (feature names)\n",
    "    text, -- doc_id\n",
    "    text, -- document content,\n",
    "    text -- label\n",
    ") cascade;\n",
    "create ordered aggregate YOUR_SCHEMA.stack_rows(\n",
    "    text[], -- header (feature names)\n",
    "    text, -- doc_id\n",
    "    text, -- document content,\n",
    "    text -- label\n",
    ")\n",
    "(\n",
    "    SFUNC = YOUR_SCHEMA.stack_rows,\n",
    "    STYPE = text -- the key in GD used to hold the data across calls\n",
    ");\n",
    "\n",
    "--3) UDT to hold document vectors\n",
    "drop type if exists YOUR_SCHEMA.doc_vectors cascade;\n",
    "create type YOUR_SCHEMA.doc_vectors\n",
    "as\n",
    "(\n",
    "    doc_id text,\n",
    "    doc_vec float8[],\n",
    "    label text\n",
    ");\n",
    "\n",
    "--4) UDF for training doc2vec model in Gensim\n",
    "drop function if exists YOUR_SCHEMA.train_doc2vec(\n",
    "    text,\n",
    "    int,\n",
    "    int,\n",
    "    int\n",
    ");\n",
    "create or replace function YOUR_SCHEMA.train_doc2vec(\n",
    "    key text,\n",
    "    vect_size int,\n",
    "    window_size int,\n",
    "    num_workers int\n",
    ")\n",
    "returns setof YOUR_SCHEMA.doc_vectors\n",
    "as\n",
    "$$\n",
    "    from unicodedata import category as uc_cat\n",
    "    from nltk.tokenize import word_tokenize, sent_tokenize\n",
    "    from nltk.stem import WordNetLemmatizer\n",
    "    from nltk.corpus import stopwords\n",
    "    import string, re\n",
    "    from gensim.models.doc2vec import TaggedDocument\n",
    "    from gensim.models import Doc2Vec\n",
    "    import pandas as pd    \n",
    "    \n",
    "    def tokenize_document(doc):\n",
    "        \"\"\"\n",
    "            Use NLTK to tokenize document\n",
    "        \"\"\"        \n",
    "        #discard stopwords\n",
    "        stop_en = stopwords.words('english')\n",
    "        \n",
    "        #Use a wordnet lemmatizer to stem the tokens\n",
    "        wnl = WordNetLemmatizer()\n",
    "        \n",
    "        #tokenize the documents using NLTK\n",
    "        tokens = [w.strip().decode('utf8') for sent in sent_tokenize(doc) for w in word_tokenize(sent)] if doc else None\n",
    "        \n",
    "        if(not tokens):\n",
    "            return [(0, None),]\n",
    "\n",
    "        #Remove control characters\n",
    "        tokens = map(lambda x: ''.join(c for c in x if uc_cat(c) != 'Cc'), tokens)\n",
    "        \n",
    "        #Remove punctuations if they are tokens by themselves\n",
    "        tokens = filter(lambda x: x not in set(list(string.punctuation.decode('utf-8'))), tokens)\n",
    "        \n",
    "        #Replace phone numbers with <PHONE>\n",
    "        phone_pat = re.compile(r'''\n",
    "                        # don't match beginning of string, number can start anywhere\n",
    "            (\\d{1,2}-?)?\n",
    "            (\\d{3})     # area code is 3 digits (e.g. '800')\n",
    "            \\D*         # optional separator is any number of non-digits\n",
    "            (\\d{3})     # trunk is 3 digits (e.g. '555')\n",
    "            \\D*         # optional separator\n",
    "            (\\d{4})     # rest of number is 4 digits (e.g. '1212')\n",
    "            \\D*         # optional separator\n",
    "            (\\d*)       # extension is optional and can be any number of digits\n",
    "            $           # end of string\n",
    "            ''', re.VERBOSE)\n",
    "        tokens = map(lambda x: u'<PHONE>' if phone_pat.match(x) else x, tokens)\n",
    "        \n",
    "        #Replace numbers with <NUM>\n",
    "        num_pat = re.compile(r'^(-|\\+)?(\\d*).?(\\d+)')\n",
    "        tokens = map(lambda x: u'<NUM>' if num_pat.match(x) else x, tokens)\n",
    "        \n",
    "        #Replace dates\n",
    "        date_pat =  re.compile(r'^(\\d{1,2})(/|-)(\\d{1,2})(/|-)(\\d{2,4})$')\n",
    "        tokens = map(lambda x: u'<DATE>' if date_pat.match(x) else x, tokens)\n",
    "        \n",
    "        #Remove tokens which are solely composed on non-alpha characters\n",
    "        non_alpha = re.compile(r'''^(\\W+)$''', re.VERBOSE)\n",
    "        tokens = filter(lambda x: not non_alpha.match(x), tokens)\n",
    "        \n",
    "        #Strip out any lingering punctuations now that the document has been tokenized by NLTK\n",
    "        tokens = map(lambda x: ''.join(c for c in x if not uc_cat(c).startswith('P')), tokens)\n",
    "        \n",
    "        #use the wordnet lemmatizer\n",
    "        lemmatized_tokens = map(lambda x: wnl.lemmatize(x), tokens)\n",
    "        \n",
    "        #filter out empty tokens and stopwords\n",
    "        lemmatized_tokens = filter(lambda x: x and x.strip() and x.strip() not in stop_en, lemmatized_tokens)\n",
    "        return lemmatized_tokens\n",
    "    \n",
    "    def train_doc2vec():\n",
    "        \"\"\"\n",
    "            Train doc2vec models in gensim\n",
    "        \"\"\"\n",
    "        if key in GD:\n",
    "            header = GD['header']\n",
    "            plpy.info('Header:',header)\n",
    "            df = pd.DataFrame(GD[key], columns=header)\n",
    "            plpy.info('Dataframe shape:',df.shape)\n",
    "            docs = [\n",
    "                    TaggedDocument(words=tokenize_document(d), tags=[tag]) \n",
    "                        for d, tag in zip(df[header[1]], df[header[0]])\n",
    "                   ]            \n",
    "            doc_ids = df[header[0]]\n",
    "            labels = df[header[2]]\n",
    "            plpy.info('Training doc2vec')\n",
    "            #Train doc2vec model in Gensim\n",
    "            model = Doc2Vec(\n",
    "                        documents=docs, \n",
    "                        min_count=1, \n",
    "                        window=window_size, \n",
    "                        size=vect_size, \n",
    "                        sample=1e-4, \n",
    "                        negative=5, \n",
    "                        workers=num_workers\n",
    "                    )\n",
    "            return zip(doc_ids, model.docvecs, labels)\n",
    "    #Invoke doc2vec\n",
    "    return train_doc2vec()\n",
    "$$ language plpythonu;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%execsql\n",
    "drop table if exists YOUR_SCHEMA.twenty_news_groups_doc2vec_representation cascade;\n",
    "create table YOUR_SCHEMA.twenty_news_groups_doc2vec_representation\n",
    "as\n",
    "(\n",
    "    select\n",
    "        (doc_vecs).*\n",
    "    from\n",
    "    (\n",
    "        select\n",
    "            YOUR_SCHEMA.train_doc2vec(\n",
    "                stacked_input_key,\n",
    "                20, -- size of the output vector\n",
    "                10, -- window_size\n",
    "                10 -- num_workers (threads)\n",
    "            ) as doc_vecs\n",
    "        from\n",
    "        (\n",
    "            select\n",
    "                YOUR_SCHEMA.stack_rows(\n",
    "                    ARRAY['doc_id', 'contents', 'label'], --header or names of input fields\n",
    "                    doc_id,\n",
    "                    contents,\n",
    "                    label\n",
    "                ) as stacked_input_key\n",
    "            from\n",
    "                YOUR_SCHEMA.twenty_news_groups\n",
    "        )q1\n",
    "    )q2\n",
    ") distributed by (doc_id);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc_id</th>\n",
       "      <th>doc_vec</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>421</td>\n",
       "      <td>[-0.0162567, 0.00640558, -0.0319764, -0.0274248, -0.0372388, 0.0265061, -0.0343747, 0.0145012, -0.00209146, -0.0405663, 0.0343613, -0.0422859, -0.00206888, -0.0618418, 0.0100281, 0.00957236, 0.00245993, -0.0346243, -0.0275735, -0.00153059]</td>\n",
       "      <td>comp.graphics</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>142</td>\n",
       "      <td>[0.0287947, -0.0838569, -0.0781008, -0.0830135, -0.0775834, 0.0868943, -0.00836284, -0.00884065, 0.0409157, -0.0411672, -0.0979297, -0.0279642, -0.00306713, -0.00927339, -0.0885007, 0.0831769, 0.0211072, -0.00763965, -0.0741872, -0.00316988]</td>\n",
       "      <td>alt.atheism</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1972</td>\n",
       "      <td>[0.0348526, -0.106675, -0.038423, -0.0955747, -0.0516444, 0.000549732, -0.162949, 0.0992188, 0.073194, -0.0691682, 0.120136, 0.00964684, -0.0750248, -0.0547911, 0.0520042, 0.134517, 0.129015, -0.0296789, 0.0156653, -0.0923599]</td>\n",
       "      <td>comp.windows.x</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1998</td>\n",
       "      <td>[-0.0464875, 0.0649271, -0.203933, -0.327844, -0.180713, 0.216827, -0.21077, 0.122838, 0.0473668, -0.033562, 0.186332, 0.0773628, 0.0975556, -0.387927, 0.130198, 0.291937, 0.147604, 0.0510774, 0.100246, -0.141307]</td>\n",
       "      <td>comp.windows.x</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>490</td>\n",
       "      <td>[-0.041306, -0.0276567, -0.150082, -0.0194698, -0.0728029, -0.00425778, -0.0793157, 0.079874, 0.01403, -0.0435103, 0.0535301, -0.0623375, 0.00359212, -0.0864903, 0.0760027, -0.000643778, 0.0876219, 0.0501357, 0.0486506, 0.0460701]</td>\n",
       "      <td>comp.graphics</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>348</td>\n",
       "      <td>[0.00667879, -0.0491869, -0.0728234, -0.00277791, -0.0360861, 0.00700159, 0.0136226, 0.0410659, -0.00317544, -0.0239412, 0.022206, -0.008782, -0.00983796, -0.0160417, 0.0463907, 0.0865696, 0.0639555, 0.0126365, 0.0989312, 0.0249212]</td>\n",
       "      <td>comp.graphics</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>56</td>\n",
       "      <td>[0.150542, -0.159149, -0.306852, -0.125841, -0.144607, 0.137373, 0.009954, 0.02002, 0.16023, -0.0807755, -0.160564, -0.0232937, 0.104897, -0.0310717, -0.223183, 0.225264, 0.114698, 0.035119, -0.188344, -0.0676671]</td>\n",
       "      <td>alt.atheism</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1316</td>\n",
       "      <td>[-0.0218237, -0.012275, -0.0668791, -0.0726829, -0.0889829, 0.0117783, -0.0310245, 0.0170602, 0.0407811, -0.0360363, 0.0462637, -0.0680719, -0.013038, -0.0113836, -0.0470533, 0.124959, 0.123977, -0.0828689, 0.0246666, 0.011108]</td>\n",
       "      <td>comp.sys.ibm.pc.hardware</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1381</td>\n",
       "      <td>[-0.0254411, -0.0150829, -0.0534905, -0.0795205, -0.119269, -0.0371484, -0.0697154, 0.0776581, 0.145638, -0.0364588, 0.0337367, -0.00233214, -0.0182445, -0.000567661, -0.0398344, 0.0782709, 0.114415, -0.099335, 0.0186272, -0.111456]</td>\n",
       "      <td>comp.sys.ibm.pc.hardware</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1778</td>\n",
       "      <td>[0.076093, -0.0202146, -0.17295, -0.144834, -0.12579, 0.042422, -0.0506727, 0.00326078, 0.0153817, -0.0408021, 0.10379, 0.0716964, 0.0486067, -0.0081311, -0.0505118, 0.196557, 0.167028, -0.106257, 0.0680318, -0.0123852]</td>\n",
       "      <td>comp.sys.mac.hardware</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  doc_id  \\\n",
       "0  421     \n",
       "1  142     \n",
       "2  1972    \n",
       "3  1998    \n",
       "4  490     \n",
       "5  348     \n",
       "6  56      \n",
       "7  1316    \n",
       "8  1381    \n",
       "9  1778    \n",
       "\n",
       "                                                                                                                                                                                                                                             doc_vec  \\\n",
       "0  [-0.0162567, 0.00640558, -0.0319764, -0.0274248, -0.0372388, 0.0265061, -0.0343747, 0.0145012, -0.00209146, -0.0405663, 0.0343613, -0.0422859, -0.00206888, -0.0618418, 0.0100281, 0.00957236, 0.00245993, -0.0346243, -0.0275735, -0.00153059]     \n",
       "1  [0.0287947, -0.0838569, -0.0781008, -0.0830135, -0.0775834, 0.0868943, -0.00836284, -0.00884065, 0.0409157, -0.0411672, -0.0979297, -0.0279642, -0.00306713, -0.00927339, -0.0885007, 0.0831769, 0.0211072, -0.00763965, -0.0741872, -0.00316988]   \n",
       "2  [0.0348526, -0.106675, -0.038423, -0.0955747, -0.0516444, 0.000549732, -0.162949, 0.0992188, 0.073194, -0.0691682, 0.120136, 0.00964684, -0.0750248, -0.0547911, 0.0520042, 0.134517, 0.129015, -0.0296789, 0.0156653, -0.0923599]                  \n",
       "3  [-0.0464875, 0.0649271, -0.203933, -0.327844, -0.180713, 0.216827, -0.21077, 0.122838, 0.0473668, -0.033562, 0.186332, 0.0773628, 0.0975556, -0.387927, 0.130198, 0.291937, 0.147604, 0.0510774, 0.100246, -0.141307]                               \n",
       "4  [-0.041306, -0.0276567, -0.150082, -0.0194698, -0.0728029, -0.00425778, -0.0793157, 0.079874, 0.01403, -0.0435103, 0.0535301, -0.0623375, 0.00359212, -0.0864903, 0.0760027, -0.000643778, 0.0876219, 0.0501357, 0.0486506, 0.0460701]              \n",
       "5  [0.00667879, -0.0491869, -0.0728234, -0.00277791, -0.0360861, 0.00700159, 0.0136226, 0.0410659, -0.00317544, -0.0239412, 0.022206, -0.008782, -0.00983796, -0.0160417, 0.0463907, 0.0865696, 0.0639555, 0.0126365, 0.0989312, 0.0249212]            \n",
       "6  [0.150542, -0.159149, -0.306852, -0.125841, -0.144607, 0.137373, 0.009954, 0.02002, 0.16023, -0.0807755, -0.160564, -0.0232937, 0.104897, -0.0310717, -0.223183, 0.225264, 0.114698, 0.035119, -0.188344, -0.0676671]                               \n",
       "7  [-0.0218237, -0.012275, -0.0668791, -0.0726829, -0.0889829, 0.0117783, -0.0310245, 0.0170602, 0.0407811, -0.0360363, 0.0462637, -0.0680719, -0.013038, -0.0113836, -0.0470533, 0.124959, 0.123977, -0.0828689, 0.0246666, 0.011108]                 \n",
       "8  [-0.0254411, -0.0150829, -0.0534905, -0.0795205, -0.119269, -0.0371484, -0.0697154, 0.0776581, 0.145638, -0.0364588, 0.0337367, -0.00233214, -0.0182445, -0.000567661, -0.0398344, 0.0782709, 0.114415, -0.099335, 0.0186272, -0.111456]            \n",
       "9  [0.076093, -0.0202146, -0.17295, -0.144834, -0.12579, 0.042422, -0.0506727, 0.00326078, 0.0153817, -0.0408021, 0.10379, 0.0716964, 0.0486067, -0.0081311, -0.0505118, 0.196557, 0.167028, -0.106257, 0.0680318, -0.0123852]                         \n",
       "\n",
       "                      label  \n",
       "0  comp.graphics             \n",
       "1  alt.atheism               \n",
       "2  comp.windows.x            \n",
       "3  comp.windows.x            \n",
       "4  comp.graphics             \n",
       "5  comp.graphics             \n",
       "6  alt.atheism               \n",
       "7  comp.sys.ibm.pc.hardware  \n",
       "8  comp.sys.ibm.pc.hardware  \n",
       "9  comp.sys.mac.hardware     "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%showsql\n",
    "select\n",
    "    *\n",
    "from\n",
    "    YOUR_SCHEMA.twenty_news_groups_doc2vec_representation\n",
    "limit 10;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Create train & test split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%execsql\n",
    "-- Set the seed to ensure the random splits are repeatable\n",
    "-- This is important if you want to measure performance \n",
    "-- of classifiers based by varying the \"num_bits\" parameter\n",
    "-- Note: You should run the setseed() statement and the following create temp table statement as a single\n",
    "-- transaction.\n",
    "select setseed(0.5);\n",
    "drop table if exists doc2vec_train_test_split cascade;\n",
    "create temp table doc2vec_train_test_split\n",
    "as\n",
    "(\n",
    "    select\n",
    "        doc_id,\n",
    "        doc_vec as feature_vector,\n",
    "        -- to create a binary classification problem\n",
    "        -- consider the task as being able to predict if a document is related to computers or not            \n",
    "        case \n",
    "            when label ~* 'comp.*' then 1 \n",
    "            else 0\n",
    "        end as label,\n",
    "        random() as splitter\n",
    "    from\n",
    "        YOUR_SCHEMA.twenty_news_groups_doc2vec_representation\n",
    ") distributed randomly;\n",
    "\n",
    "-- Training set\n",
    "drop table if exists YOUR_SCHEMA.doc2vec_training_set cascade;\n",
    "create table YOUR_SCHEMA.doc2vec_training_set\n",
    "as\n",
    "(\n",
    "    select \n",
    "        doc_id,\n",
    "        feature_vector,\n",
    "        label\n",
    "    from\n",
    "        doc2vec_train_test_split\n",
    "    where \n",
    "        splitter <= 0.75\n",
    ") distributed randomly;\n",
    "\n",
    "-- Test set\n",
    "drop table if exists YOUR_SCHEMA.doc2vec_test_set cascade;\n",
    "create table YOUR_SCHEMA.doc2vec_test_set\n",
    "as\n",
    "(\n",
    "    select \n",
    "        doc_id,\n",
    "        feature_vector,\n",
    "        label\n",
    "    from\n",
    "        doc2vec_train_test_split\n",
    "    where \n",
    "        splitter > 0.75\n",
    ") distributed randomly; "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Train classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%execsql\n",
    "drop table if exists YOUR_SCHEMA.doc2vec_logregr_mdl;\n",
    "drop table if exists YOUR_SCHEMA.doc2vec_logregr_mdl_summary;\n",
    "select \n",
    "    madlib.logregr_train( \n",
    "        'YOUR_SCHEMA.doc2vec_training_set',\n",
    "        'YOUR_SCHEMA.doc2vec_logregr_mdl',\n",
    "        'label',\n",
    "        'feature_vector',\n",
    "        NULL,\n",
    "        20,\n",
    "        'irls'\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Score using trained classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%execsql\n",
    "drop table if exists YOUR_SCHEMA.doc2vec_prediction_results cascade;\n",
    "create table YOUR_SCHEMA.doc2vec_prediction_results\n",
    "as\n",
    "(\n",
    "    select\n",
    "        doc_id,\n",
    "        label as actual_label,\n",
    "        madlib.logregr_predict(\n",
    "            coef,\n",
    "            feature_vector\n",
    "        ) as predicted_label,            \n",
    "        madlib.logregr_predict_prob(\n",
    "            coef,\n",
    "            feature_vector\n",
    "        ) as predicted_label_proba\n",
    "    from\n",
    "        YOUR_SCHEMA.doc2vec_logregr_mdl mdl,\n",
    "        YOUR_SCHEMA.doc2vec_test_set test_set\n",
    ") distributed randomly;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Check ROC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc_id</th>\n",
       "      <th>actual_label</th>\n",
       "      <th>predicted_label</th>\n",
       "      <th>predicted_label_proba</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1117</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "      <td>8.796557e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3137</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>9.659613e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>2.318031e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>307</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>4.319086e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13970</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>1.169033e-08</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  doc_id  actual_label predicted_label  predicted_label_proba\n",
       "0  1117   1             True            8.796557e-01         \n",
       "1  3137   0             False           9.659613e-02         \n",
       "2  8      0             False           2.318031e-01         \n",
       "3  307    0             False           4.319086e-02         \n",
       "4  13970  0             False           1.169033e-08         "
      ]
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sql = \"\"\"\n",
    "    select\n",
    "        doc_id,\n",
    "        actual_label,\n",
    "        predicted_label,\n",
    "        predicted_label_proba\n",
    "    from\n",
    "        YOUR_SCHEMA.doc2vec_prediction_results\n",
    "\"\"\"\n",
    "df = psql.read_sql(sql, conn)\n",
    "conn.commit()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEZCAYAAACNebLAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VFX6wPFvekhIaIYiVQFfFQRBEEER7I1VV9fdRf3Z\nG2JZ0bWujV2xLbiKuqJY1sV17WJbuygioigCir5IR6R3SM/M749zMxnCZDKE3EwmeT/Pw8PcMve+\nc2Zy33vPuefcpGAwiDHGGFNZcrwDMMYYUz9ZgjDGGBORJQhjjDERWYIwxhgTkSUIY4wxEVmCMMYY\nE5ElCJ+ISGcR2VqL2/uNiPyjmnVOFJE7Y10/kYjIQSLyYrzjqIqIXCsiT8ewXkBEWvqw/yEiki8i\n34rINyIyS0S+EpFhtb0vv4jIuSLyZgzrLRCRP0SYP76mv3kR6Sci/6zJexuy1HgH0MDVWicTVX0T\nqO6Ppz/QYhfWTxiq+g3w+3jHUY1Yvm8/Ox4tUNW+5RMi0guYJiJdVHW9j/utTbGUz6PAhcAL5TNE\nJBM4ExhQw/32BNrX8L0NliWIOBCRXOAR4EAgALwL3KSqARE5EbgHKAVmA0cDhwJHAL9T1d+IyGnA\nLUCZ9+/PQDFwGZAsIpuBBWHrtwEeA/b11p+gquMrxXQu7o8uG9ikqkeJyIXACCAJWA9cqaoqInsA\nTwN7e/NXA3NVdbSIFAGvA72As4B84EGgJZACPKSqz4hItreNbl4ZfKOql0aZPwR4WFUPqKb8Crzy\nOwZo5+3vwQjfQQHwADAMyAGuB84ADgBWAL9R1QIRGQzcBzTxyvhWVX1PRFKB8d73sxpYA2wK+34f\nxB100oCPgD+rasAry51E2c+5wG+9z9kdKALOUdV5kbYTTlXniEg+0FlEUoAJQGugLbAU+L2qrhOR\nxcAM77PfjPvt3ezF3hp4VlVv8+K8EbgA2AJMBU5V1b28q6eWuN/EW8BTuO8oG9gT+A74g6oWi0gJ\n8A/cbzoLuEVVX/PC3lNE3gI6ASXAmaqqlT7a08CdItJRVZd78/4AzFTVBV6cFwCXs/NvNxv3vR3q\nbX8y8E/gTiBXRJ5U1QtF5BLgSq8sVgNXqOqCyp9TVW+q7ntIZFbFFB8PAetU9QCgH9AbuM6rengW\n90fRF/gE98dVrvzs6j5ghKoeDNwKDFXVr3BJ4AVVvbXS+v8EVFX3AwYBF4vI3hHi2h843EsOhwPn\nAIep6kHA/cCr3nrjge9VtQfurH5Q2DbSgMnevmYDLwM3qGp/YCjwZxE5GHfQa+p9zoMBvJiqmh/+\necZHKj9vWQawRlUPwx3w7xGR9AifNQNYoaq9vPJ5ArjKK4PmwCne9/ES7uByIHAeMElEOgMjcUls\nX+BY3AGt3AO4g1V/oC+QB4yKEAPe54u2H4DDgZHe5/0Cd0JQLe9EogyYB/wR+EJVD1XVrkAB8H9h\nq89V1R6qOtmL9Rzv9zUQuElEWorIcbjfxEGq2g+XWMPP+Juo6gHeQfNi4BlVPRSX2PYGTvLWS8F9\nf/1wB/anRKSVt2wvrxx64RLQdVSiqhuBF3GJqtzFuN8F3m/3XCL/dv8KZKiqAH1wv929gduAqV5y\nONLb7xBV7QM8j0skkT5ng2YJIj5OAB4GUNUS3IH9RNyB4AdV/d5b9izuTK2y54HXReQJ3NnMfdXs\n7yjgcW+bW1S1l6ouirDeHFXd7r0+CegKfCEis7x9NBeRFl785dtbhUsC4T73/t/H28ZT3jY+BTJx\nf5ifAz1E5BPgRuAfXkxVzQ93PDuX3wlhy9/wln0LpOPOYiMpP2gsxB0gV6lqEFiMK9cBwM+qOtPb\n3jwvviNwZfofVS1T1XzgubDtDgMu9T7zN7iqv55VxECU/Qz1ln+jqiu91996sUXSzWuDmCUic4GL\ngFNUtVBVHwKmi8g1IvIo0ANoGvbeqWGvTwb6ichtwDhvXjaujF9S1fK2tUcq7f/zsNc3AOtE5M+4\nBNyu0v7Kv7+5wBzcbx/gK1Vd7L3+DncFE8k/cYkUEemBO5F621sW7bd7FPCkt+8SVT1CVT+rtO3j\ncCdaG7z1/oW7silP2J/TSFgVU3xUrmZIxn0XJeyctHeqk1XVW0XkSdyZ63m4A2nfyuuFKQ3fjojs\nhTuDq9yIvi3sdQrw7/CzJBHZU1U3elUE4Z+hrIrtpAAbK9WLt8ZVYRWLSDfcQfBI4CMRuUJVXxWR\n7sCQ8Pm4aoJylcsoGXflUq6g0vKI1Tq46ppyJRGWJ0d4bwruu6pcXVRaaZ0zyqtGRKSZt35VIp2o\npeA+Uwk7fp5ghJjK7dAGEU5E7sVdbT0FfOxtO3w727z1soBZuOQ51Vv/FG/d0krvqfyZwn8///U+\n14u4KqdOVP2bSQmbDv8eqvysqjpTRFaLyNG4hPCYl9zLt1fVb7fy30IHXDVouEjfR/hvbFuE5Q2S\nXUH4q6o/5PdwVRSISAZwCfA+rvqgu4j09JadDjRjxx90ildn3FRVH8fVs+4rImm4P+DwA2W5D4Dz\nvfc3w9WJd6sm9veB4SLS1nvf5d77wJ2pXejNb4WrForUuKhAoYic5a3bEfgeOEhELsNVQXzg/SG/\nB/T05j9deX6M5RdJVd9BLL4E9hGRft6+egCDgSleDOeISIbXQBp+V817eFVKXnxvAFdUsx+pYj+1\n5Vjc1dhzwDpcG01KhPW646qO/qKqb+MSeIa37tvA6V4bC7jfQFWNyscAo1X1Jdx3MKDS/s4BEJG+\ngOCuLnfVo7iqpNOAiWHzo/12PwTOFZEk77t5GXf1Ev638x7wB6+tDRE5H3dCtaAGMSY0SxD+yhKR\nLd6/rd7/PYCrgTZeNcBs4EdgjFe3eibwbxGZifujLiXsDEdVy7z3/0dEvsGdoZ3vVbV8BJwsIpUb\nZa8E9heR2bizwrtUdVa0wFX1feBe4AMR+Q5Xh/1bb/EoYD9vey8BS8JiDIZtowR39nmRt+67uAbJ\n6bi2lmQRmSciX+MOSg9681MizA93VaXy+wkYU3n/VUxXNz+0TN2dP2cAD4vIHGAScJ53oJiAqz76\nHtdWFF4NdhXuu5+LqyaZTUU1YKQrwmj7qS2jgbFemb6M+x2UnySExzQHd8av3m9wGK4No5uqfoI7\nEH8hIl/hvpvKZ9/lbsZVg36FO5BPYceTkkO93+9EXGP55hp8phdwfyMflVcHQbW/3TtxVymzcd/f\nW6r6OjAdd6L1iqp+iGtE/9j7Dv+PivaTRjX8dZIN911/iEgO8BfgdlUtFJE+uB9wvbr9TkRGAN+q\n6gyvAXgqcJuqvhfn0IyPROQgYJB6d8CJyDXAwao6fBe3EwBaeSdEph7zvQ1CRAYA96jqEZXm/wZ3\nB04JrkphYqT3NyaqulVEioGZXj1/Me7Msr6ZhzvbLa8nf9GSQ6MwH7hB3C2gQdytspfUYDvR2lFM\nPeLrFYR3B8P/AdtUdVDY/FRctcpBuAa4acBJqrrWt2CMMcbsEr/bIBZQUfcXbj/cbX1bvHrqz6m4\nzc0YY0w94GuCUNc7sjTColwgvFFqK+5uHWOMMfVEvPpBbMEliXI5eMMURBMMBoNJSVZ1aUy5zduK\n+H7Rehav2Exmhvtz/mXNVvILS0lPdXeVBgkyfe5K9tyjqv6C1ftlzTbKAnZDS2Xt87JJTq7ZMWn1\n+nwG9GxXK3GkpCSRlpJMSkoyqSlJpKYkk5qSTIr3+o/HSI2CrKsEUTm4H3G9PpvjbpM7HNcdPvpG\nkpJYu7bWBkhNaHl5OVYWnkQoi2AwSCCsva+0LMiq9RV3iG7YUsi2ghIqnwC9NGUBW/NLaNpk5+4t\n2woi9e2r2tJVW8nOrNmffGZ6CtsLS2mRk0GbFk1iek9xaYDM9JSYE1NhcRltWjQhNzudtNRk2rbM\nqlGs5Vo0z2bjpu3Vr1iNlORk2rXaOZaU5KSdvq+Gpq4SRBBARIYD2ao6UURG4Tq0JAETw4YSMCYu\nCopK0eWbiHZC+Ou6fMoCAe/1dmbqWvZollntgeLXdbt3oNpWULLTgTY3O52NWwtp2zKL9nlNObDb\nHrRqmc3mzQWkpSTRttWO6zdvmt7gD2jh8vJyWJsZqS+giVWi9YMI1vczxbqSCGfNdSWWsli7qYCF\nKzYzd9F6sjPd2fi2whKWrNzKqg35NMlIpaAoUnNZbCKd4Ve2raCE/Tq3CE2v31LIgd32CE0XFJXS\nvUPzHd4TJEjr5k2QTi2Ihf0uKlhZVMjLy6nXVUzG+OrHJRuYt3QjGWkVZ4y6bCNNMtOY+dOaat9f\nUFRKu1ZZrFyfz28P35vUKJcRgWCQTm1yAEhPTaZr+2akptigBKbhsQRhElIwGOT7xRt49PXvKSqu\nPFZgZHu1y6FLu1z6dN+D3Cw3Anh6WgqtWzQhCRpV9YsxsbAEYRLG6o35/LJmO29PX8KSVTtWHeQ1\nz6SkNMB5J+xHctjJfLPsDHKz08nNSrMEYMwusgRh6p2yQIDFv25lzKRvAHcHTWEVVwkD9m/DtWf3\nY9uWyiN8G2N2lyUIE3ebthVRUhpg/vJNvPXFElZv3PFgX1hcRqfWTdm4rYiWOZkc0qMNPfduRXvv\nrp4mGamNZ4B+Y+qQJQhTZwKBICWlAVZvzKe4NMArUxaiy6vuH3lU3w4cN6AjezSL7b57U/eee+5f\nvPji87z88pukpaUxZsydHH30cRx88CGhdU455TgmT3ZjOX722RRefvm/BINBiouLGT78bIYOPWqH\nba5Y8Qt33XUHycnJ7LVXV6699oYdlpeUlDBmzJ38+usKsrObcu21N9C+fYed3nfPPX8DYNKkZ/jo\no/fJzm7KmWeew6BBhzFp0jPMmDGdpKQktm7dwoYNG5g8+d3QPp599ikWLlzAnXeOCc0rLCxkxIgL\nGDHiKg4++BBWr17F3XePpqzMXd1ef/0tdOzYiSlTPuK55/5FUlIyxxxzPGec8UeCwSBjx97DggU/\nk56ezg03/IX27TuwePEi7r/f7aNDh47ceOOtJCcn88ILz/HRRx+QlJTEwIGHct55F1Ubsx8sQRhf\nFBSVsmjlFmbNX8vH366gVW4m67cUVrn+/l1a0KJpBjlZ6Qzu3Y52rWre69fUnffff5ejjz6ODz98\njxNOGFbFWq7tZ+7c2bz00vP8/e8PkpGRyZYtm7n00vPZa6+udO7cJbT2+PHjuPTSkfTu3Ye///1u\npk6dwuDBQ0PL33jjNbKyspgw4WmWLVvK2LH3Mm7c+J3e9+GHH5KT04qPPnqfJ554lkAgwGWXXcBB\nB/Xn7LPP4+yzzwPg+uuvYeTIP4W2P336NKZPn0abNm13+BTjxt1LUlJFA9fEiY/xu9/9kcMOO5yv\nvvqSCRMeZvToe5gw4RGefHISmZmZnH32GRx33AnMmvUtxcXFPPbYU/zww/c8/PAD3H33WB5//FEu\nu+wKevU6kDFj7mTatM/o2rU7H374Hk888SwAI0ZcyOGHD40as18sQZjdFggGWbuxgFc+W8SaDfks\nW7Nzhc/6LYU0y05nW0EJe+2ZS8ucDFrmZrJHs0yO7NshDlE3HC9+vICvvVt5U1KSKCvb/b5N/fdt\nze+PjP7QwVmzvqFDhw6ceurpjB59a5QE4bz55uv8/vfDycjIBCA3txlPPPEsTZs2ZcmSxbz66ouM\nGnUDqj/Ru3cfAA45ZBBffz1jhwSxZMliDjnEDQ7dqVNnli1bArDT+7744gtEDqBPn4NITXWHuo4d\nO7Jw4c/sv797SOGnn35Mbm4u/fodDLirlzfffJ2LLrqMN998PbTP55+fRK9evXf4PFdeeQ3Z2e4x\n26WlpaSnZ5CcnMxzz71McnIyGzduIBAIkpqaxpw53zFggIu5R4+e/PTTjwCMGXM/SUlJlJSUsH79\nerKzm9KmTVvGjh0f2k/5tstVjtlPliBMjRWVlPH0Oz/y1Y+R+xm0aZlFhz2yGbB/Gw7o2mqHPgom\n8b311usMG3YqHTt2Ii0tjXnzvo+4XvndY+vWrWPPPXc8GWja1B1gu3TZi1GjbtjpvVlZ2WzbtuMJ\nR/fu+/DFF58zePBQvv9+LmvXriEQCBDe6bf8fd26dWPSpGcoKCiguLiIuXPncMopp4fWmzTpGe64\nw1XxFBQUMHbsvdx662gWL14YWmfmzK9YsWI51113E3PmzA7Nz81144suW7aERx99iLvv/jsAycnJ\nfPrpJ4wbdy+DBg0mMzOT/Pztoc8KkJKSQiAQIDk5mVWrVvGnP11OTk5TunXbh5SUlNC2H3nkQUT2\npUOHjhFj9pslCBOzskCAzduKeeiVOSxbvfNVwh7NMjn3hH3p0aVlHKJrvH5/ZLfQ2X5d9R7eunUr\n06d/wcaNm3j55RfYvn07r7zyIllZ2RQXF++wbnkdfdu2bVmzZhVdu1ZcmcydO5uWLVvRvn1F4gi/\nHTk/fzs5OTk7bO+kk05m6dLFjBx5MT179kJkP5KTk0kOu785P387ubm5dOrUhdNOO4Nrr72SNm3a\n0qPHATRr5nqrL1mymJyc3NC+v/rqSzZuXM/tt9/E1q1bWL9+Pc899y8WLPiZ1atXceWVl7Js2RLm\nz1datmxFt27d+fbbmYwbdx+33Taajh07hfY/ZMgRDBlyBH/72+28++7bZGc3JT+/YriV8uRQXi7/\n/e+rvPXW64wfP45bbrmD4uJi7r57tNfGcmPofZVj9pslCFOl+cs3sXl7MT8v30RhcRmfz915uKw9\nmmVy+pCuHLxfa+tn0Ii8997bDBt2CpdffhUARUWFnHHGKQwf/n98+unHHHaYe7zL7Nmz6NJlLwBO\nPPFkJkx4mD59+pGZmcnGjRsYM+ZO7rrrvh22vc8+wnfffcuBB/blyy+/oG/f/jss//HHeRx00MFc\neeUofvrpR1avXhXxfUOHDmbTpk3k5+fz6KMT2b59G6NGXcnee3cFYObMGaGqKqg4qIOrPps8+VXO\nOuvcHfZd3ghfnhwefHAsY8c+FGqvyM/fzvXXX8MDDzxCWloamZlNSE5O5oADejNt2mccccTRfP/9\n3FCSvPHGUVxxxTV06NCRJk2yQ0njxhtH0a/fwZx55jk77L9yzH6zBGF2smTVFkY/M7PK5Z3aNOW4\n/p0Y2LNtleuYhu3tt9/g1ltHh6YzMjIZOvRIiooKycrK4vzzzyQ7uympqalcf/0tAPTseQAnn/xb\nrrnmclJSUikuLmbEiKvYe+9uO7RBjBz5J+6992+UlZXSufNeHHGEu8vpb3+7nUsuuZyOHTty++3/\n5NlnnyInJ4cbb7wVYKf3HX/88axbt42lSxdz8cXnkJaWzuWXXx06kVm+fBn9+w+ocRk89NA4yspK\nueuuOwgGg3Tu3IXrrruJ4447kZEjLyYtLY2uXbtx3HEnAvD1118yYsQFANx00+0AnH32+dx11x2k\np6eTkZHJjTf+hc8+m8Ls2bMoLS1l+vRpJCUlcemlV9CjR8/djnlX2WB9CcqPqoSColLGvzKHn5ZV\n3Hqam53OSYd0Jic7jTYtsujcNofkenalYIOyVbCyqGBlUcEG6zM18tWPq3n+w5/ZvL14p2Xj/zQ4\nNPKpMabxsQTRCH04czkff7uCLduLya80xHWLnAyO7teB4w/uZG0KxjRyliAakZLSMq78x1SKSwM7\nzO/UpinX/bFPTM80MMY0HpYgGoFf123niTfnsXR1RX1sp9ZNueGsvqSlJtuzDIwxEVmCaGCmzV3J\njB9Xh553sGLt9h0SA8Ct5/Zjr3a58QjPGJNALEE0EL+s2cZtT30VdZ37LhtIqxien2yMMWAJokGY\nMmsFz76noekhB+7JEX3ak5Xpvt70tJTQFYUxxsTKEkQC27K9mIdfm8uCXzaH5k24bghpqTbmkTFm\n91mCSFAfzFjKQy9+F5pOSoInbzgyjhEZYxoaSxAJJhgMcuG9n+wwb+zIQ2mRk1HFO4wxpmYsQSSQ\nDVsKue7RL0LT+3VuwZ+H94ljRMaYhswSRIKYOudXnn7np9D0lb8/kD5727Daxhj/WA+pBLBqQ/4O\nyeGW/zuIYwd0jmNExpjGwK4g6rmNW4u4+fEvQ9NP3WgN0caYumFXEPXY5m1FXPvItND0Q1cPjmM0\nxpjGxhJEPVVSGuCahyuSw5hLDrHB9IwxdcqqmOqZ1RvymfHjal6fujg0777LBrJH8yZxjMoY0xhZ\ngqgn/jdjKS99snCn+ZYcjDHxYgmiHrjtya/4Ze22Heade7wwuNeeJCfbwHrGmPiwBBFn785YFkoO\nHfKyGX1h3T2Q3BhjorEEESfBYJC/TJzByvX5ABy8X2suO6VnnKMyxpgKdhdTnPzrXQ0lh9ysNEsO\nxph6x64g6lhRcRkjxn0amj7sgHZccNJ+cYzIGGMi8zVBiEgS8CjQGygELlLVRWHLzwJGAaXA06r6\nmJ/xxNvrUxfxxrQloenM9BTOO3Hf+AVkjDFR+H0FcSqQoaqDRGQAMM6bV+5+YD8gH5gnIs+r6uYI\n20l4X/6waofkcM9lA2ltt68aY+oxv9sgDgPeBVDVGUC/SstnAy2A8iNl0Od44mLGvNU8/uY8AHKy\n0njqxiMtORhj6j2/ryBygfArglIRSVbVgDf9A/ANsA14VVW3+BxPnfvHS7OZs3B9aHrcFYfGMRpj\njImd3wliC5ATNh1KDiJyAHAS0BnYDjwnIqer6ivRNpiXlxNtcb2ycWthKDnktWjCxJuPqdWOb4lU\nFn6zsqhgZVHBymL3+J0gpgHDgJdF5BBgbtiyzbi2hyJVDYrIGlx1U1Rr1271JdDaVlhcytUPfQ5A\n+z2y+etFA1i/fls174pdXl5OwpSF36wsKlhZVLCyqFDTROl3gngNOEZEyoclPV9EhgPZqjpRRB4H\nPheRImAh8IzP8dSZqx6cSmmZa1L53dCucY7GGGN2na8JQlWDwIhKs+eHLZ8ATPAzhniYv3xTKDn8\n6Yze9OraKs4RGWPMrrOe1LUsGAxyz3PfAiAdm1tyMMYkLOtJXYuKisu445mvQ9NXnt4rjtEYY8zu\nsQRRS1ZtyN/h2dHDBnUhK9OK1xiTuOwIVgsWr9zCX/81MzR9wYn7cegBbeMYkTHG7D5LELupoKh0\nh+TwyDWH0yTDitUYk/jsSLYbCotLGfnAZ6Hpx/88lNQUa/c3xjQMdjTbDb+uyw+9vuWcgyw5GGMa\nFDui1dCv67bzt2dd1dLgXu3oumezOEdkjDG1yxJEDZQFAvxl4ozQ9GG92sUxGmOM8Ye1QdTAlz+s\nDr1+7NohpKelxDEaY4zxh11B7KLSsgBPvv0jAGcc0dWSgzGmwbIEsYsuuX9K6PUx/TrGLxBjjPGZ\nJYhdsK2gJPT69vP6211LxpgGzY5wu+Cjb34BoE3LLDq3tQeRGGMaNksQMQoGg0z+fDEAJw7oFOdo\njDHGf5YgYvTqZ4tCr+22VmNMY2AJIkZvT18KwMmHdiEpqfaeK22MMfWVJYgYTJ39a+j1sEFd4heI\nMcbUIUsQ1SgpLePp//0EwNH9OtidS8aYRiOmntQikg10BeYCWaq63deo6pHXvYZpgDOP3ieOkRhj\nTN2q9nRYRI4CZgOTgbbAEhE51u/A6oNAMMj/vlwGuLYHY4xpTGKpLxkDHAZsUtWVwBDgfl+jqic+\n+Hp56PWpg/eOYyTGGFP3YkkQyaq6qnxCVef5GE+9URYI8MLHCwA4fYglB2NM4xNLG8QvIjIMCIpI\nc2AksMzfsOJv4YotodcnDOgcx0iMMSY+YrmCuBQ4C+gILAQOBC72M6j64LsF6wA4aWBnkpOt34Mx\npvGJ5Qqit6oOD58hIqcBr/oTUvyVBQK8O8NdJHVrb0+KM8Y0TlUmCBH5A5ABjBaR2yq952YacIK4\n+fEvQ697dW0Vx0iMMSZ+ol1B5AKDgBzgiLD5pcAtfgYVb2s3FQJw+ak9bVgNY0yjVWWCUNUngCdE\n5ChV/agOY4qrYDAYet1v39ZxjMQYY+IrljaIIhGZDDQFkoAUoLOqdvEzsHhZsc51Et+/S4s4R2KM\nMfEVy11ME4HXccnkEeBn4DU/g4qnB16cDUCyVS0ZYxq5WBJEgao+DUwBNuJucR3iZ1DxlJOVBsCF\nw/aPcyTGGBNfsSSIQhFpCShwiKoGgWx/w4qP0rIAy1Zvo0lGCs2y0+MdjjHGxFUsCWIc8ALwJnCO\niPwAfONrVHHy49KNAGSmxzTIrTHGNGjVJghVfQk4VlW3AgcBZ+N6VzcowWAw1P4wqGfbOEdjjDHx\nF62jXB4wCtgAPIDr/1CA6xvxLtCmLgKsKwVFZaHXJwzoFMdIjDGmfohWl/IcsBXYA0gXkXeAfwNZ\nwDV1EFudeuLNHwDo030PsjLT4hyNMcbEX7QE0VVVu4pIDjAduBwYD4xT1eJYNi4iScCjQG+gELhI\nVReFLe8PjPUmVwFnx7rt2hQIBpm9cD0AQw5sX9e7N8aYeilaG8QWAK/toSXwO1W9ZxcP4KcCGao6\nCLgJ1+Ad7nHgPFU9HFdtFZdxtV/+ZGHotY29ZIwxTrQEEQx7vVpVp9dg+4fhDvyo6gygX/kCEdkH\nWA+MEpEpQEtV/bkG+9ht737lRm7945Hd4rF7Y4ypl6JVMeWIyGBcEsn2Xoe6F6vqZzFsPxfYHDZd\nKiLJqhrAtW0MxFVdLQLeEpGZqjplFz/DbskvLAm9PvZga5w2xphy0RLEL8Bo7/WKsNfgri6OjGH7\nW3CjwZYrTw7grh4WqOp8ABF5F3eFMSXaBvPycqIt3mVzF7oHA3Xr2LzWt+23RIvXT1YWFawsKlhZ\n7J5oo7keUdWyXTANGAa8LCKHAHPDli0CmorI3l7D9WDcuE9RrV27tRbCqvDax65Wa/9OzWt9237K\ny8tJqHj9ZGVRwcqigpVFhZomSr+7DL8GHCMi07zp80VkOJCtqhNF5ELgeREB+EJV/+dzPDv5dv5a\nAPp0z6vrXRtjTL3ma4Lwxm0aUWn2/LDlU4ABfsYQTX5hCWUB1xbfqU3TeIVhjDH1UixjMTVY85a4\nsZf2apdrT44zxphKqr2CEJEWwH1AV+AM4H7gWlXd6HNsvvtk1goABvZoUKOGGGNMrYjlCuIJ4Gug\nFW7ojZWOPbJrAAAUvklEQVTAJD+DqgtlgUBo9NZ9OjaPczTGGFP/xJIg9lLVx4GAqhar6i1AB5/j\n8t1VD04Nve7Uxm6FM8aYymJJEKUi0gyvZ7WIdAcC0d9Sv+myjaHRW284s0+cozHGmPoplruYbsd1\nXuskIq/jej9f4GdQfgoGg9z7n1kA7NOhGdKpRZwjMsaY+imWBPEBMBN3O2oKcKmqrvY1Kh+t3lgQ\nen3tH+3qwRhjqhJLgliG6/A2SVW/9Dke330+ZyUAx/bvSFpqo77L1xhjooolQfQETgfuEpH2wH9x\nyWKBr5H55J0vlwLQIc86xhljTDTVJgivv8NEYKKI9AMmAH+J5b31TUFRaej1oAPsudPGGBNNLB3l\n8nAd5P6Ie3DQf4Df+hyXL7Zsd8866tw2h2TrOW2MMVHFchXwHfAicI2qfuNzPL6a/8smANq1yopz\nJMYYU//FkiA6hj3DIaEtW7UNsJ7TxhgTiyoThIh8q6p9cR3lwh8/mgQEVTXF9+hqmS53Q2u0a2lX\nEMYYU51oDwzq6/2/072gIpLhZ1B++WXtdgC62xWEMcZUq9qOACIyvdJ0Mq7jXEIpC7hasuSkJGug\nNsaYGESrYvoYGOq9Dm+DKAXe8Des2vfqp4sAaNUsIS9+jDGmzkWrYjoSQEQeVNWr6y4kf0z/YRUA\nJw3sEt9AjDEmQUS7ghimqm8B34rIOZWXq+qzvkZWiwqKStm0zfWB6L9v6zhHY4wxiSHaba79gbfw\nqpkqCQIJkyAWrdwSet0kI+E6gBtjTFxEq2K63fv//PJ5IpKL6xfxQx3EVmv+++HPAAw/unucIzHG\nmMQRy1AbFwKHAjcAs4CtIvKKqv7F7+BqQ35hCSvWudtb92qbG+dojDEmccQy3vXlwHXAcGAycABw\nvJ9B1aZZP68DoENeNt06NItzNMYYkzhieiCCqm4ATgTeVtVSoImvUdWiZ/73EwDH9O8Y50iMMSax\nxJIgfhCRt4C9gQ9F5EXga3/Dqh2BQJCygBslpO8+eXGOxhhjEkssCeIC4D5ggKoWA/8GLvI1qlpS\nnhzSUpPJzkyLczTGGJNYYkkQ6cAw4AMR+Q44EkiI7shLVrnbW/extgdjjNllsSSIh4Es3JXEuUAa\n8JifQdWW979aDsDW/JI4R2KMMYknll5jB6lq77DpK0Rknl8B1aY5i9YDcO4J+8Y5EmOMSTyxXEEk\ni0hofGzvdWmU9euNtBT38Tq2bhrnSIwxJvHEcgUxDvhaRMpHcD0ZuNu/kGpPflEpTZukkZoS0928\nxhhjwlR75FTVp4HfAouAJcBpqvqUz3HttqKSMgCSk+3ZD8YYUxPRRnNNBkYC+wCfq+ojdRZVLViz\nsQCAljkJccOVMcbUO9GuIB4FzgC2AzeLyG11E1LtWLnejb+0f5eWcY7EGGMSU7QEMQQYoqo34vo+\nnF43Ie2+YDDIY5PdgLPpqdb+YIwxNRHt6FmoqkEAVV2PewZEQpg6Z2Xo9dC+7eMYiTHGJK5oCaJy\nQghEXKsemjV/LQAnH9qF3Kz0OEdjjDGJKdptrp1F5KmqplX1guo2LiJJuLaM3kAhcJGqLoqw3gRg\nvareHHPkVcgvLGH2QtdB7pAebXd3c8YY02hFSxCjKk1/WoPtnwpkqOogERmA61NxavgKInIp0LOG\n29/JD0s2hl63aZEwo5IbY0y9E+2Ro/+qhe0fBrzrbW+GiPQLXygiA3HPvp4A1Mp4GE++7UYBObZ/\nR5KSrA+EMcbUlN+3+OQCm8OmS73+FYhIW+B24Aqg1o7krZu7q4aj+3WorU0aY0yjFMtQG7tjC5AT\nNp2squWN3WcArYB3gHZAExH5SVWfjbbBvLycaItZuT6fFjkZ7Netdc2jThDVlUVjYmVRwcqigpXF\n7okpQYhINtAVmAtkqer2GLc/DfcsiZdF5BDv/QCo6nhgvLf9cwGpLjkArF27tcplpWUBygJBysoC\nUddrCPLychr8Z4yVlUUFK4sKVhYVapooq61iEpGjgNnAZKAtsEREjo1x+68BRSIyDRgLXCMiw0XE\nlyfSzdQ1ACTZ+EvGGLPbYrmCGINrbP6fqq4UkSHA88D71b3R62g3otLs+RHWq40GcX5YtAGAQXZ7\nqzHG7LaYngehqqvKJ1S13j4sqLjUNW/036/htz8YY4zfYrmC+EVEhgFB72FBI4Fl/oZVM0tXufrG\nNi2y4hyJMcYkvliuIC4FzgI64p4JcSBwiZ9B1URxSRlrNrkhvjPSUuIcjTHGJL5qryBUdQ0wvA5i\n2S0r1lXcWGUPCTLGmN1XbYIQkcVEGMlVVff2JaIaKq9eOu7gjnGOxBhjGoZY2iCGhr1Owz1+tN49\npm3z9mIA0lOteskYY2pDLFVMSyvNul9EZgJ/8yekXVdYXMrkzxcD0HefvDhHY4wxDUMsVUyHh00m\nAT2AejVM6rrNhaHX7fOy4xiJMcY0HLFUMd0Z9joIrAPO9Secmtm8zVUvHX9wJ1JT7BGjxhhTG2JJ\nEC+q6j99j2Q3bNpWBEBRaVmcIzHGmIYjltPtkb5HsZuWr9kGQPf2zeIciTHGNByxXEEsF5GPgRlA\nQflMVR3tW1S7qLzfg3WQM8aY2hNLgvgy7HW97IH2q9dJrrU9YtQYY2pNlQlCRM5V1X+p6p1VrVNf\nFBaVApCVmRbnSIwxpuGI1gZxdZ1FsZvKq5iaZafHORJjjGk4GsQ9oaWBIElJNgaTMcbUpmhtED1E\nZFGE+UlAsD6NxbTgl82kpTaIXGeMMfVGtASxADixrgKpqdIy95CgEu9hQcYYY2pHtARRHGEcpnpn\n1fp8ANq0tIcEGWNMbYpWLzOtzqLYDfOWuOdQ790uJ86RGGNMw1JlglDVK+oykJpau8kN1CedWsQ5\nEmOMaVgSvmW3LODaHrp3sGE2jDGmNiV8gvjF60WdZqO4GmNMrUr4o2qpd/dSdhPrRW2MMbUp4RPE\nRm+o7yYZsQwrZYwxJlYJnyA2byu2TnLGGOODhD+ypiQn2TDfxhjjg4ROEFu2F1MWCNLBnkNtjDG1\nLqETRHknue2FpXGOxBhjGp6EThDrNrtOcv0kL86RGGNMw5PQCeKnZRsBG4fJGGP8kNAJoqCoDID9\nu7SMcyTGGNPwJHSCWL5mGwDZmdYHwhhjaltCJ4jSsgBJQFKSPUnOGGNqW8ImiMJid+dSy9zMOEdi\njDENU8ImiKISNwZTXnNLEMYY4wdfK+9FJAl4FOgNFAIXqeqisOXDgauBEmCuql4e67bLvEeNNm+a\nUZshG2OM8fh9BXEqkKGqg4CbgHHlC0QkExgNDFHVwUBzERkW64bLAkHADbVhjDGm9vmdIA4D3gVQ\n1RlAv7BlRcAgVS3yplNxVxkx2ZJfDEBKiiUIY4zxg98JIhfYHDZdKiLJAKoaVNW1ACJyJZCtqh/G\nuuFCrw9EeV8IY4wxtcvvDgRbgJyw6WRVDZRPeG0U9wHdgdNi2WBenttck9WuD8QB3fNC8xqbxvq5\nI7GyqGBlUcHKYvf4nSCmAcOAl0XkEGBupeWPAwWqemqsG1y7disA3y9YC0BRYUloXmOSl5fTKD93\nJFYWFawsKlhZVKhpovQ7QbwGHCMi07zp8707l7KBb4Dzgaki8gkQBB5U1cmxbHjdJtdcYc+CMMYY\nf/iaIFQ1CIyoNHt+bey/uNS1PXRobc+CMMYYPyRsR7myMnebq/WkNsYYfyRsgigf6jvdnkdtjDG+\nSNijazOvB7W1QRhjjD8SNkGUlgVo1jTdRnI1xhifJGyCKCkNkJaSsOEbY0y9l5BH2EAgyMatRaRa\ngjDGGN8k5BF23tINAOQXlcY5EmOMabgSMkGs3+w6yfWTvDhHYowxDVdCJoiSUjec076dWsQ5EmOM\nabgSMkEsXunGV0mzPhDGGOObhDzCZqan7PC/McaY2peQCaIs4KqYmtnjRo0xxjcJmSBKvXGYUu1x\no8YY45uETBAr1m4HIMX6QRhjjG8S8ggbxF1BZGX4/TgLY4xpvBIyQSxbvY2kJMiwRmpjjPFNwiWI\nYDDo/R/nQIwxpoFLuASxdlMBAPt3sU5yxhjjp4RLEB/O/AWw50AYY4zfEi5BrNqYD8DAHm3jHIkx\nxjRsCZcgmmamAdClXU6cIzHGmIYt4RLE/F82AZCZbre4GmOMnxIuQZRfQTTJsDYIY4zxU8IliJKy\nAE2bpJGSnHChG2NMQkm4o+yGLUWkptgYTMYY47eEShDBYJCikjLrJGeMMXUgoRJEYXEZYENsGGNM\nXUioBLFqvRvF1QbpM8YY/yVUgigqcVcQ+3W2YTaMMcZvCZUgvpu/FoAUa6Q2xhjfJVSCmPPzOgBy\nstLjHIkxxjR8CZUgspu4tofeXVvFORJjjGn4EipBfPPTGsCG2TDGmLqQUAmi/GFBudlWxWSMMX5L\nqARRWhYkO9OuHowxpi4kVIIAyGveJN4hGGNMo5BwCaJtq6x4h2CMMY2Cr/U1IpIEPAr0BgqBi1R1\nUdjy3wC3AiXA06o6sbpt7tkq26dojTHGhPP7CuJUIENVBwE3AePKF4hIqjd9NDAUuERE8nyOxxhj\nTIz8ThCHAe8CqOoMoF/Ysv2An1V1i6qWAJ8Dh1e3wX1tmA1jjKkTfieIXGBz2HSpiCRXsWwr0Ky6\nDbbMyai96IwxxlTJ73tGtwA5YdPJqhoIW5YbtiwH2BRtY2+OPcUGYQqTl5dT/UqNhJVFBSuLClYW\nu8fvK4hpwIkAInIIMDds2Y9ANxFpLiLpuOql6T7HY4wxJkZJQR8fzxZ2F1Mvb9b5wEFAtqpOFJGT\ngNuBJOBJVX3Mt2CMMcbsEl8ThDHGmMSVcB3ljDHG1A1LEMYYYyKyBGGMMSaiejk0qh9DdCSqGMpi\nOHA1rizmqurlcQnUZ9WVQ9h6E4D1qnpzHYdYZ2L4TfQHxnqTq4CzVbW4zgOtAzGUxVnAKKAUd6xo\n8DfCiMgA4B5VPaLS/F0+btbXKwgboqNCtLLIBEYDQ1R1MNBcRIbFJ0zfVVkO5UTkUqBnXQcWB9WV\nxePAeap6OG4kg851HF9dqq4s7geOxI3qcK2IVNsZN5GJyJ+BJ4CMSvNrdNysrwmi1ofoSGDRyqII\nGKSqRd50Ku4sqiGKVg6IyECgPzCh7kOrc1WWhYjsA6wHRonIFKClqv4cjyDrSNTfBTAbaAGUPyeg\nod+2uQD4bYT5NTpu1tcEUetDdCSwKstCVYOquhZARK7E9S/5MA4x1oUqy0FE2uL601yB61PT0EX7\n+9gDGAg8hDtbPFpEhtZteHUqWlkA/AB8g+uk+5aqbqnL4Oqaqr6Gq06rrEbHzfqaIGp1iI4EF60s\nEJEkEbkfOAo4ra6Dq0PRyuEMoBXwDnAjcKaInFPH8dWlaGWxHligqvNVtRR3dl35rLohqbIsROQA\n4CRcFVsXoI2InF7nEdYPNTpu1tcEYUN0VIhWFuDqmzNU9dSwqqaGqMpyUNXxqtpfVY8E7gH+o6rP\nxifMOhHtN7EIaCoie3vTg3Fn0Q1VtLLYDOQDRaoaBNbgqpsag8pX0jU6btbLntQ2REeFaGWBu3T+\nGpjqLQsCD6rq5LqO02/V/SbC1jsXkEZyF1NVfx9DgXu9ZV+o6jV1H2XdiKEsLgUuwLXXLQQu9q6s\nGiwR6Qw8r6qDvLsca3zcrJcJwhhjTPzV1yomY4wxcWYJwhhjTESWIIwxxkRkCcIYY0xEliCMMcZE\nZAnCGGNMRPVyNFfTOHn3b8+nomNXEq5vx29UdUUV77kdCKrq6N3Y77m4gcyWevvMBD4FLg/vtR7j\ntu4EvlbVt0TkY6/zHiLyrar2rWmM3jY+ATrghklIwvWMXQicVT7kShXvuxjYoqov7M7+TeNjCcLU\nNyt290BaQ5NV9QIIdb76FBgJjN+Vjajq7WGTQ8Pm19ZnukBVyztGIiKv4IazvinKewYBn9TS/k0j\nYgnCJAQR6YE7WGcDrYGxqvpw2PJU4Cmghzfrn17v0da4EV47AAHgZlX9KNq+VDUoIl8A+3jbPh93\nEA7geq9fARRX2t+jqvqkiDwNTAH6eu+drqoDRSSA+3tbDhyoqmtFpAXwPdAJOAa401tnMa7H78YI\n4YWqhUUkBzc435fe9BlenJm40Usvwg37fDJwhIisxI1uukvlYRova4Mw9U17EflWRGZ5/1/rzb8I\n+KuqDsCN7z+m0vsG4Ya2Pgh3sB3kzX8QN6xAf+AUYIKIZEcLQERaAScAn4tIT+BmYLCq9saN7XNH\nhP0dGraJoKpeDaCqA8PmBYAXcYMLApwOvIYbH+hu4Fhve+8D91UR3hNe2fyKG0vnfeAB76rnEuAk\nVe2DG2rjz97B/w3gNlX9oCblYRovu4Iw9U1VVUzXAseLyI24cXcqH9S+B/YRkXdxo7re4M0/GhAR\n+as3nQJ0BeZUev8pIvIt7qQpCXhFVV8QkZHAG6paPvLl47grh7ur2F91JgEP4MYPGg7cAgzAXUV8\n4h3ok3GjskZyoapO9Z5/8TLwTvnYQiJyGvAbERFc9VakMYdiLQ9jLEGYhPES7qD5JvBf4A/hC1V1\ng3e2fzRuiOdZXrVUMnBk+QFeRNrhHsNZWagNopLKV9lJQKqqboywv/2r+xCq+o2ItBSRfkB7Vf1S\nRE4GpqrqqV6M6ew4hHXl/aOq00VkPPBvEemFq1L6GngW134yB9eGEunzxFIexlgVk6l3qnrgz1G4\napI38Rp/vbNtvNe/ASap6ju4Z3RvxdWzf4x3oPQO4HOArF2IZwpwsog096Yvxp3pR9pfx0rvDX94\nTfjn+g+uHeC/3vQMYKCIdPemb8c9KrM647zPMgLXXlKmqmNwDdIn4K4OwF1JlJ8M7m55mEbEEoSp\nb6oaXvgOYJqIzMTV+S8G9gpb/g5QICI/4BptX1HVH4CrgENEZDbwPO6W0O2xBqOqc3HVSZ+JyDzc\nU7j+AvwPyI+wv/D43wBmi0hGpfmTgN7e/6jqatyQ1C96cR6Iq1KrbIeyUdViL5bbcI+anC0iimtI\n30rFs6g/BG72qqCu3J3yMI2LDfdtjDEmIruCMMYYE5ElCGOMMRFZgjDGGBORJQhjjDERWYIwxhgT\nkSUIY4wxEVmCMMYYE5ElCGOMMRH9PzSIDzbhcxURAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1214de1d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "fpr, tpr, thresholds = metrics.roc_curve(\n",
    "                            df['actual_label'].tolist(),\n",
    "                            df['predicted_label_proba'].tolist(), \n",
    "                            pos_label=1\n",
    "                        )\n",
    "auc = metrics.auc(fpr, tpr)\n",
    "rocdf = pd.DataFrame(np.transpose([fpr, tpr, thresholds]), columns = ['fpr','tpr','thresholds'])\n",
    "rocdf.plot(x='fpr',y='tpr', label='AUC:{0}'.format(auc))\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Logistic regression model on Paragraph Vector')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the ROC curve above, we observe that 20-dimensional doc2vec encoding of the newsgroups dataset achieves an accuracy of 0.91, which is comparable to the performance of a 634-dimensional tf-idf vector representation. By increasing the doc2vec size to 100, we get equal performance as the tf-idf representation (0.94)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}