01_Drug_Review_Dataset_Exploration.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "01_Drug_Review_Dataset_Exploration_.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/BrittonWinterrose/Drug_Review_NLP/blob/master/01_Drug_Review_Dataset_Exploration.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "OruMCvwf2ZEL"
      },
      "cell_type": "markdown",
      "source": [
        "# Drug Review Dataset Exploration\n",
        "## [Dataset](https://archive.ics.uci.edu/ml/datasets/Drug+Review+Dataset+%28Drugs.com%29)\n",
        "\n",
        "This was from the first day of looking at the dataset and exploring any type of sentiment analysis or regression. "
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "u0Hnncbo2ZDi"
      },
      "cell_type": "markdown",
      "source": [
        "## Import the Dataset"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "c34d3a41-cc8c-4925-edce-913e37da641d",
        "id": "7cRPjNAz2ZDd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "# Getting started with drug data\n",
        "# http://archive.ics.uci.edu/ml/datasets/Drug+Review+Dataset+%28Drugs.com%29\n",
        "\n",
        "!wget http://archive.ics.uci.edu/ml/machine-learning-databases/00462/drugsCom_raw.zip"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2019-01-01 00:46:51--  http://archive.ics.uci.edu/ml/machine-learning-databases/00462/drugsCom_raw.zip\n",
            "Resolving archive.ics.uci.edu (archive.ics.uci.edu)... 128.195.10.249\n",
            "Connecting to archive.ics.uci.edu (archive.ics.uci.edu)|128.195.10.249|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 42989872 (41M) [application/zip]\n",
            "Saving to: ‘drugsCom_raw.zip’\n",
            "\n",
            "drugsCom_raw.zip    100%[===================>]  41.00M  17.8MB/s    in 2.3s    \n",
            "\n",
            "2019-01-01 00:46:53 (17.8 MB/s) - ‘drugsCom_raw.zip’ saved [42989872/42989872]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "667a3f00-4815-4792-c44b-8efd16997173",
        "id": "flXwghY72ZDa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "!unzip drugsCom_raw.zip"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Archive:  drugsCom_raw.zip\n",
            "  inflating: drugsComTest_raw.tsv    \n",
            "  inflating: drugsComTrain_raw.tsv   \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "cc0e5d48-6a99-4773-f74e-4269958e1729",
        "id": "f9wQxYpZ2ZDU",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        }
      },
      "cell_type": "code",
      "source": [
        "!head -n5 drugsComTrain_raw.tsv"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\tdrugName\tcondition\treview\trating\tdate\tusefulCount\r\n",
            "206461\tValsartan\tLeft Ventricular Dysfunction\t\"\"\"It has no side effect, I take it in combination of Bystolic 5 Mg and Fish Oil\"\"\"\t9.0\tMay 20, 2012\t27\r\n",
            "95260\tGuanfacine\tADHD\t\"\"\"My son is halfway through his fourth week of Intuniv. We became concerned when he began this last week, when he started taking the highest dose he will be on. For two days, he could hardly get out of bed, was very cranky, and slept for nearly 8 hours on a drive home from school vacation (very unusual for him.) I called his doctor on Monday morning and she said to stick it out a few days. See how he did at school, and with getting up in the morning. The last two days have been problem free. He is MUCH more agreeable than ever. He is less emotional (a good thing), less cranky. He is remembering all the things he should. Overall his behavior is better. \r\n",
            "We have tried many different medications and so far this is the most effective.\"\"\"\t8.0\tApril 27, 2010\t192\r\n",
            "92703\tLybrel\tBirth Control\t\"\"\"I used to take another oral contraceptive, which had 21 pill cycle, and was very happy- very light periods, max 5 days, no other side effects. But it contained hormone gestodene, which is not available in US, so I switched to Lybrel, because the ingredients are similar. When my other pills ended, I started Lybrel immediately, on my first day of period, as the instructions said. And the period lasted for two weeks. When taking the second pack- same two weeks. And now, with third pack things got even worse- my third period lasted for two weeks and now it&#039;s the end of the third week- I still have daily brown discharge.\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "7bc6e097-f8ea-4d95-98a8-3fedad78212b",
        "id": "0Mr2c6FM2ZDQ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 142
        }
      },
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from scipy import stats\n",
        "import os\n",
        "import re\n",
        "\n",
        "\n",
        "df = pd.read_table('drugsComTrain_raw.tsv')\n",
        "df.head()\n",
        "\n",
        "### Thought flow for Depression Confidence Intervals\n",
        "\"\"\"\n",
        "I want to take the df, filter by condition, drug, confidence interval, sample size cutoff)\n",
        "Then loop through all the drugs for a specific condition and calculate their\n",
        "mean, top limit, and bottom limit. \n",
        "\"\"\"\n",
        "# Create Confidence Interval Function\n",
        "def confidence_interval (data, ci_percent):\n",
        "  data = np.array(data) # Makes sure our data is in a numpy array\n",
        "  mean = np.mean(data)\n",
        "  n = len(data)\n",
        "  stderr = stats.sem(data)\n",
        "  interval = stderr * stats.t.ppf((1 + ci_percent) / 2., n - 1)\n",
        "  return (mean, mean - interval, mean + interval)\n",
        "\n",
        "\n",
        "def condition_compare (df, condition_id, ci_percent, sample_size_cutoff):\n",
        "  output_names = [\"Drug Name\", \"Sample Mean\", \"Lower Bound\", \"Upper Bound\", \"Sample Size\"]\n",
        "  drug_compare = []\n",
        "  data = df[df.condition == condition_id]\n",
        "  for drug in data.drugName.unique():\n",
        "    one_drug = data[data.drugName == drug].rating\n",
        "    if one_drug.size > sample_size_cutoff:\n",
        "      mean, ilower, iupper= confidence_interval(one_drug, ci_percent)\n",
        "      entry = [drug, mean, ilower, iupper, one_drug.size]\n",
        "      drug_compare.append(entry)\n",
        "  return pd.DataFrame(drug_compare, columns=output_names)\n",
        "\n",
        "\n",
        "df2 = condition_compare(df, \"Depression\", 0.95, 10).sort_values(by=\"Sample Mean\", ascending=False)\n",
        "df2.head(3)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Drug Name</th>\n",
              "      <th>Sample Mean</th>\n",
              "      <th>Lower Bound</th>\n",
              "      <th>Upper Bound</th>\n",
              "      <th>Sample Size</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>62</th>\n",
              "      <td>Niacin</td>\n",
              "      <td>9.857143</td>\n",
              "      <td>9.647474</td>\n",
              "      <td>10.066812</td>\n",
              "      <td>14</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>47</th>\n",
              "      <td>Tramadol</td>\n",
              "      <td>9.288462</td>\n",
              "      <td>8.934000</td>\n",
              "      <td>9.642923</td>\n",
              "      <td>52</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>68</th>\n",
              "      <td>Clomipramine</td>\n",
              "      <td>9.181818</td>\n",
              "      <td>8.106160</td>\n",
              "      <td>10.257476</td>\n",
              "      <td>11</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       Drug Name  Sample Mean  Lower Bound  Upper Bound  Sample Size\n",
              "62        Niacin     9.857143     9.647474    10.066812           14\n",
              "47      Tramadol     9.288462     8.934000     9.642923           52\n",
              "68  Clomipramine     9.181818     8.106160    10.257476           11"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "E3uD5nyG2ZDQ"
      },
      "cell_type": "markdown",
      "source": [
        "## Now an attempt at some sentiment analysis using scikit-learn\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "95pgbDWU2ZDM",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# For this first attempt I followed this tutorial. Simple, but lacking somewhat.\n",
        "# Nevertheless it was reproduceable using my dataset. \n",
        "# https://towardsdatascience.com/sentiment-analysis-with-python-part-1-5ce197074184\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.metrics import accuracy_score"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "f2ee821f-d715-45ff-f961-04f2b84ab158",
        "id": "F-hIspO22ZDH",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 142
        }
      },
      "cell_type": "code",
      "source": [
        "# I combined the pre-split test & train data. Didn't have to but felt right. \n",
        "df_train = pd.read_table('drugsComTrain_raw.tsv')\n",
        "df_test = pd.read_table('drugsComTest_raw.tsv')\n",
        "\n",
        "df_main = pd.concat([df_train, df_test], axis=0)\n",
        "df_main.head(3)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>drugName</th>\n",
              "      <th>condition</th>\n",
              "      <th>review</th>\n",
              "      <th>rating</th>\n",
              "      <th>date</th>\n",
              "      <th>usefulCount</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>206461</td>\n",
              "      <td>Valsartan</td>\n",
              "      <td>Left Ventricular Dysfunction</td>\n",
              "      <td>\"It has no side effect, I take it in combinati...</td>\n",
              "      <td>9.0</td>\n",
              "      <td>May 20, 2012</td>\n",
              "      <td>27</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>95260</td>\n",
              "      <td>Guanfacine</td>\n",
              "      <td>ADHD</td>\n",
              "      <td>\"My son is halfway through his fourth week of ...</td>\n",
              "      <td>8.0</td>\n",
              "      <td>April 27, 2010</td>\n",
              "      <td>192</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>92703</td>\n",
              "      <td>Lybrel</td>\n",
              "      <td>Birth Control</td>\n",
              "      <td>\"I used to take another oral contraceptive, wh...</td>\n",
              "      <td>5.0</td>\n",
              "      <td>December 14, 2009</td>\n",
              "      <td>17</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Unnamed: 0    drugName                     condition  \\\n",
              "0      206461   Valsartan  Left Ventricular Dysfunction   \n",
              "1       95260  Guanfacine                          ADHD   \n",
              "2       92703      Lybrel                 Birth Control   \n",
              "\n",
              "                                              review  rating  \\\n",
              "0  \"It has no side effect, I take it in combinati...     9.0   \n",
              "1  \"My son is halfway through his fourth week of ...     8.0   \n",
              "2  \"I used to take another oral contraceptive, wh...     5.0   \n",
              "\n",
              "                date  usefulCount  \n",
              "0       May 20, 2012           27  \n",
              "1     April 27, 2010          192  \n",
              "2  December 14, 2009           17  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "metadata": {
        "id": "lSyXZwPLyfL4",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### Some RegEx to clean this text up. "
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "2e600dae-4c05-4e60-a890-3a00ecd63600",
        "id": "C0ITH5s32ZDE",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "# Clean up text with RegEx\n",
        "pd.set_option('display.width', 1000)\n",
        "rx_pat = r\"(\\\\r)|(\\\\n)|(\\\\t)|(\\\\f)|(\\.)|(\\;)|(\\:)|(\\!)|(\\')|(\\?)|(\\,)|(\\\")|(\\()|(\\))|(\\[)|(\\])|(&#039;)|(\\d\\s)|(\\d)|(\\/)\"\n",
        "rx_pat_wSpace = r\"(\\-)|(\\\\)|(\\s{2,})\"\n",
        "    \n",
        "df_main['review'].replace(regex=True,inplace=True,to_replace=rx_pat, value=r'')\n",
        "df_main['review'].replace(regex=True,inplace=True,to_replace=rx_pat_wSpace, value=r' ')\n",
        "df_main.review.head(5)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    It has no side effect I take it in combination...\n",
              "1    My son is halfway through his fourth week of I...\n",
              "2    I used to take another oral contraceptive whic...\n",
              "3    This is my first time using any form of birth ...\n",
              "4    Suboxone has completely turned my life around ...\n",
              "5    nd day on mg started to work with rock hard er...\n",
              "6    He pulled out but he cummed a bit in me I took...\n",
              "7    Abilify changed my life There is hope I was on...\n",
              "8     I Ve had nothing but problems with the Kepper...\n",
              "9    I had been on the pill for many years When my ...\n",
              "Name: review, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "9b4b83a9-0342-4d46-ebef-10b1876a6760",
        "id": "J19j9W5B2ZC_",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "cell_type": "code",
      "source": [
        "# Inspect the cleaned text. \n",
        "df_main['review'] = df_main['review'].str.lower()\n",
        "\n",
        "df_main['review'].head(5)\n",
        "# Nailed it. "
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    it has no side effect i take it in combination...\n",
              "1    my son is halfway through his fourth week of i...\n",
              "2    i used to take another oral contraceptive whic...\n",
              "3    this is my first time using any form of birth ...\n",
              "4    suboxone has completely turned my life around ...\n",
              "Name: review, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "zS3HmeD-2ZC8",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# VECTORIZE IT (One Hot Encode It)\n",
        "# Each word becomes one feature (column)\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "\n",
        "cv = CountVectorizer(binary=True)\n",
        "cv.fit(df_main['review'])\n",
        "\n",
        "# Define my X & create my matrix with n things and n features\n",
        "X = cv.transform(df_main['review'])\n",
        "\n",
        "# Define my y with \n",
        "y = df_main[\"rating\"]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "5c294842-ab5d-46fa-bf79-ca483e7bb33f",
        "id": "RVpj-vxL2ZC3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Does the matrix dimensions look right?\n",
        "X"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<215063x71335 sparse matrix of type '<class 'numpy.int64'>'\n",
              "\twith 12366059 stored elements in Compressed Sparse Row format>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "b38f8dcb-221a-42d7-9ae2-2b986d75e836",
        "id": "GYiDars-2ZCy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "cell_type": "code",
      "source": [
        "y.head()"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    9.0\n",
              "1    8.0\n",
              "2    5.0\n",
              "3    8.0\n",
              "4    9.0\n",
              "Name: rating, dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "7bc7ea6b-82f6-4826-a2dc-b8df661c99a3",
        "id": "jbZDpxxJ2ZCm",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "np.size(X,0)"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "215063"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "5fALFKn62ZCk",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Binned by rating (same as the research paper)\n",
        "y_rank = []\n",
        "for i in y:\n",
        "  if i <= 4:\n",
        "    y_rank.append(-1)\n",
        "  elif i >= 7:\n",
        "    y_rank.append(1)\n",
        "  else:\n",
        "    y_rank.append(0)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "6a87f2bb-c000-42a8-cf2e-8685877f355e",
        "id": "vpZLRMuK2ZCf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Make sure they're the same number of n. \n",
        "y_rank = np.asarray(y_rank)\n",
        "np.size(y_rank,0)"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "215063"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "ca2761ba-b22c-48e4-f1ca-5e33bbc1291f",
        "id": "5jPPS0nk2ZCc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        }
      },
      "cell_type": "code",
      "source": [
        "np.random.seed()\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import accuracy_score\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "X_train, X_val, y_train, y_val = train_test_split(X, y_rank, train_size = 0.7)\n",
        "\n",
        "for c in [0.01, 0.05, 0.25, 0.5, 1]:\n",
        "    \n",
        "    lr = LogisticRegression(C=c)\n",
        "    lr.fit(X_train, y_train)\n",
        "    print (\"Accuracy for C=%s: %s\" \n",
        "           % (c, accuracy_score(y_val, lr.predict(X_val))))\n",
        "    \n",
        "# Accuracy for C=0.01: 0.7125497915342768\n",
        "# Accuracy for C=0.05: 0.7175405694446597\n",
        "# Accuracy for C=0.25: 0.7197724701250794\n",
        "# Accuracy for C=0.5: 0.7199584618484477"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_split.py:2179: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
            "  FutureWarning)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
            "  FutureWarning)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/logistic.py:460: FutureWarning: Default multi_class will be changed to 'auto' in 0.22. Specify the multi_class option to silence this warning.\n",
            "  \"this warning.\", FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Accuracy for C=0.01: 0.7822191912459896\n",
            "Accuracy for C=0.05: 0.7928672174088253\n",
            "Accuracy for C=0.25: 0.8028022752987493\n",
            "Accuracy for C=0.5: 0.8072815759698694\n",
            "Accuracy for C=1: 0.8132643097382166\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "866df636-4e85-4355-9b22-afb0ef4babaa",
        "id": "VashOGVa2ZCY",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 156
        }
      },
      "cell_type": "code",
      "source": [
        "# Pick a model with a C value. \n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y_rank, train_size = 0.823)\n",
        "\n",
        "final_model = LogisticRegression(C=1)\n",
        "final_model.fit(X_train, y_train)\n",
        "print (\"Final Accuracy: %s\" \n",
        "       % accuracy_score(y_test, final_model.predict(X_test)))"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_split.py:2179: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
            "  FutureWarning)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
            "  FutureWarning)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/logistic.py:460: FutureWarning: Default multi_class will be changed to 'auto' in 0.22. Specify the multi_class option to silence this warning.\n",
            "  \"this warning.\", FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Final Accuracy: 0.819502456195655\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "c9f5148e-fccb-4a71-a4be-ea38ecc29637",
        "id": "5mmrkXAt2ZCT",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 391
        }
      },
      "cell_type": "code",
      "source": [
        "# Inspect the weights of each token. \n",
        "feature_to_coef = {\n",
        "    word: coef for word, coef in zip(\n",
        "        cv.get_feature_names(), final_model.coef_[0]\n",
        "    )\n",
        "}\n",
        "for best_positive in sorted(\n",
        "    feature_to_coef.items(), \n",
        "    key=lambda x: x[1], \n",
        "    reverse=True)[:10]:\n",
        "    print (best_positive)\n",
        "print(\"\\n\")\n",
        "    \n",
        "#     ('excellent', 0.9288812418118644)\n",
        "#     ('perfect', 0.7934641227980576)\n",
        "#     ('great', 0.675040909917553)\n",
        "#     ('amazing', 0.6160398142631545)\n",
        "#     ('superb', 0.6063967799425831)\n",
        "    \n",
        "for best_negative in sorted(\n",
        "    feature_to_coef.items(), \n",
        "    key=lambda x: x[1])[:10]:\n",
        "    print (best_negative)\n",
        "    \n",
        "#     ('worst', -1.367978497228895)\n",
        "#     ('waste', -1.1684451288279047)\n",
        "#     ('awful', -1.0277001734353677)\n",
        "#     ('poorly', -0.8748317895742782)\n",
        "#     ('boring', -0.8587249740682945)"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "('rejecting', 2.4807666897364)\n",
            "('association', 2.359297784471058)\n",
            "('proair', 2.319277235606814)\n",
            "('outfits', 2.3175035087548577)\n",
            "('ponytail', 2.270379700558579)\n",
            "('disappointed', 2.2315833736272137)\n",
            "('bac', 2.131347660810012)\n",
            "('scam', 2.129333979450242)\n",
            "('doubting', 2.1255704237188664)\n",
            "('advertise', 2.0964402402231)\n",
            "\n",
            "\n",
            "('lifesaver', -2.9898448202957866)\n",
            "('saver', -2.6957205162040636)\n",
            "('saved', -2.507537430528717)\n",
            "('doable', -2.453727059971392)\n",
            "('blessing', -2.306927404512549)\n",
            "('excellent', -2.300258083089314)\n",
            "('endocrine', -2.2506912210059955)\n",
            "('miracle', -2.1964994014355055)\n",
            "('dip', -2.1444478560138642)\n",
            "('aggravate', -2.0894909343327517)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "GbZeITpSzPSY",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### The text tokens represented here were not congruent with the expected sentiment. Things like \"miracle\" and \"aggravate\" being so close together were suspect and caused me to seek a deeper understanding of NLP best practices. "
      ]
    }
  ]
}