Notebooks/2_ETL_DS2_NLP.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  **EDA Dataset 2 and NLP Natural Language Processing** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Natural Language Processing (NLP) is a field of artificial intelligence (AI) that focuses on the interaction between computers and human language. It involves the development of algorithms and models to enable computers to understand, interpret, and generate human language.\n",
    "\n",
    "The purpose of this notebook is to convert the reviews of each user into a dicrete column called \"sentiment_analysis\". The idea is to assign the value of '0' if th review is bad, '1' if it is neutral and '2' if it is positive. If there is not a review, by default we are going to assing '1'."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Data Load:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import json\n",
    "import ast\n",
    "\n",
    "# Let's open the file \"user_reviews\" in .json format and save it into a dataframe called \"df_user_reviews\"\n",
    "data = []\n",
    "with open('Dataset/user_reviews.json', 'r', encoding='utf-8') as file:\n",
    "    for line in file.readlines():\n",
    "        data.append(ast.literal_eval(line))\n",
    "\n",
    "df_user_reviews = pd.DataFrame(data)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Exploratory Data Analysis (EDA):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_url</th>\n",
       "      <th>reviews</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td>[{'funny': '', 'posted': 'Posted November 5, 2...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td>[{'funny': '', 'posted': 'Posted June 24, 2014...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>evcentric</td>\n",
       "      <td>http://steamcommunity.com/id/evcentric</td>\n",
       "      <td>[{'funny': '', 'posted': 'Posted February 3.',...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>doctr</td>\n",
       "      <td>http://steamcommunity.com/id/doctr</td>\n",
       "      <td>[{'funny': '', 'posted': 'Posted October 14, 2...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>maplemage</td>\n",
       "      <td>http://steamcommunity.com/id/maplemage</td>\n",
       "      <td>[{'funny': '3 people found this review funny',...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25794</th>\n",
       "      <td>76561198306599751</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198306...</td>\n",
       "      <td>[{'funny': '', 'posted': 'Posted May 31.', 'la...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25795</th>\n",
       "      <td>Ghoustik</td>\n",
       "      <td>http://steamcommunity.com/id/Ghoustik</td>\n",
       "      <td>[{'funny': '', 'posted': 'Posted June 17.', 'l...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25796</th>\n",
       "      <td>76561198310819422</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198310...</td>\n",
       "      <td>[{'funny': '1 person found this review funny',...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25797</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td>[{'funny': '', 'posted': 'Posted July 21.', 'l...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25798</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td>[{'funny': '1 person found this review funny',...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>25799 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id                                           user_url  \\\n",
       "0      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "1                js41637               http://steamcommunity.com/id/js41637   \n",
       "2              evcentric             http://steamcommunity.com/id/evcentric   \n",
       "3                  doctr                 http://steamcommunity.com/id/doctr   \n",
       "4              maplemage             http://steamcommunity.com/id/maplemage   \n",
       "...                  ...                                                ...   \n",
       "25794  76561198306599751  http://steamcommunity.com/profiles/76561198306...   \n",
       "25795           Ghoustik              http://steamcommunity.com/id/Ghoustik   \n",
       "25796  76561198310819422  http://steamcommunity.com/profiles/76561198310...   \n",
       "25797  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "25798        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "\n",
       "                                                 reviews  \n",
       "0      [{'funny': '', 'posted': 'Posted November 5, 2...  \n",
       "1      [{'funny': '', 'posted': 'Posted June 24, 2014...  \n",
       "2      [{'funny': '', 'posted': 'Posted February 3.',...  \n",
       "3      [{'funny': '', 'posted': 'Posted October 14, 2...  \n",
       "4      [{'funny': '3 people found this review funny',...  \n",
       "...                                                  ...  \n",
       "25794  [{'funny': '', 'posted': 'Posted May 31.', 'la...  \n",
       "25795  [{'funny': '', 'posted': 'Posted June 17.', 'l...  \n",
       "25796  [{'funny': '1 person found this review funny',...  \n",
       "25797  [{'funny': '', 'posted': 'Posted July 21.', 'l...  \n",
       "25798  [{'funny': '1 person found this review funny',...  \n",
       "\n",
       "[25799 rows x 3 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_user_reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 25799 entries, 0 to 25798\n",
      "Data columns (total 3 columns):\n",
      " #   Column    Non-Null Count  Dtype \n",
      "---  ------    --------------  ----- \n",
      " 0   user_id   25799 non-null  object\n",
      " 1   user_url  25799 non-null  object\n",
      " 2   reviews   25799 non-null  object\n",
      "dtypes: object(3)\n",
      "memory usage: 604.8+ KB\n"
     ]
    }
   ],
   "source": [
    "df_user_reviews.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that this dataframe has 25,799 entries and 3 columns. Also, we don't have nulls that need to be addressed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first sight is that we have different reviews per user_id. Also, theres is a user_url per user_id. By this way, we can observe that the column \"reviews\" has a list of dictionaries. Let's check the keys of the dictionaries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'funny': '',\n",
       "  'posted': 'Posted November 5, 2011.',\n",
       "  'last_edited': '',\n",
       "  'item_id': '1250',\n",
       "  'helpful': 'No ratings yet',\n",
       "  'recommend': True,\n",
       "  'review': 'Simple yet with great replayability. In my opinion does \"zombie\" hordes and team work better than left 4 dead plus has a global leveling system. Alot of down to earth \"zombie\" splattering fun for the whole family. Amazed this sort of FPS is so rare.'},\n",
       " {'funny': '',\n",
       "  'posted': 'Posted July 15, 2011.',\n",
       "  'last_edited': '',\n",
       "  'item_id': '22200',\n",
       "  'helpful': 'No ratings yet',\n",
       "  'recommend': True,\n",
       "  'review': \"It's unique and worth a playthrough.\"},\n",
       " {'funny': '',\n",
       "  'posted': 'Posted April 21, 2011.',\n",
       "  'last_edited': '',\n",
       "  'item_id': '43110',\n",
       "  'helpful': 'No ratings yet',\n",
       "  'recommend': True,\n",
       "  'review': 'Great atmosphere. The gunplay can be a bit chunky at times but at the end of the day this game is definitely worth it and I hope they do a sequel...so buy the game so I get a sequel!'}]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_user_reviews.iloc[0,2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that we have 7 keys in the dictionaries. The idea is to convert those keys into columns of the dataframe. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Data Transformation:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to separate the list in the column \"reviews\" for each user_id. What we are going to do is to expand each user_id row with each element of the list. For this purpose, we use the function \"explode\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_url</th>\n",
       "      <th>reviews</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td>{'funny': '', 'posted': 'Posted November 5, 20...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td>{'funny': '', 'posted': 'Posted July 15, 2011....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td>{'funny': '', 'posted': 'Posted April 21, 2011...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td>{'funny': '', 'posted': 'Posted June 24, 2014....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td>{'funny': '', 'posted': 'Posted September 8, 2...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25797</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td>{'funny': '', 'posted': 'Posted July 10.', 'la...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25797</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td>{'funny': '', 'posted': 'Posted July 8.', 'las...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25798</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td>{'funny': '1 person found this review funny', ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25798</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td>{'funny': '', 'posted': 'Posted July 20.', 'la...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25798</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td>{'funny': '', 'posted': 'Posted July 2.', 'las...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59333 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id                                           user_url  \\\n",
       "0      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "0      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "0      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "1                js41637               http://steamcommunity.com/id/js41637   \n",
       "1                js41637               http://steamcommunity.com/id/js41637   \n",
       "...                  ...                                                ...   \n",
       "25797  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "25797  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "25798        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "25798        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "25798        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "\n",
       "                                                 reviews  \n",
       "0      {'funny': '', 'posted': 'Posted November 5, 20...  \n",
       "0      {'funny': '', 'posted': 'Posted July 15, 2011....  \n",
       "0      {'funny': '', 'posted': 'Posted April 21, 2011...  \n",
       "1      {'funny': '', 'posted': 'Posted June 24, 2014....  \n",
       "1      {'funny': '', 'posted': 'Posted September 8, 2...  \n",
       "...                                                  ...  \n",
       "25797  {'funny': '', 'posted': 'Posted July 10.', 'la...  \n",
       "25797  {'funny': '', 'posted': 'Posted July 8.', 'las...  \n",
       "25798  {'funny': '1 person found this review funny', ...  \n",
       "25798  {'funny': '', 'posted': 'Posted July 20.', 'la...  \n",
       "25798  {'funny': '', 'posted': 'Posted July 2.', 'las...  \n",
       "\n",
       "[59333 rows x 3 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_user_reviews_exploded = df_user_reviews.explode('reviews')\n",
    "df_user_reviews_exploded"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we can check that the user_id rows were expanded according to the number of list elements. For instance, the user_id 76561197970982479 was expanded three times.\n",
    "\n",
    "Now we need to convert the keys of each ditionary into other columns in the dataframe. For this purpose, we are going to use the function json_normalize for Pandas as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>funny</th>\n",
       "      <th>posted</th>\n",
       "      <th>last_edited</th>\n",
       "      <th>item_id</th>\n",
       "      <th>helpful</th>\n",
       "      <th>recommend</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td></td>\n",
       "      <td>Posted November 5, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>1250</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Simple yet with great replayability. In my opi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td></td>\n",
       "      <td>Posted July 15, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>22200</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>It's unique and worth a playthrough.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td></td>\n",
       "      <td>Posted April 21, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>43110</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Great atmosphere. The gunplay can be a bit chu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td></td>\n",
       "      <td>Posted June 24, 2014.</td>\n",
       "      <td></td>\n",
       "      <td>251610</td>\n",
       "      <td>15 of 20 people (75%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>I know what you think when you see this title ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td></td>\n",
       "      <td>Posted September 8, 2013.</td>\n",
       "      <td></td>\n",
       "      <td>227300</td>\n",
       "      <td>0 of 1 people (0%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>For a simple (it's actually not all that simpl...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59328</th>\n",
       "      <td></td>\n",
       "      <td>Posted July 10.</td>\n",
       "      <td></td>\n",
       "      <td>70</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>a must have classic from steam definitely wort...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59329</th>\n",
       "      <td></td>\n",
       "      <td>Posted July 8.</td>\n",
       "      <td></td>\n",
       "      <td>362890</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>this game is a perfect remake of the original ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59330</th>\n",
       "      <td>1 person found this review funny</td>\n",
       "      <td>Posted July 3.</td>\n",
       "      <td></td>\n",
       "      <td>273110</td>\n",
       "      <td>1 of 2 people (50%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>had so much fun plaing this and collecting res...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59331</th>\n",
       "      <td></td>\n",
       "      <td>Posted July 20.</td>\n",
       "      <td></td>\n",
       "      <td>730</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>:D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59332</th>\n",
       "      <td></td>\n",
       "      <td>Posted July 2.</td>\n",
       "      <td></td>\n",
       "      <td>440</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>so much fun :D</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59333 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  funny                     posted  \\\n",
       "0                                         Posted November 5, 2011.   \n",
       "1                                            Posted July 15, 2011.   \n",
       "2                                           Posted April 21, 2011.   \n",
       "3                                            Posted June 24, 2014.   \n",
       "4                                        Posted September 8, 2013.   \n",
       "...                                 ...                        ...   \n",
       "59328                                              Posted July 10.   \n",
       "59329                                               Posted July 8.   \n",
       "59330  1 person found this review funny             Posted July 3.   \n",
       "59331                                              Posted July 20.   \n",
       "59332                                               Posted July 2.   \n",
       "\n",
       "      last_edited item_id                                          helpful  \\\n",
       "0                    1250                                   No ratings yet   \n",
       "1                   22200                                   No ratings yet   \n",
       "2                   43110                                   No ratings yet   \n",
       "3                  251610  15 of 20 people (75%) found this review helpful   \n",
       "4                  227300     0 of 1 people (0%) found this review helpful   \n",
       "...           ...     ...                                              ...   \n",
       "59328                  70                                   No ratings yet   \n",
       "59329              362890                                   No ratings yet   \n",
       "59330              273110    1 of 2 people (50%) found this review helpful   \n",
       "59331                 730                                   No ratings yet   \n",
       "59332                 440                                   No ratings yet   \n",
       "\n",
       "      recommend                                             review  \n",
       "0          True  Simple yet with great replayability. In my opi...  \n",
       "1          True               It's unique and worth a playthrough.  \n",
       "2          True  Great atmosphere. The gunplay can be a bit chu...  \n",
       "3          True  I know what you think when you see this title ...  \n",
       "4          True  For a simple (it's actually not all that simpl...  \n",
       "...         ...                                                ...  \n",
       "59328      True  a must have classic from steam definitely wort...  \n",
       "59329      True  this game is a perfect remake of the original ...  \n",
       "59330      True  had so much fun plaing this and collecting res...  \n",
       "59331      True                                                 :D  \n",
       "59332      True                                     so much fun :D  \n",
       "\n",
       "[59333 rows x 7 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_user_reviews_normalized = pd.json_normalize(df_user_reviews_exploded['reviews'])\n",
    "df_user_reviews_normalized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, with the \"reviews\" column transformed into a Dataframe, we need to concatenate it with \"df_user_reviews_exploded\" using the fuunction concat from Pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We need to get rid of the indexes in both dataframes first:\n",
    "\n",
    "df_user_reviews_exploded.reset_index(drop=True, inplace=True)\n",
    "df_user_reviews_normalized.reset_index(drop=True, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_url</th>\n",
       "      <th>funny</th>\n",
       "      <th>posted</th>\n",
       "      <th>last_edited</th>\n",
       "      <th>item_id</th>\n",
       "      <th>helpful</th>\n",
       "      <th>recommend</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted November 5, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>1250</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Simple yet with great replayability. In my opi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 15, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>22200</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>It's unique and worth a playthrough.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted April 21, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>43110</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Great atmosphere. The gunplay can be a bit chu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted June 24, 2014.</td>\n",
       "      <td></td>\n",
       "      <td>251610</td>\n",
       "      <td>15 of 20 people (75%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>I know what you think when you see this title ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted September 8, 2013.</td>\n",
       "      <td></td>\n",
       "      <td>227300</td>\n",
       "      <td>0 of 1 people (0%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>For a simple (it's actually not all that simpl...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59328</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 10.</td>\n",
       "      <td></td>\n",
       "      <td>70</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>a must have classic from steam definitely wort...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59329</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 8.</td>\n",
       "      <td></td>\n",
       "      <td>362890</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>this game is a perfect remake of the original ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59330</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td>1 person found this review funny</td>\n",
       "      <td>Posted July 3.</td>\n",
       "      <td></td>\n",
       "      <td>273110</td>\n",
       "      <td>1 of 2 people (50%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>had so much fun plaing this and collecting res...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59331</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 20.</td>\n",
       "      <td></td>\n",
       "      <td>730</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>:D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59332</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 2.</td>\n",
       "      <td></td>\n",
       "      <td>440</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>so much fun :D</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59333 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id                                           user_url  \\\n",
       "0      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "1      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "2      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "3                js41637               http://steamcommunity.com/id/js41637   \n",
       "4                js41637               http://steamcommunity.com/id/js41637   \n",
       "...                  ...                                                ...   \n",
       "59328  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59329  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59330        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59331        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59332        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "\n",
       "                                  funny                     posted  \\\n",
       "0                                         Posted November 5, 2011.   \n",
       "1                                            Posted July 15, 2011.   \n",
       "2                                           Posted April 21, 2011.   \n",
       "3                                            Posted June 24, 2014.   \n",
       "4                                        Posted September 8, 2013.   \n",
       "...                                 ...                        ...   \n",
       "59328                                              Posted July 10.   \n",
       "59329                                               Posted July 8.   \n",
       "59330  1 person found this review funny             Posted July 3.   \n",
       "59331                                              Posted July 20.   \n",
       "59332                                               Posted July 2.   \n",
       "\n",
       "      last_edited item_id                                          helpful  \\\n",
       "0                    1250                                   No ratings yet   \n",
       "1                   22200                                   No ratings yet   \n",
       "2                   43110                                   No ratings yet   \n",
       "3                  251610  15 of 20 people (75%) found this review helpful   \n",
       "4                  227300     0 of 1 people (0%) found this review helpful   \n",
       "...           ...     ...                                              ...   \n",
       "59328                  70                                   No ratings yet   \n",
       "59329              362890                                   No ratings yet   \n",
       "59330              273110    1 of 2 people (50%) found this review helpful   \n",
       "59331                 730                                   No ratings yet   \n",
       "59332                 440                                   No ratings yet   \n",
       "\n",
       "      recommend                                             review  \n",
       "0          True  Simple yet with great replayability. In my opi...  \n",
       "1          True               It's unique and worth a playthrough.  \n",
       "2          True  Great atmosphere. The gunplay can be a bit chu...  \n",
       "3          True  I know what you think when you see this title ...  \n",
       "4          True  For a simple (it's actually not all that simpl...  \n",
       "...         ...                                                ...  \n",
       "59328      True  a must have classic from steam definitely wort...  \n",
       "59329      True  this game is a perfect remake of the original ...  \n",
       "59330      True  had so much fun plaing this and collecting res...  \n",
       "59331      True                                                 :D  \n",
       "59332      True                                     so much fun :D  \n",
       "\n",
       "[59333 rows x 9 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Concat both dataframes:\n",
    "df_user_reviews_expanded = pd.concat([df_user_reviews_exploded.drop('reviews',axis=1),df_user_reviews_normalized], axis=1)\n",
    "df_user_reviews_expanded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 59333 entries, 0 to 59332\n",
      "Data columns (total 9 columns):\n",
      " #   Column       Non-Null Count  Dtype \n",
      "---  ------       --------------  ----- \n",
      " 0   user_id      59333 non-null  object\n",
      " 1   user_url     59333 non-null  object\n",
      " 2   funny        59305 non-null  object\n",
      " 3   posted       59305 non-null  object\n",
      " 4   last_edited  59305 non-null  object\n",
      " 5   item_id      59305 non-null  object\n",
      " 6   helpful      59305 non-null  object\n",
      " 7   recommend    59305 non-null  object\n",
      " 8   review       59305 non-null  object\n",
      "dtypes: object(9)\n",
      "memory usage: 4.1+ MB\n"
     ]
    }
   ],
   "source": [
    "df_user_reviews_expanded.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we have unnested the initial dataframe into something that is easier to manipulate. Now, let's move to  application of NLP."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Sentiment Analysis:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To do a sentiment analysis of a text we can use the library [vaderSentiment](https://github.com/cjhutto/vaderSentiment). \n",
    "\n",
    "The VADER (Valence Aware Dictionary and sEntiment Reasoner) library is a tool for analyzing and quantifying the sentiment or emotional tone of text data. It is specifically designed to work with text data that is often found in social media, online reviews, and informal communication, where traditional sentiment analysis methods may struggle due to the use of slang, abbreviations, emojis, and other language nuances.\n",
    "\n",
    "It has a function that is called \"sentiment.polarity_scores\". It can be programmed as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Import the library\n",
    "from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentiment of text 1: {'neg': 0.0, 'neu': 0.73, 'pos': 0.27, 'compound': 0.5719}\n",
      "Sentiment of text 2: {'neg': 0.508, 'neu': 0.492, 'pos': 0.0, 'compound': -0.4767}\n"
     ]
    }
   ],
   "source": [
    "sentiment = SentimentIntensityAnalyzer()\n",
    "text_1 = \"The book was a perfect balance between wrtiting style and plot.\"\n",
    "text_2 =  \"The pizza tastes terrible.\"\n",
    "sent_1 = sentiment.polarity_scores(text_1)\n",
    "sent_2 = sentiment.polarity_scores(text_2)\n",
    "print(\"Sentiment of text 1:\", sent_1)\n",
    "print(\"Sentiment of text 2:\", sent_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "According to the [documentation](https://github.com/cjhutto/vaderSentiment), this is the interpretation of the scoring:\n",
    "\n",
    "* The ``compound`` score is computed by summing the valence scores of each word in the lexicon, adjusted according to the rules, and then normalized to be between -1 (most extreme negative) and +1 (most extreme positive). This is the most useful metric if you want a single unidimensional measure of sentiment for a given sentence. Calling it a 'normalized, weighted composite score' is accurate. \n",
    " \n",
    "  It is also useful for researchers who would like to set standardized thresholds for classifying sentences as either positive, neutral, or negative.  \n",
    "  Typical threshold values (used in the literature cited on this page) are:\n",
    "\n",
    "  - **positive sentiment**: ``compound`` score >=  0.05\n",
    "  - **neutral  sentiment**: (``compound`` score > -0.05) and (``compound`` score < 0.05)\n",
    "  - **negative sentiment**: ``compound`` score <= -0.05\n",
    " \n",
    "  **NOTE:** The ``compound`` score is the one most commonly used for sentiment analysis by most researchers, including the authors.\n",
    "\n",
    "* The ``pos``, ``neu``, and ``neg`` scores are *ratios for proportions of text that fall in each category* (so these should all add up to be 1... or close to it with float operation).  These are the most useful metrics if you want to analyze the context & presentation of how sentiment is conveyed or embedded in rhetoric for a given sentence. For example, different writing styles may embed strongly positive or negative sentiment within varying proportions of neutral text -- i.e., some writing styles may reflect a penchant for strongly flavored rhetoric, whereas other styles may use a great deal of neutral text while still conveying a similar overall (compound) sentiment. As another example: researchers analyzing information presentation in journalistic or editorical news might desire to establish whether the proportions of text (associated with a topic or named entity, for example) are balanced with similar amounts of positively and negatively framed text versus being \"biased\" towards one polarity or the other for the topic/entity.\n",
    "\n",
    "  * **IMPORTANTLY:** these proportions represent the \"raw categorization\" of each lexical item (e.g., words, emoticons/emojis, or initialisms) into positve, negative, or neutral classes; they **do not** account for the VADER rule-based enhancements such as word-order sensitivity for sentiment-laden multi-word phrases, degree modifiers, word-shape amplifiers, punctuation amplifiers, negation polarity switches, or contrastive conjunction sensitivity.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have understood how we can leverage this library for our analysis, let's try to use it with our first review in the dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Simple yet with great replayability. In my opinion does \"zombie\" hordes and team work better than left 4 dead plus has a global leveling system. Alot of down to earth \"zombie\" splattering fun for the whole family. Amazed this sort of FPS is so rare.'"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's do the exercise with the first row of reviews\n",
    "testSA = df_user_reviews_expanded.iloc[0,8]\n",
    "text = testSA\n",
    "text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentiment of text: {'neg': 0.074, 'neu': 0.692, 'pos': 0.234, 'compound': 0.8481}\n"
     ]
    }
   ],
   "source": [
    "# We obtain the sentiment scores\n",
    "sent_1 = sentiment.polarity_scores(text)\n",
    "print(\"Sentiment of text:\", sent_1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8481"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We are going to use the compound score to classify the review in the categories 0,1 and 2. We can found this value\n",
    "# in the 'compound' key of the dictionary\n",
    "sent_1['compound']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it's time to do the same for all the dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.8481, 0.2263, 0.9117, 0.9899, 0.9958]"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentiment_score = []\n",
    "for text in df_user_reviews_expanded['review']:\n",
    "    if isinstance(text, str):\n",
    "        sent = sentiment.polarity_scores(text)\n",
    "        sentiment_score.append(sent['compound'])\n",
    "    else:\n",
    "        # Handle cases where 'review' is not a string (e.g., NaN values)\n",
    "        sentiment_score.append(0.0)  \n",
    "\n",
    "\n",
    "sentiment_score[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a copy of our original expanded data set in order to add the column called \"sentiment_score\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_url</th>\n",
       "      <th>funny</th>\n",
       "      <th>posted</th>\n",
       "      <th>last_edited</th>\n",
       "      <th>item_id</th>\n",
       "      <th>helpful</th>\n",
       "      <th>recommend</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted November 5, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>1250</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Simple yet with great replayability. In my opi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 15, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>22200</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>It's unique and worth a playthrough.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted April 21, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>43110</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Great atmosphere. The gunplay can be a bit chu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted June 24, 2014.</td>\n",
       "      <td></td>\n",
       "      <td>251610</td>\n",
       "      <td>15 of 20 people (75%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>I know what you think when you see this title ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted September 8, 2013.</td>\n",
       "      <td></td>\n",
       "      <td>227300</td>\n",
       "      <td>0 of 1 people (0%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>For a simple (it's actually not all that simpl...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59328</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 10.</td>\n",
       "      <td></td>\n",
       "      <td>70</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>a must have classic from steam definitely wort...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59329</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 8.</td>\n",
       "      <td></td>\n",
       "      <td>362890</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>this game is a perfect remake of the original ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59330</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td>1 person found this review funny</td>\n",
       "      <td>Posted July 3.</td>\n",
       "      <td></td>\n",
       "      <td>273110</td>\n",
       "      <td>1 of 2 people (50%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>had so much fun plaing this and collecting res...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59331</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 20.</td>\n",
       "      <td></td>\n",
       "      <td>730</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>:D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59332</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 2.</td>\n",
       "      <td></td>\n",
       "      <td>440</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>so much fun :D</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59333 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id                                           user_url  \\\n",
       "0      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "1      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "2      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "3                js41637               http://steamcommunity.com/id/js41637   \n",
       "4                js41637               http://steamcommunity.com/id/js41637   \n",
       "...                  ...                                                ...   \n",
       "59328  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59329  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59330        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59331        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59332        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "\n",
       "                                  funny                     posted  \\\n",
       "0                                         Posted November 5, 2011.   \n",
       "1                                            Posted July 15, 2011.   \n",
       "2                                           Posted April 21, 2011.   \n",
       "3                                            Posted June 24, 2014.   \n",
       "4                                        Posted September 8, 2013.   \n",
       "...                                 ...                        ...   \n",
       "59328                                              Posted July 10.   \n",
       "59329                                               Posted July 8.   \n",
       "59330  1 person found this review funny             Posted July 3.   \n",
       "59331                                              Posted July 20.   \n",
       "59332                                               Posted July 2.   \n",
       "\n",
       "      last_edited item_id                                          helpful  \\\n",
       "0                    1250                                   No ratings yet   \n",
       "1                   22200                                   No ratings yet   \n",
       "2                   43110                                   No ratings yet   \n",
       "3                  251610  15 of 20 people (75%) found this review helpful   \n",
       "4                  227300     0 of 1 people (0%) found this review helpful   \n",
       "...           ...     ...                                              ...   \n",
       "59328                  70                                   No ratings yet   \n",
       "59329              362890                                   No ratings yet   \n",
       "59330              273110    1 of 2 people (50%) found this review helpful   \n",
       "59331                 730                                   No ratings yet   \n",
       "59332                 440                                   No ratings yet   \n",
       "\n",
       "      recommend                                             review  \n",
       "0          True  Simple yet with great replayability. In my opi...  \n",
       "1          True               It's unique and worth a playthrough.  \n",
       "2          True  Great atmosphere. The gunplay can be a bit chu...  \n",
       "3          True  I know what you think when you see this title ...  \n",
       "4          True  For a simple (it's actually not all that simpl...  \n",
       "...         ...                                                ...  \n",
       "59328      True  a must have classic from steam definitely wort...  \n",
       "59329      True  this game is a perfect remake of the original ...  \n",
       "59330      True  had so much fun plaing this and collecting res...  \n",
       "59331      True                                                 :D  \n",
       "59332      True                                     so much fun :D  \n",
       "\n",
       "[59333 rows x 9 columns]"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_user_reviews_expanded1 = df_user_reviews_expanded.copy()\n",
    "df_user_reviews_expanded1 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_user_reviews_expanded1['sentiment_score'] = sentiment_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_url</th>\n",
       "      <th>funny</th>\n",
       "      <th>posted</th>\n",
       "      <th>last_edited</th>\n",
       "      <th>item_id</th>\n",
       "      <th>helpful</th>\n",
       "      <th>recommend</th>\n",
       "      <th>review</th>\n",
       "      <th>sentiment_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted November 5, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>1250</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Simple yet with great replayability. In my opi...</td>\n",
       "      <td>0.8481</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 15, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>22200</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>It's unique and worth a playthrough.</td>\n",
       "      <td>0.2263</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted April 21, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>43110</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Great atmosphere. The gunplay can be a bit chu...</td>\n",
       "      <td>0.9117</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted June 24, 2014.</td>\n",
       "      <td></td>\n",
       "      <td>251610</td>\n",
       "      <td>15 of 20 people (75%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>I know what you think when you see this title ...</td>\n",
       "      <td>0.9899</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted September 8, 2013.</td>\n",
       "      <td></td>\n",
       "      <td>227300</td>\n",
       "      <td>0 of 1 people (0%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>For a simple (it's actually not all that simpl...</td>\n",
       "      <td>0.9958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59328</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 10.</td>\n",
       "      <td></td>\n",
       "      <td>70</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>a must have classic from steam definitely wort...</td>\n",
       "      <td>0.5574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59329</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 8.</td>\n",
       "      <td></td>\n",
       "      <td>362890</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>this game is a perfect remake of the original ...</td>\n",
       "      <td>0.9786</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59330</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td>1 person found this review funny</td>\n",
       "      <td>Posted July 3.</td>\n",
       "      <td></td>\n",
       "      <td>273110</td>\n",
       "      <td>1 of 2 people (50%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>had so much fun plaing this and collecting res...</td>\n",
       "      <td>0.7827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59331</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 20.</td>\n",
       "      <td></td>\n",
       "      <td>730</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>:D</td>\n",
       "      <td>0.5106</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59332</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 2.</td>\n",
       "      <td></td>\n",
       "      <td>440</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>so much fun :D</td>\n",
       "      <td>0.8349</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59333 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id                                           user_url  \\\n",
       "0      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "1      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "2      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "3                js41637               http://steamcommunity.com/id/js41637   \n",
       "4                js41637               http://steamcommunity.com/id/js41637   \n",
       "...                  ...                                                ...   \n",
       "59328  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59329  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59330        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59331        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59332        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "\n",
       "                                  funny                     posted  \\\n",
       "0                                         Posted November 5, 2011.   \n",
       "1                                            Posted July 15, 2011.   \n",
       "2                                           Posted April 21, 2011.   \n",
       "3                                            Posted June 24, 2014.   \n",
       "4                                        Posted September 8, 2013.   \n",
       "...                                 ...                        ...   \n",
       "59328                                              Posted July 10.   \n",
       "59329                                               Posted July 8.   \n",
       "59330  1 person found this review funny             Posted July 3.   \n",
       "59331                                              Posted July 20.   \n",
       "59332                                               Posted July 2.   \n",
       "\n",
       "      last_edited item_id                                          helpful  \\\n",
       "0                    1250                                   No ratings yet   \n",
       "1                   22200                                   No ratings yet   \n",
       "2                   43110                                   No ratings yet   \n",
       "3                  251610  15 of 20 people (75%) found this review helpful   \n",
       "4                  227300     0 of 1 people (0%) found this review helpful   \n",
       "...           ...     ...                                              ...   \n",
       "59328                  70                                   No ratings yet   \n",
       "59329              362890                                   No ratings yet   \n",
       "59330              273110    1 of 2 people (50%) found this review helpful   \n",
       "59331                 730                                   No ratings yet   \n",
       "59332                 440                                   No ratings yet   \n",
       "\n",
       "      recommend                                             review  \\\n",
       "0          True  Simple yet with great replayability. In my opi...   \n",
       "1          True               It's unique and worth a playthrough.   \n",
       "2          True  Great atmosphere. The gunplay can be a bit chu...   \n",
       "3          True  I know what you think when you see this title ...   \n",
       "4          True  For a simple (it's actually not all that simpl...   \n",
       "...         ...                                                ...   \n",
       "59328      True  a must have classic from steam definitely wort...   \n",
       "59329      True  this game is a perfect remake of the original ...   \n",
       "59330      True  had so much fun plaing this and collecting res...   \n",
       "59331      True                                                 :D   \n",
       "59332      True                                     so much fun :D   \n",
       "\n",
       "       sentiment_score  \n",
       "0               0.8481  \n",
       "1               0.2263  \n",
       "2               0.9117  \n",
       "3               0.9899  \n",
       "4               0.9958  \n",
       "...                ...  \n",
       "59328           0.5574  \n",
       "59329           0.9786  \n",
       "59330           0.7827  \n",
       "59331           0.5106  \n",
       "59332           0.8349  \n",
       "\n",
       "[59333 rows x 10 columns]"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_user_reviews_expanded1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the \"sentiment_score\" to do the final classification into 0,1,2 considering the following rules as the documentation suggests:\n",
    "\n",
    "  - **positive sentiment**: ``compound`` score >=  0.05\n",
    "  - **neutral  sentiment**: (``compound`` score > -0.05) and (``compound`` score < 0.05)\n",
    "  - **negative sentiment**: ``compound`` score <= -0.05"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [],
   "source": [
    "sentiment_analysis = []\n",
    "for score in df_user_reviews_expanded1['sentiment_score']:\n",
    "    if score >= 0.05:\n",
    "        sent = 2\n",
    "    elif score > -0.05 and score < 0.05:\n",
    "        sent = 1\n",
    "    else:\n",
    "        sent = 0\n",
    "    sentiment_analysis.append(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_url</th>\n",
       "      <th>funny</th>\n",
       "      <th>posted</th>\n",
       "      <th>last_edited</th>\n",
       "      <th>item_id</th>\n",
       "      <th>helpful</th>\n",
       "      <th>recommend</th>\n",
       "      <th>review</th>\n",
       "      <th>sentiment_score</th>\n",
       "      <th>sentiment_analysis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted November 5, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>1250</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Simple yet with great replayability. In my opi...</td>\n",
       "      <td>0.8481</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 15, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>22200</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>It's unique and worth a playthrough.</td>\n",
       "      <td>0.2263</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted April 21, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>43110</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>Great atmosphere. The gunplay can be a bit chu...</td>\n",
       "      <td>0.9117</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted June 24, 2014.</td>\n",
       "      <td></td>\n",
       "      <td>251610</td>\n",
       "      <td>15 of 20 people (75%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>I know what you think when you see this title ...</td>\n",
       "      <td>0.9899</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted September 8, 2013.</td>\n",
       "      <td></td>\n",
       "      <td>227300</td>\n",
       "      <td>0 of 1 people (0%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>For a simple (it's actually not all that simpl...</td>\n",
       "      <td>0.9958</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59328</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 10.</td>\n",
       "      <td></td>\n",
       "      <td>70</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>a must have classic from steam definitely wort...</td>\n",
       "      <td>0.5574</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59329</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 8.</td>\n",
       "      <td></td>\n",
       "      <td>362890</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>this game is a perfect remake of the original ...</td>\n",
       "      <td>0.9786</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59330</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td>1 person found this review funny</td>\n",
       "      <td>Posted July 3.</td>\n",
       "      <td></td>\n",
       "      <td>273110</td>\n",
       "      <td>1 of 2 people (50%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>had so much fun plaing this and collecting res...</td>\n",
       "      <td>0.7827</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59331</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 20.</td>\n",
       "      <td></td>\n",
       "      <td>730</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>:D</td>\n",
       "      <td>0.5106</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59332</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 2.</td>\n",
       "      <td></td>\n",
       "      <td>440</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>so much fun :D</td>\n",
       "      <td>0.8349</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59333 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id                                           user_url  \\\n",
       "0      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "1      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "2      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "3                js41637               http://steamcommunity.com/id/js41637   \n",
       "4                js41637               http://steamcommunity.com/id/js41637   \n",
       "...                  ...                                                ...   \n",
       "59328  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59329  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59330        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59331        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59332        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "\n",
       "                                  funny                     posted  \\\n",
       "0                                         Posted November 5, 2011.   \n",
       "1                                            Posted July 15, 2011.   \n",
       "2                                           Posted April 21, 2011.   \n",
       "3                                            Posted June 24, 2014.   \n",
       "4                                        Posted September 8, 2013.   \n",
       "...                                 ...                        ...   \n",
       "59328                                              Posted July 10.   \n",
       "59329                                               Posted July 8.   \n",
       "59330  1 person found this review funny             Posted July 3.   \n",
       "59331                                              Posted July 20.   \n",
       "59332                                               Posted July 2.   \n",
       "\n",
       "      last_edited item_id                                          helpful  \\\n",
       "0                    1250                                   No ratings yet   \n",
       "1                   22200                                   No ratings yet   \n",
       "2                   43110                                   No ratings yet   \n",
       "3                  251610  15 of 20 people (75%) found this review helpful   \n",
       "4                  227300     0 of 1 people (0%) found this review helpful   \n",
       "...           ...     ...                                              ...   \n",
       "59328                  70                                   No ratings yet   \n",
       "59329              362890                                   No ratings yet   \n",
       "59330              273110    1 of 2 people (50%) found this review helpful   \n",
       "59331                 730                                   No ratings yet   \n",
       "59332                 440                                   No ratings yet   \n",
       "\n",
       "      recommend                                             review  \\\n",
       "0          True  Simple yet with great replayability. In my opi...   \n",
       "1          True               It's unique and worth a playthrough.   \n",
       "2          True  Great atmosphere. The gunplay can be a bit chu...   \n",
       "3          True  I know what you think when you see this title ...   \n",
       "4          True  For a simple (it's actually not all that simpl...   \n",
       "...         ...                                                ...   \n",
       "59328      True  a must have classic from steam definitely wort...   \n",
       "59329      True  this game is a perfect remake of the original ...   \n",
       "59330      True  had so much fun plaing this and collecting res...   \n",
       "59331      True                                                 :D   \n",
       "59332      True                                     so much fun :D   \n",
       "\n",
       "       sentiment_score  sentiment_analysis  \n",
       "0               0.8481                   2  \n",
       "1               0.2263                   2  \n",
       "2               0.9117                   2  \n",
       "3               0.9899                   2  \n",
       "4               0.9958                   2  \n",
       "...                ...                 ...  \n",
       "59328           0.5574                   2  \n",
       "59329           0.9786                   2  \n",
       "59330           0.7827                   2  \n",
       "59331           0.5106                   2  \n",
       "59332           0.8349                   2  \n",
       "\n",
       "[59333 rows x 11 columns]"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_user_reviews_expanded1['sentiment_analysis'] = sentiment_analysis\n",
    "df_user_reviews_expanded1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>user_url</th>\n",
       "      <th>funny</th>\n",
       "      <th>posted</th>\n",
       "      <th>last_edited</th>\n",
       "      <th>item_id</th>\n",
       "      <th>helpful</th>\n",
       "      <th>recommend</th>\n",
       "      <th>sentiment_analysis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted November 5, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>1250</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 15, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>22200</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>76561197970982479</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561197970...</td>\n",
       "      <td></td>\n",
       "      <td>Posted April 21, 2011.</td>\n",
       "      <td></td>\n",
       "      <td>43110</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted June 24, 2014.</td>\n",
       "      <td></td>\n",
       "      <td>251610</td>\n",
       "      <td>15 of 20 people (75%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>js41637</td>\n",
       "      <td>http://steamcommunity.com/id/js41637</td>\n",
       "      <td></td>\n",
       "      <td>Posted September 8, 2013.</td>\n",
       "      <td></td>\n",
       "      <td>227300</td>\n",
       "      <td>0 of 1 people (0%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59328</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 10.</td>\n",
       "      <td></td>\n",
       "      <td>70</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59329</th>\n",
       "      <td>76561198312638244</td>\n",
       "      <td>http://steamcommunity.com/profiles/76561198312...</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 8.</td>\n",
       "      <td></td>\n",
       "      <td>362890</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59330</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td>1 person found this review funny</td>\n",
       "      <td>Posted July 3.</td>\n",
       "      <td></td>\n",
       "      <td>273110</td>\n",
       "      <td>1 of 2 people (50%) found this review helpful</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59331</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 20.</td>\n",
       "      <td></td>\n",
       "      <td>730</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59332</th>\n",
       "      <td>LydiaMorley</td>\n",
       "      <td>http://steamcommunity.com/id/LydiaMorley</td>\n",
       "      <td></td>\n",
       "      <td>Posted July 2.</td>\n",
       "      <td></td>\n",
       "      <td>440</td>\n",
       "      <td>No ratings yet</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>59333 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id                                           user_url  \\\n",
       "0      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "1      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "2      76561197970982479  http://steamcommunity.com/profiles/76561197970...   \n",
       "3                js41637               http://steamcommunity.com/id/js41637   \n",
       "4                js41637               http://steamcommunity.com/id/js41637   \n",
       "...                  ...                                                ...   \n",
       "59328  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59329  76561198312638244  http://steamcommunity.com/profiles/76561198312...   \n",
       "59330        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59331        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "59332        LydiaMorley           http://steamcommunity.com/id/LydiaMorley   \n",
       "\n",
       "                                  funny                     posted  \\\n",
       "0                                         Posted November 5, 2011.   \n",
       "1                                            Posted July 15, 2011.   \n",
       "2                                           Posted April 21, 2011.   \n",
       "3                                            Posted June 24, 2014.   \n",
       "4                                        Posted September 8, 2013.   \n",
       "...                                 ...                        ...   \n",
       "59328                                              Posted July 10.   \n",
       "59329                                               Posted July 8.   \n",
       "59330  1 person found this review funny             Posted July 3.   \n",
       "59331                                              Posted July 20.   \n",
       "59332                                               Posted July 2.   \n",
       "\n",
       "      last_edited item_id                                          helpful  \\\n",
       "0                    1250                                   No ratings yet   \n",
       "1                   22200                                   No ratings yet   \n",
       "2                   43110                                   No ratings yet   \n",
       "3                  251610  15 of 20 people (75%) found this review helpful   \n",
       "4                  227300     0 of 1 people (0%) found this review helpful   \n",
       "...           ...     ...                                              ...   \n",
       "59328                  70                                   No ratings yet   \n",
       "59329              362890                                   No ratings yet   \n",
       "59330              273110    1 of 2 people (50%) found this review helpful   \n",
       "59331                 730                                   No ratings yet   \n",
       "59332                 440                                   No ratings yet   \n",
       "\n",
       "      recommend  sentiment_analysis  \n",
       "0          True                   2  \n",
       "1          True                   2  \n",
       "2          True                   2  \n",
       "3          True                   2  \n",
       "4          True                   2  \n",
       "...         ...                 ...  \n",
       "59328      True                   2  \n",
       "59329      True                   2  \n",
       "59330      True                   2  \n",
       "59331      True                   2  \n",
       "59332      True                   2  \n",
       "\n",
       "[59333 rows x 9 columns]"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columns_to_eliminate = ['review','sentiment_score']\n",
    "df_user_reviews_final = df_user_reviews_expanded1.drop(columns=columns_to_eliminate)\n",
    "df_user_reviews_final"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Export the final DataFrame to a CSV file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_user_reviews_final.to_csv('user_reviews_final.csv', index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}