amazon-personalize-workshop-part-2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Amazon Personalize Workshop Part 2 - View Campaign and Interactions\n",
    "> In the first part, you successfully built and deployed a recommendation model using deep learning with Amazon Personalize. This notebook will expand on that and will walk you through adding the ability to react to real time behavior of users. If their intent changes while browsing a movie, you will see revised recommendations based on that behavior. It will also showcase demo code for simulating user behavior selecting movies before the recommendations are returned.\n",
    "- toc: true\n",
    "- badges: true\n",
    "- comments: true\n",
    "- categories: [amazonpersonalize, movie, realtime]\n",
    "- image: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we start with just importing libraries that we need to interact with Personalize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Imports\n",
    "import boto3\n",
    "import json\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "import uuid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The line below will retrieve your shared variables from the first notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "%store -r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup and Config\n",
    "# Recommendations from Event data\n",
    "personalize = boto3.client('personalize')\n",
    "personalize_runtime = boto3.client('personalize-runtime')\n",
    "\n",
    "# Establish a connection to Personalize's Event Streaming\n",
    "personalize_events = boto3.client(service_name='personalize-events')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating an Event Tracker\n",
    "\n",
    "Before your recommendation system can respond to real time events you will need an event tracker, the code below will generate one and can be used going forward with this lab. Feel free to name it something more clever."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arn:aws:personalize:us-east-1:284105231590:event-tracker/a1b7dce4\n",
      "f252168d-a73d-467a-b2e7-66071f5d6d78\n"
     ]
    }
   ],
   "source": [
    "response = personalize.create_event_tracker(\n",
    "    name='MovieClickTracker',\n",
    "    datasetGroupArn=dataset_group_arn\n",
    ")\n",
    "print(response['eventTrackerArn'])\n",
    "print(response['trackingId'])\n",
    "TRACKING_ID = response['trackingId']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "event_tracker_arn = response['eventTrackerArn']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configuring Source Data\n",
    "\n",
    "Above you'll see your tracking ID and this has been assigned to a variable so no further action is needed by you. The lines below are going to setup the data used for recommendations so you can render the list of movies later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First load items into memory\n",
    "items = pd.read_csv('./ml-100k/u.item', sep='|', usecols=[0,1], encoding='latin-1', names=['ITEM_ID', 'TITLE'], index_col='ITEM_ID')\n",
    "\n",
    "def get_movie_title(movie_id):\n",
    "    \"\"\"\n",
    "    Takes in an ID, returns a title\n",
    "    \"\"\"\n",
    "    movie_id = int(movie_id)-1\n",
    "    return items.loc[movie_id]['TITLE']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting Recommendations\n",
    "\n",
    "First we will render the recommendations again from the previous notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>OriginalRecs</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Usual Suspects, The (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Raiders of the Lost Ark (1981)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Silence of the Lambs, The (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Casablanca (1942)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Fugitive, The (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Shawshank Redemption, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Amadeus (1984)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Braveheart (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Truth About Cats &amp; Dogs, The (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Sling Blade (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>One Flew Over the Cuckoo's Nest (1975)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Aladdin (1992)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Back to the Future (1985)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Time to Kill, A (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Seven (Se7en) (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>E.T. the Extra-Terrestrial (1982)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Princess Bride, The (1987)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Sleepers (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Long Kiss Goodnight, The (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Monty Python and the Holy Grail (1974)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Willy Wonka and the Chocolate Factory (1971)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>GoodFellas (1990)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    OriginalRecs\n",
       "0                        Schindler's List (1993)\n",
       "1                            Pulp Fiction (1994)\n",
       "2                     Usual Suspects, The (1995)\n",
       "3                 Raiders of the Lost Ark (1981)\n",
       "4               Silence of the Lambs, The (1991)\n",
       "5                              Casablanca (1942)\n",
       "6                           Fugitive, The (1993)\n",
       "7               Shawshank Redemption, The (1994)\n",
       "8                                 Amadeus (1984)\n",
       "9                              Braveheart (1995)\n",
       "10           Truth About Cats & Dogs, The (1996)\n",
       "11                            Sling Blade (1996)\n",
       "12        One Flew Over the Cuckoo's Nest (1975)\n",
       "13                                Aladdin (1992)\n",
       "14                     Back to the Future (1985)\n",
       "15                        Time to Kill, A (1996)\n",
       "16                          Seven (Se7en) (1995)\n",
       "17             E.T. the Extra-Terrestrial (1982)\n",
       "18                    Princess Bride, The (1987)\n",
       "19                         Godfather, The (1972)\n",
       "20                               Sleepers (1996)\n",
       "21               Long Kiss Goodnight, The (1996)\n",
       "22        Monty Python and the Holy Grail (1974)\n",
       "23  Willy Wonka and the Chocolate Factory (1971)\n",
       "24                             GoodFellas (1990)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simulating User Behavior\n",
    "\n",
    "The lines below provide a code sample that simulates a user interacting with a particular item, you will then get recommendations that differ from those when you started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "session_dict = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def send_movie_click(USER_ID, ITEM_ID):\n",
    "    \"\"\"\n",
    "    Simulates a click as an event\n",
    "    to send an event to Amazon Personalize's Event Tracker\n",
    "    \"\"\"\n",
    "    # Configure Session\n",
    "    try:\n",
    "        session_ID = session_dict[USER_ID]\n",
    "    except:\n",
    "        session_dict[USER_ID] = str(uuid.uuid1())\n",
    "        session_ID = session_dict[USER_ID]\n",
    "        \n",
    "    # Configure Properties:\n",
    "    event = {\n",
    "    \"itemId\": str(ITEM_ID),\n",
    "    }\n",
    "    event_json = json.dumps(event)\n",
    "        \n",
    "    # Make Call\n",
    "    personalize_events.put_events(\n",
    "    trackingId = TRACKING_ID,\n",
    "    userId= USER_ID,\n",
    "    sessionId = session_ID,\n",
    "    eventList = [{\n",
    "        'sentAt': int(time.time()),\n",
    "        'eventType': 'EVENT_TYPE',\n",
    "        'properties': event_json\n",
    "        }]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Immediately below this line will update the tracker as if the user has clicked a particular title.\n",
    "\n",
    "\n",
    "If the table generated by the cells below does not shift the recommendations simply try another random 3 digit number in the cell above and run both cells again. You'll see a third column generated of recommendations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pick a movie, we will use ID 270 or Gattaca\n",
    "movie_to_click = 270\n",
    "movie_title_clicked = get_movie_title(movie_to_click)\n",
    "send_movie_click(USER_ID=str(user_id), ITEM_ID=movie_to_click)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After executing this block you will see the alterations in the recommendations now that you have event tracking enabled and that you have sent the events to the service."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recommendations for user:  839\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>OriginalRecs</th>\n",
       "      <th>Full Monty, The (1997)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "      <td>Professional, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Usual Suspects, The (1995)</td>\n",
       "      <td>Amistad (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Raiders of the Lost Ark (1981)</td>\n",
       "      <td>Chasing Amy (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Silence of the Lambs, The (1991)</td>\n",
       "      <td>Muppet Treasure Island (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Casablanca (1942)</td>\n",
       "      <td>Empire Strikes Back, The (1980)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Fugitive, The (1993)</td>\n",
       "      <td>River Wild, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Shawshank Redemption, The (1994)</td>\n",
       "      <td>Midnight in the Garden of Good and Evil (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Amadeus (1984)</td>\n",
       "      <td>Last of the Mohicans, The (1992)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Braveheart (1995)</td>\n",
       "      <td>Marvin's Room (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Truth About Cats &amp; Dogs, The (1996)</td>\n",
       "      <td>Seven (Se7en) (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Sling Blade (1996)</td>\n",
       "      <td>Spitfire Grill, The (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>One Flew Over the Cuckoo's Nest (1975)</td>\n",
       "      <td>Citizen Ruth (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Aladdin (1992)</td>\n",
       "      <td>Hoodlum (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Back to the Future (1985)</td>\n",
       "      <td>Henry V (1989)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Time to Kill, A (1996)</td>\n",
       "      <td>I.Q. (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Seven (Se7en) (1995)</td>\n",
       "      <td>Dr. Strangelove or: How I Learned to Stop Worr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>E.T. the Extra-Terrestrial (1982)</td>\n",
       "      <td>Starship Troopers (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Princess Bride, The (1987)</td>\n",
       "      <td>Santa Clause, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "      <td>Ref, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Sleepers (1996)</td>\n",
       "      <td>Apocalypse Now (1979)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Long Kiss Goodnight, The (1996)</td>\n",
       "      <td>Star Trek: First Contact (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Monty Python and the Holy Grail (1974)</td>\n",
       "      <td>Free Willy (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Willy Wonka and the Chocolate Factory (1971)</td>\n",
       "      <td>Princess Bride, The (1987)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>GoodFellas (1990)</td>\n",
       "      <td>Shanghai Triad (Yao a yao yao dao waipo qiao) ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    OriginalRecs  \\\n",
       "0                        Schindler's List (1993)   \n",
       "1                            Pulp Fiction (1994)   \n",
       "2                     Usual Suspects, The (1995)   \n",
       "3                 Raiders of the Lost Ark (1981)   \n",
       "4               Silence of the Lambs, The (1991)   \n",
       "5                              Casablanca (1942)   \n",
       "6                           Fugitive, The (1993)   \n",
       "7               Shawshank Redemption, The (1994)   \n",
       "8                                 Amadeus (1984)   \n",
       "9                              Braveheart (1995)   \n",
       "10           Truth About Cats & Dogs, The (1996)   \n",
       "11                            Sling Blade (1996)   \n",
       "12        One Flew Over the Cuckoo's Nest (1975)   \n",
       "13                                Aladdin (1992)   \n",
       "14                     Back to the Future (1985)   \n",
       "15                        Time to Kill, A (1996)   \n",
       "16                          Seven (Se7en) (1995)   \n",
       "17             E.T. the Extra-Terrestrial (1982)   \n",
       "18                    Princess Bride, The (1987)   \n",
       "19                         Godfather, The (1972)   \n",
       "20                               Sleepers (1996)   \n",
       "21               Long Kiss Goodnight, The (1996)   \n",
       "22        Monty Python and the Holy Grail (1974)   \n",
       "23  Willy Wonka and the Chocolate Factory (1971)   \n",
       "24                             GoodFellas (1990)   \n",
       "\n",
       "                               Full Monty, The (1997)  \n",
       "0                            Professional, The (1994)  \n",
       "1                                             unknown  \n",
       "2                                      Amistad (1997)  \n",
       "3                                  Chasing Amy (1997)  \n",
       "4                       Muppet Treasure Island (1996)  \n",
       "5                     Empire Strikes Back, The (1980)  \n",
       "6                              River Wild, The (1994)  \n",
       "7      Midnight in the Garden of Good and Evil (1997)  \n",
       "8                    Last of the Mohicans, The (1992)  \n",
       "9                                Marvin's Room (1996)  \n",
       "10                               Seven (Se7en) (1995)  \n",
       "11                         Spitfire Grill, The (1996)  \n",
       "12                                Citizen Ruth (1996)  \n",
       "13                                     Hoodlum (1997)  \n",
       "14                                     Henry V (1989)  \n",
       "15                                        I.Q. (1994)  \n",
       "16  Dr. Strangelove or: How I Learned to Stop Worr...  \n",
       "17                           Starship Troopers (1997)  \n",
       "18                           Santa Clause, The (1994)  \n",
       "19                                    Ref, The (1994)  \n",
       "20                              Apocalypse Now (1979)  \n",
       "21                    Star Trek: First Contact (1996)  \n",
       "22                                  Free Willy (1993)  \n",
       "23                         Princess Bride, The (1987)  \n",
       "24  Shanghai Triad (Yao a yao yao dao waipo qiao) ...  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "    campaignArn = campaign_arn,\n",
    "    userId = str(user_id),\n",
    ")\n",
    "\n",
    "print(\"Recommendations for user: \", user_id)\n",
    "\n",
    "item_list = get_recommendations_response['itemList']\n",
    "\n",
    "recommendation_list = []\n",
    "\n",
    "for item in item_list:\n",
    "    title = get_movie_title(item['itemId'])\n",
    "    recommendation_list.append(title)\n",
    "    \n",
    "new_rec_DF = pd.DataFrame(recommendation_list, columns = [movie_title_clicked])\n",
    "\n",
    "recommendations_df = recommendations_df.join(new_rec_DF)\n",
    "recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "You can see now that recommendations are altered by changing the movie that a user interacts with, this system can be modified to any application where users are interacting with a collection of items. These tools are available at any time to pull down and start exploring what is possible with the data you have.\n",
    "\n",
    "Execute the cell below to store values needed for the cleanup notebook.\n",
    "\n",
    "Finally when you are ready to remove the items from your account, open the `Cleanup.ipynb` notebook and execute the steps there.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stored 'event_tracker_arn' (str)\n"
     ]
    }
   ],
   "source": [
    "%store event_tracker_arn"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_python3",
   "language": "python",
   "name": "conda_python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}