aulas/cd_aula_25.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "082836c0-55fe-4030-a540-8f82ac4d262a",
   "metadata": {
    "id": "7daaa14a"
   },
   "source": [
    "# Sistemas de recomendação "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e86a34f-936d-4d57-812a-710017e084c9",
   "metadata": {},
   "source": [
    "<b>Filtragem colaborativa:</b> Predições (filtros) em relação aos interesses de um usuário através de coleções de preferências e interesses de vários outros usuários (colaborativo)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa230189-4f78-4386-a657-f0c25fc01cb3",
   "metadata": {},
   "source": [
    "<b>Filtragem baseada em conteúdo:</b> O algoritmo faz predições (filtros) baseado na semelhança entre itens que o usuário já manifestou interesse ou preferência."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3fc3312-18ee-4dad-b7f8-7dc8d579d7d1",
   "metadata": {},
   "source": [
    "<b>Sistema de recomendação híbrido:</b> combina os dois sistemas acima"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adac43b0-de58-4136-aaab-49348d1d98a2",
   "metadata": {},
   "source": [
    "<center>\n",
    "    <img width=\"250px\" src=\"../assets/recomendation.png\">\n",
    "</center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f18bf1f1-a244-497f-befe-c8fb9f97a69d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a063003e-e2b1-45b7-8d16-57ea8c5b2ba0",
   "metadata": {},
   "source": [
    "- http://guidetodatamining.com/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5b8c9f8-26bc-4784-8076-01cd70598336",
   "metadata": {},
   "source": [
    "## Exemplo de filtragem colaborativa (distância euclideana)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b68ff0c7-773f-418a-9b1b-fbbf8aabb395",
   "metadata": {},
   "outputs": [],
   "source": [
    "users = {\"Angelica\": {\"Blues Traveler\": 3.5, \"Broken Bells\": 2.0, \"Norah Jones\": 4.5, \"Phoenix\": 5.0, \"Slightly Stoopid\": 1.5, \"The Strokes\": 2.5, \"Vampire Weekend\": 2.0},\n",
    "         \"Bill\":{\"Blues Traveler\": 2.0, \"Broken Bells\": 3.5, \"Deadmau5\": 4.0, \"Phoenix\": 2.0, \"Slightly Stoopid\": 3.5, \"Vampire Weekend\": 3.0},\n",
    "         \"Chan\": {\"Blues Traveler\": 5.0, \"Broken Bells\": 1.0, \"Deadmau5\": 1.0, \"Norah Jones\": 3.0, \"Phoenix\": 5, \"Slightly Stoopid\": 1.0},\n",
    "         \"Dan\": {\"Blues Traveler\": 3.0, \"Broken Bells\": 4.0, \"Deadmau5\": 4.5, \"Phoenix\": 3.0, \"Slightly Stoopid\": 4.5, \"The Strokes\": 4.0, \"Vampire Weekend\": 2.0},\n",
    "         \"Hailey\": {\"Broken Bells\": 4.0, \"Deadmau5\": 1.0, \"Norah Jones\": 4.0, \"The Strokes\": 4.0, \"Vampire Weekend\": 1.0},\n",
    "         \"Jordyn\":  {\"Broken Bells\": 4.5, \"Deadmau5\": 4.0, \"Norah Jones\": 5.0, \"Phoenix\": 5.0, \"Slightly Stoopid\": 4.5, \"The Strokes\": 4.0, \"Vampire Weekend\": 4.0},\n",
    "         \"Sam\": {\"Blues Traveler\": 5.0, \"Broken Bells\": 2.0, \"Norah Jones\": 3.0, \"Phoenix\": 5.0, \"Slightly Stoopid\": 4.0, \"The Strokes\": 5.0},\n",
    "         \"Veronica\": {\"Blues Traveler\": 3.0, \"Norah Jones\": 5.0, \"Phoenix\": 4.0, \"Slightly Stoopid\": 2.5, \"The Strokes\": 3.0}\n",
    "        }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "abbf0b72-6695-42ba-9258-433e0803ddac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Norah Jones', 'Phoenix', 'Slightly Stoopid', 'Blues Traveler', 'Deadmau5', 'Vampire Weekend', 'Broken Bells', 'The Strokes']\n"
     ]
    }
   ],
   "source": [
    "# get unique musics list\n",
    "musics = set()\n",
    "for user_key in users:\n",
    "    for music_key in users[user_key]:\n",
    "        musics.add(music_key)\n",
    "musics = list(musics)\n",
    "print(musics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6865f0e8-e741-4641-b40e-8097c1b8aca6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[4.5, 5.0, 1.5, 3.5, nan, 2.0, 2.0, 2.5], [nan, 2.0, 3.5, 2.0, 4.0, 3.0, 3.5, nan], [3.0, 5, 1.0, 5.0, 1.0, nan, 1.0, nan], [nan, 3.0, 4.5, 3.0, 4.5, 2.0, 4.0, 4.0], [4.0, nan, nan, nan, 1.0, 1.0, 4.0, 4.0], [5.0, 5.0, 4.5, nan, 4.0, 4.0, 4.5, 4.0], [3.0, 5.0, 4.0, 5.0, nan, nan, 2.0, 5.0], [5.0, 4.0, 2.5, 3.0, nan, nan, nan, 3.0]]\n"
     ]
    }
   ],
   "source": [
    "matrix = []\n",
    "for user_key in users:\n",
    "    row = []\n",
    "    for music in musics:\n",
    "        if music in users[user_key]:\n",
    "            row.append(users[user_key][music])\n",
    "        else:\n",
    "            row.append(np.nan)\n",
    "    matrix.append(row)\n",
    "print(matrix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "092248de-74f7-42ca-862d-5d0c96969fa7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Norah Jones</th>\n",
       "      <th>Phoenix</th>\n",
       "      <th>Slightly Stoopid</th>\n",
       "      <th>Blues Traveler</th>\n",
       "      <th>Deadmau5</th>\n",
       "      <th>Vampire Weekend</th>\n",
       "      <th>Broken Bells</th>\n",
       "      <th>The Strokes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>User</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Angelica</th>\n",
       "      <td>4.5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.5</td>\n",
       "      <td>3.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bill</th>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Chan</th>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dan</th>\n",
       "      <td>NaN</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hailey</th>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jordyn</th>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sam</th>\n",
       "      <td>3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Veronica</th>\n",
       "      <td>5.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Norah Jones  Phoenix  Slightly Stoopid  Blues Traveler  Deadmau5  \\\n",
       "User                                                                         \n",
       "Angelica          4.5      5.0               1.5             3.5       NaN   \n",
       "Bill              NaN      2.0               3.5             2.0       4.0   \n",
       "Chan              3.0      5.0               1.0             5.0       1.0   \n",
       "Dan               NaN      3.0               4.5             3.0       4.5   \n",
       "Hailey            4.0      NaN               NaN             NaN       1.0   \n",
       "Jordyn            5.0      5.0               4.5             NaN       4.0   \n",
       "Sam               3.0      5.0               4.0             5.0       NaN   \n",
       "Veronica          5.0      4.0               2.5             3.0       NaN   \n",
       "\n",
       "          Vampire Weekend  Broken Bells  The Strokes  \n",
       "User                                                  \n",
       "Angelica              2.0           2.0          2.5  \n",
       "Bill                  3.0           3.5          NaN  \n",
       "Chan                  NaN           1.0          NaN  \n",
       "Dan                   2.0           4.0          4.0  \n",
       "Hailey                1.0           4.0          4.0  \n",
       "Jordyn                4.0           4.5          4.0  \n",
       "Sam                   NaN           2.0          5.0  \n",
       "Veronica              NaN           NaN          3.0  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(matrix, columns=musics)\n",
    "df['User'] = users.keys()\n",
    "df.set_index('User', inplace=True)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c58432ea-b691-4a8d-a253-3f4b2c218bcb",
   "metadata": {},
   "source": [
    "Qual a distância euclideana entre Hailey e Veronica?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b5bdf632-6cd3-40aa-bbab-7278c1701f16",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Norah Jones</th>\n",
       "      <th>The Strokes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>User</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Hailey</th>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Veronica</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Norah Jones  The Strokes\n",
       "User                              \n",
       "Hailey            4.0          4.0\n",
       "Veronica          5.0          3.0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[['Hailey', 'Veronica']].dropna(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b31a3fc4-c319-4d44-82ab-ac00538af0f8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Resposta: 1.4142135623730951\n"
     ]
    }
   ],
   "source": [
    "def euclidean_distance(v1, v2):\n",
    "    square_sum = 0\n",
    "    for i in range(len(v1)):\n",
    "        square_sum += (v1[i] - v2[i]) ** 2\n",
    "    return square_sum ** 0.5\n",
    "\n",
    "print(\"Resposta:\", euclidean_distance([4, 4], [3,5]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55c2884b-ec1c-4afd-982f-e98d61a76fc4",
   "metadata": {},
   "source": [
    "Qual a distância euclideana entre Hailey e Jordyn?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d6dda294-5d68-4e9d-99ab-c337afd6f5c9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Norah Jones</th>\n",
       "      <th>Deadmau5</th>\n",
       "      <th>Vampire Weekend</th>\n",
       "      <th>Broken Bells</th>\n",
       "      <th>The Strokes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>User</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Hailey</th>\n",
       "      <td>4.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jordyn</th>\n",
       "      <td>5.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Norah Jones  Deadmau5  Vampire Weekend  Broken Bells  The Strokes\n",
       "User                                                                     \n",
       "Hailey          4.0       1.0              1.0           4.0          4.0\n",
       "Jordyn          5.0       4.0              4.0           4.5          4.0"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[['Hailey', 'Jordyn']].dropna(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2bda1be0-ebdf-4c7b-a43d-26d2cca31d35",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Resposta: 4.387482193696061\n"
     ]
    }
   ],
   "source": [
    "print(\"Resposta:\", euclidean_distance([4,1,1,4,4], [4.5,4,4,4,5]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5537af9f-a417-46f8-9eff-6de0f17ee45a",
   "metadata": {},
   "source": [
    "Quem tem gosto mais próximo de Hailey? Jordyn ou Veronica?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e82490e6-2413-4231-9a7d-1b422a290b00",
   "metadata": {},
   "source": [
    "<b>Resposta:</b> Veronica."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1804cf52-ec92-4df0-83df-1de0409e3ac9",
   "metadata": {},
   "source": [
    "## Outras distâncias"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d46ce6ca-f823-4565-88e7-f4ac24e8b053",
   "metadata": {},
   "source": [
    "- https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.cdist.html#scipy.spatial.distance.cdist"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f968d81-15e8-43ac-a17f-29929c01d5e2",
   "metadata": {},
   "source": [
    "### Manhattan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2584e118-2990-4be6-a877-99f25e2dc306",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[2.]]\n",
      "[[7.5]]\n"
     ]
    }
   ],
   "source": [
    "from scipy.spatial.distance import cdist\n",
    "\n",
    "print(cdist([[4, 4]], [[3,5]], metric=\"cityblock\"))\n",
    "print(cdist([[4,1,1,4,4]], [[4.5,4,4,4,5]], metric=\"cityblock\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04d4f540-50bd-4b60-9f0d-ad15e2c4144b",
   "metadata": {},
   "source": [
    "### Minkowski p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "54a27f60-63f7-4715-93a1-384ac7755c9b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1.18920712]]\n",
      "[[3.5734567]]\n"
     ]
    }
   ],
   "source": [
    "# p = 1, manhattan\n",
    "# p = 2, euclideana\n",
    "# p = 3, chebyshev\n",
    "\n",
    "print(cdist([[4, 4]], [[3,5]], metric=\"minkowski\", p=4))\n",
    "print(cdist([[4,1,1,4,4]], [[4.5,4,4,4,5]], metric=\"minkowski\", p=4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57398903-e265-4000-931b-35309ed17756",
   "metadata": {},
   "source": [
    "### Hamming"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "81244e90-d9bf-412b-915b-ee381f3fee68",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1.]]\n",
      "[[0.8]]\n"
     ]
    }
   ],
   "source": [
    "print(cdist([[4, 4]], [[3,5]], metric=\"hamming\"))\n",
    "print(cdist([[4,1,1,4,4]], [[4.5,4,4,4,5]], metric=\"hamming\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d09e60b1-e8ad-4a22-a3a7-0eb708abf2cc",
   "metadata": {},
   "source": [
    "Observe que, se considerarmos a distância de Hamming, Jordyn tem gosto mais semelhante a Hailey do que Veronica."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c68b795-a7b4-4d20-b977-e7d4a93cb5d3",
   "metadata": {},
   "source": [
    "## Medida de similaridade"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7e15e111-32cd-4d11-abc3-5529c9541037",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.029857499854668124\n",
      "0.0920074878366165\n"
     ]
    }
   ],
   "source": [
    "from scipy.spatial.distance import cosine\n",
    "\n",
    "print(cosine([4, 4], [3,5]))\n",
    "print(cosine([4,1,1,4,4], [4.5,4,4,4,5]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d0b031c-797b-4863-8fc7-aa0644c743ca",
   "metadata": {},
   "source": [
    "Distâncias, no contexto de espaços métricos (matemática), satisfazem as propriedades de reflexividade e de desigualdade triangular. Outras funções comparativas que não satisfazem as propriedades de uma distância (métrica) são denominadas medidas de similaridade. Há uma confusão natural entre os dois termos em áreas fora da matemática."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36fba458-3a0a-4425-bc1b-b6480d68b4c2",
   "metadata": {},
   "source": [
    "### Exemplo de filtragem baseada em conteúdo"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fc58005-8f58-400f-b464-f1c440a3b446",
   "metadata": {},
   "source": [
    "Angelica deu nota 5 para Phoenix. Assumindo por hipótese que items parecidos com Phoenix devem também ser de interesse de Angelica, qual item é mais próximo a Phoenix?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ea66c740-5b7e-4e0c-a31d-647c5bbf5901",
   "metadata": {},
   "outputs": [],
   "source": [
    "dft = df.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "d976ade8-9a24-4fc3-9dcc-5bde491694ac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distância entre Phoenix e Norah Jones: 3.0413812651491097\n",
      "Distância entre Phoenix e Slightly Stoopid: 6.020797289396148\n",
      "Distância entre Phoenix e Blues Traveler: 1.8027756377319946\n",
      "Distância entre Phoenix e Deadmau5: 4.8218253804964775\n",
      "Distância entre Phoenix e Vampire Weekend: 3.4641016151377544\n",
      "Distância entre Phoenix e Broken Bells: 6.123724356957945\n",
      "Distância entre Phoenix e The Strokes: 3.0413812651491097\n"
     ]
    }
   ],
   "source": [
    "for music in dft.index:\n",
    "    if music != 'Phoenix':\n",
    "        v1, v2 = dft.loc[['Phoenix', music]].dropna(axis=1).values\n",
    "        d = euclidean_distance(v1, v2)\n",
    "        print(f\"Distância entre Phoenix e {music}:\", d)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0849c093-53c8-487b-a10e-fc4971dc21cf",
   "metadata": {},
   "source": [
    "Possivelmente Angelica irá gostar mais de Blues Traveler."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "015d9c01-5a2b-40ba-a27f-f8b4148593dc",
   "metadata": {},
   "source": [
    "Aqui, temos que ter atenção, porque Angelica já ouviu Blues traveler e avaliou com 3.5. Sistemas de recomendação geralmente são utilizados para recomendar novos conteúdos."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95797db6-1b04-4088-b681-3cfea3293eb4",
   "metadata": {},
   "source": [
    "## Apriori"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd9e40d5-a59b-456c-a832-75f654c9a16c",
   "metadata": {},
   "source": [
    "Dataset:\n",
    "- http://archive.ics.uci.edu/dataset/352/online+retail\n",
    "\n",
    "Referência:\n",
    "- https://www.geeksforgeeks.org/implementing-apriori-algorithm-in-python/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "755bd010-0066-4aa1-a8fd-df71c2c5fbf8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlxtend.frequent_patterns import apriori, association_rules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "7ed43348-01d7-45a2-be9a-52e6999b6d62",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>InvoiceNo</th>\n",
       "      <th>StockCode</th>\n",
       "      <th>Description</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>InvoiceDate</th>\n",
       "      <th>UnitPrice</th>\n",
       "      <th>CustomerID</th>\n",
       "      <th>Country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>536365</td>\n",
       "      <td>85123A</td>\n",
       "      <td>WHITE HANGING HEART T-LIGHT HOLDER</td>\n",
       "      <td>6</td>\n",
       "      <td>1/12/2010 08:26</td>\n",
       "      <td>2,55</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>536365</td>\n",
       "      <td>71053</td>\n",
       "      <td>WHITE METAL LANTERN</td>\n",
       "      <td>6</td>\n",
       "      <td>1/12/2010 08:26</td>\n",
       "      <td>3,39</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>536365</td>\n",
       "      <td>84406B</td>\n",
       "      <td>CREAM CUPID HEARTS COAT HANGER</td>\n",
       "      <td>8</td>\n",
       "      <td>1/12/2010 08:26</td>\n",
       "      <td>2,75</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>536365</td>\n",
       "      <td>84029G</td>\n",
       "      <td>KNITTED UNION FLAG HOT WATER BOTTLE</td>\n",
       "      <td>6</td>\n",
       "      <td>1/12/2010 08:26</td>\n",
       "      <td>3,39</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>536365</td>\n",
       "      <td>84029E</td>\n",
       "      <td>RED WOOLLY HOTTIE WHITE HEART.</td>\n",
       "      <td>6</td>\n",
       "      <td>1/12/2010 08:26</td>\n",
       "      <td>3,39</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  InvoiceNo StockCode                          Description  Quantity  \\\n",
       "0    536365    85123A   WHITE HANGING HEART T-LIGHT HOLDER         6   \n",
       "1    536365     71053                  WHITE METAL LANTERN         6   \n",
       "2    536365    84406B       CREAM CUPID HEARTS COAT HANGER         8   \n",
       "3    536365    84029G  KNITTED UNION FLAG HOT WATER BOTTLE         6   \n",
       "4    536365    84029E       RED WOOLLY HOTTIE WHITE HEART.         6   \n",
       "\n",
       "       InvoiceDate UnitPrice  CustomerID         Country  \n",
       "0  1/12/2010 08:26      2,55     17850.0  United Kingdom  \n",
       "1  1/12/2010 08:26      3,39     17850.0  United Kingdom  \n",
       "2  1/12/2010 08:26      2,75     17850.0  United Kingdom  \n",
       "3  1/12/2010 08:26      3,39     17850.0  United Kingdom  \n",
       "4  1/12/2010 08:26      3,39     17850.0  United Kingdom  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv('../datasets/online_retail/online_retail.csv')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "819b58ca-0bad-4d71-ac51-da0340ce6dbc",
   "metadata": {},
   "outputs": [],
   "source": [
    "data['Description'] = data['Description'].str.strip() # clear spaces\n",
    "data.dropna(axis = 0, subset =['InvoiceNo'], inplace = True) # dropna\n",
    "data['InvoiceNo'] = data['InvoiceNo'].astype('str') # Inv to str\n",
    "data = data[~data['InvoiceNo'].str.contains('C')] # not C"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "e03c1cb9-ff9f-4ff4-b896-19c0795ec555",
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_basket(country):\n",
    "    return data[data['Country'] == country].groupby(\n",
    "        ['InvoiceNo', 'Description'])['Quantity'].sum().unstack().reset_index().fillna(0).set_index('InvoiceNo')\n",
    "\n",
    "basket_France = create_basket(\"France\")\n",
    "basket_UK = create_basket(\"United Kingdom\")\n",
    "basket_Por = create_basket(\"Portugal\")\n",
    "basket_Sweden = create_basket(\"Sweden\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "61aeccd8-b63e-493c-aa85-5b788377d412",
   "metadata": {},
   "outputs": [],
   "source": [
    "def hot_encode(x):\n",
    "    if(x<= 0):\n",
    "        return False\n",
    "    if(x>= 1):\n",
    "        return True\n",
    "\n",
    "basket_France = basket_France.applymap(hot_encode)\n",
    "basket_UK = basket_UK.applymap(hot_encode)\n",
    "basket_Por = basket_Por.applymap(hot_encode)\n",
    "basket_Sweden = basket_Sweden.applymap(hot_encode)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "b17f6065-7e5f-41e9-b872-51e27b80d47e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>antecedents</th>\n",
       "      <th>consequents</th>\n",
       "      <th>antecedent support</th>\n",
       "      <th>consequent support</th>\n",
       "      <th>support</th>\n",
       "      <th>confidence</th>\n",
       "      <th>lift</th>\n",
       "      <th>leverage</th>\n",
       "      <th>conviction</th>\n",
       "      <th>zhangs_metric</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>(JUMBO BAG WOODLAND ANIMALS)</td>\n",
       "      <td>(POSTAGE)</td>\n",
       "      <td>0.076531</td>\n",
       "      <td>0.765306</td>\n",
       "      <td>0.076531</td>\n",
       "      <td>1.000</td>\n",
       "      <td>1.306667</td>\n",
       "      <td>0.017961</td>\n",
       "      <td>inf</td>\n",
       "      <td>0.254144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>258</th>\n",
       "      <td>(RED TOADSTOOL LED NIGHT LIGHT, PLASTERS IN TI...</td>\n",
       "      <td>(POSTAGE)</td>\n",
       "      <td>0.051020</td>\n",
       "      <td>0.765306</td>\n",
       "      <td>0.051020</td>\n",
       "      <td>1.000</td>\n",
       "      <td>1.306667</td>\n",
       "      <td>0.011974</td>\n",
       "      <td>inf</td>\n",
       "      <td>0.247312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>271</th>\n",
       "      <td>(RED TOADSTOOL LED NIGHT LIGHT, PLASTERS IN TI...</td>\n",
       "      <td>(POSTAGE)</td>\n",
       "      <td>0.053571</td>\n",
       "      <td>0.765306</td>\n",
       "      <td>0.053571</td>\n",
       "      <td>1.000</td>\n",
       "      <td>1.306667</td>\n",
       "      <td>0.012573</td>\n",
       "      <td>inf</td>\n",
       "      <td>0.247978</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>301</th>\n",
       "      <td>(SET/20 RED RETROSPOT PAPER NAPKINS, SET/6 RED...</td>\n",
       "      <td>(SET/6 RED SPOTTY PAPER PLATES)</td>\n",
       "      <td>0.102041</td>\n",
       "      <td>0.127551</td>\n",
       "      <td>0.099490</td>\n",
       "      <td>0.975</td>\n",
       "      <td>7.644000</td>\n",
       "      <td>0.086474</td>\n",
       "      <td>34.897959</td>\n",
       "      <td>0.967949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>300</th>\n",
       "      <td>(SET/20 RED RETROSPOT PAPER NAPKINS, SET/6 RED...</td>\n",
       "      <td>(SET/6 RED SPOTTY PAPER CUPS)</td>\n",
       "      <td>0.102041</td>\n",
       "      <td>0.137755</td>\n",
       "      <td>0.099490</td>\n",
       "      <td>0.975</td>\n",
       "      <td>7.077778</td>\n",
       "      <td>0.085433</td>\n",
       "      <td>34.489796</td>\n",
       "      <td>0.956294</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           antecedents  \\\n",
       "44                        (JUMBO BAG WOODLAND ANIMALS)   \n",
       "258  (RED TOADSTOOL LED NIGHT LIGHT, PLASTERS IN TI...   \n",
       "271  (RED TOADSTOOL LED NIGHT LIGHT, PLASTERS IN TI...   \n",
       "301  (SET/20 RED RETROSPOT PAPER NAPKINS, SET/6 RED...   \n",
       "300  (SET/20 RED RETROSPOT PAPER NAPKINS, SET/6 RED...   \n",
       "\n",
       "                         consequents  antecedent support  consequent support  \\\n",
       "44                         (POSTAGE)            0.076531            0.765306   \n",
       "258                        (POSTAGE)            0.051020            0.765306   \n",
       "271                        (POSTAGE)            0.053571            0.765306   \n",
       "301  (SET/6 RED SPOTTY PAPER PLATES)            0.102041            0.127551   \n",
       "300    (SET/6 RED SPOTTY PAPER CUPS)            0.102041            0.137755   \n",
       "\n",
       "      support  confidence      lift  leverage  conviction  zhangs_metric  \n",
       "44   0.076531       1.000  1.306667  0.017961         inf       0.254144  \n",
       "258  0.051020       1.000  1.306667  0.011974         inf       0.247312  \n",
       "271  0.053571       1.000  1.306667  0.012573         inf       0.247978  \n",
       "301  0.099490       0.975  7.644000  0.086474   34.897959       0.967949  \n",
       "300  0.099490       0.975  7.077778  0.085433   34.489796       0.956294  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Building the model\n",
    "frq_items = apriori(\n",
    "    basket_France, \n",
    "    min_support = 0.05, # min percent of a itemset\n",
    "    use_colnames = True)\n",
    "  \n",
    "# Collecting the inferred rules in a dataframe\n",
    "rules = association_rules(frq_items, metric =\"lift\", min_threshold = 1)\n",
    "rules = rules.sort_values(['confidence', 'lift'], ascending =[False, False])\n",
    "rules.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "093f0a0a-b666-4c99-992c-6b8f14bec835",
   "metadata": {},
   "source": [
    "Definições\n",
    "- https://s2.smu.edu/~mhahsler/crawler_test/michael.hahsler.net/research/association_rules/measures.html\n",
    "\n",
    "Interpretações e um exemplo simples\n",
    "- https://codinginfinite.com/apriori-algorithm-numerical-example/\n",
    "- https://medium.com/@bernardo.costa/uma-introdu%C3%A7%C3%A3o-ao-algoritmo-apriori-60b11293aa5a"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "426a4885-10ef-4a12-8105-b6f6a739c83a",
   "metadata": {},
   "source": [
    "Cada regra de associação é definido com um antecedente (antecedents) e um consequente (consequents). Isso quer dizer que numa cesta de compras, dado que os antecedentes estejam, há chances do consequente estar. \n",
    "\n",
    "ANTECEDENTE -> CONSEQUENTE\n",
    "\n",
    "Podemos citar alguns exemplos:\n",
    "- Pão -> Margarina\n",
    "- Café, Açucar -> Filtro"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e2127b6-1675-430a-adac-08870c958b19",
   "metadata": {},
   "source": [
    "<b>Support:</b> O suporte de um conjunto X (que pode ser antecedents ou consequentes) é a proporção do conjunto de dados T que contém X. Isto é: \n",
    "$$Supp(X) = \\frac{|\\{t ∈ T : X ⊆ t\\}|}{|T|}$$\n",
    "\n",
    "<b>Confidence:</b> A confiança de uma regra $A \\rightarrow B$ é a  probabilidade de B numa transação dado que na transação também está A. Isto é: $$Conf(A \\rightarrow B) = \\frac{Supp(A ∪ B)}{Supp(A)}$$\n",
    "\n",
    "<b>Lift:</b> O lift de uma regra $A \\rightarrow B$ é definido por: $$Lift(A \\rightarrow B) = \\frac{Conf(A \\rightarrow B)}{Supp(B)}$$\n",
    "\n",
    "<b>Leverage</b> Leverage é a diferença entre a frequência observada de A e B e a frequencia que deveria ser esperada se A e B forem independentes. $$Lev(A \\rightarrow B) = Supp(A ∪ B) - Supp(A) * Supp(B)$$\n",
    "\n",
    "<b>Conviction</b> A conviction de uma regra $A \\rightarrow B$ é definido por $$Conv(A \\rightarrow B) = \\frac{(1 - Supp(B))}{(1 - Conf(A \\rightarrow B))}$$"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}