unit_8_py/lab-unsupervised-learning-intro/imb_solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a1db8e1c-870e-4c35-ac37-a46bb7732f05",
   "metadata": {},
   "source": [
    "![logo_ironhack_blue 7](https://user-images.githubusercontent.com/23629340/40541063-a07a0a8a-601a-11e8-91b5-2f13e4e6b441.png)\n",
    "\n",
    "# Lab | Unsupervised learning intro\n",
    "\n",
    "#### Instructions \n",
    "\n",
    "\n",
    "It's the moment to perform clustering on the songs you collected. Remember that the ultimate goal of this little project is to improve the recommendations of artists. Clustering the songs will allow the recommendation system to limit the scope of the recommendations to only songs that belong to the same cluster - songs with similar audio features.\n",
    "\n",
    "The experiments you did with the `Spotify` API and the Billboard web scraping will allow you to create a pipeline such that when the user enters a song, you:\n",
    "\n",
    "1. Check whether or not the song is in the Billboard Hot 200.\n",
    "2. Collect the audio features from the `Spotify` API.\n",
    "\n",
    "After that, you want to send the `Spotify` audio features of the submitted song to the clustering model, which should return a cluster number.\n",
    "\n",
    "We want to have as many songs as possible to create the clustering model, so we will add the songs you collected to a bigger dataset available on Kaggle containing 160 thousand songs.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "69a65c1f-2522-46c0-a110-0361e1105086",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 📚 Basic libraries\n",
    "import pandas as pd # dataframe managment\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore') # ignore warnings\n",
    "\n",
    "# 🧩 New libraries\n",
    "import spotipy # Spotify API for developers\n",
    "from spotipy.oauth2 import SpotifyClientCredentials # Spotify user credentiials\n",
    "\n",
    "# Machine Learning\n",
    "from sklearn.cluster import KMeans # To make the clusters\n",
    "from sklearn.preprocessing import StandardScaler # data scaler\n",
    "from sklearn.metrics import pairwise_distances_argmin_min # sklearn metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3c6eb7bb-5823-4456-b1a3-603be505ee2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specific function for this lab.~ From Xisca's script\n",
    "def recommend_song():\n",
    "    \n",
    "    # get song id\n",
    "    song_name = input('Choose a song: ')\n",
    "    results = sp.search(q=f'track:{song_name}', limit=1)\n",
    "    track_id = results['tracks']['items'][0]['id']\n",
    "    \n",
    "    # get song features with the obtained id\n",
    "    audio_features = sp.audio_features(track_id)\n",
    "    \n",
    "    # create dataframe\n",
    "    df_ = pd.DataFrame(audio_features)\n",
    "    new_features = df_[x.columns]\n",
    "    \n",
    "    # scale features\n",
    "    scaled_x = scaler.transform(new_features)\n",
    "    \n",
    "    # predict cluster\n",
    "    cluster = kmeans.predict(scaled_x)\n",
    "    \n",
    "    # filter dataset to predicted cluster\n",
    "    filtered_df = scaled_df[scaled_df['cluster'] == cluster[0]][x.columns]\n",
    "    \n",
    "    # get closest song from filtered dataset\n",
    "    closest, _ = pairwise_distances_argmin_min(scaled_x, filtered_df)\n",
    "    \n",
    "    # return it in a readable way\n",
    "    print('\\n [You might like this song:]')\n",
    "    return ' - '.join([scaled_df.loc[closest]['song'].values[0], scaled_df.loc[closest]['artist'].values[0]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66b79812-9c3e-400b-9ed3-a7e0818f7bf6",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h1 style=\"color: #00BFFF;\">01 | Data Extraction</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6a48cb0-da9b-477a-918f-925df32e4864",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">API acces</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "02652914-00d1-4695-9265-f5581c767b92",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Don't copy this, those are my credientials :P\n",
    "sp = spotipy.Spotify(auth_manager=SpotifyClientCredentials(client_id=\"d565de5f22d343be949f165e317b3448\",\n",
    "                                                           client_secret=\"9ec0e5075f784243b25dedcf5d91f149\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a1d11017-7845-4ffa-92e5-f9c3fc4eff86",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist</th>\n",
       "      <th>album</th>\n",
       "      <th>song</th>\n",
       "      <th>danceability</th>\n",
       "      <th>energy</th>\n",
       "      <th>key</th>\n",
       "      <th>loudness</th>\n",
       "      <th>mode</th>\n",
       "      <th>speechiness</th>\n",
       "      <th>acousticness</th>\n",
       "      <th>instrumentalness</th>\n",
       "      <th>liveness</th>\n",
       "      <th>valence</th>\n",
       "      <th>tempo</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Marea</td>\n",
       "      <td>Coces al Aire 1997-2007</td>\n",
       "      <td>Corazon de mimbre</td>\n",
       "      <td>0.521</td>\n",
       "      <td>0.928</td>\n",
       "      <td>9</td>\n",
       "      <td>-3.548</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0837</td>\n",
       "      <td>0.0235</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0961</td>\n",
       "      <td>0.323</td>\n",
       "      <td>90.555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Passion Pit</td>\n",
       "      <td>Gossamer</td>\n",
       "      <td>Take a Walk</td>\n",
       "      <td>0.566</td>\n",
       "      <td>0.755</td>\n",
       "      <td>11</td>\n",
       "      <td>-5.526</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0368</td>\n",
       "      <td>0.0338</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.3150</td>\n",
       "      <td>0.445</td>\n",
       "      <td>101.006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Fleetwood Mac</td>\n",
       "      <td>Tango In the Night (Deluxe Edition)</td>\n",
       "      <td>Everywhere - 2017 Remaster</td>\n",
       "      <td>0.730</td>\n",
       "      <td>0.487</td>\n",
       "      <td>4</td>\n",
       "      <td>-10.991</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0303</td>\n",
       "      <td>0.2580</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.0852</td>\n",
       "      <td>0.731</td>\n",
       "      <td>114.965</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          artist                                album  \\\n",
       "0          Marea              Coces al Aire 1997-2007   \n",
       "1    Passion Pit                             Gossamer   \n",
       "2  Fleetwood Mac  Tango In the Night (Deluxe Edition)   \n",
       "\n",
       "                         song  danceability  energy  key  loudness  mode  \\\n",
       "0           Corazon de mimbre         0.521   0.928    9    -3.548     0   \n",
       "1                 Take a Walk         0.566   0.755   11    -5.526     1   \n",
       "2  Everywhere - 2017 Remaster         0.730   0.487    4   -10.991     1   \n",
       "\n",
       "   speechiness  acousticness  instrumentalness  liveness  valence    tempo  \n",
       "0       0.0837        0.0235              0.00    0.0961    0.323   90.555  \n",
       "1       0.0368        0.0338              0.00    0.3150    0.445  101.006  \n",
       "2       0.0303        0.2580              0.01    0.0852    0.731  114.965  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('playlist.csv') # extracting my old playlist df\n",
    "df.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9a90c86c-07db-4b5b-b266-9a6fb159b1c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will make the previous dataframe bigger with another 2 more playlist:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c22a8ead-9894-498e-9f9b-d5a051063975",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #00BFFF;\">Playlist track</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "593cbc48-5db8-4166-b0f0-31c00c807a67",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Selecting the playlists; \"Noodles Quarantine Playlist\"\n",
    "playlist = sp.user_playlist_tracks(\"spotify\", \"7uot6g7fHhwHEgRByIC94m\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d538c5b5-3190-47f6-9619-5019add8e540",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Getting artists from the playlist</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3b78986e-db91-44e5-b49a-0f65691d2546",
   "metadata": {},
   "outputs": [],
   "source": [
    "artists = []\n",
    "for i in range(len(playlist[\"items\"])):\n",
    "    artist = playlist[\"items\"][i][\"track\"][\"artists\"][0][\"name\"]\n",
    "    if artist not in artists:\n",
    "        artists.append(artist)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7af63694-61d7-4cde-bbca-1e0ceed53af7",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Getting all albums for each artist</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e66fa068-694a-4abb-8cd9-bf0507ebd58f",
   "metadata": {},
   "outputs": [],
   "source": [
    "albums = []\n",
    "for i in range(len(playlist[\"items\"])):\n",
    "    album = playlist[\"items\"][i][\"track\"][\"album\"][\"name\"]\n",
    "    if album not in albums:\n",
    "        albums.append(album)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb225fab-ae8f-48a3-bd67-febbd47388f2",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Getting all songs from the playlist</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "099922cb-fa6c-4d1f-8020-51010cd9dda0",
   "metadata": {},
   "outputs": [],
   "source": [
    "tracks_title = []\n",
    "for i in range(0,len(playlist[\"items\"])):\n",
    "    title = playlist[\"items\"][i][\"track\"][\"name\"]\n",
    "    tracks_title.append(title)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "952328f3-6a53-4877-8d4c-c6dc7210809b",
   "metadata": {},
   "source": [
    "<h3 style=\"color: #008080;\">Getting song features from the playlist</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "c94479d5-d18e-49bd-a205-c88ee0db6c4b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "tracks = playlist['items']\n",
    "\n",
    "while playlist['next']:\n",
    "    playlist = sp.next(playlist)\n",
    "    tracks.extend(playlist['items'])\n",
    "\n",
    "track_ids = [track['track']['id'] for track in tracks]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "014672b2-245c-4a52-8fb8-6335c3cb5a1e",
   "metadata": {},
   "outputs": [],
   "source": [
    "features = sp.audio_features(track_ids[:100]) # getting features for each track\n",
    "features_df = pd.DataFrame(features) # making it a df\n",
    "\n",
    "features_df = features_df.drop([\"type\", \"id\", \"uri\", \"track_href\", \"analysis_url\", \"duration_ms\", \"time_signature\"], axis=1) # dropping some ugly columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4f5ac68-0328-46b0-b3d3-f4b692f4ea11",
   "metadata": {},
   "source": [
    "<h3 style=\"color: #008080;\">Dictionaries rule</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ac41e30b-69bf-406c-9872-2ade97397446",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now, we will make a dictionary with our lists, assigning the future column name as key\n",
    "ids = {\n",
    "    'artist': artists,\n",
    "    'album': albums,\n",
    "    'song': tracks_title,\n",
    "}\n",
    "\n",
    "ids = pd.DataFrame(ids)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a272f220-4079-42c0-88bd-19c18a2f38ef",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Pandas rule the world -> making it a dataframe</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "788b85e9-36b6-4b05-8c1a-009dd319f9ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "df2 = pd.concat([ids, features_df], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac446d3a-1189-4fbe-8da9-fdd207e83e7a",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #00BFFF;\">Playlist track</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "879e4c8d-2fde-456b-9b9f-0b61135a2fcf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Selecting the playlists; \"Biopsias\"\n",
    "playlist2 = sp.user_playlist_tracks(\"spotify\", \"7nnB4G2MgfyucLZlxctqjw\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01176681-f21e-4547-b75a-cfdfdccc4fba",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Getting artists from the playlist</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "78efbf5e-b004-4710-855b-0bc4459c6631",
   "metadata": {},
   "outputs": [],
   "source": [
    "artists2 = []\n",
    "for i in range(len(playlist2[\"items\"])):\n",
    "    artist = playlist2[\"items\"][i][\"track\"][\"artists\"][0][\"name\"]\n",
    "    if artist not in artists:\n",
    "        artists2.append(artist)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebb44a63-2f39-4ab1-afa0-4fb60a409da1",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Getting all albums for each artist</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "56b9b6c5-ac17-4948-81d6-dd3eb6caf892",
   "metadata": {},
   "outputs": [],
   "source": [
    "albums2 = []\n",
    "for i in range(len(playlist2[\"items\"])):\n",
    "    album = playlist2[\"items\"][i][\"track\"][\"album\"][\"name\"]\n",
    "    if album not in albums:\n",
    "        albums2.append(album)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1fe2acf-7fa3-40ea-afc5-c4d34e647927",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Getting all songs from the playlist</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "f03fc1e7-853d-41b7-b9bd-1c4dc42164d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "tracks_title2 = []\n",
    "for i in range(0,len(playlist2[\"items\"])):\n",
    "    title = playlist2[\"items\"][i][\"track\"][\"name\"]\n",
    "    tracks_title2.append(title)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ce4474a-a45e-4903-be01-2c9f9718bd2f",
   "metadata": {},
   "source": [
    "<h3 style=\"color: #008080;\">Getting song features from the playlist</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "2b4347e5-847a-438f-91fb-f0b2fc7b5a8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "tracks2 = playlist2['items']\n",
    "\n",
    "while playlist2['next']:\n",
    "    playlist2 = sp.next(playlist2)\n",
    "    tracks2.extend(playlist2['items'])\n",
    "\n",
    "track_ids2 = [track['track']['id'] for track in tracks2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f46bc415-7e7b-4eac-9193-fb329b91331c",
   "metadata": {},
   "outputs": [],
   "source": [
    "features2 = sp.audio_features(track_ids2[:100]) # getting features for each track\n",
    "features_df2 = pd.DataFrame(features2) # making it a df\n",
    "\n",
    "features_df2 = features_df2.drop([\"type\", \"id\", \"uri\", \"track_href\", \"analysis_url\", \"duration_ms\", \"time_signature\"], axis=1) # dropping some ugly columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e504666-008f-4973-88ae-54a1d01d471d",
   "metadata": {},
   "source": [
    "<h3 style=\"color: #008080;\">Dictionaries rule</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f904c40c-fd29-4a43-8436-a8e424b76b09",
   "metadata": {},
   "outputs": [],
   "source": [
    "ids2 = {\n",
    "    'artist': artists,\n",
    "    'album': albums,\n",
    "    'song': tracks_title,\n",
    "}\n",
    "\n",
    "ids2 = pd.DataFrame(ids2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d892e12-d924-4d0c-897c-8ebbb5dc82e7",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Pandas rule the world -> making it a dataframe</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "6bdad456-9a1b-48fa-95dd-a25c4bacb2fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "df3 = pd.concat([ids2, features_df2], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "589fdd98-fd3c-4e82-89e4-5e30fee82dcb",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #00BFFF;\">Playlist track (from Kaggle dataset)</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "a07fa754-0339-4e09-8b1d-39fab2b47dc8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>artists</th>\n",
       "      <th>duration_ms</th>\n",
       "      <th>release_date</th>\n",
       "      <th>year</th>\n",
       "      <th>acousticness</th>\n",
       "      <th>danceability</th>\n",
       "      <th>energy</th>\n",
       "      <th>instrumentalness</th>\n",
       "      <th>liveness</th>\n",
       "      <th>loudness</th>\n",
       "      <th>speechiness</th>\n",
       "      <th>tempo</th>\n",
       "      <th>valence</th>\n",
       "      <th>mode</th>\n",
       "      <th>key</th>\n",
       "      <th>popularity</th>\n",
       "      <th>explicit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0gNNToCW3qjabgTyBSjt3H</td>\n",
       "      <td>!Que Vida! - Mono Version</td>\n",
       "      <td>['Love']</td>\n",
       "      <td>220560</td>\n",
       "      <td>11/1/66</td>\n",
       "      <td>1966</td>\n",
       "      <td>0.5250</td>\n",
       "      <td>0.600</td>\n",
       "      <td>0.540</td>\n",
       "      <td>0.00305</td>\n",
       "      <td>0.100</td>\n",
       "      <td>-11.803</td>\n",
       "      <td>0.0328</td>\n",
       "      <td>125.898</td>\n",
       "      <td>0.547</td>\n",
       "      <td>1</td>\n",
       "      <td>9</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0tMgFpOrXZR6irEOLNWwJL</td>\n",
       "      <td>\"40\"</td>\n",
       "      <td>['U2']</td>\n",
       "      <td>157840</td>\n",
       "      <td>2/28/83</td>\n",
       "      <td>1983</td>\n",
       "      <td>0.2280</td>\n",
       "      <td>0.368</td>\n",
       "      <td>0.480</td>\n",
       "      <td>0.70700</td>\n",
       "      <td>0.159</td>\n",
       "      <td>-11.605</td>\n",
       "      <td>0.0306</td>\n",
       "      <td>150.166</td>\n",
       "      <td>0.338</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2ZywW3VyVx6rrlrX75n3JB</td>\n",
       "      <td>\"40\" - Live</td>\n",
       "      <td>['U2']</td>\n",
       "      <td>226200</td>\n",
       "      <td>8/20/83</td>\n",
       "      <td>1983</td>\n",
       "      <td>0.0998</td>\n",
       "      <td>0.272</td>\n",
       "      <td>0.684</td>\n",
       "      <td>0.01450</td>\n",
       "      <td>0.946</td>\n",
       "      <td>-9.728</td>\n",
       "      <td>0.0505</td>\n",
       "      <td>143.079</td>\n",
       "      <td>0.279</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       id                       name   artists  duration_ms  \\\n",
       "0  0gNNToCW3qjabgTyBSjt3H  !Que Vida! - Mono Version  ['Love']       220560   \n",
       "1  0tMgFpOrXZR6irEOLNWwJL                       \"40\"    ['U2']       157840   \n",
       "2  2ZywW3VyVx6rrlrX75n3JB                \"40\" - Live    ['U2']       226200   \n",
       "\n",
       "  release_date  year  acousticness  danceability  energy  instrumentalness  \\\n",
       "0      11/1/66  1966        0.5250         0.600   0.540           0.00305   \n",
       "1      2/28/83  1983        0.2280         0.368   0.480           0.70700   \n",
       "2      8/20/83  1983        0.0998         0.272   0.684           0.01450   \n",
       "\n",
       "   liveness  loudness  speechiness    tempo  valence  mode  key  popularity  \\\n",
       "0     0.100   -11.803       0.0328  125.898    0.547     1    9          26   \n",
       "1     0.159   -11.605       0.0306  150.166    0.338     1    8          21   \n",
       "2     0.946    -9.728       0.0505  143.079    0.279     1    8          41   \n",
       "\n",
       "   explicit  \n",
       "0         0  \n",
       "1         0  \n",
       "2         0  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df4 = pd.read_csv('data.csv') # extracting the songs from kaggle datasets\n",
    "df4.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "8f9075d4-0348-4a49-8e79-b88748fd7981",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['id', 'name', 'artists', 'duration_ms', 'release_date', 'year',\n",
       "       'acousticness', 'danceability', 'energy', 'instrumentalness',\n",
       "       'liveness', 'loudness', 'speechiness', 'tempo', 'valence', 'mode',\n",
       "       'key', 'popularity', 'explicit'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df4.columns # getting it's columns to change it to t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "543547da-1b08-4e59-a117-fd9518e259c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "df4 = df4[['artists', 'name', 'danceability', 'energy', 'key', 'loudness', 'mode', 'speechiness', 'acousticness', 'instrumentalness', 'liveness', 'valence', 'tempo']]\n",
    "df4 = df4.rename(columns={'name': 'song', 'artists': 'artist'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "c12677e9-7662-44d0-874e-791f633dfdea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist</th>\n",
       "      <th>song</th>\n",
       "      <th>danceability</th>\n",
       "      <th>energy</th>\n",
       "      <th>key</th>\n",
       "      <th>loudness</th>\n",
       "      <th>mode</th>\n",
       "      <th>speechiness</th>\n",
       "      <th>acousticness</th>\n",
       "      <th>instrumentalness</th>\n",
       "      <th>liveness</th>\n",
       "      <th>valence</th>\n",
       "      <th>tempo</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>['Love']</td>\n",
       "      <td>!Que Vida! - Mono Version</td>\n",
       "      <td>0.600</td>\n",
       "      <td>0.540</td>\n",
       "      <td>9</td>\n",
       "      <td>-11.803</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0328</td>\n",
       "      <td>0.5250</td>\n",
       "      <td>0.00305</td>\n",
       "      <td>0.100</td>\n",
       "      <td>0.547</td>\n",
       "      <td>125.898</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>['U2']</td>\n",
       "      <td>\"40\"</td>\n",
       "      <td>0.368</td>\n",
       "      <td>0.480</td>\n",
       "      <td>8</td>\n",
       "      <td>-11.605</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0306</td>\n",
       "      <td>0.2280</td>\n",
       "      <td>0.70700</td>\n",
       "      <td>0.159</td>\n",
       "      <td>0.338</td>\n",
       "      <td>150.166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>['U2']</td>\n",
       "      <td>\"40\" - Live</td>\n",
       "      <td>0.272</td>\n",
       "      <td>0.684</td>\n",
       "      <td>8</td>\n",
       "      <td>-9.728</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0505</td>\n",
       "      <td>0.0998</td>\n",
       "      <td>0.01450</td>\n",
       "      <td>0.946</td>\n",
       "      <td>0.279</td>\n",
       "      <td>143.079</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     artist                       song  danceability  energy  key  loudness  \\\n",
       "0  ['Love']  !Que Vida! - Mono Version         0.600   0.540    9   -11.803   \n",
       "1    ['U2']                       \"40\"         0.368   0.480    8   -11.605   \n",
       "2    ['U2']                \"40\" - Live         0.272   0.684    8    -9.728   \n",
       "\n",
       "   mode  speechiness  acousticness  instrumentalness  liveness  valence  \\\n",
       "0     1       0.0328        0.5250           0.00305     0.100    0.547   \n",
       "1     1       0.0306        0.2280           0.70700     0.159    0.338   \n",
       "2     1       0.0505        0.0998           0.01450     0.946    0.279   \n",
       "\n",
       "     tempo  \n",
       "0  125.898  \n",
       "1  150.166  \n",
       "2  143.079  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df4.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed8a6d24-8e1e-457f-be02-d310667ba7ea",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #00BFFF;\">Pandas rule the world -> making it a big dataframe</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "5a9dcfcb-4a0f-4959-8b57-09913adad5b4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(170022, 14)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.concat([df, df2, df3, df4])\n",
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9aa5bc0e-d589-4b87-880a-19ecd2d8ef69",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h1 style=\"color: #00BFFF;\">02 | Data Pre-processing</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb0caee1-6c75-40d7-8b03-caf4e7525296",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Scaling</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "23ee83cd-5759-45b4-8197-3a43fd89c66a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>danceability</th>\n",
       "      <th>energy</th>\n",
       "      <th>key</th>\n",
       "      <th>loudness</th>\n",
       "      <th>mode</th>\n",
       "      <th>speechiness</th>\n",
       "      <th>acousticness</th>\n",
       "      <th>instrumentalness</th>\n",
       "      <th>liveness</th>\n",
       "      <th>valence</th>\n",
       "      <th>tempo</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.521</td>\n",
       "      <td>0.928</td>\n",
       "      <td>9</td>\n",
       "      <td>-3.548</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0837</td>\n",
       "      <td>0.0235</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0961</td>\n",
       "      <td>0.323</td>\n",
       "      <td>90.555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.566</td>\n",
       "      <td>0.755</td>\n",
       "      <td>11</td>\n",
       "      <td>-5.526</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0368</td>\n",
       "      <td>0.0338</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.3150</td>\n",
       "      <td>0.445</td>\n",
       "      <td>101.006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.730</td>\n",
       "      <td>0.487</td>\n",
       "      <td>4</td>\n",
       "      <td>-10.991</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0303</td>\n",
       "      <td>0.2580</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.0852</td>\n",
       "      <td>0.731</td>\n",
       "      <td>114.965</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   danceability  energy  key  loudness  mode  speechiness  acousticness  \\\n",
       "0         0.521   0.928    9    -3.548     0       0.0837        0.0235   \n",
       "1         0.566   0.755   11    -5.526     1       0.0368        0.0338   \n",
       "2         0.730   0.487    4   -10.991     1       0.0303        0.2580   \n",
       "\n",
       "   instrumentalness  liveness  valence    tempo  \n",
       "0              0.00    0.0961    0.323   90.555  \n",
       "1              0.00    0.3150    0.445  101.006  \n",
       "2              0.01    0.0852    0.731  114.965  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# selecting numericals\n",
    "x = data[['danceability', 'energy', 'key', 'loudness', 'mode', 'speechiness', 'acousticness', 'instrumentalness', 'liveness', 'valence', 'tempo']]\n",
    "x.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "a63e907a-5da6-4705-8a68-d57b149141d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# standarize the data\n",
    "scaler = StandardScaler()\n",
    "x_prep = scaler.fit_transform(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78eb1115-df02-4674-9cbe-948caa82db14",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Train and Predict</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "c30fa493-5a7c-4fe6-8d39-acbc6070e32e",
   "metadata": {},
   "outputs": [],
   "source": [
    "kmeans = KMeans(n_clusters=3, random_state=42)\n",
    "kmeans.fit(x_prep)\n",
    "clusters = kmeans.predict(x_prep)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40d69cc7-abaa-4325-8d01-ce0957741d49",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">New master dataframe</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "0a97c57b-8fcb-4ea4-ac82-181267c51d73",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>danceability</th>\n",
       "      <th>energy</th>\n",
       "      <th>key</th>\n",
       "      <th>loudness</th>\n",
       "      <th>mode</th>\n",
       "      <th>speechiness</th>\n",
       "      <th>acousticness</th>\n",
       "      <th>instrumentalness</th>\n",
       "      <th>liveness</th>\n",
       "      <th>valence</th>\n",
       "      <th>tempo</th>\n",
       "      <th>song</th>\n",
       "      <th>artist</th>\n",
       "      <th>cluster</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.097730</td>\n",
       "      <td>1.642075</td>\n",
       "      <td>1.080778</td>\n",
       "      <td>1.379646</td>\n",
       "      <td>-1.558937</td>\n",
       "      <td>-0.069045</td>\n",
       "      <td>-1.246305</td>\n",
       "      <td>-0.523426</td>\n",
       "      <td>-0.625565</td>\n",
       "      <td>-0.796846</td>\n",
       "      <td>-0.859234</td>\n",
       "      <td>Corazon de mimbre</td>\n",
       "      <td>Marea</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.158926</td>\n",
       "      <td>0.995260</td>\n",
       "      <td>1.649709</td>\n",
       "      <td>1.030621</td>\n",
       "      <td>0.641463</td>\n",
       "      <td>-0.381925</td>\n",
       "      <td>-1.218961</td>\n",
       "      <td>-0.523426</td>\n",
       "      <td>0.612650</td>\n",
       "      <td>-0.331871</td>\n",
       "      <td>-0.519101</td>\n",
       "      <td>Take a Walk</td>\n",
       "      <td>Passion Pit</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.094296</td>\n",
       "      <td>-0.006744</td>\n",
       "      <td>-0.341549</td>\n",
       "      <td>0.066300</td>\n",
       "      <td>0.641463</td>\n",
       "      <td>-0.425288</td>\n",
       "      <td>-0.623754</td>\n",
       "      <td>-0.491093</td>\n",
       "      <td>-0.687222</td>\n",
       "      <td>0.758152</td>\n",
       "      <td>-0.064799</td>\n",
       "      <td>Everywhere - 2017 Remaster</td>\n",
       "      <td>Fleetwood Mac</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   danceability    energy       key  loudness      mode  speechiness  \\\n",
       "0     -0.097730  1.642075  1.080778  1.379646 -1.558937    -0.069045   \n",
       "1      0.158926  0.995260  1.649709  1.030621  0.641463    -0.381925   \n",
       "2      1.094296 -0.006744 -0.341549  0.066300  0.641463    -0.425288   \n",
       "\n",
       "   acousticness  instrumentalness  liveness   valence     tempo  \\\n",
       "0     -1.246305         -0.523426 -0.625565 -0.796846 -0.859234   \n",
       "1     -1.218961         -0.523426  0.612650 -0.331871 -0.519101   \n",
       "2     -0.623754         -0.491093 -0.687222  0.758152 -0.064799   \n",
       "\n",
       "                         song         artist  cluster  \n",
       "0           Corazon de mimbre          Marea        1  \n",
       "1                 Take a Walk    Passion Pit        1  \n",
       "2  Everywhere - 2017 Remaster  Fleetwood Mac        1  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scaled_df = pd.DataFrame(x_prep, columns=x.columns)\n",
    "scaled_df = scaled_df.reset_index(drop=True)\n",
    "\n",
    "data = data.reset_index(drop=True)\n",
    "\n",
    "\n",
    "scaled_df['song'] = data['song']\n",
    "scaled_df['artist'] = data['artist']\n",
    "scaled_df['cluster'] = clusters\n",
    "scaled_df.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f11d1a5e-3a9d-4eb6-8022-ca6270fc812f",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h3 style=\"color: #008080;\">Dropping duplicates</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "1c934bec-36d4-454f-86d5-d4fc7bfecb50",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaled_df = scaled_df.drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "887b7362-aae8-49d8-bbd4-5460cd6f24ce",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h1 style=\"color: #00BFFF;\">03 | Reporting</h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "c10c0bf4-0e9b-4be4-994f-33fc3ed4d8d4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Choose a song:  Toxic\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " [You might like this song:]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Everything Dies - Mariachi El Bronx'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommend_song()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}