04. K-Means Clustering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Partitioning Method\n",
    "## KMeans\n",
    "* Minimizing creteria : within-cluster-sum-of-squares.\n",
    "\n",
    "  \n",
    "\n",
    "* The centroids are chosen in such a way that it minimizes within cluster sum of squares.\n",
    "\n",
    "* The k-means algorithm divides a set of $N$ samples $X$ into $K$ disjoint clusters $C$, each described by the mean  of the samples in the cluster. $\\mu$\n",
    "\n",
    "<img src=\"https://cssanalytics.files.wordpress.com/2013/11/cluster-image.png\" width=\"300px\">\n",
    "\n",
    "##### KMeans Algorithm\n",
    "1. Initialize k centroids.\n",
    "2. Select at random K points, the centroids(not necessary from the dataset)\n",
    "3. Assign each data to the nearest centroid, this step will create clusters.\n",
    "4. Compute and place the new centroid of each cluster.  \n",
    "5. Reassign each data point to the new closest centroid. If any new reassignment, Repeat steps 4 otherwise go to Finish  \n",
    "\n",
    "Animated Implementation of the Algorithm - http://tech.nitoyon.com/en/blog/2013/11/07/k-means/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Initialization Trap\n",
    "Depending on initialisation of Centroid we get different clusters for the same data  \n",
    "Solution -> K-Means++\n",
    "\n",
    "<img src=\"Image/kmeans1.png\" width=\"300\" />\n",
    "<font size=1><center>Case 1</font>\n",
    "    \n",
    "<img src=\"Image/kmeans2.png\" width=\"300\" />\n",
    "<font size=1><center>Case 2</font>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Choosing the Right Number of Clusters\n",
    "- WCSS -> Within Cluster Sum of Squares  \n",
    "- Elbow Method to find the optimal number of clusters  \n",
    "<img src=\"Image/elbow.jpg\" width=\"300\" />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl8leWZ//HPlZ0sLIEksskaFNRqNSIoqKAiVVttp452EWzt0Kq4O7/uY6fLTLepu3ZcWtFa1KIOVlxKFVEUkeACKkICgiBIgqwJBAhcvz/OHTjEkLDk5DlJvu/X67zOOfezXeco+eZ+njv3Y+6OiIhIIqVEXYCIiLR9ChsREUk4hY2IiCScwkZERBJOYSMiIgmnsBERkYRT2Ijsg5ldamaz4t67mQ2Msqbm0pyfxcyWmdmZzbEvabsUNtKuhR+UW82sKu5xR9R1we6wczP7Q732C0L7A/u5n5fM7DsJKVJkPylsROCL7p4b95gYdUFxlgAXmVlaXNs4YHFE9YgcFIWNyIE5x8yWmtlaM/udmaUAmFmKmf3EzJabWYWZPWhmncKySWZ2Q3jdM/RKrgjvB5rZOjOzfRzvE2ABcHZYPx84GXgqfiUzG2Zmr5nZBjN7x8xOD+2/AkYCdzTQazvTzMrMbL2Z3VlXQ2OfJSy/JCz71Mx+fIjfp7QTChuRA/NloAQ4Hjgf+HZovzQ8RgH9gVyg7gf7TOD08Po0YGl4BjgVeMUbnzfqQWK9GYCLganAtrqFZtYTmAb8EsgHbgQeN7MCd/8x8AowsYFe23nAicCxwL8SAq2xz2JmQ4C7gUuAHkBXoFcjtYsAChsRgP8LPYK6x781su5v3H2du38E3AJ8LbR/A/iDuy919yrgh8DF4fTXTGBk6AWdCvwWOCVsd1pY3pgngdND72IcsfCJ903gGXd/xt13uft0oBQ4p4n9/trdN4TPMgM4bj8+y1eBp939ZXffBvwU2NXEcUQUNiLABe7eOe5xbyPrroh7vZzYb/eE5+X1lqUBRe6+BKgi9sN8JPA0sMrMjmA/wsbdtxLrufwE6Obur9ZbpQ9wYXxgAiOA7o3tl9gpujpbiPVgGv0sYdnu78Ddq4FPmziOCGlNryIicXoD74XXhwOrwutVxH7oE7esFlgT3s8k1ivIcPePzWwmsV5KF+Dt/Tjug8CLwH82sGwF8JC776tHdqBTuzf2WVYDg+sWmFk2sVNpIo1Sz0bkwPy7mXUxs97ANcCjoX0ycJ2Z9TOzXOC/gEfdvTYsnwlMBF4O718CrgJmufvO/TjuTOAs4PYGlv0F+KKZnW1mqWaWZWanm1ndtZQ1xK697K/GPssU4DwzG2FmGcDP0c8R2Q/6n0QE/l7v72yebGTdqcA8Yr2RacD9of1PwEPEwuRDoIZYmNSZCeSxJ2xmAdlx7xvlMS+4+7oGlq0gNljhR0AlsZ7Ov7Pn3/etwFfDqLPb9uNw+/ws7v4ecCXwV2K9nPXAyv35DNK+mW6eJiIiiaaejYiIJJzCRkREEk5hIyIiCaewERGRhNPf2QTdunXzvn37Rl2GiEirMm/evLXuXtDUegqboG/fvpSWlkZdhohIq2Jmy5teS6fRRESkBShsREQk4RQ2IiKScAobERFJOIWNiIgknMJGREQSTmEjIiIJp7A5RK+Vr+Wul8qjLkNEJKkpbA7RS4sr+f3zi1j+aXXUpYiIJC2FzSH6zsh+pKWmcPdLS6IuRUQkaSlsDlFhXhZfO7E3j7+5klUbtkZdjohIUlLYNIMJpw3AHe55eWnUpYiIJCWFTTPo2bkD/3J8Lya/8REVm2uiLkdEJOkobJrJ5acPYMfOXdz/yodRlyIiknQUNs2kb7ccvnRsDx56fTnrq7dHXY6ISFJR2DSjK0YNZMv2nfz5tWVRlyIiklQUNs1oUFEeY486jAde/ZBNNTuiLkdEJGkobJrZxNED2VRTy0Oz9+vmdSIi7YLCppkd3bMTo44o4P5ZH7Jle23U5YiIJAWFTQJMHD2QddXbmfzGiqhLERFJCgqbBDihTz7D+3flnpeXULNjZ9TliIhELmFhY2a9zWyGmS00s/fM7JrQ/jMz+9jM3g6Pc+K2+aGZlZvZIjM7O659bGgrN7MfxLX3M7M5ZlZmZo+aWUZozwzvy8Pyvon6nPty1eiBrNm0jSnzVrb0oUVEkk4ieza1wA3uPhgYBlxpZkPCspvd/bjweAYgLLsYOAoYC9xlZqlmlgrcCXwBGAJ8LW4/vwn7KgbWA5eF9suA9e4+ELg5rNeihg/oyucP78zdLy1hx85dLX14EZGkkrCwcffV7v5meL0ZWAj0bGST84FH3H2bu38IlANDw6Pc3Ze6+3bgEeB8MzNgNDAlbD8JuCBuX5PC6ynAGWH9FmNmXDV6IB9v2MrUt1e15KFFRJJOi1yzCaexPg/MCU0TzWy+mf3JzLqEtp5A/BX1laFtX+1dgQ3uXluvfa99heUbw/r165pgZqVmVlpZWXlIn7Eho44oZEj3jtw1o5ydu7zZ9y8i0lokPGzMLBd4HLjW3TcBdwMDgOOA1cD/1K3awOZ+EO2N7WvvBvd73L3E3UsKCgoa/RwHo653s3RtNc8sWN3s+xcRaS0SGjZmlk4saB529ycA3H2Nu+90913AvcROk0GsZ9I7bvNewKpG2tcCnc0srV77XvsKyzsB65r30+2fs486jIGFudw5o5xd6t2ISDuVyNFoBtwPLHT3P8S1d49b7cvAu+H1U8DFYSRZP6AYeAOYCxSHkWcZxAYRPOXuDswAvhq2Hw9MjdvX+PD6q8CLYf0Wl5JiXDlqAB98spkXPqiIogQRkcglsmdzCnAJMLreMOffmtkCM5sPjAKuA3D394DHgPeB54ArQw+oFpgIPE9skMFjYV2A7wPXm1k5sWsy94f2+4Guof16YPdw6Sh88XM9ODw/mzteLCOizBMRiZTph19MSUmJl5aWJmz/j7zxET94YgEPfnsopw5q/utDIiJRMLN57l7S1HqaQaCFfOX4XnTvlMUdL5ZHXYqISItT2LSQjLQUvntqf95Yto45Sz+NuhwRkRalsGlBFw89nG65GdwxQ70bEWlfFDYtKCs9le+M7M8rZWt5Z8WGqMsREWkxCpsW9s1hfejUIV29GxFpVxQ2LSw3M41vn9KP6e+vYeHqTVGXIyLSIhQ2Ebj05L7kZqZxp3o3ItJOKGwi0Ck7nUuG92HagtUsqayKuhwRkYRT2ETkshH9yExL4e6XlkRdiohIwilsItItN5OvD+3Dk299zIp1W6IuR0QkoRQ2EZpwan9SzfjjTPVuRKRtU9hE6LBOWXy1pBd/K13JJxtroi5HRCRhFDYRu/y0Aex0595XlkZdiohIwihsItY7P5sLjuvJw3OW82nVtqjLERFJCIVNErhi1AC21e7i/lkfRl2KiEhCKGySwICCXM45pjsPzl7Oxi07oi5HRKTZKWySxMRRA6naVsuk2cuiLkVEpNkpbJLE4O4dOXNwEX969UOqttVGXY6ISLNS2CSRiaMHsmHLDh5+fXnUpYiINCuFTRI5rndnRhZ3495XPqRmx86oyxERaTYKmyQzcdRA1lZt49G5K6IuRUSk2ShsksxJ/bsytG8+f5y5hO21u6IuR0SkWShsktCVoweyemMNT7y5MupSRESahcImCZ1a3I3P9erE3TOXULtTvRsRaf0UNknIzJg4aiDLP93C0/NXR12OiMghU9gkqTMHF3FEUR53zChn1y6PuhwRkUOisElSKSnGlaMHUl5RxfPvfRJ1OSIih0Rhk8TOPaY7/brlcMeMctzVuxGR1kthk8RSU4wrTh/Ae6s28dKiyqjLERE5aAqbJHfB53vSs3MHbnuxTL0bEWm1EhY2ZtbbzGaY2UIze8/Mrgnt+WY23czKwnOX0G5mdpuZlZvZfDM7Pm5f48P6ZWY2Pq79BDNbELa5zcyssWO0RumpKXzv9AG89dEGZi/5NOpyREQOSiJ7NrXADe4+GBgGXGlmQ4AfAC+4ezHwQngP8AWgODwmAHdDLDiAm4CTgKHATXHhcXdYt267saF9X8dolS48oReFeZncMaM86lJERA5KwsLG3Ve7+5vh9WZgIdATOB+YFFabBFwQXp8PPOgxrwOdzaw7cDYw3d3Xuft6YDowNizr6O6zPXZ+6cF6+2roGK1SVnoqE07tz2tLPmXe8nVRlyMicsBa5JqNmfUFPg/MAYrcfTXEAgkoDKv1BOJnn1wZ2hprX9lAO40co9X6+kmHk5+TwR0vqncjIq1PwsPGzHKBx4Fr3X1TY6s20OYH0X4gtU0ws1IzK62sTO7RXtkZaVw2oh8zFlXy7scboy5HROSAJDRszCydWNA87O5PhOY14RQY4bkitK8Eesdt3gtY1UR7rwbaGzvGXtz9HncvcfeSgoKCg/uQLeiS4X3Iy0rjTl27EZFWJpGj0Qy4H1jo7n+IW/QUUDeibDwwNa59XBiVNgzYGE6BPQ+MMbMuYWDAGOD5sGyzmQ0LxxpXb18NHaNV65iVzqUn9+XZdz9h8ZrNUZcjIrLfEtmzOQW4BBhtZm+HxznAr4GzzKwMOCu8B3gGWAqUA/cCVwC4+zrgF8Dc8Ph5aAO4HLgvbLMEeDa07+sYrd63TulHdkYqd6l3IyKtiOkPBWNKSkq8tLQ06jL2y389s5D7XlnKizecTt9uOVGXIyLtmJnNc/eSptbTDAKt0HdG9iMtNYU/zlwSdSkiIvtFYdMKFeZlcfGJvXn8zZV8vGFr1OWIiDRJYdNKffe0AbjDPerdiEgroLBppXp27sC/HN+LyXNXULG5JupyREQapbBpxS4/fQC1O3dx/ysfRl2KiEijFDatWN9uOXzx2B489Ppy1ldvj7ocEZF9Uti0cleOGsiW7Tv586vq3YhI8lLYtHKDivIYe9RhPPDaMjbV7Ii6HBGRBils2oArRw1kU00tD81eHnUpIiINUti0Acf06sToIwv548wlrNO1GxFJQgqbNuKHXziSLdt3cvP0xVGXIiLyGQqbNqK4KI9vnnQ4D89ZzqJPNCO0iCQXhU0bcu2Zg8jNTOOX095HE6yKSDJR2LQhXXIyuPbMQbxStpYXP2jwfnEiIpFQ2LQxlwzvQ/+CHH41bSHba3dFXY6ICKCwaXPSU1P46blDWLq2mode11BoEUkOCps26PQjCjh1UAG3/nOxhkKLSFJQ2LRBZsZPzh1M9fad3PJPDYUWkegpbNqoQUV5fOOkw3l4zkcsXqOh0CISLYVNG3btmYPIyUjlF09rKLSIREth04bl52RwTRgKPWORhkKLSHQUNm3cuDAU+pdPL2THTg2FFpFoKGzauPTUFH5y7uDYUGjNCi0iEVHYtAOjjihkZHE3bvnnYt3RU0QiobBpB8yMn543hKpttRoKLSKRUNi0E7Gh0H34y5yPKNNQaBFpYQqbduS6swaRnZHKL6Yt1FBoEWlRjYaNmZ1oZofFvR9nZlPN7DYzy098edKc8nMyuOaMYl5eXMlLiyqjLkdE2pGmejb/C2wHMLNTgV8DDwIbgXsSW5okwrjhfenfLYdfTHtfQ6FFpMU0FTap7r4uvL4IuMfdH3f3nwIDE1uaJEJGWgo/PncwSyur+YtmhRaRFtJk2JhZWnh9BvBi3LK0BtaXVmD0kXVDocs0FFpEWkRTYTMZmGlmU4GtwCsAZjaQ2Km0fTKzP5lZhZm9G9f2MzP72MzeDo9z4pb90MzKzWyRmZ0d1z42tJWb2Q/i2vuZ2RwzKzOzR80sI7RnhvflYXnf/f422onYrNBD2Fyzg1tfKIu6HBFpBxoNG3f/FXAD8AAwwvcMYUoBrmpi3w8AYxtov9ndjwuPZwDMbAhwMXBU2OYuM0s1s1TgTuALwBDga2FdgN+EfRUD64HLQvtlwHp3HwjcHNaTeo44LI+vn3Q4D72+nPIKDYUWkcRqajRaNjDP3Z9092ozO8LMrgOOdvc3G9vW3V8G1jW2TpzzgUfcfZu7fwiUA0PDo9zdl7r7duAR4HwzM2A0MCVsPwm4IG5fk8LrKcAZYX2p57ozw1DopxdGXYqItHFNnUZ7DugLu0+dzQb6A1ea2X8f5DEnmtn8cJqtS2jrCayIW2dlaNtXe1dgg7vX1mvfa19h+caw/meY2QQzKzWz0srK9jcUuGtuJtecUczMxZWaFVpEEqqpsOni7nUn9ccDk939KmKntc47iOPdDQwAjgNWA/8T2hvqefhBtDe2r882ut/j7iXuXlJQUNBY3W3WuOF96dcth18+raHQIpI4TYVN/A/p0cB0gHBK64B/Mrn7Gnff6e67gHuJnSaDWM+kd9yqvYBVjbSvBTrHjZSra99rX2F5J/b/dF67k5GWwo/PGcySymoe1lBoEUmQpsJmvpn9PlynGQj8A8DMOh/Mwcyse9zbLwN1I9WeAi4OI8n6AcXAG8BcoDiMPMsgNojgqTBQYQbw1bD9eGBq3L7Gh9dfBV6MG9ggDThjcCEjBnbj5n+WsWGLhkKLSPNrKmz+jVgvoi8wxt23hPYhwO8b29DMJhO7xnOEma00s8uA35rZAjObD4wCrgNw9/eAx4D3iV0nujL0gGqBicDzwELgsbAuwPeB682snNg1mftD+/1A19B+PbB7uLQ0zMz4yXmD2Vyzg1v+qaHQItL8rLFf+s2sACiM+wFf1340sMbd28xV9ZKSEi8tLY26jEj9+MkFPDJ3Bc9fO5KBhXlRlyMirYCZzXP3kqbWa6pnczvQrYH2nsCtB1OYJK/rw6zQv5ymodAi0ryaCptj3H1m/UZ3fx74XGJKkqjUDYV+aZGGQotI82oqbNIPcpm0UuOG96Vv12x+NW2hhkKLSLNpKmzK4ucvq2NmXwCWJqYkiVJsVughlFdU8dc5H0Vdjoi0EU3N3HwtMM3M/hWYF9pKgOEc3B91Sitw5uBCThnYlZv/uZjzj+tB5+yMqEsSkVauqZ7NucQmtnwV6BMeM4HPufviBNcmEambFXrTVs0KLSLNo6mw6UVs1uTfEuvRbAfWANkJrksiNrh7Ry4eejgPzV5OeUVV1OWISCvX1C0GbnT3k4Ei4EfEpn35NvCumb3fAvVJhK4/axAd0lP51TT9pxaRQ9NUz6ZOB6AjsXnGOhGbh2xOooqS5NAtN5OrzyhmxqJKXtJQaBE5BE3dz+YeM3sVeJTYoIDXgAvDTMnfaokCJVrjT44Nhf7ltIXUaii0iBykpno2hwOZwCfAx8RmVN6Q6KIkeWSkpfCjcwbHhkK/oaHQInJwmrpmMxY4kT2Tbt4AzDWzf5jZfya6OEkOZw0p4uQBXfnD9MVs3LIj6nJEpBVq8pqNx7wLPAM8S2wY9ADgmgTXJknCzPjpeRoKLSIHr6lrNleb2SNmtgJ4mdgfci4CvgLkt0B9kiQGd+/IRScezoOzl7GkUkOhReTANNWz6QtMAYa6e393v8Td73L3d8LdNqUduWFM3VBozQotIgemqWs217v7FHdf3VIFSfLqlpvJVWcM5MUPKpi5uM3cykhEWsD+/p2NCBAbCt2naza/fPp9DYUWkf2msJEDkpmWyo/OGUxZRRWTNRRaRPaTwkYO2JghRQzvr6HQIrL/FDZywOqGQm/YuoPbXtRQaBFpmsJGDsqQHh25+MTeTHptGUs1FFpEmqCwkYN2w5gjyNJQaBHZDwobOWjdcjO5avRAXviggpc1FFpEGqGwkUNy6Sl9OTw/m19O01BoEdk3hY0ckrqh0IvXVDF57oqoyxGRJKWwkUN29lFFDOufzx/+sYiNWzUUWkQ+S2Ejhyx+KPTtmhVaRBqgsJFmcVSPTlxU0psHNBRaRBqgsJFmc8OYI+iQnso37pvDvOXroy5HRJKIwkaaTUFeJpMnDCMjLYWL/nc298/6EHePuiwRSQIJCxsz+5OZVZjZu3Ft+WY23czKwnOX0G5mdpuZlZvZfDM7Pm6b8WH9MjMbH9d+gpktCNvcZmbW2DGkZRzdsxNPTRzB6CML+cXT73PFw2+yqUaDBkTau0T2bB4AxtZr+wHwgrsXAy+E9wBfAIrDYwJwN8SCA7gJOAkYCtwUFx53h3XrthvbxDGkhXTqkM7/XnICPzl3MNPfX8OXbp/Fe6s2Rl2WiEQoYWHj7i8D6+o1nw9MCq8nARfEtT/oMa8Dnc2sO3A2MN3d17n7emA6MDYs6+jusz12nubBevtq6BjSgsyM74zszyMThlGzYxdfvus1Hp37kU6ribRTLX3Npqjurp/huTC09wTi/yJwZWhrrH1lA+2NHeMzzGyCmZWaWWllpaZbSYSSvvlMu3oEJ/XL5/uPL+DGv81n6/adUZclIi0sWQYIWANtfhDtB8Td73H3EncvKSgoONDNZT91zc3kgW8N5dozi3nirZVccOerLNHwaJF2paXDZk04BUZ4rgjtK4Hecev1AlY10d6rgfbGjiERSk0xrj1zEA9+eyiVVdv40u2z+Ps7q5reUETahJYOm6eAuhFl44Gpce3jwqi0YcDGcArseWCMmXUJAwPGAM+HZZvNbFgYhTau3r4aOoYkgZHFBUy7egRHdu/IVZPf4qap77KtVqfVRNq6RA59ngzMBo4ws5Vmdhnwa+AsMysDzgrvAZ4BlgLlwL3AFQDuvg74BTA3PH4e2gAuB+4L2ywBng3t+zqGJInunTrwyIRh/NvIfkyavZx//eNsVq7fEnVZIpJAptFBMSUlJV5aWhp1Ge3Oc+9+wr//7R1SUoybLzqW0UcWRV2SiBwAM5vn7iVNrZcsAwSknRp79GE8ffUIenXpwLcfKOW3z32g++KItEEKG4lcn645PH75yXxt6OHc9dISvnHfHCo21URdlog0I4WNJIWs9FT++yvH8Id/PZb5Kzdyzm2zmL3k06jLEpFmorCRpPKV43sxdeIpdOyQxjfue507Z5Sza5euK4q0dgobSTqDivJ4auIIzv1cD373/CIumzSX9dXboy5LRA6BwkaSUm5mGrddfBy/OP8oZpWv5bzbZ/H2ig1RlyUiB0lhI0nLzLhkeF+mfO9kAC7842tMem2ZJvMUaYUUNpL0ju3dmWlXj+DU4gJueuo9Jk5+i6pttVGXJSIHQGEjrULn7AzuHVfC98ceybMLVvOl22fxwSeboi5LRPaTwkZajZQU4/LTB/DXfxvG5m21XHDnq0yZt7LpDUUkcgobaXWG9e/KtKtH8PneXbjxb+/w/SnzqdmhyTxFkpnCRlqlwrwsHrpsKFeOGsCjpSv48l2v8eHa6qjLEpF9UNhIq5WWmsK/n30kf770RFZv3MoXb5/FswtWR12WiDRAYSOt3qgjC5l29UgGFOZy+cNv8vO/v8/2Wk3mKZJMFDbSJvTs3IG/fXc4l57clz+9+iEX3TObFet0jxyRZKGwkTYjIy2Fn33pKO78+vGUraniC7e+wqNzP9IfgYokAYWNtDnnfq47z14zkqN7duT7jy/gskmlumWBSMQUNtIm9c7P5q/fGcZ/nDeEV8vXMuaWl/n7O6uiLkuk3VLYSJuVkmJ8e0Q/pl09kj5dc7hq8ltM/OubmkFaJAIKG2nzBhbm8vj3hnPjmEE8/94njLnlZV78YE3UZYm0KwobaRfSUlOYOLqY/7vyFLrmZPDtB0r5f1PeYXPNjqhLE2kXFDbSrhzVoxNTJ57C5acPYMq8lYy95RVeW7I26rJE2jyFjbQ7mWmpfH/skfzte8NJTzW+fu8c/vPv72l+NZEEUthIu3VCn3yeuWYk44f34c+vLuOc217hrY/WR12WSJuksJF2LTsjjf88/2j+ctlJ1Gzfyb/c/Rq/f36RprsRaWYKGxFgRHE3nrvuVL5yfC/umFHOBXe+qpuziTQjhY1I0DErnd9feCz3jiuhYnMNX7x9Fne9VM7OXZruRuRQKWxE6jlrSBH/uO40zhpSxG+fW8SFf9S9ckQOlcJGpAH5ORnc+fXjufXi4yivqOKcW1/hwdnL2KVejshBUdiI7IOZcf5xPfnHdacxtF8+/zH1Pcb96Q1WbdgadWkirY7CRqQJh3XK4oFvnch/ffkY3vxoPWff/DJT5q3UrQtEDkAkYWNmy8xsgZm9bWaloS3fzKabWVl47hLazcxuM7NyM5tvZsfH7Wd8WL/MzMbHtZ8Q9l8etrWW/5TSlpgZXz/pcJ675lQGd+/IjX97hwkPzaNy87aoSxNpFaLs2Yxy9+PcvSS8/wHwgrsXAy+E9wBfAIrDYwJwN8TCCbgJOAkYCtxUF1BhnQlx241N/MeR9uDwrtlMnjCMn5w7mJmLKzn7lpd5dsHqqMsSSXrJdBrtfGBSeD0JuCCu/UGPeR3obGbdgbOB6e6+zt3XA9OBsWFZR3ef7bHzHA/G7UvkkKWmGN8Z2Z9pV42gZ+cOXP7wm1zzyFts3KJJPUX2JaqwceAfZjbPzCaEtiJ3Xw0QngtDe09gRdy2K0NbY+0rG2j/DDObYGalZlZaWVl5iB9J2pviojyeuOJkrjtzENPmr2bMLTN5aVFF1GWJJKWowuYUdz+e2CmyK83s1EbWbeh6ix9E+2cb3e9x9xJ3LykoKGiqZpHPSE9N4Zozi3nyilPomJXOpX+ey4+eXED1ttqoSxNJKpGEjbuvCs8VwJPErrmsCafACM91vyKuBHrHbd4LWNVEe68G2kUS5phenfj7VSP47qn9mfzGR4y99WXe+HBd1GWJJI0WDxszyzGzvLrXwBjgXeApoG5E2Xhganj9FDAujEobBmwMp9meB8aYWZcwMGAM8HxYttnMhoVRaOPi9iWSMFnpqfzwnME89t3hGMZF98zmZ0+9x6yytVRsqtFQaWnX0iI4ZhHwZBiNnAb81d2fM7O5wGNmdhnwEXBhWP8Z4BygHNgCfAvA3deZ2S+AuWG9n7t73a+SlwMPAB2AZ8NDpEWc2DefZ68ZyX8/u5AHXlvGA68tA6BTh3QGFeVSXJTHoMJcBhXlUVyUR7fcDDQ6X9o6029bMSUlJV5aWhp1GdLGVG7exuI1m8OjirLwelPNnms6XbLTYwFUFAKoMI/ioly65WZGWLnI/jGzeXF/wrJPUfRsRNqNgrxMCvIyOWVgt91t7h5CqIrFazZTVhELoqlvr2JzXAjl52R0CUVNAAAMCUlEQVRQHHpAu3tERXnk52RE8VFEDonCRqSFmRmFHbMo7JjFiOK9Q2jNpj09obI1VSyu2MyTb31MVdzotm65GRQX7h1Ag4py6ZytEJLkpbARSRJmxmGdsjisUxanDtozFN/dWb2xZncA1fWEpsxbSfX2nbvXK8jLjAVQOA03qCiPQYV5dMpOj+LjiOxFYSOS5MyMHp070KNzB04/onB3u7uzancI7bkm9FjpCrbEhVBhXiaDivIYWJhLcV0YFebSRafjpAUpbERaKTOjZ+cO9OzcgVFxIbRrl/Pxhq27e0B1PaL6IdQtNyMWQKEnVPdao+MkERQ2Im1MSorROz+b3vnZjD6yaHf7rl3Oqo1bKauoojycjiurqOL/3vqYzXHXhDpnp1NcmMvA0AOq6w0VdcxUCMlBU9iItBMpKUavLtn06pK9V0+obmBCWUXdNaEqyis288yC1Wzcumdy0bzMNAYW5cYCqDBv9+senTqQkqIQksYpbETaufiBCSOL9x6YsLZqO2UVmymvqNo9OOHFDyp4rHTPXLfZGakMLNxzGq6uN9SrSzapCiEJFDYi0iAz2/13QicP6LbXsnXV22MBFHpD5RVVvFq+life/Hj3OplpKQwoqDsNFzst178gh+6dssjL0gi59kZhIyIHLD8ng6H98hnaL3+v9o1bd1AeTsPVnZIrXbaeqW/vPRduXmYaPTp3oHvnLLp36kDP8Ny9cxY9O3fgsE5ZZKaltuRHkgRT2IhIs+nUIZ0T+nThhD5d9mqv2lZLeUUVH63bwqoNW1m9YSurNtawasNW5q/cyLrq7Z/ZV7fcjFggdaoLpPhw6kBBXqZO07UiChsRSbjczDSO692Z43p3bnD51u07Wb1xK6tDANU9r9pYw9LKamaVrd3rD1gB0lKMoo5Z9KjXK+reKRZQPTp3oEt2ukbQJQmFjYhErkNGKv0LculfkNvgcndnU01tLJA21PDxhq17vX57xQaee7eG7Tt37bVdVnoKPUIQxZ470KNTFoUdM8nPyaRrTgb5ORlkZ6QqlBJMYSMiSc/M6NQhnU4d0jnysI4NrrNrl7O2ehurN9SweuNWPt5Qw+rQS/p4w1ZeLqukYvM2GproPjMthfwQPPk5GSGEMumam0GX7NCWu2dZx6x0Dfc+QAobEWkTUlKMwrwsCvOyOHYfp+t27NzFmk01VG7exrrq7XxavZ111dtZH/f60+rtLPu0mnVV2z9z6q5OaorRJTs9Lpwyyc/JoMvuoArPIaC6ZGeQnhrJjZGThsJGRNqN9NSU3X/Yuj9qduxk/ZbtfFoVC6K6MNoTTrHQWvjJJtZVb2fDlh373FfHrDS65mbu1XvKy0ojJzON3PDIqfcce51KTmYamWkprfpUn8JGRGQfstJTw4CDDvu1fu3OXWzYuiMWSrsDahvrqnewrnrb7t7TinVbeHvFBqpqatm6o+HeU33pqUZOZho5GSGIdgdVaqwta09Q5WSmkbf7deru4KpbHsU1KoWNiEgzSUtNoVtuZuwuq0VNrw+wc5dTvb2WqppaqrfVUrWtluptO6natoOqbTt3t8Xa937euHUHqzZsjb2vqaVqe22D16TqSzHIydgTRv/15WM4qX/XQ/vwTVDYiIhEKDXF6JiVTsdmmFXB3dm6Y2csnGrqQqv2M4FVva2Wzbtf72yRGR0UNiIibYSZkZ2RRnZGGoV5UVezt/Y9PEJERFqEwkZERBJOYSMiIgmnsBERkYRT2IiISMIpbEREJOEUNiIiknAKGxERSTjz/ZnboB0ws0pgedR1HKJuwNqoi0gi+j720HexN30fezuU76OPuxc0tZLCpg0xs1J3L4m6jmSh72MPfRd70/ext5b4PnQaTUREEk5hIyIiCaewaVvuibqAJKPvYw99F3vT97G3hH8fumYjIiIJp56NiIgknMJGREQSTmHTBphZbzObYWYLzew9M7sm6pqiZmapZvaWmT0ddS1RM7POZjbFzD4I/48Mj7qmqJjZdeHfyLtmNtnMsqKuqSWZ2Z/MrMLM3o1ryzez6WZWFp67JOLYCpu2oRa4wd0HA8OAK81sSMQ1Re0aYGHURSSJW4Hn3P1I4Fja6fdiZj2Bq4ESdz8aSAUujraqFvcAMLZe2w+AF9y9GHghvG92Cps2wN1Xu/ub4fVmYj9MekZbVXTMrBdwLnBf1LVEzcw6AqcC9wO4+3Z33xBtVZFKAzqYWRqQDayKuJ4W5e4vA+vqNZ8PTAqvJwEXJOLYCps2xsz6Ap8H5kRbSaRuAf4fsCvqQpJAf6AS+HM4rXifmeVEXVQU3P1j4PfAR8BqYKO7/yPaqpJCkbuvhtgvrkBhIg6isGlDzCwXeBy41t03RV1PFMzsPKDC3edFXUuSSAOOB+52988D1SToNEmyC9cizgf6AT2AHDP7ZrRVtR8KmzbCzNKJBc3D7v5E1PVE6BTgS2a2DHgEGG1mf4m2pEitBFa6e11Pdwqx8GmPzgQ+dPdKd98BPAGcHHFNyWCNmXUHCM8ViTiIwqYNMDMjdk5+obv/Iep6ouTuP3T3Xu7el9jF3xfdvd3+9urunwArzOyI0HQG8H6EJUXpI2CYmWWHfzNn0E4HS9TzFDA+vB4PTE3EQdISsVNpcacAlwALzOzt0PYjd38mwpokeVwFPGxmGcBS4FsR1xMJd59jZlOAN4mN4HyLdjZtjZlNBk4HupnZSuAm4NfAY2Z2GbFAvjAhx9Z0NSIikmg6jSYiIgmnsBERkYRT2IiISMIpbEREJOEUNiIiknAKG2kXzMzN7H/i3t9oZj9rpn0/YGZfbY59NXGcC8OszTMaWDbIzJ4xs/KwzmNmVmRmpx/szNdmdq2ZZR965SIKG2k/tgFfMbNuURcSz8xSD2D1y4Ar3H1UvX1kAdOITUkzMMz+fTdQcIjlXUtsssr9doCfR9oRhY20F7XE/oDvuvoL6vdMzKwqPJ9uZjNDL2Gxmf3azL5hZm+Y2QIzGxC3mzPN7JWw3nlh+1Qz+52ZzTWz+Wb23bj9zjCzvwILGqjna2H/75rZb0LbfwAjgD+a2e/qbfJ1YLa7/72uwd1nuPu78SuZ2c/M7Ma49++aWV8zyzGzaWb2Tmi7yMyuJjZ/2Iy6npSZjTGz2Wb2ppn9LczFh5ktM7P/MLNZwIVmdrWZvR8+8yNN/HeRdkIzCEh7cicw38x+ewDbHAsMJjYt+1LgPncfarEb1F1F7Ld/gL7AacAAYj+gBwLjiM0sfKKZZQKvmlndLMNDgaPd/cP4g5lZD+A3wAnAeuAfZnaBu//czEYDN7p7ab0ajwYOZeLRscAqdz831NDJ3Tea2fXAKHdfG3qEPwHOdPdqM/s+cD3w87CPGncfEbZfBfRz921m1vkQ6pI2RD0baTfCTNgPEruB1v6aG+4XtA1YAtSFxQJiAVPnMXff5e5lxELpSGAMMC5MITQH6AoUh/XfqB80wYnAS2GyyFrgYWL3o0mkBcR6Zr8xs5HuvrGBdYYBQ4gF5tvE5tDqE7f80bjX84lNj/NNYj1KEYWNtDu3ELv2EX9Pl1rCv4UwQWNG3LJtca93xb3fxd5nBurP++SAAVe5+3Hh0S/u/inV+6jP9veDxHmPWE+oKbs/Z5AF4O6Lw/YLgP8Op+waqmt63GcZ4u6XxS2P/zznEutFngDMCzcqk3ZOYSPtiruvAx4jFjh1lrHnh/X5QPpB7PpCM0sJ13H6A4uA54HLw+0f6kaMNXXjsjnAaWbWLVxs/xows4lt/gqcbGbn1jWY2VgzO6beessItxcws+OJ3del7tTdFnf/C7Gbi9XdgmAzkBdevw6cEk4PEmZOHlS/EDNLAXq7+wxiN7DrDOQ2Ub+0A/qNQ9qj/wEmxr2/F5hqZm8Quwf7vnodjVlELBSKgO+5e42Z3UfsVNubocdUSRO33HX31Wb2Q2AGsd7EM+7e6JTv7r41DEq4xcxuAXYQO5V1DbFTd3UeZ89pvbnA4tB+DPA7M9sVtr08tN8DPGtmq919lJldCkwO158gdg1nMXtLBf5iZp1C/Te389tQS6BZn0VEJOF0Gk1ERBJOYSMiIgmnsBERkYRT2IiISMIpbEREJOEUNiIiknAKGxERSbj/D4Ho5HtzQHXCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "\n",
    "# Importing the dataset\n",
    "dataset = pd.read_csv(\"Data/Unsupervised/Mall_Customers.csv\")\n",
    "X = dataset.iloc[:, [3, 4]].values\n",
    "\n",
    "# Using the elbow method to find the optimal number of clusters\n",
    "from sklearn.cluster import KMeans\n",
    "wcss = []\n",
    "for i in range(1, 11):\n",
    "    k_means = KMeans(n_clusters = i, init = 'k-means++', random_state = 42)\n",
    "    k_means.fit(X)\n",
    "    wcss.append(k_means.inertia_)\n",
    "plt.plot(range(1, 11), wcss)\n",
    "plt.title('Elbow Method')\n",
    "plt.xlabel('Number of Clusters')\n",
    "plt.ylabel('WCSS')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJztnXuYFOWV8H9nBhCY0agRER0IoCYajCKMCl42RFTiJahoVgZcickq7pqokMToijdQo5/fA5pNYj40UVcnoAJulEgErzHegBEkeIt4C6MIatQICAMz5/vjrXJqeqq7q+/V0+f3PP309FvVVae6evq85z03UVUMwzAMI5GqUgtgGIZhxBNTEIZhGEYopiAMwzCMUExBGIZhGKGYgjAMwzBCMQVhGIZhhGIKwigJInKViNxdajnyhThuF5GPRWRpqeUxjHxgCsIoGCIyQUSWi8hGEVknIotE5Mg8Hn+giKiIdMvXMXPgSOBYoE5VDy3GCUXkbRE5phjnMioTUxBGQRCRqcBNwHVAX2AA8Gvg5FLKFSTPiuUrwNuquimPxywLPOvJfku6IHZTjbwjIl8CpgPnq+oCVd2kqttU9UFV/WnI/qNEpDlh7IvZsYgc6lki/xSR9SIy09vtz97zJ56VMtLb//si8oq33POwiHwlcFwVkfNF5HXgde/HbZaIbBCRT0VklYgckOS69hSRB0TkHyKyRkTO8cZ/ANwGjPTkuDrJ+8/x5PpMRF4WkWEBmfYJ7HeHiFzj/b2biCwUkU+88z4lIlUichdO6T7onfNib/+xIvKSt/8TIrJ/wmf6U+8aN4nIb0Wkr2fZfSYij4jILoH9R4jIM96xXhSRUYFtT4jItSLyNLAZGCwi3xORN71jvSUiE8M+B6OMUFV72COvD+DbwHagW4p9rgLu9v4eBTQnbH8bOMb7+1ng37y/a4ER3t8DAQ2eBzgFWAPsD3QDpgHPBLYrsATYFegFjAGagJ0B8d7XL4nMT+KsoJ7AUOADYLS37XvAX1Jc73eBd4FDvPPsA3wlINM+gX3vAK7x/v458Bugu/c4CpDEz8h7/VVgE26pqztwsfdZ9Ajs/xzOotsL2AC8ABwM7AA8Blzp7bsX8BFwAm4ieaz3uo+3/Qng78AQ73P+EvBP4Gve9n7AkFJ/F+2R28MsCKMQfBn4UFW35+l424B9RGQ3Vd2oqs+l2Hcy8HNVfcU7/3XA0KAV4W3/h6p+7h17R2A/3A/vK6q6LvGgItIf52f4mapuUdWVOKvh3yJew78D/0dVl6ljjaq+E+F923A/tl9RZ4U9parJCqidAfxRVZeo6jbg/+KU4OGBff5bVder6rvAU8DzqrpCVbcC9+OUBcCZwEOq+pCqtqnqEmA5TmH43KGqL3mf83agDThARHqp6jpVfSnKB2PEF1MQRiH4CNgtj2v8P8DNjl8VkWUiclKKfb8C3Owti3wC/AM3Y98rsM9a/w9VfQz4JfArYL2IzBaRnUKOuyfwD1X9LDD2TsJxU9EfeCPivkFuxFkBi73lm0tS7LunJxMAqtqGu9agjOsDf38e8rrW+/srwHf9z9H7LI/EKSuf4Oe4CaegzgPWicgfRWS/iNdoxBRTEEYheBbYglvuicImoLf/QkSqgT7+a1V9XVUbgN2BG4B5IlKDW5pJZC0wWVV3Djx6qeozgX06vE9Vf6Gqw3HLJV8FOvlJgPeAXUVkx8DYANyyURTWAnsn2baZwPUDewRk+0xVf6yqg4HvAFNFZHTYdXgyBv0tglNMUWVMlPeuhM+xRlWvD+yT+Dk+rKrH4pTIq8CtWZzXiBGmIIy8o6qfAlcAvxKRU0Skt4h0F5HjReT/hLzlb0BPETlRRLrj/AY7+BtF5EwR6ePNiD/xhltxPoA2YHDgWL8BLhWRId57vyQi300mq4gcIiKHeefdhFNsrSHXtBZ4Bvi5iPQUkQNxlk1jtE+F24CfiMhwzzG+T2DZayUwQUSqReTbwDcD8p3k7Su4Nf7WgHzrE679XuBEERntXc+Pga2e3JlyN/AdERnjydVTXDBBXdjOnrN7rKe4twIbCfkcjfLCFIRREFR1JjAV92P/AW5G+kPgf0P2/RT4T9yP6Lu4H+pgVNO3gZdEZCNwMzDe8wNsBq4FnvaWQUao6v04K2OuiPwTWA0cn0LUnXAz3Y9xyzMf4dbuw2jAOcbfw63XX+mtzadFVe/zZP098Bnuc9jV23whzjr4BJhIx89oX+AR3A/us8CvVfUJb9vPgWnetf9EVV/D+Q7+G/jQO+Z3VLUliowJ8q7FhST/F+3376ck/82owimk93DLet/E3VOjjPGjIQzDMAyjA2ZBGIZhGKGYgjAMwzBCMQVhGIZhhGIKwjAMwwglDlUws2a33XbTgQMHlloMwzCMsqKpqelDVe2Tbr+yVhADBw5k+fLlpRbDMAyjrBCRKGVebInJMAzDCMcUhGEYhhGKKQjDMAwjlIIpCBH5nbgmLKsDY7uKyBIRed173sUbFxH5hbgmLKv8RiqGYRhG6SikBXEHroZOkEuAR1V1X+BR7zW4Wjn7eo9zgVsKKJdhGIYRgYIpCFX9M65oV5CTgTu9v++kvRz0ycD/eI1UngN2FpF+xIRWYCEww3u2EpWGYVQCxQ5z7et361LVdSKyuze+F4HmI7hKnnsBYZ29zsVZGQwYMKCw0uKUwRjgeVyJ0RrgMOBhoLrgZzcMwygdcXFSS8hYaJlZVZ2tqvWqWt+nT9o8j5xZhFMOGz2BNnqvFyXsZ1aGYRhdjWJbEOtFpJ9nPfTDNU0HZzH0D+xXh6srX3JW4CyHIJtwHV78vpdmZRiG0RUptgXxADDJ+3sS8IfA+FleNNMI4NOwxvGl4GDcD36QGmBo4HVUK8OoNLqqXdlVr8tIpGAWhIjMAUbhmtc3A1cC1wP3isgPgL8DfivIh4ATcM3ZNwNnF0quTDkeZw0kWgfBFmVRrAyj0uiqdmVXvS4jjIIpCK/JfBijEwfUtbU7v1Cy5EI17qu/CPeDPxSnHIL/Cr6VsTEwlmhlGJVG0K6EjnZlOU8buup1GWHExUkda6pxX/1p3nPiPMm3Mmpx3vZaOlsZRqWRyq4sZ7rqdRlhlHU117gQxcowypVW3J1dgbMVo97ZQtqV2cqUD8xeriRMQeQJ38owI7srkct6exTvVbFlygeFui4jjpiCSEIp52hGXMhlvb1QdmWpfQBmL1cSpiBCKPUczYgLucanFcKujEPMnNnLlYI5qUOwvAbDESULxieX3IBM3puJTIaRG2ZBhBCHOZoRB6Kut+dic2b6XvMBGMXDFEQIcY3TyNQvYn6UXIm63p6LXyDT9wZlegHY7o0tSiKbYWSPKYgQ4jhHy3SeaX6UfBFlvT0XmzOb91bjvo03YXfYKCTmgwjBn6PNAaZ7z6X+t8vUL2J+lGzI1o+Qi18g2/faHbaaUIXHLIgkxC1OI9N5pvlRMqVUOQ/ZvrfS77DZyMXAFESZkKlfJK5+lPhSyJyHVN6gbPMKcr3D5e6hKnU+SGVgCqJMyHSeGUc/SrwpVM5DlJluNvZqLne4K8y+K92CKg6mIMqETOeZlu+aKYWyuRYBz9H+Y7bRe53rTDeXO9wVZt9mIxcDUxBlRKbzzLj5UeJNoWyuJsJnui+Q+53J9g53hdm32cjFwBSEYQCZzcgzWb9PFlmzPRdhcySb2XfcfBZmIxcDUxCG8QVRZuSZrt8niyQv5Q9ZprPvuPoszEYuNKYgYkDc5mZGKtKt3yfezWSWQilj9jOdfXcFn4WRDaYgSkxc52ZGMlKt3x9P57vZJ8lxSv2vl8nsuyv4LIxssEzqEmP5sOVGqsznsLu5DtghZP9heZarUFnFrcA2oHvCuEUMVQKlnsZUPDY3KzdSrd9fR+e7uQUYDGwI2T9fFMoO9Y/7HNASGLeIoUrBFESJsWjuciPV+n3Y3awFZnrbCxVtUygfgX/coNLrAUwFriRaKXPzrpUzpiBKjEVzlyPJ1u+T3c2TAu8pBIWyQ8OOuw2nJPLd58KII6YgSoxFc3clSnU3C2WH5nJci3zqCpiCiAEWzd2VKMXdLJQdmstxzbvWFTAFYRhFpRDr8oWyXHI5rnnX8ktp/DmmIAyjaORnXb6trY05c+Ywa9Ys1q5dS//+/ZkyZQoNDQ1UVeV7dp6tRWTetfxROn+OqGpBT1BI6uvrdfny5aUWI2ssxqPSWAg00DnKaQ5Rf4Db2toYN24cjzzyCJs2tS/h1NTUcOyxxzJ//nyqquKS3uR/w827lhu5f28SEZEmVa1Pt19cvkkVhz8naMAFDDZ4r61pYlcm1bp8NObMmdNJOQBs2rSJJUuWMHfu3JylzB++9TGN9kguI3Ny/95kS0UriFJ2tM0mg9o68JY7ufSudsyaNauTcvDZtGkTM2fOzFo6I67k/r3Jlsg+CBGpAbaoapf4XSp1lHamMR6lltfIB7mvy69duzbl9ubm5uzFM2JK6fw5SS0IEakSkQki8kcR2QC8CqwTkZdE5EYR2Tfbk4rIFO84q0Vkjoj0FJFBIvK8iLwuIveISI9sjx+FYtVAamtro7Gxkfr6evr27Ut9fT2NjY0c1NaW0ZzAajZ1BfyooDnAdO85MxXfv3//lNvr6uqyF8+IKbl/b7JGVUMfwJPA5cCBQFVgfFfgNGA+cGay96c47l7AW0Av7/W9wPe85/He2G+A/0h3rOHDh2u2TFdVSTigqOqMrI/YmdbWVj355JO1pqZGcb/rCmhNTY2efMopenRrq9Z6561V1dGqur2E8hrx5+677+70fQp+rxobG0stolEGAMs1wu91Kh/EMep+f1apaltAofxDVeer6mnAPVnqpW5ALxHpBvTGlbw8Gpjnbb8TOCXLY0cim1W9TH0AqRyKjyxZwtlz50aeE5RuFdKIEw0NDRxzzDHU1HT8NvhRTOPHjy+RZOWKefZSkkp7AIJb7BoHnOr9nTiRzfgBXIhbJfkAaAR2A9YEtvcHVid577nAcmD5gAEDstag29XN2KPO4DPdX1V1+PDhoTM9/5GJBZTN+Y2uSWtrqzY2Nurw4cO1b9++Onz4cG1sbNTW1tZSi1ZmVO5/FREtiKR5ECJyHPBr4HXgXW+4DtgH+E9VXZyNQhKRXXDLU2cAnwD3ea+vVNV9vH36Aw+p6jdSHSvXPIhMorSziUTu27cvGzZsSHr+vn378v777xdEXiNOWMZLPMl/fkG5EDUPIlUU0824Zaa3Ew48CHgI2D9L2Y4B3lLVD7zjLQAOB3YWkW6quh2niN7L8viRKXRPrf79+6dUEJk6FK1mUzli8WfxxepFpSOVD6IbEBYz9y6d20tlwt+BESLSW0QEGA28DDwOnO7tMwn4Qw7nyIgoq5BhPoDepPYBTJkypdNasU9NTQ1Tp07NXFijSORrbdriz+KLefbSkcqC+B2wTETmAn7wdX9gPPDbbE+oqs+LyDzgBVxH9xXAbOCPwFwRucYby/ocmRB1fnccnbViD288GQ0NDdx3331JyyKYQzGu5HPWb7PU+GL1otKRshaTiHwdGIsLTRWcRfGAqr5cHPFSk49aTGGrkD1xcbzjaV8tXui9Dv6r1wBzSf1v3tbWxty5c5k5cybNzc3U1dUxdepUxo8fH6OaOUZH8rk2HXasKN8cozhUpmcvHz4IPEXwsojs6l7qx/kSMC6Eze+24EKr/kD7vHEFsDlhv82knwdWVVUxYcIEJkyYkB+BjSKQz1l/NranUTzMs5eKVJnUA0RkrpdF/TywVEQ2eGMDiyVgoQlbhfQJrhaX02qlRXbnStjdrgJWAy0ZHmtxyHtavPFMSbyzLdidNgpJKgviHuAmYKJ69ZdEpBr4Ls4+HlF48QpPcBVyY8h2f954KeWxWmkxM/nA/1Y8S7vd2Ir7l1gMvI+zAqKQre2ZSOKd7e3JsA2700ahSLUIvpuq3qOB4nyq2qqqc4EvF1604hCscnImzv8QxLcSSlgNJSMsZiYf+Hf7OyHbPgauiXicVtwPeOISUza2Z+Kd3eTJYne6dHR9Wz2VBdEkIr/Glb0IRjFNwk2Lugz+KuTxuJofyayEclittJiZfFGNS/QP45kI7/dn/M/RcYkpW9sz7M4mYne6eFSGrZ5KQZwF/AC4mvYoprXAgxQpBLXY5Kuzrx8X0eT9XQ0Mj3isKDm3qfaxTsD55AjgsZDxEbgZY6q75M/4gz/q/hTjf0L2T0fYnU3E7nTxCFp00NGC6zoKOqmCUNUW4BbvUTHkaiUE543Bn4Za0s8vosxJ0u1jkd35ZBrwS9xSjs/OOAviZlLPHMNm/K3A/cAJIfunI/HO+j6IFpxPw+50cakMWz2rQHwRuSLfgsSZTFYaw+aNEG2FOIr/IN0+5eIrKQ964BzSl+MS/i/Hyx+l4x14GrfyGvx2JIuPayE7X0HinZ3ryTYXu9OloJziGrMncke5BP4d963s8mS60phqpTjd/CLKnCTKPuXgKykfetDxqz6DaJkz/oz/KTqHuWY70wy7s3anS0Nl2OpJFYSI/DPZJqBXYcSJH5muNKZaKU43v4jiPzAfQykJRiWF5UMkfjsexrnwbqCzo9ruWCZs27aN5uZmtmzZUmpRAvwCNzlowU0kegJ/K6lEifTs2ZO6ujq6d8+ufF4qC+IT4BBVXZ+4QURSN8btQmS60ujPKxJ9EFHmF1HmJJUxb4kjyaKSEgl+O6qBK3E+C7tjudDc3MyOO+7IwIEDcTU+jXSoKh999BHNzc0MGjQoq2OkUhD/A3wF6KQggN9ndbYyJNMZezASyq9G2A0YRvoopihRVPmKtDIyJcy71A1nUG8LjCV+O+yO5YMtW7aYcsgQEeHLX/4yH3yQLFw7PamimKal2PazrM9YZmQzY8/FBxDlveZjKAVhtuR2YDCwgdTfDrtj+cCUQ+bk+pllFMUkIlfldLYyxKKCDEdY1EotMJN4fDvimNUbR5my5/3332f8+PHsvffefP3rX+eEE07gb3/7G2+//TYHHHBA5ONcdtll9O/fn9ra2gJKmx8yjWIaC1xVADlijc3/jOS2pO9rKOW3I45ZvXGUKXtUlVNPPZVJkyYxd+5cAFauXMn69evp379/Rsf6zne+ww9/+EP23XffQoiaVzJVEGbjGRVKnH0JccjqTczvby2pTPnuAv7444/TvXt3zjvvvC/Ghg51vqa33377i7GXXnqJs88+m5aWFtra2pg/f34nRTBiRPnUOc1UQQwviBSGURbE1ZYsdVZvmLWwe8lkKoTtsnr1aoYPT//z95vf/IYLL7yQiRMn0tLSQmtreS+rZeSDUNU2qLxMasOIN6XO6g3L738P2KEkMpWyovHIkSO57rrruOGGG3jnnXfo1au8U8ay7Xn573mVwjCMHPD9I7W4VWC/8lexci2S9WXcsyQypbKnsmXIkCE0NTWl3W/ChAk88MAD9OrVizFjxvDYY2HFHsuHVB3l/pnk8RnuzlcsXSs2wyh/Sh1rF68Ir0LYU0cffTRbt27l1ltv/WJs2bJlPPnkkx32e/PNNxk8eDAXXHABY8eOZdWqVTmctfSksiA+AfZV1Z0SHjvi2iZUJP76ZgMuR7bBe21Kwigtvn9kGu2RVcUimQXj+2uKK1Mh7CkR4f7772fJkiXsvffeDBkyhKuuuoo99+w4V77nnns44IADGDp0KK+++ipnnXVWp2NdfPHF1NXVsXnzZurq6rjqqqtykKywiKqGbxC5BnhAVZeGbLshDsly9fX1unz58qKecyFOKQQzq2tx86O4uS6D5DuqwzA64n/DChPh9corr7D//vvHRJryIuyzE5EmVa1P917LpM6QUseLZEPXikg34km8IrziJU35ksoHMTDVG8VRl2+B4k6p40WywfpUG4aRDal8EDeKyHwROUtEhojI7iIyQESOFpEZuC4p0W2+LkKp40WyoRBRHYZhdH1SLTF9V0S+DkwEvg/0w/U2fAV4CLhWVeNUnL0oxDmfNhnWQ6KrYp4lo7CkzKRW1ZeBy4okS9lQbuub1kOiK2KeJaPwZNty1CgjytHqMdIRh/pLRlcn20xqo8woZZS8UQjMs1Rs8lHue/PmzZx44onst99+DBkyhEsuuaTAUueGKYgSUsyM7NZWWLgQZsxwz2VeQ8woy3i68sUv9z1q1CjeeOMNXn75Za677jrWrw9ruJman/zkJ7z66qusWLGCp59+mkWL4htPGHmJSUT2ANZrssw6IyOKuYLc2gpjxsDzz8OmTVBTA4cdBg8/DNVmSpQp5llKRWsrLFoEK1bAwQfD8cfn9l3PV7nv3r17861vfQuAHj16MGzYMJqbm7MXrMBEUhAisgvwJi6J+A+5nlREdgZuAw7AheZ/H3gNuAcYCLwN/KuqfpzrueJKMVeQFy1yymGjd7KNG93rRYvgpJPa/5mamtzf1dUwfHju/1T5IN//6F0H8ywloxATokKU+/7kk0948MEHufDCC7MTqghEtSAmAktwVVxzVhDAzcCfVPV0EekB9Ab+C3hUVa8XkUuAS4Aum7FdzIzsFSvcP0qHc22ClSvdD+6YMfDccx33qa0tvZVhlk86yi2erjikmxAVkpEjR3LttdfS3NzMuHHjknaN2759Ow0NDVxwwQUMHjy4sELlQFQfxNnAD4H+ItIvlxOKyE7AvwC/BVDVFlX9BDgZuNPb7U7glFzOE3eKuYJ88MHux7XDuWpg6ND2f6ZEBRL8p0ok6M9YsACmTYPRo+GKK6ClJX9yB//RVaPLlOhjMf9LZZFqQpQt+S73fe6557Lvvvty0UUXZS9UMVDVlA+gHnjY+3sq8F/p3pPmeEOBpcAduIn0bbjfxk8S9vs4yfvPBZYDywcMGKDlynZVHa2qtaoq3vNobzzv59quOnq0am2tqoh7Hj3ajU+f7sbcT3DHh4jqjBnJjxX2nl12Ud26NT9yh8mWTqbE60u1zSgfXn755cj7Pvhg5+9nba0bz5a2tjY99NBDdfbs2V+MLV26VJ944gl96623dMiQIaqq+sYbb2hbW5uqql544YU6a9asTse67LLLdNy4cdra2pq9QBkQ9tkByzXC73UUC+IHeLN94C7gzBx1UjdgGHCLqh6MW1mJHOulqrNVtV5V6/v06ZOjKKWjmBX8q6vdssycOTB9unv2l2nCrAsf38oIkmi+J/Lxx3DNNbnL3NoK27ZB9+6ZyZRoaeTLCjHKh+OPd0uRtbUg0r5cenwO/vt8lftubm7m2muv5eWXX2bYsGEMHTqU2267LXvBCk0q7YHzDbwFdA+M3Q+MiqJ9khxzD+DtwOujgD/inNT9vLF+wGvpjjV8+PB8KNiKxp9h19R0nHHV1ITPtFNZHP5j9OjSy+RbGvmwQozSk4kFoeru24MPuvv84IOVfR9zsSDSOam3AYep6rbA2KQcFdL7IrJWRL6mqq8Bo4GXvcck4HrvOR/OcCMNvnWxaBG88AJs3w7dusGwYeERQ77FkcyCADj88PTnTRWdFOYX6dEDpk6FK6+MJlPQ0gjb9o1vOCvBP39ra+kcm0b+qa52983uXW6kq8W0DdjgvxaRk1R1YR7O+yOg0YtgehPnBK8C7hWRHwB/B76bh/MYEcjkn8k335MtM+2yi3NapyJddFKYk3HbNqckwqKXgjIFj+cvKSRuO/RQ+MUvYOnS9rHdd0/u2LQfGaNSybQW03Rc0m9OqOpKnPM7kdG5HtsoHP6s/6ijnJXQrRsccICzPJ57zo1Nm+Z+xIOz80RLJF0YYjqLIJHqanjoIef7ePppOOKIdjmg3UJaudIdo7UVJk5sVwgbNzoFtMMOsCVQnzjVOQ2jEshUQUhBpDBiT7JZ/2WXwbhx6fcL5i6kCkM86aT0FkGYbCec0L7/0qXwzDPt50y0kK6+uvP5t26FQYPggw+indMwKoFMazFNLogURtHINlJn0SJnJQSjgZ57rj0ayD/upEluFp8qaigscqp37/bZum8RTJkC3/qWe37oIbctTPawSKWgbGGfQRgTJoRHehlGpZKRBaGqSwsliFF4cslMbmoKn/W/8EJ7NnYyv0TiWv5xx3UOX+3Rw437ciZaBE8/7bYF/Qa+7E1Nnc/ryxbmP0h2rT16mGPTMIJYNdcKIl1OQLps5DC2b0+fG5G4lr94ceeM661b3XgyOZ95xj3CZE8m22OPhVtJw4e72PhEGYcNCz+OYUB+yn0DfPvb3+aggw5iyJAhnHfeeSnrNZUaUxAVRKq1f9+6aGhwoaQNDe61/91NNuvu1i38uD5hSUph1sjmzW7Gn0zOLVs6OpCDsieT7cknO18HhCdSjRhh/gYjOar5K/d977338uKLL7J69Wo++OAD7rvvvgJInB/SLjGJiOCK9Q1W1ekiMgDYw5abyo9U0UHpIov8WXfie/1Zd+Jxe/aE00+HM87oHMWUyhpJJmfPnu45WZRRTU24kgrLZwjmfviRTVYltquR337d+Sr3DbDTTjsBrmBfS0sL7ic2nkSxIH4NjMSV+gb4DPhVwSQyCkaqEgTpCpylmnWHbTviCLjjDvejnPjDm+z/oaoq+bkOP9w9wmQ//ngnS+KyUdh1+PiRTdOmhctolDN+t5UG4ErveQy5tOTKtNz3ypUrWb58OXV1daH7jRkzht13350dd9yR008/PWu5Ck0UJ/VhqjpMRFYAqOrHXoKbUWYkmzlDe92joG8gOEMPy7iurnavjz8+sxm5Jmk51daWXs5057/nHpg3r6OlEYyQMiqB0vXrjlru++GHH2bLli1MnDiRxx57jGOPPbagcmVNulocuE+2GnjBe90HWBGljkehH1aLKXcyrXuUj5pFV1wRXsPpyiujy5vs/Fu3uoqyhaowa5SGzGoxTVdXIzn4cyGqOiPVm1LyyCOP6FFHHRW6LVjNVVV1zZo1evPNN+ugQYP00UcfTXncO+64Q88///ys5YpCoau5/gJXoG93EbkW+AtwXUG0lVF0UtU9Sgx/bW11SWZPPdUxmujpp13+Q9S8inQWRBR5k0VihUVItbS0R0gZlUD+u60cffTRbN26lVtvvfWLsWXLlvHkk0922O/NN99k8ODBXHDBBYwdO5ZVq1Z12L5x40bWrVsHOB/EQw89xH7OWvGVAAAgAElEQVT77Ze1XIUmrYJQ1UbgYuDnwDrgFFWNr9vdyIiodY/8KKcbbuj8A7xlCzQ2hkcMhZHOB5GpvEEfw4oVLiIqyObNuTWLMcoNv193La74Qy259uvOV7nvTZs2MXbsWA488EAOOuggdt999w6O77iR0gchIlXAKlU9AHi1OCIZxSRZZFOyaqepOsYFM5hTJZt1tiDagDnMnj2LW25ZS//+/ZkyZQoNDQ1UJWiNgw92PoWgkgj6GMKup1cvJ/eMGdbXujIoTL/uPffck3vvvTd02+rVqwG49NJLufTSS5Meo2/fvixbtiwnOYpJumqubSLyoogMUNW/F0soo3iE1T2KWu00jFQZzD4df5zbgHHAI6xb506wYcMGJk+ezLx585g/f34HJXHccc66SVwS87Oww7K0W1pg1izra11ZWL/ufBAliqkf8JKILMV1fwNAVccWTCqjaIRFDLW2wplndsyJ2L69c7XTqqpwv0FLS+pqrh1zKuYAjxD4agHOFF+yZAlz585lwoQJX4wvXuyWwIJs3gzf/z6MH+9kT7Rytm+3Pg+GkQ1RFMTVBZfCKCmJ1U5nzAjPZB48GDZsaJ+J9+kDb73V+Xi//z3cfHPyGXvHnhKzSFQOPps2bWLmzJkdFERY3aWtW50P5A9/cJZOog+i83Gtz4NhRCGtglDVJ0WkL3CIN7RUVTekeo9R3oSt49fWwsyZ7kc+WV8FcFbGunXtlkayTGa/f8P116/tZBEEaW5u7vA6lQN840Z33urq9qzsMCwvwjCiEaXUxr8CNwJP4EIC/ltEfqqq8wosm1EikvVj8DOO/R/6lpbO/gARN6MPkjhjD1Zr3batP4GmhZ1IzERN5zdIpRh8gj4LwzCSE2WJ6TLgEN9qEJE+uEVjUxBlSKpe0D5RaxWF5Ryopu/M1rHu0xRcm5HOy0w1NTVMnTq1w9jw4c4CSLeMlAo/L8KWmAwjNVES5aoSlpQ+ivg+I2akq9gaJEqtorCqrFu3Qr9+4TWTgu9rX75qAI4hMbGppqaGY489lvHjx3cYP+44p4BywfIijGzIV7lvn7Fjx2b1vmISxYL4k4g8jAs3ATgDF2BslBnpKrYGCbM0/GP4Y8l8BxMmuOJ5yayPjgqpClgAzKVfv5m0tTVTV1fH1KlTGT9+PKpVnfIxEs/bzfsWR1leAus1bWSOeuW+J02axNy5cwFYuXIl69evp3///hkfb8GCBdQmqy4ZI6I4qX8qIuOAI3E+iNmqen/BJTPyTrpe0D5hnecOPdRtC+ZG9OkTfp50ndk6Z1JXARM499wJXHVVajnC8jG2b2+PsErWtMjHek1XCFHWUjMgn+W+N27cyMyZM5k9ezb/+q//mrVMxSCKk3oQ8JCqLvBe9xKRgar6dqGFM/JLqn4Q0P4/NXeuq68UjER65hn3d3Bs+3bXpyHR35CuM1vUWkxhFk9YPkYwwuqee+C++zo7yqG9xtSVV1qSXJcml966Sci03PfEiRNpaWkJ7RZ3+eWX8+Mf/5jevXtnJUsxieJLuA+X7urT6o0ZZUaqfhBB/0RjY+fubWEd3bZsgT33zLwzW6rudEGSdZZLPKcfYXXSSfDVryYvBxJWY8rogqSr6FhARo4cyXXXXccNN9zAO++8Q69evTpsX7lyJWvWrOHUU08tuCz5IIoPopuqfvEvp6ot1g+iPEkVnbRwYeq+0mEd3cJyI6JY8um60/lEzccInjPsPcFzmO+hAoi6lpoBQ4YMYd689IGbEyZM4LDDDuOPf/wjY8aM4bbbbuPoo4/+Yvuzzz5LU1MTAwcOZPv27WzYsIFRo0bxxBNPZCVXwUlXDxxYAowNvD4ZeDRKLfFCP6wfRP6YPt31Vwjr01Bbq3r00e6RSx8In6g9G7LpPZFpfwujPMioH8SDD7ovS+KX+MEHsz5/W1ubHnrooTp79uwvxpYuXapPPPFEh34Qb7zxhra1tamq6oUXXqizZs1KeszEPhKFIpd+EFEsiPOARhH5Jc5JvRY4K/VbjHIjWR/oYF9pyE8f51Q9G4ITvGx6RweztP/yF+fU/trX3K9EsAOdfy158mEacSJZpmcOkQl+ue+LLrqI66+/np49ezJw4EBuuummDvvdc8893H333XTv3p099tiDK664IterKSmiyTyGiTuK1Hr7f1ZYkaJTX1+vy5cvL7UYXYIC+PWSMmOGcxQHv3oiMH26y7vIhcTr6N3b+R22bUsdkWUVXuPNK6+8wv777x/9DX7ERa6zmS5A2GcnIk2qWp/uvUktCBH5Dq4XxDve0FTgNBF5B7hQVUPKtBlxp6XFza6ffhqOOML9IPuO20xn60EyiSoMs1YyqY8UPNeBB7qxlSvd+Jo1HSOwNm3quBydLCLLKrx2MRIrUBpZkWqJ6VpgBICInASciUt7PRj4DTCm4NIZeaWlBfbYAz7+2L1+7DH45S/h/ffblUQ2/1OZWh9hPRui1kcKnmvjRnd81WjtSn0So7HAKrwaRhipwlxVVf2KN+OA36pqk6reBiRJkTLizDXXtCsHn48/duPgfnwXLnRLQFH7S0PmUYWLF3fOU9i6NX3f6MSe2P5YJsoBnG/Fj8rysQgnw+hMKgtCPL/DZmA08OvAtp7hbzHizNNPh48/80xuPohMowqbmsL7RqfqROfL99RTqduepqNHDzj8cPd3og/CsqsNoyOpLIibcA1dlwOvqOpyABE5GFiX64lFpFpEVojIQu/1IBF5XkReF5F7SpZrke00ugw44ojw8cMPzy23yPcpBEk1I0/2kaaqpeTLl4lySGhnTY8e8LOfOUtl0SKYMgW+9S33/NBD+fFhduGvj1GJpIqBBfbC+RyqAmP9gAFRYmjTHHsq8Htgoff6XmC89/dvgP9Id4y850FkE3hfRqTKPwjLgxBRnTEj/XEz/diuuio83+Lqq5OfI1WeRrLHoEHhMhXqNnfxr09JySgPwuhAwfIgVPVd4N2EsXxYD3XAiThH+FQREeBowO8teSdwFXBLrudKSWLoTWtr9HKnZUiPHs4hfc01blnp8MPbo5jS1WlKRaYRUFEzqYOkypDu1s2phOBsvaYGZs0Kz7hOzBrP123OpFpuWdOKq+e8Ajd9PB6ogAjS999/n4suuohly5axww47fJEH0aNHD0466SRWr14d6TijRo1i3bp1X5ThWLx4MbvvvnshRc+eKFok3w9cs6HhwChgIbAbsCawvT+wOsl7z8Utey0fMGBA9mo1bLo3eHD20+gyp5iz31wypHv0CLcWBg+OfrxcrKVUFOq4sWK7qo5W1VpVFe95tDdeQEptQbS1temIESP0lltu+WJsxYoV+uc//znjjOhvfvObumzZskKIGUqhM6nzihcyu0FVm0RklD8csmtoBp+qzgZmg0uUy1qQqKVCKyS8Jdc8iEKfy3/P1VfDDTd09EVkWhMqF2spFYU6bqxYBDwP+Ne40Xu9CIiTlZRnKyef5b7LiSjlvncNGf5MVVO0mk/JEcBYETkBFw21E84hvrOIdFPV7UAd8F6Wx49GslKhfmOBCgxvKWZuUTbnqq52GdjPPJO+X3YqClCJoaDHjRUr6NwddhMunCUuCqIVl6X1PE62GuAw4GGyVhL5LPcNcPbZZ1NdXc1pp53GtGnTkM5NUmJBFAviBdySz8e4mf7OwDoR2QCco6pNmZxQVS8FLgXwLIifqOpEEbkPOB2YC0wC/pDJcTMmm1KhRsnJh6VTKGupmFZYyTgY94Mb9AXVAHGykkpo5YwcOZJrr72W5uZmxo0bF2o9NDY2stdee/HZZ59x2mmncdddd3HWWTEtb5duDQovazrw+jhgJi7L+vko61gpjj2K9iimwcBSYA2u38QO6d6fUxSThZwYRuaUgw9iujrZgr8Woqo5+IIeeeQRPeqoo0K3Jfog1qxZozfffLMOGjRIH3300ZTHvf322/X888/PXrAI5OKDiNIwqF5VHw4olMXAv6jqc0BO7eNV9QlVPcn7+01VPVRV91HV76pqSE+wPOJP9+bMcVXi5szp2tXaLEC/+LTiQjBmeM9d4SOvxi3VzAGme885LN0UBN/KCZKjlXP00UezdetWbr311i/Gli1bxpNPPtlhvzfffJPBgwdzwQUXMHbsWFatWtVh+/bt2/nwww8B2LZtGwsXLuSAAw7IXrACE2WJ6R8i8jPc0g/AGcDHIlJNx05z5UelFPQqZqlWw1GAdfDYUI1bqonrv83xuM868bPPwReUr3LfW7duZcyYMWzbto3W1laOOeYYzjnnnOwFKzBpy32LyG7AlcCROB/EX4CrgU9xCXNrCi1kMkpW7jtd6dI8N0zPWc65c2H+/M7t4ObM6frKMUeyvpULcaUtg2v1tcDduB/YCsshyJXMy33jfA4rcZZDBX/OBSn37aOqHwI/SrK5ZMqhZKSbjcdltp5Y9jQRK1+alpxuZVi0z0Zc/YANdD2rIm7E3copE9L6IETkqyIyW0QWi8hj/qMYwsWSdEWLStgwPamcYXS5AP38E3Yrn34aJk2K4MYJWwfviQve3ojL8glG1xhGDInipL4PNx+aBvw08KhMUpUujbK9WITJ4VNb2wUD9PNPslSZxkZoaHDWRVIl4a+D1+IWZmuBPYHE0As/h8AwYkgUJ/V2VS1sTaRyIl26bFzSaaM0mTYHdUpS1X9KW2fJj/YJroO34tpuxTmHwDACRLEgHhSR/xSRfiKyq/8ouGRxxU+Xra11jZQTZ+PptpdSziOOgDvuaE87NlIS/AjDSGsY+uvg02hfD0+0KnKMrjGMQhLFgpjkPQeXlRSX2FZ5pEuXzSSdtpDRThWR1ltYgh/hPffAvHk5lukKsyriHl1ToZVbDY8o2XRxfeS9H0QxsUzusqIib1eJsqbDKHU1V59169bpGWecoYMHD9b9999fjz/+eH3ttdcyPs7tt9+u7777bsbvu/zyy3XJkiWdxh9//HE98cQTQ99TkExqETnaex4X9iiaBuuqhIXIPPWUK1dqWc6xo9IS74GONY3KKOqqra2NxsZG6uvr6du3L/X19TQ2NtKWafPyBFSVU089lVGjRvHGG2/w8ssvc91117F+/fqMj3XHHXfw3nvh9UiTFfgDmD59Osccc0zG58uWVD6Ib3rP3wl5WHRxroSFyLS0uFrWKcNjjFLhJ95Pm1YhbpxUlVtjSltbG+PGjWPy5Mk0NTWxYcMGmpqamDx5MqeddlpOSiJZye+jjjqKG2+8kUMOOYQDDzyQK6+8EnBlwPfff3/OOecchgwZwnHHHcfnn3/OvHnzWL58ORMnTmTo0KF8/vnnDBw4kOnTp3PkkUdy3333sXLlSkaMGMGBBx7IqaeeyscffwzA9773PebNmwfAn/70J/bbbz+OPPJIFixYkMOnlpykCkJVr/Sezw55fL8g0lQSYY2cwSmJUuRNGEYiBahpVGjmzJnDI488wqaEydemTZtYsmQJc+fOTfLO9CQr+b148WJef/11li5dysqVK2lqauLPf/4zAK+//jrnn38+L730EjvvvDPz58/n9NNP/8KqWbly5Red5Xr27Mlf/vIXxo8fz1lnncUNN9zAqlWr+MY3vsHVV1/d4ZxbtmzhnHPO4cEHH+Spp57i/fffz/q6UpFqiWlqqkdBpKkk/BCZHj06bytF3oRhJBKWyxHzqKtZs2Z1Ug4+mzZtYubMmXk/5+LFi1m8eDEHH3www4YN49VXX+X1118HYNCgQV80Fho+fHiH5kKJnHHGGQB8+umnfPLJJ3zzm24RZ9KkSV8oHJ9XX32VQYMGse+++yIinHnmmXm/LkgdxbSj9/w14BDgAe/1d4A/h76jksk0IilVizTLci49Fr1TllFXa9euTbm9ubk562MPGTLki+WdIKrKpZdeyuTJkzuMv/322+ywQ3vB6+rqaj7//POkx68JW1FIQTGaDKVaYrpaVa/G9Ysepqo/VtUf43pJ1xVcsnLCL9rT0OBanqVNs/XwW6QddVTp8yaMdvxKrA24MpUN3utKdAsl5nLEWDkA9O/fP+X2urrsf7qSlfzeaaed+N3vfsdGL6Py3XffZcOGDSmPteOOO/LZZ5+FbvvSl77ELrvswlNPPQXAXXfd9YU14bPffvvx1ltv8cYbbwBuaa0QRMmDGAAEpre0AAMLIk25EtbfOmWabYBi5SvEpcJsOVAufZeNTkyZMoXJkyeHLjPV1NQwdWr2q+OhJb/3GshNV9zEzjvszMiRIwGora3l7rvvpjrF/9f3vvc9zjvvPHr16sWzzz7bafudd97Jeeedx+bNmxk8eDC33357h+09e/Zk9uzZnHjiiey2224ceeSRrF69OutrS0q6OFjgMuBF4CrcfGol8F9RYmgL/YhNHsT06S443gWsuoeI6owcWljlk4oM4s+BAnQkM3Ijah5Ea2urnnzyyVpTU6O44FwFtKamRk855RRtbW3Nj0Btqvqqqjap6jLv+VVvPGYUtKOcql4LfB/Xk/oT4GxVvS7/qqqMCYtIipMfIS4VZsuFcone6Yod63KkqqqKBQsWMHv2bIYPH07fvn0ZPnw4s2fPZv78+VRVRakuFIFPcSG/ftRsm/f60/wcPi5EWWICZzWs8/cXkQGq+veCSVVu+BFJiY0D4uJHSFVh1vpBdKYAHcnyTlfuWJcjVVVVTJgwgQkTJhTuJJvp3E+zzRvfuXCnLTZpFYSI/Ai3tLQe97UUnNl2YGFFKyPiVvco0d9w4IHxqDBbLsQ1eicYWbUN85OUkt64EJ+gkqjyxrsQUSyIC4GvqepHhRamrIlLf+uwNmiHHuoeS5fG08KJI3HrSJZoMXSnY+gItGc5x0XmPKOqRQntjMSXcFabv8xU5b3+UimF6oymaSmdjigKYi1dbmWtCxMWUbV0Kdx9t1NicbBwjMxJjKxKVA4QTz9JnujZsycfffQRX/7yl+OhJAT4Ku6XcTPOcviSNx4TVJWPPvqInj17Zn2MKAriTeAJEfkjgX5Yqpr/lEQjd5L5G/761/YiQkb5EVYXCaAHbrkpjn6SPFJXV0dzczMffPBBqUUJ51OclzZm9OzZM6fcjygK4u/eo4f3MOJMXDraGfnFj6xK7EY3FfdfWWo/SYEzz7t3786gQYPyd0AjEmkVhLpsakSkRlWTNDk2YkPcI6qM7EgWWXUl8XCeW0RVlyRKFNNI4Le4Ul0DROQgYLKq/mehhTOyIG4RVUZ+iGtkFSTPPF+Ik6+S61mVOVGWmG7CzQ8eAFDVF0XkXwoqlZEbcYmoMvJL3CKrfML8Ixtxy18bMKuijImUVqiqiSUSLWfTMKLS1TOewzLPewLvUXbd6IyORFEQa0XkcEBFpIeI/AR4pcBydTkK1QbRiDmVUBk2rG/EngRiHj1i3o3O6EyUJabzgJuBvYB3cUbi+YUUqqvht0EMdrrasGEDkydPZt68efmtEWPEi0qoDBvmH2kFzqRz1JUF05UVUYr1faiqE1W1r6r2UdUzc8mqFpH+IvK4iLwiIi+JyIXe+K4iskREXveed8n2HHGjkG0QjZhThn2dsyKxb8RJlF03OqMzaRWEiAwWkQdF5AMR2SAifxCRwTmcczvwY1XdHxgBnC8iXwcuAR5V1X2BR73XXYJStEE0YsLBdK7P05vcZtLl4NPwrYo5wHTv2RzUZUeUJabfA78CTvVej8fd7sOyOaGqrsPLOVTVz0TkFdzy1cnAKG+3O4EngJ9lc464Ucg2iEbMOQ6XyBacH/TwxrOhnHIO4hp1ZUQmysK3qOpdqrrde9yNi0vIGREZiJtjPQ/09ZSHr0R2z8c54kAh2yAaMWcxrhRGkBZvPBuCPg2LDjIKTBQF8biIXCIiA0XkKyJyMfBHz2ewa7YnFpFaYD5wkar+M4P3nSsiy0VkeWzrsiQwZcqUpA3Jc22DaMScMB/EZrL3QVSKT8OIBVEUxBnAZOBx3LLPf+A6zDUBy7M5qYh0xymHRlVd4A2vF5F+3vZ+uBSbTqjqbFWtV9X6Pn36ZHP6otPQ0MAxxxzTSUnU1NRw7LHHMn78+BJJZhScfHenCzleW+82Gj+xEOouTYn8TpJrvfCMT+hq9d4J/ENVLwqM3wh8pKrXi8glwK6qenGqY9XX1+vy5VnpqKLT1tbG3LlzmTlzJs3NzdTV1TF16lTGjx9vIa5dmXz7DBKO19a7jXG9xvHI5x2j5PzJh4VQdwEK4HcSkSZVrU+7XzIFISKHAGtV9X3v9VnAacA7wFWq+o8sBTsSeAr4K+39mP4Ld/n3AgNw1WO/m+4c5aQg8kpixzirtRRvWoBrgKeBI3ChoLnURfYrp66Exk8amfybyaFRcjU1NcyePbuwrTdTUeAKrxXDQlyCZTCnpBYXKpRlAEA+FMQLwDGq+g+v9tJc4Ec443h/VT09O9HyR0UqiLCOcYcd5gr0mZKIHwWOOqqvr6epqSnp9uHDh1OS/5FyiraKOzNwWfjBn2rBhQ9Py+6QURVEKtuzOjCDPwOYrarzVfVyYJ/sxDJyJtgxTtU9P/+8GzfixyLgOTpGHT1H3qKOYhtCXS7RVuWQU5JvP1YGpFQQIuLnSYwGHgtsi5I/YRSCZB3jVloYSyxpIjzq6IX8HD62IdTlEG1VLnWywmpdFSkrPZWCmAM8KSJ/AD7H+Q0QkX2o1B7Vra2wcCHMmOGeW0vwTfI7xgWxjnHxJdlXZHt+Dl+0EOpMZ9olnPVGplysnBJmpSe1BFT1WhF5FOgHLNZ2Z0UVzhdRWcRl7d86xpUXyb4aebLBGxoauO/e+3hk4SNsagtEMVXVcOwxeQqhzsafkKwDXpy+pqmsnLhlf5coKz3l11RVnwsZ+1vhxIkpra1w9dXw1FPQ0uLGgmv/2TbmyTQayd//qKPg8MOhWzcYNsyimOLMcNySQGJV02H5OXxVVRULfrCAuQ/PZebWmTTTTB11TO0+lfHfTxFCnUmEUTYVaePcAc8nWZ/vOFk5JcZ8CenwLYegcvDx1/6zURCZWiTJ9r/sMlMOcaYIM+mqF6uY0DKBCQTCWVuAVcDYkDdkahFkO9OOey2mcrBySoxl0KTDjxpKVA4AvXtnv/afaTSSRS+VJ4VcP/b9Aq8BOyRsS1UxNtO193LwJ2SDVZxNiymIdIRFDfn06AHHZVmWM9NoJIteKl8SeyXkSzn4ETiNwJaE7akqxmYaYVTCKJqCU4h704UwBZGOsKghn5YWWJxlWc5Mo5Eseqn8KGSMfaJfIJFUFWMztQhspl2xmIJIhx811COkNsLmzdnP4P3j1taCiHtOFY2U6f5GaSl0jH2YFRAkVcXYbCwCm2lXJOakTkd1tXMcX3013HBDR19ELjN4/7iLFjklM3Ro6mikTPc38kO29YQK3Ys6LAInSBSLIM4RRkYsKHo113xS1FpMccmDMIpHLvWEClA/J6lsGwPytGUop1GRRK3FZBZEVGwGX3lEsQKSWRiFjrGvBh6ivUrsSFzOxUuYRWDkDVMQmVBd7XIesk2MM8qLdPH/qSyMQsfYtwInBI6/FLMajLxjTmrDSMbBuHyCIMH8glT5BIWO/CmXOkJGWWMKwjCScRydG/sE8wvS5RMUMvKnHKqlGmWPKQjDSMZiYFvCWDC/oJQZxl01uzkTyqGXQ5ljCsIwkhE2Sw/mF5Qyw7grZzdHoVx6OZQ55qQ2jGQki0T6Bm7GugJX+P5HuA7rxYweqvRchkLnmRiAKQjDSE5YJNKhwC9wUUOl7rUc92qphaScejmUMaYgILwvA2TWq8HoeoTN0luBM7GZa6mxXg5FwRREWIb0oYe6bUuXWtZ0pZM4S5+BzVzjgPVyKAqmIIJ9FsA9P/OM+3vLlvaxbLrHZdoxzog/YTPX3nT0S2RSswmyr/dUyVS6D6ZImIII67OwJbG4Ppl3j7PaTV2T44DuCWPdgZuBZWTul8il3lOlU8k+mCJhYa5hfRZ69nSPIJlWbs22A1xrKyxcCDNmuOfW1mjbjOKwGJcLEeRz4Fmyy2q2jGgjxpgF4fdZSOWD6N0b9t4bmpra35POCkjVAS6ZFZLK6gCzSOLAClwuRJCtIftF9UtYNI4RY0xBJKvSCm7shRdgwQJ44w3XEyLqD7NvmWwMLFans0LC/CFBqyPZNiseWDzCfBC+sRlcmYwaUWPROEaMsSUmaK/SOm2ae66ubh8bNswph0yXirLpAJfK6rCe1PEgLIP5cO+RTVZzpWdEG7HGLIh0ZLNUBNn1j0hndWRqkRj5J1n0DCFjUVb+LBrHiDGmINKRzVKRT6b9I8L8IUGrI9U2o3gki55JFlHTQntjnyNw1V2DVWItGseIKdZyNB3FDlf1cyfCrI5U24x40gLsAXwcGNsFeJ/OpcQNo0hEbTkaKwUhIt/GRZRXA7ep6vWp9i9aT2r7YTay5Qpc9nUil+MaCRlGCSi7ntQiUg38CjgWaAaWicgDqvpyaSXDWo0a2fN0kvFniiqFYWRFnKKYDgXWqOqbqtoCzAVOLrFMhpEbRyQZP7yoUhhGVsRJQewFrA28bvbGOiAi54rIchFZ/sEHHxRNOMPIimk4n0OQXbxxw4g5cVIQEjLWyUGiqrNVtV5V6/v06VMEsQwjB3rgHNKXA6O9Z3NQG2VCbHwQOIuhf+B1HfBeiWQxjPzRA3NIG2VJnCyIZcC+IjJIRHoA44EHSiyTYRhGxRIbC0JVt4vID2kvdPw7VX2pxGIZhmFULLFREACq+hDwUKnlMAzDMOK1xGQYhmHECFMQhmEYRiimIAzDMIxQTEEYhmEYoZiCMAzDMEKJVTXXTBGRD4B3SnDq3YAPS3DefGLXEA/sGuJBV7gGiH4dX1HVtKUoylpBlAoRWR6lVG6csWuIB3YN8aArXAPk/zpsickwDMMIxRSEYRiGEYopiOyYXWoB8oBdQzywa4gHXeEaIM/XYT4IwzAMIxSzIAzDMIxQTEEYhmEYoZiCSGbwj54AAAbjSURBVIGI9BeRx0XkFRF5SUQu9MZ3FZElIvK695zYVDJ2iEi1iKwQkYXe60Ei8rx3Dfd4PThijYjsLCLzRORV756MLLd7ISJTvO/SahGZIyI9434vROR3IrJBRFYHxkI/d3H8QkTWiMgqERlWOsnbSXINN3rfpVUicr+I7BzYdql3Da+JyJjSSN2RsGsIbPuJiKiI7Oa9zst9MAWRmu3Aj1V1f2AEcL6IfB24BHhUVfcFHvVex50LgVcCr28AZnnX8DHwg5JIlRk3A39S1f2Ag3DXUzb3QkT2Ai4A6lX1AFzfk/HE/17cAXw7YSzZ5348sK/3OBe4pUgypuMOOl/DEuAAVT0Q+BtwKYD3Pz4eGOK959ciUl08UZNyB52vARHpDxwL/D0wnJ/7oKr2iPgA/uDdiNeAft5YP+C1UsuWRu463D/x0cBCXP/vD4Fu3vaRwMOlljPNNewEvIUXWBEYL5t7AewFrAV2xfViWQiMKYd7AQwEVqf73IH/BzSE7VfqR+I1JGw7FWj0/r4UuDSw7WFgZKnlT3YNwDzchOltYLd83gezICIiIgOBg4Hngb6qug7Ae969dJJF4ibgYqDNe/1l4BNV3e69bsb9eMWZwcAHwO3eUtltIlJDGd0LVX0X+L+4md464FOgifK7F5D8c/eVoE+5XM/3gUXe32VzDSIyFnhXVV9M2JSXazAFEQERqQXmAxep6j9LLU8miMhJwAZVbQoOh+wa93jnbsAw4BZVPRjYRIyXk8Lw1ulPBgYBewI1uKWAROJ+L1JRdt8tEbkMt5zc6A+F7Ba7axCR3sBlwBVhm0PGMr4GUxBpEJHuOOXQqKoLvOH1ItLP294P2FAq+SJwBDBWRN4G5uKWmW4CdhYRv+VsHfBeacSLTDPQrKrPe6/n4RRGOd2LY4C3VPUDVd0GLAAOp/zuBST/3JuB/oH9Yn09IjIJOAmYqN5aDOVzDXvjJhsvev/fdcALIrIHeboGUxApEBEBfgu8oqozA5seACZ5f0/C+SZiiapeqqp1qjoQ53h7TFUnAo8Dp3u7xfoaAFT1fWCtiHzNGxoNvEwZ3Qvc0tIIEentfbf8ayire+GR7HN/ADjLi6IZAXzqL0XFDRH5NvAzYKyqbg5segAYLyI7iMggnKN3aSlkTIWq/lVVd1fVgd7/dzMwzPtfyc99KLXTJc4P4EicWbYKWOk9TsCt4T8KvO4971pqWSNezyhgoff3YNyXfg1wH7BDqeWLIP9QYLl3P/4X2KXc7gVwNfAqsBq4C9gh7vcCmIPzmWzzfoR+kOxzxy1t/Ap4A/grLmIrrtewBrdO7/9v/yaw/2XeNbwGHF9q+ZNdQ8L2t2l3UuflPlipDcMwDCMUW2IyDMMwQjEFYRiGYYRiCsIwDMMIxRSEYRiGEYopCMMwDCMUUxBGl0FETvUqWu5XgnO/7VfSjDJuGOWAKQijK9EA/AWXEGgYRo6YgjC6BF69rCNwCVDjA+OjROSJQB+JRi+L2Z/dXy0iL4jIX33LQ0SuEpGfBI6x2ivWiIj8r4g0eT0dzs1AvoHieljc6r13sYj08rbtIyKPiMiLnix7exmwN3rn/quInBG4nidF5F4R+ZuIXC8iE0Vkqbff3t5+fURkvogs8x5H5PgRGxWIKQijq3AKrlfE34B/JDRIORi4CPg6Lms5+GP5oaoOw9XL/wnp+b6qDgfqgQtE5MsZyLgv8CtVHQJ8ApzmjTd64wfhajOtA8bhMscPwtVwutGvfeSNXQh8A/g34KuqeihwG/Ajb5+bcT0mDvHOc1sGchoGYArC6Do04IoR4j03BLYtVdVmVW3DlVQYGNjmF2BsShhPxgUi8iLwHK4Y2r4ZyPiWqq4Mnk9EdgT2UtX7AVR1i7q6QEcCc1S1VVXXA08Ch3jvXaaq61R1K66UwmJv/K+BazgG+KWIrMTV5dnJO5dhRKZb+l0MI954s/ijgQNERHGd2lRELvZ22RrYvZWO3/utIePb6Th56umdZxTuh3ekqm4WkSf8bRFJlKMX4WWZSTGeeJy2wOs22q+hypPz8wzkM4wOmAVhdAVOB/5HVb+irrJlf1z3uSOzPN7buFLieEtVg7zxLwEfe8phP1wb2pxQ11+kWURO8c63g1fn/8/AGeJ6ifcB/oXMKoouBn7ovxCRobnKalQepiCMrkADcH/C2HxgQpbHmw/s6i3P/AeuXzHAn4BuIrIKmIFbZsoH/4ZbuloFPAPsgbueVcCLwGPAxerKOEflAqBeXMP6l4Hz8iSrUUFYNVfDMAwjFLMgDMMwjFBMQRiGYRihmIIwDMMwQjEFYRiGYYRiCsIwDMMIxRSEYRiGEYopCMMwDCOU/w+u7BwShJUdyQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Fitting K-Means to the dataset\n",
    "k_means = KMeans(n_clusters = 5, init = 'k-means++', random_state = 42)\n",
    "y_kmeans = k_means.fit_predict(X)\n",
    "\n",
    "# Visualising the clusters\n",
    "size = 25\n",
    "plt.scatter(X[y_kmeans == 0, 0], X[y_kmeans == 0, 1], s = size, c = 'cyan', label = 'Cls 1')\n",
    "plt.scatter(X[y_kmeans == 1, 0], X[y_kmeans == 1, 1], s = size, c = 'blue', label = 'Cls 2')\n",
    "plt.scatter(X[y_kmeans == 2, 0], X[y_kmeans == 2, 1], s = size, c = 'yellow', label = 'Cls 3')\n",
    "plt.scatter(X[y_kmeans == 3, 0], X[y_kmeans == 3, 1], s = size, c = 'red', label = 'Cls 4')\n",
    "plt.scatter(X[y_kmeans == 4, 0], X[y_kmeans == 4, 1], s = size, c = 'magenta', label = 'Cls 5')\n",
    "plt.scatter(k_means.cluster_centers_[:, 0], k_means.cluster_centers_[:, 1], s = 60, c = 'black', label = 'Centroid')\n",
    "plt.title('Clusters of customers')\n",
    "plt.xlabel('Annual Income')\n",
    "plt.ylabel('Spending Score -> (1-100)')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "k_means = KMeans(n_clusters = 5, init = 'k-means++', random_state = 42)\n",
    "y_kmeans = k_means.fit_predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0,\n",
       "       3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 1,\n",
       "       3, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 4, 2, 1, 2, 4, 2, 4, 2,\n",
       "       1, 2, 4, 2, 4, 2, 4, 2, 4, 2, 1, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2,\n",
       "       4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2,\n",
       "       4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2, 4, 2,\n",
       "       4, 2])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_kmeans"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Difference Between fit and fit_predict method\n",
    "* fit : Creates an affinity matrix for X using the selected affinity, then applies clustering to this affinity matrix\n",
    "* fit_predict : Performs clustering on X and returns cluster labels\n",
    "\n",
    "Note: fit_predict is equivalent to getting the labels_ attribute from the object after fit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3\n",
      " 0 3 0 3 0 3 1 3 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 2 4 2 1 2 4 2 4 2 1 2 4 2 4 2 4 2 4 2 1 2 4 2 4 2\n",
      " 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4\n",
      " 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2]\n",
      "[3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3\n",
      " 0 3 0 3 0 3 1 3 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 2 4 2 1 2 4 2 4 2 1 2 4 2 4 2 4 2 4 2 1 2 4 2 4 2\n",
      " 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2 4\n",
      " 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2]\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "\n",
    "# Importing the dataset\n",
    "dataset = pd.read_csv(\"Data/Unsupervised/Mall_Customers.csv\")\n",
    "X = dataset.iloc[:, [3, 4]].values\n",
    "\n",
    "# Using the elbow method to find the optimal number of clusters\n",
    "from sklearn.cluster import KMeans\n",
    "k_means = KMeans(n_clusters = 5, init = 'k-means++', random_state = 42)\n",
    "k_means.fit(X)\n",
    "print(k_means.labels_)\n",
    "\n",
    "k_means_1 = KMeans(n_clusters = 5, init = 'k-means++', random_state = 42)\n",
    "print(k_means_1.fit_predict(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Measuring Performance of Clusters\n",
    "\n",
    "* silhoutte_score\n",
    "* calinski_harabaz_score\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## silhoutte_score\n",
    "\n",
    "* The Silhouette Coefficient is calculated using the mean intra-cluster distance (a) and the mean nearest-cluster distance (b) for each sample.\n",
    "* The Silhouette Coefficient for a sample is (b - a) / max(a, b). To clarify, b is the distance between a sample and the nearest cluster that the sample is not a part of.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2 0.2968969162503008\n",
      "3 0.46761358158775435\n",
      "4 0.4931963109249047\n",
      "5 0.553931997444648\n",
      "6 0.53976103063432\n",
      "7 0.5314247434775297\n",
      "8 0.45732611752686836\n",
      "9 0.4553729779390266\n",
      "10 0.44973539403790147\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import silhouette_score\n",
    "\n",
    "for i in range(2, 11):\n",
    "    kmeans = KMeans(n_clusters=i)\n",
    "    kmeans.fit(X)\n",
    "    labels = kmeans.predict(X)\n",
    "    print (i, silhouette_score(X,labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## calinski_harabaz_score\n",
    "* The score is defined as ratio between the within-cluster dispersion and the between-cluster dispersion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2 96.25619260242958\n",
      "3 151.5570090006799\n",
      "4 174.0644335433684\n",
      "5 247.35899338037282\n",
      "6 242.29567848269156\n",
      "7 254.6990131602406\n",
      "8 268.6382937788907\n",
      "9 271.55855058789604\n",
      "10 260.21051597750545\n",
      "11 263.3694322261491\n",
      "12 274.63027931386483\n",
      "13 279.20082460649155\n",
      "14 269.77731802817084\n",
      "15 285.6552409968533\n",
      "16 287.3853382632505\n",
      "17 282.77226080808555\n",
      "18 293.1301552479272\n",
      "19 303.38538133683005\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import calinski_harabasz_score\n",
    "\n",
    "for i in range(2, 20):\n",
    "    kmeans = KMeans(n_clusters=i)\n",
    "    kmeans.fit(X)\n",
    "    labels = kmeans.predict(X)\n",
    "    print (i, calinski_harabasz_score(X,labels))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}