Project Week 4 (1.-Data Cleaning) [Nadim SAAD DAFT NOV21].ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Descriptive Statistics Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Context"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![img](./diamonds.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The dataset we will be using is comprised of approximately 54k rows and 11 different columns. As always, a row represents a single observation (in this case a diamond) and each of the columns represent a different feature of a diamond.\n",
    "\n",
    "The following codebook was provided together with the dataset to clarify what each column represents:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Column  | Description  |\n",
    "|---|---|\n",
    "| Price  | Price in US dollars (326-18,823)  |\n",
    "| Carat  | Weight of the diamond (0.2--5.01)  |\n",
    "| Cut  | Quality of the cut (Fair, Good, Very Good, Premium, Ideal)  |\n",
    "| Color  | Diamond colour, from J (worst) to D (best)  |\n",
    "| Clarity  | A measurement of how clear the diamond is (I1 (worst), SI2, SI1, VS2, VS1, VVS2, VVS1, IF (best))   |\n",
    "| x  | Length in mm (0--10.74)  |\n",
    "| y  | Width in mm (0--58.9)  |\n",
    "| z  | Depth in mm (0--31.8)  |\n",
    "| Depth  | Total depth percentage = z / mean(x, y) = 2 * z / (x + y) (43--79)  |\n",
    "| Table  | Width of top of diamond relative to widest point (43--95)  |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Libraries\n",
    "Pandas and numpy will be needed for the analysis of the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First import the data from the .csv file provided and assign it to a variable named `diamonds` and drop the column with the index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds=pd.read_csv(r'/Users/nadimsaad/Desktop/IRONHACK/Module 2/Week 4/Day 20/Project 4/diamonds.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Taking the first look at the data.\n",
    "Let's see how the data looks by using pandas methods like `head()`, `info()` and `describe()`. \n",
    "\n",
    "**First, use the `head` method.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.23</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>E</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.5</td>\n",
       "      <td>55.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.95</td>\n",
       "      <td>3.98</td>\n",
       "      <td>2.43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0.21</td>\n",
       "      <td>Premium</td>\n",
       "      <td>E</td>\n",
       "      <td>SI1</td>\n",
       "      <td>59.8</td>\n",
       "      <td>61.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.89</td>\n",
       "      <td>3.84</td>\n",
       "      <td>2.31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0.23</td>\n",
       "      <td>Good</td>\n",
       "      <td>E</td>\n",
       "      <td>VS1</td>\n",
       "      <td>56.9</td>\n",
       "      <td>65.0</td>\n",
       "      <td>327</td>\n",
       "      <td>4.05</td>\n",
       "      <td>4.07</td>\n",
       "      <td>2.31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.29</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>VS2</td>\n",
       "      <td>62.4</td>\n",
       "      <td>58.0</td>\n",
       "      <td>334</td>\n",
       "      <td>4.20</td>\n",
       "      <td>4.23</td>\n",
       "      <td>2.63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.31</td>\n",
       "      <td>Good</td>\n",
       "      <td>J</td>\n",
       "      <td>SI2</td>\n",
       "      <td>63.3</td>\n",
       "      <td>58.0</td>\n",
       "      <td>335</td>\n",
       "      <td>4.34</td>\n",
       "      <td>4.35</td>\n",
       "      <td>2.75</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  carat      cut color clarity  depth  table  price     x     y  \\\n",
       "0           1   0.23    Ideal     E     SI2   61.5   55.0    326  3.95  3.98   \n",
       "1           2   0.21  Premium     E     SI1   59.8   61.0    326  3.89  3.84   \n",
       "2           3   0.23     Good     E     VS1   56.9   65.0    327  4.05  4.07   \n",
       "3           4   0.29  Premium     I     VS2   62.4   58.0    334  4.20  4.23   \n",
       "4           5   0.31     Good     J     SI2   63.3   58.0    335  4.34  4.35   \n",
       "\n",
       "      z  \n",
       "0  2.43  \n",
       "1  2.31  \n",
       "2  2.31  \n",
       "3  2.63  \n",
       "4  2.75  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.head(5)\n",
    "# we have 11 columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 53940 entries, 0 to 53939\n",
      "Data columns (total 11 columns):\n",
      " #   Column      Non-Null Count  Dtype  \n",
      "---  ------      --------------  -----  \n",
      " 0   Unnamed: 0  53940 non-null  int64  \n",
      " 1   carat       53940 non-null  float64\n",
      " 2   cut         53940 non-null  object \n",
      " 3   color       53940 non-null  object \n",
      " 4   clarity     53940 non-null  object \n",
      " 5   depth       53940 non-null  float64\n",
      " 6   table       53940 non-null  float64\n",
      " 7   price       53940 non-null  int64  \n",
      " 8   x           53940 non-null  float64\n",
      " 9   y           53940 non-null  float64\n",
      " 10  z           53940 non-null  float64\n",
      "dtypes: float64(6), int64(2), object(3)\n",
      "memory usage: 4.5+ MB\n"
     ]
    }
   ],
   "source": [
    "diamonds.info()\n",
    "# we have 2 int variable columns (unnamed and price)\n",
    "# 6 float variable columns (carat, depth, table, x, y, z)\n",
    "# 3 object variable columns (cut, color, clarity)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>26970.500000</td>\n",
       "      <td>0.797940</td>\n",
       "      <td>61.749405</td>\n",
       "      <td>57.457184</td>\n",
       "      <td>3932.799722</td>\n",
       "      <td>5.731157</td>\n",
       "      <td>5.734526</td>\n",
       "      <td>3.538734</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>15571.281097</td>\n",
       "      <td>0.474011</td>\n",
       "      <td>1.432621</td>\n",
       "      <td>2.234491</td>\n",
       "      <td>3989.439738</td>\n",
       "      <td>1.121761</td>\n",
       "      <td>1.142135</td>\n",
       "      <td>0.705699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>326.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>13485.750000</td>\n",
       "      <td>0.400000</td>\n",
       "      <td>61.000000</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>950.000000</td>\n",
       "      <td>4.710000</td>\n",
       "      <td>4.720000</td>\n",
       "      <td>2.910000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>26970.500000</td>\n",
       "      <td>0.700000</td>\n",
       "      <td>61.800000</td>\n",
       "      <td>57.000000</td>\n",
       "      <td>2401.000000</td>\n",
       "      <td>5.700000</td>\n",
       "      <td>5.710000</td>\n",
       "      <td>3.530000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>40455.250000</td>\n",
       "      <td>1.040000</td>\n",
       "      <td>62.500000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>5324.250000</td>\n",
       "      <td>6.540000</td>\n",
       "      <td>6.540000</td>\n",
       "      <td>4.040000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>53940.000000</td>\n",
       "      <td>5.010000</td>\n",
       "      <td>79.000000</td>\n",
       "      <td>95.000000</td>\n",
       "      <td>18823.000000</td>\n",
       "      <td>10.740000</td>\n",
       "      <td>58.900000</td>\n",
       "      <td>31.800000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Unnamed: 0         carat         depth         table         price  \\\n",
       "count  53940.000000  53940.000000  53940.000000  53940.000000  53940.000000   \n",
       "mean   26970.500000      0.797940     61.749405     57.457184   3932.799722   \n",
       "std    15571.281097      0.474011      1.432621      2.234491   3989.439738   \n",
       "min        1.000000      0.200000     43.000000     43.000000    326.000000   \n",
       "25%    13485.750000      0.400000     61.000000     56.000000    950.000000   \n",
       "50%    26970.500000      0.700000     61.800000     57.000000   2401.000000   \n",
       "75%    40455.250000      1.040000     62.500000     59.000000   5324.250000   \n",
       "max    53940.000000      5.010000     79.000000     95.000000  18823.000000   \n",
       "\n",
       "                  x             y             z  \n",
       "count  53940.000000  53940.000000  53940.000000  \n",
       "mean       5.731157      5.734526      3.538734  \n",
       "std        1.121761      1.142135      0.705699  \n",
       "min        0.000000      0.000000      0.000000  \n",
       "25%        4.710000      4.720000      2.910000  \n",
       "50%        5.700000      5.710000      3.530000  \n",
       "75%        6.540000      6.540000      4.040000  \n",
       "max       10.740000     58.900000     31.800000  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the first 5 rows of the dataset using the `head` method. This by itself doesn't tell us much about the data that we have, but we can have a first look at the features (columns) and some of the values that each one takes.\n",
    "\n",
    "**What do you see? Make some comments about the values you see in each column, comparing them with the codebook. Is that what you would expect for these variables?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dataset lines are 53940\n",
    "# Here we can see most of the coobkoot infos i.e. price ranging from 326 to 18823 $\n",
    "# Quality of the cut, the color and the clarity do not appear with the describe function\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is very important to know the amount of data we have, because everything will depend on that, from the quality of the analysis to the choice of our infrastracture.\n",
    "\n",
    "**Check the shape of the data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(53940, 11)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.shape # 53940 rows and 11 columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `clarity` column is confusing because we are not diamond experts. Let's create a new column with a new scale that is more understandable for us.\n",
    "\n",
    "**Create a new column with numbers from 0 to 7. The lowest would be 0 with value `I1` and the greatest 7 with value `IF`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        SI2\n",
       "1        SI1\n",
       "2        VS1\n",
       "3        VS2\n",
       "4        SI2\n",
       "        ... \n",
       "53935    SI1\n",
       "53936    SI1\n",
       "53937    SI1\n",
       "53938    SI2\n",
       "53939    SI2\n",
       "Name: clarity, Length: 53940, dtype: object"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds['clarity']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.23</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>E</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.5</td>\n",
       "      <td>55.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.95</td>\n",
       "      <td>3.98</td>\n",
       "      <td>2.43</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0.21</td>\n",
       "      <td>Premium</td>\n",
       "      <td>E</td>\n",
       "      <td>SI1</td>\n",
       "      <td>59.8</td>\n",
       "      <td>61.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.89</td>\n",
       "      <td>3.84</td>\n",
       "      <td>2.31</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0.23</td>\n",
       "      <td>Good</td>\n",
       "      <td>E</td>\n",
       "      <td>VS1</td>\n",
       "      <td>56.9</td>\n",
       "      <td>65.0</td>\n",
       "      <td>327</td>\n",
       "      <td>4.05</td>\n",
       "      <td>4.07</td>\n",
       "      <td>2.31</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.29</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>VS2</td>\n",
       "      <td>62.4</td>\n",
       "      <td>58.0</td>\n",
       "      <td>334</td>\n",
       "      <td>4.20</td>\n",
       "      <td>4.23</td>\n",
       "      <td>2.63</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.31</td>\n",
       "      <td>Good</td>\n",
       "      <td>J</td>\n",
       "      <td>SI2</td>\n",
       "      <td>63.3</td>\n",
       "      <td>58.0</td>\n",
       "      <td>335</td>\n",
       "      <td>4.34</td>\n",
       "      <td>4.35</td>\n",
       "      <td>2.75</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53935</th>\n",
       "      <td>53936</td>\n",
       "      <td>0.72</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>D</td>\n",
       "      <td>SI1</td>\n",
       "      <td>60.8</td>\n",
       "      <td>57.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.75</td>\n",
       "      <td>5.76</td>\n",
       "      <td>3.50</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53936</th>\n",
       "      <td>53937</td>\n",
       "      <td>0.72</td>\n",
       "      <td>Good</td>\n",
       "      <td>D</td>\n",
       "      <td>SI1</td>\n",
       "      <td>63.1</td>\n",
       "      <td>55.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.69</td>\n",
       "      <td>5.75</td>\n",
       "      <td>3.61</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53937</th>\n",
       "      <td>53938</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>D</td>\n",
       "      <td>SI1</td>\n",
       "      <td>62.8</td>\n",
       "      <td>60.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.66</td>\n",
       "      <td>5.68</td>\n",
       "      <td>3.56</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53938</th>\n",
       "      <td>53939</td>\n",
       "      <td>0.86</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>6.15</td>\n",
       "      <td>6.12</td>\n",
       "      <td>3.74</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53939</th>\n",
       "      <td>53940</td>\n",
       "      <td>0.75</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>D</td>\n",
       "      <td>SI2</td>\n",
       "      <td>62.2</td>\n",
       "      <td>55.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.83</td>\n",
       "      <td>5.87</td>\n",
       "      <td>3.64</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>53940 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0  carat        cut color clarity  depth  table  price     x  \\\n",
       "0               1   0.23      Ideal     E     SI2   61.5   55.0    326  3.95   \n",
       "1               2   0.21    Premium     E     SI1   59.8   61.0    326  3.89   \n",
       "2               3   0.23       Good     E     VS1   56.9   65.0    327  4.05   \n",
       "3               4   0.29    Premium     I     VS2   62.4   58.0    334  4.20   \n",
       "4               5   0.31       Good     J     SI2   63.3   58.0    335  4.34   \n",
       "...           ...    ...        ...   ...     ...    ...    ...    ...   ...   \n",
       "53935       53936   0.72      Ideal     D     SI1   60.8   57.0   2757  5.75   \n",
       "53936       53937   0.72       Good     D     SI1   63.1   55.0   2757  5.69   \n",
       "53937       53938   0.70  Very Good     D     SI1   62.8   60.0   2757  5.66   \n",
       "53938       53939   0.86    Premium     H     SI2   61.0   58.0   2757  6.15   \n",
       "53939       53940   0.75      Ideal     D     SI2   62.2   55.0   2757  5.83   \n",
       "\n",
       "          y     z  clarity_scale  \n",
       "0      3.98  2.43              1  \n",
       "1      3.84  2.31              2  \n",
       "2      4.07  2.31              4  \n",
       "3      4.23  2.63              3  \n",
       "4      4.35  2.75              1  \n",
       "...     ...   ...            ...  \n",
       "53935  5.76  3.50              2  \n",
       "53936  5.75  3.61              2  \n",
       "53937  5.68  3.56              2  \n",
       "53938  6.12  3.74              1  \n",
       "53939  5.87  3.64              1  \n",
       "\n",
       "[53940 rows x 12 columns]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds['clarity_scale']= np.where(diamonds[\"clarity\"] == 'I1', 0,\n",
    "                                    np.where(diamonds[\"clarity\"] == 'SI2', 1,\n",
    "                                    np.where(diamonds[\"clarity\"] == 'SI1', 2,\n",
    "                                    np.where(diamonds[\"clarity\"] == 'VS2', 3,\n",
    "                                    np.where(diamonds[\"clarity\"] == 'VS1', 4,\n",
    "                                    np.where(diamonds[\"clarity\"] == 'VVS2',5,\n",
    "                                    np.where(diamonds[\"clarity\"] == 'VVS1',6,\n",
    "                                    np.where(diamonds[\"clarity\"] == 'IF',7,8))))))))\n",
    "diamonds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "G    11292\n",
      "E     9797\n",
      "F     9542\n",
      "H     8304\n",
      "D     6775\n",
      "I     5422\n",
      "J     2808\n",
      "Name: color, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(diamonds['color'].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It makes sense to do the same with the `color` column.\n",
    "\n",
    "**Do the same with values from 0 to 6. Read the codebook to see the match**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.23</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>E</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.5</td>\n",
       "      <td>55.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.95</td>\n",
       "      <td>3.98</td>\n",
       "      <td>2.43</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0.21</td>\n",
       "      <td>Premium</td>\n",
       "      <td>E</td>\n",
       "      <td>SI1</td>\n",
       "      <td>59.8</td>\n",
       "      <td>61.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.89</td>\n",
       "      <td>3.84</td>\n",
       "      <td>2.31</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0.23</td>\n",
       "      <td>Good</td>\n",
       "      <td>E</td>\n",
       "      <td>VS1</td>\n",
       "      <td>56.9</td>\n",
       "      <td>65.0</td>\n",
       "      <td>327</td>\n",
       "      <td>4.05</td>\n",
       "      <td>4.07</td>\n",
       "      <td>2.31</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.29</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>VS2</td>\n",
       "      <td>62.4</td>\n",
       "      <td>58.0</td>\n",
       "      <td>334</td>\n",
       "      <td>4.20</td>\n",
       "      <td>4.23</td>\n",
       "      <td>2.63</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.31</td>\n",
       "      <td>Good</td>\n",
       "      <td>J</td>\n",
       "      <td>SI2</td>\n",
       "      <td>63.3</td>\n",
       "      <td>58.0</td>\n",
       "      <td>335</td>\n",
       "      <td>4.34</td>\n",
       "      <td>4.35</td>\n",
       "      <td>2.75</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53935</th>\n",
       "      <td>53936</td>\n",
       "      <td>0.72</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>D</td>\n",
       "      <td>SI1</td>\n",
       "      <td>60.8</td>\n",
       "      <td>57.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.75</td>\n",
       "      <td>5.76</td>\n",
       "      <td>3.50</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53936</th>\n",
       "      <td>53937</td>\n",
       "      <td>0.72</td>\n",
       "      <td>Good</td>\n",
       "      <td>D</td>\n",
       "      <td>SI1</td>\n",
       "      <td>63.1</td>\n",
       "      <td>55.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.69</td>\n",
       "      <td>5.75</td>\n",
       "      <td>3.61</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53937</th>\n",
       "      <td>53938</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>D</td>\n",
       "      <td>SI1</td>\n",
       "      <td>62.8</td>\n",
       "      <td>60.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.66</td>\n",
       "      <td>5.68</td>\n",
       "      <td>3.56</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53938</th>\n",
       "      <td>53939</td>\n",
       "      <td>0.86</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>6.15</td>\n",
       "      <td>6.12</td>\n",
       "      <td>3.74</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53939</th>\n",
       "      <td>53940</td>\n",
       "      <td>0.75</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>D</td>\n",
       "      <td>SI2</td>\n",
       "      <td>62.2</td>\n",
       "      <td>55.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.83</td>\n",
       "      <td>5.87</td>\n",
       "      <td>3.64</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>53940 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0  carat        cut color clarity  depth  table  price     x  \\\n",
       "0               1   0.23      Ideal     E     SI2   61.5   55.0    326  3.95   \n",
       "1               2   0.21    Premium     E     SI1   59.8   61.0    326  3.89   \n",
       "2               3   0.23       Good     E     VS1   56.9   65.0    327  4.05   \n",
       "3               4   0.29    Premium     I     VS2   62.4   58.0    334  4.20   \n",
       "4               5   0.31       Good     J     SI2   63.3   58.0    335  4.34   \n",
       "...           ...    ...        ...   ...     ...    ...    ...    ...   ...   \n",
       "53935       53936   0.72      Ideal     D     SI1   60.8   57.0   2757  5.75   \n",
       "53936       53937   0.72       Good     D     SI1   63.1   55.0   2757  5.69   \n",
       "53937       53938   0.70  Very Good     D     SI1   62.8   60.0   2757  5.66   \n",
       "53938       53939   0.86    Premium     H     SI2   61.0   58.0   2757  6.15   \n",
       "53939       53940   0.75      Ideal     D     SI2   62.2   55.0   2757  5.83   \n",
       "\n",
       "          y     z  clarity_scale  color_scale  \n",
       "0      3.98  2.43              1            1  \n",
       "1      3.84  2.31              2            1  \n",
       "2      4.07  2.31              4            1  \n",
       "3      4.23  2.63              3            5  \n",
       "4      4.35  2.75              1            6  \n",
       "...     ...   ...            ...          ...  \n",
       "53935  5.76  3.50              2            0  \n",
       "53936  5.75  3.61              2            0  \n",
       "53937  5.68  3.56              2            0  \n",
       "53938  6.12  3.74              1            4  \n",
       "53939  5.87  3.64              1            0  \n",
       "\n",
       "[53940 rows x 13 columns]"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds['color_scale']= np.where(diamonds[\"color\"] == 'D', 0,\n",
    "                                    np.where(diamonds[\"color\"] == 'E', 1,\n",
    "                                    np.where(diamonds[\"color\"] == 'F', 2,\n",
    "                                    np.where(diamonds[\"color\"] == 'G', 3,\n",
    "                                    np.where(diamonds[\"color\"] == 'H', 4,\n",
    "                                    np.where(diamonds[\"color\"] == 'I', 5,\n",
    "                                    np.where(diamonds[\"color\"] == 'J', 6,7)))))))\n",
    "diamonds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    11292\n",
      "1     9797\n",
      "2     9542\n",
      "3     8304\n",
      "4     6775\n",
      "5     5422\n",
      "6     2808\n",
      "Name: color_scale, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(diamonds['color_scale'].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the `info` method, we can see the features of the dataset, and the amount of observations (rows) that have a non-null value and the types of the features. \n",
    "\n",
    "**Now use the `info` method and comparing with the shape, comment on what you see**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 53940 entries, 0 to 53939\n",
      "Data columns (total 13 columns):\n",
      " #   Column         Non-Null Count  Dtype  \n",
      "---  ------         --------------  -----  \n",
      " 0   Unnamed: 0     53940 non-null  int64  \n",
      " 1   carat          53940 non-null  float64\n",
      " 2   cut            53940 non-null  object \n",
      " 3   color          53940 non-null  object \n",
      " 4   clarity        53940 non-null  object \n",
      " 5   depth          53940 non-null  float64\n",
      " 6   table          53940 non-null  float64\n",
      " 7   price          53940 non-null  int64  \n",
      " 8   x              53940 non-null  float64\n",
      " 9   y              53940 non-null  float64\n",
      " 10  z              53940 non-null  float64\n",
      " 11  clarity_scale  53940 non-null  int64  \n",
      " 12  color_scale    53940 non-null  int64  \n",
      "dtypes: float64(6), int64(4), object(3)\n",
      "memory usage: 5.4+ MB\n"
     ]
    }
   ],
   "source": [
    "diamonds.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the last line of the info output, you have some information about the types of the columns. As you know, it is a good idea to check if the types of each column is what you expect. If a column has the right type, we will be able to do all the operations that we want to do. \n",
    "\n",
    "For instance, if we have a column that is a `date` with a `string` format, we will have the data but we won't be able to do a simple operation, such as format the date the way that we would like.\n",
    "\n",
    "Changing the data type to the one we needs can help us to solve a lot of problems in our data.\n",
    "\n",
    "**Check the types of each column and comment if it matches with the expected**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we have 6 float columns, 4 integer and 3 object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. A deeper look: checking the basic statistics.\n",
    "\n",
    "The `describe` method gives us an overview of our data. From here we can see all the descriptive metrics for our variables.\n",
    "\n",
    "**Use the `describe` method and comment on what you see**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53932.000000</td>\n",
       "      <td>53933.000000</td>\n",
       "      <td>53932.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>26970.500000</td>\n",
       "      <td>0.797940</td>\n",
       "      <td>61.749405</td>\n",
       "      <td>57.457184</td>\n",
       "      <td>3932.799722</td>\n",
       "      <td>5.732007</td>\n",
       "      <td>5.734159</td>\n",
       "      <td>3.539735</td>\n",
       "      <td>3.051020</td>\n",
       "      <td>2.594197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>15571.281097</td>\n",
       "      <td>0.474011</td>\n",
       "      <td>1.432621</td>\n",
       "      <td>2.234491</td>\n",
       "      <td>3989.439738</td>\n",
       "      <td>1.119670</td>\n",
       "      <td>1.113273</td>\n",
       "      <td>0.692051</td>\n",
       "      <td>1.647136</td>\n",
       "      <td>1.701105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>326.000000</td>\n",
       "      <td>3.730000</td>\n",
       "      <td>3.680000</td>\n",
       "      <td>1.070000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>13485.750000</td>\n",
       "      <td>0.400000</td>\n",
       "      <td>61.000000</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>950.000000</td>\n",
       "      <td>4.710000</td>\n",
       "      <td>4.720000</td>\n",
       "      <td>2.910000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>26970.500000</td>\n",
       "      <td>0.700000</td>\n",
       "      <td>61.800000</td>\n",
       "      <td>57.000000</td>\n",
       "      <td>2401.000000</td>\n",
       "      <td>5.700000</td>\n",
       "      <td>5.710000</td>\n",
       "      <td>3.530000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>40455.250000</td>\n",
       "      <td>1.040000</td>\n",
       "      <td>62.500000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>5324.250000</td>\n",
       "      <td>6.540000</td>\n",
       "      <td>6.540000</td>\n",
       "      <td>4.040000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>53940.000000</td>\n",
       "      <td>5.010000</td>\n",
       "      <td>79.000000</td>\n",
       "      <td>95.000000</td>\n",
       "      <td>18823.000000</td>\n",
       "      <td>10.740000</td>\n",
       "      <td>19.278421</td>\n",
       "      <td>8.060000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Unnamed: 0         carat         depth         table         price  \\\n",
       "count  53940.000000  53940.000000  53940.000000  53940.000000  53940.000000   \n",
       "mean   26970.500000      0.797940     61.749405     57.457184   3932.799722   \n",
       "std    15571.281097      0.474011      1.432621      2.234491   3989.439738   \n",
       "min        1.000000      0.200000     43.000000     43.000000    326.000000   \n",
       "25%    13485.750000      0.400000     61.000000     56.000000    950.000000   \n",
       "50%    26970.500000      0.700000     61.800000     57.000000   2401.000000   \n",
       "75%    40455.250000      1.040000     62.500000     59.000000   5324.250000   \n",
       "max    53940.000000      5.010000     79.000000     95.000000  18823.000000   \n",
       "\n",
       "                  x             y             z  clarity_scale   color_scale  \n",
       "count  53932.000000  53933.000000  53932.000000   53940.000000  53940.000000  \n",
       "mean       5.732007      5.734159      3.539735       3.051020      2.594197  \n",
       "std        1.119670      1.113273      0.692051       1.647136      1.701105  \n",
       "min        3.730000      3.680000      1.070000       0.000000      0.000000  \n",
       "25%        4.710000      4.720000      2.910000       2.000000      1.000000  \n",
       "50%        5.700000      5.710000      3.530000       3.000000      3.000000  \n",
       "75%        6.540000      6.540000      4.040000       4.000000      4.000000  \n",
       "max       10.740000     19.278421      8.060000       7.000000      6.000000  "
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You have probably noticed that the columns x, y and z have a minimum value of 0. This means that there are one or more rows (or observations) in our dataset that are supposedly representing a diamond that has lenght, width or depth of 0. Considering that we're talking about a physical object, this is impossible!\n",
    "\n",
    "Now let's proceed to check the rows that have a value of 0 in any of the x, y or z columns. By doing this we want to check if the data we are missing can be obtained using the data that we do have.\n",
    "\n",
    "**Check the columns with `x`, `y` and `z` with value 0 in all of them and comment what you see**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [Unnamed: 0, carat, cut, color, clarity, depth, table, price, x, y, z, clarity_scale, color_scale]\n",
       "Index: []"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[diamonds['x']==0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [Unnamed: 0, carat, cut, color, clarity, depth, table, price, x, y, z, clarity_scale, color_scale]\n",
       "Index: []"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[diamonds['y']==0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2207</th>\n",
       "      <td>2208</td>\n",
       "      <td>1.00</td>\n",
       "      <td>Premium</td>\n",
       "      <td>G</td>\n",
       "      <td>SI2</td>\n",
       "      <td>59.1</td>\n",
       "      <td>59.0</td>\n",
       "      <td>3142</td>\n",
       "      <td>6.55</td>\n",
       "      <td>6.48</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2314</th>\n",
       "      <td>2315</td>\n",
       "      <td>1.01</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>I1</td>\n",
       "      <td>58.1</td>\n",
       "      <td>59.0</td>\n",
       "      <td>3167</td>\n",
       "      <td>6.66</td>\n",
       "      <td>6.60</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4791</th>\n",
       "      <td>4792</td>\n",
       "      <td>1.10</td>\n",
       "      <td>Premium</td>\n",
       "      <td>G</td>\n",
       "      <td>SI2</td>\n",
       "      <td>63.0</td>\n",
       "      <td>59.0</td>\n",
       "      <td>3696</td>\n",
       "      <td>6.50</td>\n",
       "      <td>6.47</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5471</th>\n",
       "      <td>5472</td>\n",
       "      <td>1.01</td>\n",
       "      <td>Premium</td>\n",
       "      <td>F</td>\n",
       "      <td>SI2</td>\n",
       "      <td>59.2</td>\n",
       "      <td>58.0</td>\n",
       "      <td>3837</td>\n",
       "      <td>6.50</td>\n",
       "      <td>6.47</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10167</th>\n",
       "      <td>10168</td>\n",
       "      <td>1.50</td>\n",
       "      <td>Good</td>\n",
       "      <td>G</td>\n",
       "      <td>I1</td>\n",
       "      <td>64.0</td>\n",
       "      <td>61.0</td>\n",
       "      <td>4731</td>\n",
       "      <td>7.15</td>\n",
       "      <td>7.04</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11182</th>\n",
       "      <td>11183</td>\n",
       "      <td>1.07</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>F</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.6</td>\n",
       "      <td>56.0</td>\n",
       "      <td>4954</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.62</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11963</th>\n",
       "      <td>11964</td>\n",
       "      <td>1.00</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>H</td>\n",
       "      <td>VS2</td>\n",
       "      <td>63.3</td>\n",
       "      <td>53.0</td>\n",
       "      <td>5139</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13601</th>\n",
       "      <td>13602</td>\n",
       "      <td>1.15</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>G</td>\n",
       "      <td>VS2</td>\n",
       "      <td>59.2</td>\n",
       "      <td>56.0</td>\n",
       "      <td>5564</td>\n",
       "      <td>6.88</td>\n",
       "      <td>6.83</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15951</th>\n",
       "      <td>15952</td>\n",
       "      <td>1.14</td>\n",
       "      <td>Fair</td>\n",
       "      <td>G</td>\n",
       "      <td>VS1</td>\n",
       "      <td>57.5</td>\n",
       "      <td>67.0</td>\n",
       "      <td>6381</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24394</th>\n",
       "      <td>24395</td>\n",
       "      <td>2.18</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI2</td>\n",
       "      <td>59.4</td>\n",
       "      <td>61.0</td>\n",
       "      <td>12631</td>\n",
       "      <td>8.49</td>\n",
       "      <td>8.45</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24520</th>\n",
       "      <td>24521</td>\n",
       "      <td>1.56</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>G</td>\n",
       "      <td>VS2</td>\n",
       "      <td>62.2</td>\n",
       "      <td>54.0</td>\n",
       "      <td>12800</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26123</th>\n",
       "      <td>26124</td>\n",
       "      <td>2.25</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>SI1</td>\n",
       "      <td>61.3</td>\n",
       "      <td>58.0</td>\n",
       "      <td>15397</td>\n",
       "      <td>8.52</td>\n",
       "      <td>8.42</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26243</th>\n",
       "      <td>26244</td>\n",
       "      <td>1.20</td>\n",
       "      <td>Premium</td>\n",
       "      <td>D</td>\n",
       "      <td>VVS1</td>\n",
       "      <td>62.1</td>\n",
       "      <td>59.0</td>\n",
       "      <td>15686</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27112</th>\n",
       "      <td>27113</td>\n",
       "      <td>2.20</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI1</td>\n",
       "      <td>61.2</td>\n",
       "      <td>59.0</td>\n",
       "      <td>17265</td>\n",
       "      <td>8.42</td>\n",
       "      <td>8.37</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27429</th>\n",
       "      <td>27430</td>\n",
       "      <td>2.25</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI2</td>\n",
       "      <td>62.8</td>\n",
       "      <td>59.0</td>\n",
       "      <td>18034</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27503</th>\n",
       "      <td>27504</td>\n",
       "      <td>2.02</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>VS2</td>\n",
       "      <td>62.7</td>\n",
       "      <td>53.0</td>\n",
       "      <td>18207</td>\n",
       "      <td>8.02</td>\n",
       "      <td>7.95</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27739</th>\n",
       "      <td>27740</td>\n",
       "      <td>2.80</td>\n",
       "      <td>Good</td>\n",
       "      <td>G</td>\n",
       "      <td>SI2</td>\n",
       "      <td>63.8</td>\n",
       "      <td>58.0</td>\n",
       "      <td>18788</td>\n",
       "      <td>8.90</td>\n",
       "      <td>8.85</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49556</th>\n",
       "      <td>49557</td>\n",
       "      <td>0.71</td>\n",
       "      <td>Good</td>\n",
       "      <td>F</td>\n",
       "      <td>SI2</td>\n",
       "      <td>64.1</td>\n",
       "      <td>60.0</td>\n",
       "      <td>2130</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49557</th>\n",
       "      <td>49558</td>\n",
       "      <td>0.71</td>\n",
       "      <td>Good</td>\n",
       "      <td>F</td>\n",
       "      <td>SI2</td>\n",
       "      <td>64.1</td>\n",
       "      <td>60.0</td>\n",
       "      <td>2130</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51506</th>\n",
       "      <td>51507</td>\n",
       "      <td>1.12</td>\n",
       "      <td>Premium</td>\n",
       "      <td>G</td>\n",
       "      <td>I1</td>\n",
       "      <td>60.4</td>\n",
       "      <td>59.0</td>\n",
       "      <td>2383</td>\n",
       "      <td>6.71</td>\n",
       "      <td>6.67</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0  carat        cut color clarity  depth  table  price     x  \\\n",
       "2207         2208   1.00    Premium     G     SI2   59.1   59.0   3142  6.55   \n",
       "2314         2315   1.01    Premium     H      I1   58.1   59.0   3167  6.66   \n",
       "4791         4792   1.10    Premium     G     SI2   63.0   59.0   3696  6.50   \n",
       "5471         5472   1.01    Premium     F     SI2   59.2   58.0   3837  6.50   \n",
       "10167       10168   1.50       Good     G      I1   64.0   61.0   4731  7.15   \n",
       "11182       11183   1.07      Ideal     F     SI2   61.6   56.0   4954  0.00   \n",
       "11963       11964   1.00  Very Good     H     VS2   63.3   53.0   5139  0.00   \n",
       "13601       13602   1.15      Ideal     G     VS2   59.2   56.0   5564  6.88   \n",
       "15951       15952   1.14       Fair     G     VS1   57.5   67.0   6381  0.00   \n",
       "24394       24395   2.18    Premium     H     SI2   59.4   61.0  12631  8.49   \n",
       "24520       24521   1.56      Ideal     G     VS2   62.2   54.0  12800  0.00   \n",
       "26123       26124   2.25    Premium     I     SI1   61.3   58.0  15397  8.52   \n",
       "26243       26244   1.20    Premium     D    VVS1   62.1   59.0  15686  0.00   \n",
       "27112       27113   2.20    Premium     H     SI1   61.2   59.0  17265  8.42   \n",
       "27429       27430   2.25    Premium     H     SI2   62.8   59.0  18034  0.00   \n",
       "27503       27504   2.02    Premium     H     VS2   62.7   53.0  18207  8.02   \n",
       "27739       27740   2.80       Good     G     SI2   63.8   58.0  18788  8.90   \n",
       "49556       49557   0.71       Good     F     SI2   64.1   60.0   2130  0.00   \n",
       "49557       49558   0.71       Good     F     SI2   64.1   60.0   2130  0.00   \n",
       "51506       51507   1.12    Premium     G      I1   60.4   59.0   2383  6.71   \n",
       "\n",
       "          y    z  clarity_scale  color_scale  \n",
       "2207   6.48  0.0              1            0  \n",
       "2314   6.60  0.0              0            3  \n",
       "4791   6.47  0.0              1            0  \n",
       "5471   6.47  0.0              1            2  \n",
       "10167  7.04  0.0              0            0  \n",
       "11182  6.62  0.0              1            2  \n",
       "11963  0.00  0.0              3            3  \n",
       "13601  6.83  0.0              3            0  \n",
       "15951  0.00  0.0              4            0  \n",
       "24394  8.45  0.0              1            3  \n",
       "24520  0.00  0.0              3            0  \n",
       "26123  8.42  0.0              2            5  \n",
       "26243  0.00  0.0              6            4  \n",
       "27112  8.37  0.0              2            3  \n",
       "27429  0.00  0.0              1            3  \n",
       "27503  7.95  0.0              3            3  \n",
       "27739  8.85  0.0              1            0  \n",
       "49556  0.00  0.0              1            2  \n",
       "49557  0.00  0.0              1            2  \n",
       "51506  6.67  0.0              0            0  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[diamonds['z']==0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we have 20 rows that have a value of 0 in some or all the aforementioned columns.\n",
    "Most of them (12) are missing the z value, which we can obtain using the columns depth, x and y. \n",
    "\n",
    "20 rows with issues represent just 0.03% of our data (20 out of 53940) so it wouldn't be a big deal to remove them. Still, lets try to keep all the data we have. \n",
    "\n",
    "For those 12 rows, we will create a function that applies the formula given in the codebook and get the value of z. We will drop the other rows (8), since they are missing all 3 values or 2 of them.\n",
    "\n",
    "**Create a function named `calculate_z` that applies the function in the codebook to one single row you give to the function**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [Unnamed: 0, carat, cut, color, clarity, depth, table, price, x, y, z, clarity_scale, color_scale]\n",
       "Index: []"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[diamonds['z']==0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.31014"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def calculate_z(i):\n",
    "    z=diamonds['depth'][i]*((diamonds['x'][i])+(diamonds['y'][i]))/(2*100)\n",
    "    return z\n",
    "calculate_z(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2207,\n",
       " 2314,\n",
       " 4791,\n",
       " 5471,\n",
       " 10167,\n",
       " 13601,\n",
       " 24394,\n",
       " 26123,\n",
       " 27112,\n",
       " 27503,\n",
       " 27739,\n",
       " 51506]"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#find the index of rows to fix\n",
    "condition_z = (diamonds['z']==0) & (diamonds['y']!=0) & (diamonds['x']!=0)\n",
    "n_ind = diamonds.loc[lambda x: condition_z==True].index.tolist()\n",
    "n_ind\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2207\n",
      "2314\n",
      "4791\n",
      "5471\n",
      "10167\n",
      "13601\n",
      "24394\n",
      "26123\n",
      "27112\n",
      "27503\n",
      "27739\n",
      "51506\n"
     ]
    }
   ],
   "source": [
    "for i in n_ind:\n",
    "    print(i)\n",
    "    x=(calculate_z(i))\n",
    "    diamonds.at[i,'z']=x\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [Unnamed: 0, carat, cut, color, clarity, depth, table, price, x, y, z, clarity_scale, color_scale]\n",
       "Index: []"
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[diamonds['z']==0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "dd=pd.DataFrame()\n",
    "for i in range(len (diamonds['z'])):\n",
    "    \n",
    "    if diamonds['x'][i]!=0 and diamonds['z'][i]==0:\n",
    "                print(i)\n",
    "                \n",
    "   diamonds[(diamonds['x']!=0) & (diamonds['y']!=0) & (diamonds['z']==0)] = diamonds.apply(lambda x: calculate_z(diamonds['x'], diamonds['y'], diamonds['depth']))\n",
    "diamonds[diamonds['z']==0]\n",
    "                \n",
    "                     \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>11182</th>\n",
       "      <td>11183</td>\n",
       "      <td>1.07</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>F</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.6</td>\n",
       "      <td>56.0</td>\n",
       "      <td>4954</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.62</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11963</th>\n",
       "      <td>11964</td>\n",
       "      <td>1.00</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>H</td>\n",
       "      <td>VS2</td>\n",
       "      <td>63.3</td>\n",
       "      <td>53.0</td>\n",
       "      <td>5139</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15951</th>\n",
       "      <td>15952</td>\n",
       "      <td>1.14</td>\n",
       "      <td>Fair</td>\n",
       "      <td>G</td>\n",
       "      <td>VS1</td>\n",
       "      <td>57.5</td>\n",
       "      <td>67.0</td>\n",
       "      <td>6381</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24520</th>\n",
       "      <td>24521</td>\n",
       "      <td>1.56</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>G</td>\n",
       "      <td>VS2</td>\n",
       "      <td>62.2</td>\n",
       "      <td>54.0</td>\n",
       "      <td>12800</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26243</th>\n",
       "      <td>26244</td>\n",
       "      <td>1.20</td>\n",
       "      <td>Premium</td>\n",
       "      <td>D</td>\n",
       "      <td>VVS1</td>\n",
       "      <td>62.1</td>\n",
       "      <td>59.0</td>\n",
       "      <td>15686</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27429</th>\n",
       "      <td>27430</td>\n",
       "      <td>2.25</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI2</td>\n",
       "      <td>62.8</td>\n",
       "      <td>59.0</td>\n",
       "      <td>18034</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49556</th>\n",
       "      <td>49557</td>\n",
       "      <td>0.71</td>\n",
       "      <td>Good</td>\n",
       "      <td>F</td>\n",
       "      <td>SI2</td>\n",
       "      <td>64.1</td>\n",
       "      <td>60.0</td>\n",
       "      <td>2130</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49557</th>\n",
       "      <td>49558</td>\n",
       "      <td>0.71</td>\n",
       "      <td>Good</td>\n",
       "      <td>F</td>\n",
       "      <td>SI2</td>\n",
       "      <td>64.1</td>\n",
       "      <td>60.0</td>\n",
       "      <td>2130</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0  carat        cut color clarity  depth  table  price    x  \\\n",
       "11182       11183   1.07      Ideal     F     SI2   61.6   56.0   4954  0.0   \n",
       "11963       11964   1.00  Very Good     H     VS2   63.3   53.0   5139  0.0   \n",
       "15951       15952   1.14       Fair     G     VS1   57.5   67.0   6381  0.0   \n",
       "24520       24521   1.56      Ideal     G     VS2   62.2   54.0  12800  0.0   \n",
       "26243       26244   1.20    Premium     D    VVS1   62.1   59.0  15686  0.0   \n",
       "27429       27430   2.25    Premium     H     SI2   62.8   59.0  18034  0.0   \n",
       "49556       49557   0.71       Good     F     SI2   64.1   60.0   2130  0.0   \n",
       "49557       49558   0.71       Good     F     SI2   64.1   60.0   2130  0.0   \n",
       "\n",
       "          y    z  clarity_scale  color_scale  \n",
       "11182  6.62  0.0              1            2  \n",
       "11963  0.00  0.0              3            3  \n",
       "15951  0.00  0.0              4            0  \n",
       "24520  0.00  0.0              3            0  \n",
       "26243  0.00  0.0              6            4  \n",
       "27429  0.00  0.0              1            3  \n",
       "49556  0.00  0.0              1            2  \n",
       "49557  0.00  0.0              1            2  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[diamonds['z']==0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(diamonds['y'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Apply it just to the rows with incorrect values**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "calculate_z() takes 1 positional argument but 4 were given",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-136-fd2b7779009e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdiamonds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'z'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwhere\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdiamonds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'z'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m==\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m&\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mdiamonds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'x'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m!=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m&\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mdiamonds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'y'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m!=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcalculate_z\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdiamonds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'depth'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'x'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'y'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdiamonds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'z'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: calculate_z() takes 1 positional argument but 4 were given"
     ]
    }
   ],
   "source": [
    "diamonds['z']= np.where((diamonds['z']==0) & (diamonds['x']!=0) & (diamonds['y']!=0), calculate_z(diamonds, 'depth', 'x','y'), diamonds['z'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds[diamonds['z']==0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.23</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>E</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.5</td>\n",
       "      <td>55.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.95</td>\n",
       "      <td>3.98</td>\n",
       "      <td>2.43</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0.21</td>\n",
       "      <td>Premium</td>\n",
       "      <td>E</td>\n",
       "      <td>SI1</td>\n",
       "      <td>59.8</td>\n",
       "      <td>61.0</td>\n",
       "      <td>326</td>\n",
       "      <td>3.89</td>\n",
       "      <td>3.84</td>\n",
       "      <td>2.31</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0.23</td>\n",
       "      <td>Good</td>\n",
       "      <td>E</td>\n",
       "      <td>VS1</td>\n",
       "      <td>56.9</td>\n",
       "      <td>65.0</td>\n",
       "      <td>327</td>\n",
       "      <td>4.05</td>\n",
       "      <td>4.07</td>\n",
       "      <td>2.31</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.29</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>VS2</td>\n",
       "      <td>62.4</td>\n",
       "      <td>58.0</td>\n",
       "      <td>334</td>\n",
       "      <td>4.20</td>\n",
       "      <td>4.23</td>\n",
       "      <td>2.63</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.31</td>\n",
       "      <td>Good</td>\n",
       "      <td>J</td>\n",
       "      <td>SI2</td>\n",
       "      <td>63.3</td>\n",
       "      <td>58.0</td>\n",
       "      <td>335</td>\n",
       "      <td>4.34</td>\n",
       "      <td>4.35</td>\n",
       "      <td>2.75</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53935</th>\n",
       "      <td>53936</td>\n",
       "      <td>0.72</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>D</td>\n",
       "      <td>SI1</td>\n",
       "      <td>60.8</td>\n",
       "      <td>57.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.75</td>\n",
       "      <td>5.76</td>\n",
       "      <td>3.50</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53936</th>\n",
       "      <td>53937</td>\n",
       "      <td>0.72</td>\n",
       "      <td>Good</td>\n",
       "      <td>D</td>\n",
       "      <td>SI1</td>\n",
       "      <td>63.1</td>\n",
       "      <td>55.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.69</td>\n",
       "      <td>5.75</td>\n",
       "      <td>3.61</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53937</th>\n",
       "      <td>53938</td>\n",
       "      <td>0.70</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>D</td>\n",
       "      <td>SI1</td>\n",
       "      <td>62.8</td>\n",
       "      <td>60.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.66</td>\n",
       "      <td>5.68</td>\n",
       "      <td>3.56</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53938</th>\n",
       "      <td>53939</td>\n",
       "      <td>0.86</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI2</td>\n",
       "      <td>61.0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>6.15</td>\n",
       "      <td>6.12</td>\n",
       "      <td>3.74</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53939</th>\n",
       "      <td>53940</td>\n",
       "      <td>0.75</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>D</td>\n",
       "      <td>SI2</td>\n",
       "      <td>62.2</td>\n",
       "      <td>55.0</td>\n",
       "      <td>2757</td>\n",
       "      <td>5.83</td>\n",
       "      <td>5.87</td>\n",
       "      <td>3.64</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>53939 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0  carat        cut color clarity  depth  table  price     x  \\\n",
       "0               1   0.23      Ideal     E     SI2   61.5   55.0    326  3.95   \n",
       "1               2   0.21    Premium     E     SI1   59.8   61.0    326  3.89   \n",
       "2               3   0.23       Good     E     VS1   56.9   65.0    327  4.05   \n",
       "3               4   0.29    Premium     I     VS2   62.4   58.0    334  4.20   \n",
       "4               5   0.31       Good     J     SI2   63.3   58.0    335  4.34   \n",
       "...           ...    ...        ...   ...     ...    ...    ...    ...   ...   \n",
       "53935       53936   0.72      Ideal     D     SI1   60.8   57.0   2757  5.75   \n",
       "53936       53937   0.72       Good     D     SI1   63.1   55.0   2757  5.69   \n",
       "53937       53938   0.70  Very Good     D     SI1   62.8   60.0   2757  5.66   \n",
       "53938       53939   0.86    Premium     H     SI2   61.0   58.0   2757  6.15   \n",
       "53939       53940   0.75      Ideal     D     SI2   62.2   55.0   2757  5.83   \n",
       "\n",
       "          y     z  clarity_scale  color_scale  \n",
       "0      3.98  2.43              1            1  \n",
       "1      3.84  2.31              2            1  \n",
       "2      4.07  2.31              4            1  \n",
       "3      4.23  2.63              3            5  \n",
       "4      4.35  2.75              1            6  \n",
       "...     ...   ...            ...          ...  \n",
       "53935  5.76  3.50              2            0  \n",
       "53936  5.75  3.61              2            0  \n",
       "53937  5.68  3.56              2            0  \n",
       "53938  6.12  3.74              1            4  \n",
       "53939  5.87  3.64              1            0  \n",
       "\n",
       "[53939 rows x 13 columns]"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# I deleted the unique remaining row in which x and z equal zero for which we wont be able to calculate the z)\n",
    "\n",
    "diamonds.drop(labels=11182, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53932.000000</td>\n",
       "      <td>53933.000000</td>\n",
       "      <td>53932.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>26970.500000</td>\n",
       "      <td>0.797940</td>\n",
       "      <td>61.749405</td>\n",
       "      <td>57.457184</td>\n",
       "      <td>3932.799722</td>\n",
       "      <td>5.732007</td>\n",
       "      <td>5.734159</td>\n",
       "      <td>3.539735</td>\n",
       "      <td>3.051020</td>\n",
       "      <td>2.594197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>15571.281097</td>\n",
       "      <td>0.474011</td>\n",
       "      <td>1.432621</td>\n",
       "      <td>2.234491</td>\n",
       "      <td>3989.439738</td>\n",
       "      <td>1.119670</td>\n",
       "      <td>1.113273</td>\n",
       "      <td>0.692051</td>\n",
       "      <td>1.647136</td>\n",
       "      <td>1.701105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>326.000000</td>\n",
       "      <td>3.730000</td>\n",
       "      <td>3.680000</td>\n",
       "      <td>1.070000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>13485.750000</td>\n",
       "      <td>0.400000</td>\n",
       "      <td>61.000000</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>950.000000</td>\n",
       "      <td>4.710000</td>\n",
       "      <td>4.720000</td>\n",
       "      <td>2.910000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>26970.500000</td>\n",
       "      <td>0.700000</td>\n",
       "      <td>61.800000</td>\n",
       "      <td>57.000000</td>\n",
       "      <td>2401.000000</td>\n",
       "      <td>5.700000</td>\n",
       "      <td>5.710000</td>\n",
       "      <td>3.530000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>40455.250000</td>\n",
       "      <td>1.040000</td>\n",
       "      <td>62.500000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>5324.250000</td>\n",
       "      <td>6.540000</td>\n",
       "      <td>6.540000</td>\n",
       "      <td>4.040000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>53940.000000</td>\n",
       "      <td>5.010000</td>\n",
       "      <td>79.000000</td>\n",
       "      <td>95.000000</td>\n",
       "      <td>18823.000000</td>\n",
       "      <td>10.740000</td>\n",
       "      <td>19.278421</td>\n",
       "      <td>8.060000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Unnamed: 0         carat         depth         table         price  \\\n",
       "count  53940.000000  53940.000000  53940.000000  53940.000000  53940.000000   \n",
       "mean   26970.500000      0.797940     61.749405     57.457184   3932.799722   \n",
       "std    15571.281097      0.474011      1.432621      2.234491   3989.439738   \n",
       "min        1.000000      0.200000     43.000000     43.000000    326.000000   \n",
       "25%    13485.750000      0.400000     61.000000     56.000000    950.000000   \n",
       "50%    26970.500000      0.700000     61.800000     57.000000   2401.000000   \n",
       "75%    40455.250000      1.040000     62.500000     59.000000   5324.250000   \n",
       "max    53940.000000      5.010000     79.000000     95.000000  18823.000000   \n",
       "\n",
       "                  x             y             z  clarity_scale   color_scale  \n",
       "count  53932.000000  53933.000000  53932.000000   53940.000000  53940.000000  \n",
       "mean       5.732007      5.734159      3.539735       3.051020      2.594197  \n",
       "std        1.119670      1.113273      0.692051       1.647136      1.701105  \n",
       "min        3.730000      3.680000      1.070000       0.000000      0.000000  \n",
       "25%        4.710000      4.720000      2.910000       2.000000      1.000000  \n",
       "50%        5.700000      5.710000      3.530000       3.000000      3.000000  \n",
       "75%        6.540000      6.540000      4.040000       4.000000      4.000000  \n",
       "max       10.740000     19.278421      8.060000       7.000000      6.000000  "
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we leave the other 8 values as they are, it would negatively affect our analysis, because these are data that do not make logical sense. Therefore it is better to consider those values as NaN values, since they are probably the result of a mistake or error during process of measuring and storing these values in a dataset.\n",
    "\n",
    "To replace them we can use the pandas .replace() method and np.NaN.\n",
    "\n",
    "**Replace the zero values in the `z` column for a NaN**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds['x']=np.where(diamonds['x'] == '0', np.nan, diamonds['x'])\n",
    "diamonds['y']=np.where(diamonds['y'] == '0', np.nan, diamonds['y'])\n",
    "diamonds['z']=np.where(diamonds['z'] == '0', np.nan, diamonds['z'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds['x'].replace(0,np.nan,inplace=True)\n",
    "diamonds['y'].replace(0,np.nan,inplace=True)\n",
    "diamonds['z'].replace(0,np.nan,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "# Bonus: check the new z values\n",
    "Since we need to be 100% sure of our data, let's create a function that validates our z. To do so, we will use the same formula, but this time we will calculate the value of depth with the new value assigned to z.\n",
    "\n",
    "**Create a function named `validate_z` that compares the `z`  in cells above with the one thrown by the formula and run it with the rows you changed in the cells above**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the data again with the `describe()` method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The minimum value for x, y and z should now be a positive number, as it should be for the physical measurements of an object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's finish by checking for NaN values in the data. Since we introduced them ourselves using 'replace', we will surely find some, but there may be more that are unrelated to the x, y and z columns. Checking NaNs is a fundamental part of data cleaning and it's always better to do this kind of operations before proceeding with analysis.\n",
    "\n",
    "**Check how many NaNs do you have, comment what you would do with those values, and then do so**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed: 0       0\n",
       "carat            0\n",
       "cut              0\n",
       "color            0\n",
       "clarity          0\n",
       "depth            0\n",
       "table            0\n",
       "price            0\n",
       "x                8\n",
       "y                7\n",
       "z                8\n",
       "clarity_scale    0\n",
       "color_scale      0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 142,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [],
   "source": [
    "# deleting the rows where all x, y and z values are equal to 0\n",
    "\n",
    "indexNames = diamonds[ (diamonds['x'] == 0) & (diamonds['y'] == 0 ) & (diamonds['z'] == 0) ].index\n",
    "diamonds.drop(indexNames , inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed: 0       0\n",
       "carat            0\n",
       "cut              0\n",
       "color            0\n",
       "clarity          0\n",
       "depth            0\n",
       "table            0\n",
       "price            0\n",
       "x                8\n",
       "y                7\n",
       "z                8\n",
       "clarity_scale    0\n",
       "color_scale      0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Checking for outliers\n",
    "Now we are going to revisit the summary table to check for outliers.\n",
    "\n",
    "**Use the `describe` method again and comment on what you see. After that, check if you have any outliers** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53932.000000</td>\n",
       "      <td>53933.000000</td>\n",
       "      <td>53932.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "      <td>53940.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>26970.500000</td>\n",
       "      <td>0.797940</td>\n",
       "      <td>61.749405</td>\n",
       "      <td>57.457184</td>\n",
       "      <td>3932.799722</td>\n",
       "      <td>5.732007</td>\n",
       "      <td>5.734159</td>\n",
       "      <td>3.539735</td>\n",
       "      <td>3.051020</td>\n",
       "      <td>2.594197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>15571.281097</td>\n",
       "      <td>0.474011</td>\n",
       "      <td>1.432621</td>\n",
       "      <td>2.234491</td>\n",
       "      <td>3989.439738</td>\n",
       "      <td>1.119670</td>\n",
       "      <td>1.113273</td>\n",
       "      <td>0.692051</td>\n",
       "      <td>1.647136</td>\n",
       "      <td>1.701105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>326.000000</td>\n",
       "      <td>3.730000</td>\n",
       "      <td>3.680000</td>\n",
       "      <td>1.070000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>13485.750000</td>\n",
       "      <td>0.400000</td>\n",
       "      <td>61.000000</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>950.000000</td>\n",
       "      <td>4.710000</td>\n",
       "      <td>4.720000</td>\n",
       "      <td>2.910000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>26970.500000</td>\n",
       "      <td>0.700000</td>\n",
       "      <td>61.800000</td>\n",
       "      <td>57.000000</td>\n",
       "      <td>2401.000000</td>\n",
       "      <td>5.700000</td>\n",
       "      <td>5.710000</td>\n",
       "      <td>3.530000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>40455.250000</td>\n",
       "      <td>1.040000</td>\n",
       "      <td>62.500000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>5324.250000</td>\n",
       "      <td>6.540000</td>\n",
       "      <td>6.540000</td>\n",
       "      <td>4.040000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>53940.000000</td>\n",
       "      <td>5.010000</td>\n",
       "      <td>79.000000</td>\n",
       "      <td>95.000000</td>\n",
       "      <td>18823.000000</td>\n",
       "      <td>10.740000</td>\n",
       "      <td>19.278421</td>\n",
       "      <td>8.060000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Unnamed: 0         carat         depth         table         price  \\\n",
       "count  53940.000000  53940.000000  53940.000000  53940.000000  53940.000000   \n",
       "mean   26970.500000      0.797940     61.749405     57.457184   3932.799722   \n",
       "std    15571.281097      0.474011      1.432621      2.234491   3989.439738   \n",
       "min        1.000000      0.200000     43.000000     43.000000    326.000000   \n",
       "25%    13485.750000      0.400000     61.000000     56.000000    950.000000   \n",
       "50%    26970.500000      0.700000     61.800000     57.000000   2401.000000   \n",
       "75%    40455.250000      1.040000     62.500000     59.000000   5324.250000   \n",
       "max    53940.000000      5.010000     79.000000     95.000000  18823.000000   \n",
       "\n",
       "                  x             y             z  clarity_scale   color_scale  \n",
       "count  53932.000000  53933.000000  53932.000000   53940.000000  53940.000000  \n",
       "mean       5.732007      5.734159      3.539735       3.051020      2.594197  \n",
       "std        1.119670      1.113273      0.692051       1.647136      1.701105  \n",
       "min        3.730000      3.680000      1.070000       0.000000      0.000000  \n",
       "25%        4.710000      4.720000      2.910000       2.000000      1.000000  \n",
       "50%        5.700000      5.710000      3.530000       3.000000      3.000000  \n",
       "75%        6.540000      6.540000      4.040000       4.000000      4.000000  \n",
       "max       10.740000     19.278421      8.060000       7.000000      6.000000  "
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I don't have zero values anymore and my data is basically ready to be plotted"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To manage these outliers, we are going to filter our DataFrame, we're going to take all the values that have a price higher than the 75th percentile.\n",
    "\n",
    "**Look for that quantile and filter the dataframe to clearly see the outliers. What do you think?**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our dataset is really big and the outliers are really far apart from the rest of the values. To see this more clearly we will use a boxplot, which plots the median, 25th and 75th quartile, the maximum and minimum, as well as any outliers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAEkCAYAAAD+cD+8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAZyklEQVR4nO3df5Bd5X3f8c9He7ElERPDZS2IQF2TTXAaHMBcY2LGkIC23TjYZNpkBmaI1gnTnbrp7lq1axunYyZ/xOO6OEVSZtLuGMLVhMFjY9ykrrxlReJgdwjuCuRIRErZEglpBWi5Ghsa8UN399s/9sJIO9Lq157nuefu+zWzs/uce8TzGY04+ui55z7HESEAAAAUb1nuAAAAAEsFxQsAACARihcAAEAiFC8AAIBEKF4AAACJULwAAAASKax42b7U9l/Z3mX7GdsjreMX2B63/Wzr+/lFZQAAAGgnLmofL9sXS7o4Ip6y/S5J2yT9hqRPSDoUEV+2/XlJ50fE5woJAQAA0EYKW/GKiBci4qnWz69K2iVptaRbJdVbp9U1V8YWZHusoJgAFsD/ewCwuCopJrHdI+lqSU9KWhURL0hz5cz2e07wawYlDUrS8uXLr6nVamyxD6T3Su4AOHMXXnhh9PT05I4BLDnbtm17OSK6j/da4cXL9k9J+pakT0XEK7ZP6ddFxKikUUmq1WoxMTFRXEgAx2X72dwZcOZ6enrEtRNIz/beE71W6KcabZ+judL1YEQ80jr8Uuv+r7fuAztYZAYAAIB2UeSnGi3pPkm7IuKPjnrpLyQNtH4ekPTnRWUAAABoJ0W+1Xi9pN+WtMP29taxL0j6sqRv2L5T0vOSfqvADAAAAG2jsOIVET+QdKIbum4ual4AAIB2xc71AAAAiVC8AAAAEqF4oVCNRkPDw8NqNBq5owBAKXDd7GwULxSqXq9rx44d2rx5c+4oAFAKXDc7G8ULhWk0GhobG1NEaGxsjH+9AcBJcN3sfBQvFKZer2t2dlaSNDMzw7/egLNke73tZ2zvtP2Q7eW5M2Fxcd3sfBQvFGbr1q1qNpuSpGazqfHx8cyJgPKyvVrSsKRaRFwhqUvSbXlTYbFx3ex8FC8UZu3atapU5raKq1Qq6uvry5wIKL2KpBW2K5JWSjqQOQ8WGdfNzkfxQmEGBga0bNncH7Guri6tW7cucyKgvCJiStI9mnvixwuSfhIRj84/z/ag7QnbE9PT06lj4ixx3ex8FC8Uplqtqr+/X7bV39+varWaOxJQWrbPl3SrpPdK+hlJ59q+Y/55ETEaEbWIqHV3d6eOibPEdbPzUbxQqIGBAb3//e/nX23A2Vsr6R8iYjoijkh6RNKHM2dCAbhudrYiH5INqFqtauPGjbljAJ3geUnX2V4p6TXNPfN2Im8kFIHrZmdjxQsASiAinpT0sKSnJO3Q3PV7NGsoAKeNFS8AKImIuFvS3blzADhzrHgBAAAkQvECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJFJY8bJ9v+2Dtncedewq239je7vtCdvXFjU/AABAuylyxesBSf3zjn1F0h9ExFWSvtgaAwAALAmFFa+IeFzSofmHJZ3X+vmnJR0oan4AAIB2k/oh2Z+S9D9t36O50vfhE51oe1DSoCStWbMmSTgAAIAipb65/pOS1kfEpZLWS7rvRCdGxGhE1CKi1t3dnSwgAABAUVIXrwFJj7R+/qYkbq4HAABLRuridUDSja2fb5L0bOL5AQAAsinsHi/bD0n6FUkX2t4v6W5J/0rSBtsVSa+rdQ8XAADAUlBY8YqI20/w0jVFzQkAANDO2LkeAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJELxAgAASITiBQAAkAjFCwBKwPbltrcf9fWK7U/lzoXF12g0NDw8rEajkTsKCkDxAoASiIi/j4irIuIqzT0B5LCkb+dNhSLU63Xt2LFDmzdvzh0FBaB4AUD53Czp/0bE3txBsLgajYbGxsYUERobG2PVqwNRvACgfG6T9NDxXrA9aHvC9sT09HTiWDhb9Xpds7OzkqSZmRlWvToQxQsASsT2OyR9XNI3j/d6RIxGRC0iat3d3WnD4axt3bpVzWZTktRsNjU+Pp45ERYbxQsAyuXXJD0VES/lDoLFt3btWlUqFUlSpVJRX19f5kRYbBQvACiX23WCtxlRfgMDA1q2bO6v5q6uLq1bty5zIiw2ihcAlITtlZL6JD2SOwuKUa1W1d/fL9vq7+9XtVrNHQmLrJI7AADg1ETEYUn8TdzhBgYGtGfPHla7OhQrXgAAAIlQvAAAaCNsoNrZKF4AALQJNlDtfBQvAADaBBuodj6KFwAAbYINVDsfxQsAgDbBBqqdr7DiZft+2wdt75x3fMj239t+xvZXipofAICyYQPVzlfkitcDkvqPPmD7VyXdKumXIuIXJd1T4PwAAJQKG6h2vsI2UI2Ix233zDv8SUlfjog3WuccLGp+AADKiA1UO1vqe7x+XtJHbD9p+69tf/BEJ9oetD1he2J6ejphRAAA8qlWq9q4cSOrXR0qdfGqSDpf0nWS/r2kb9j28U6MiNGIqEVErbu7O2VGAACyaTQaGh4eZg+vDpW6eO2X9EjM+aGkWUkXJs4AAEDbYuf6zpa6eP03STdJku2fl/QOSS8nzgAAQFs6euf67373u6x6daAit5N4SNITki63vd/2nZLul3RZa4uJr0saiIgoKgMAAGVSr9d15MgRSdKRI0dY9epARX6q8fYTvHRHUXMCAFBm4+Pjems9IiL06KOPav369ZlTYTGxcz0AAG1i1apVC45RfhQvAADaxEsvvbTgGOVH8QIAoE185CMfOWZ8ww03ZEqColC8AABoE2+++eYx4zfeeCNTEhSF4gUAQJv4/ve/v+AY5UfxAgCgTczOzi44RvlRvAAAaBPzn6J3gqfqocQoXgAAtIkPfehDC45RfhQvAADaxJ49e44Z7927N08QFIbiBQBAm3jhhReOGR84cCBTEhSF4gUAQJvgHq/OR/ECgJKw/W7bD9vebXuX7V/OnQmL68Ybb1xwjPKjeAFAeWyQNBYR75N0paRdmfNgkd1yyy3HjD/2sY9lSoKiULwAoARsnyfpBkn3SVJEvBkRP84aCotuw4YNx4zvvffePEFQGIoXAJTDZZKmJf2p7adtf832ublDYXHt27dvwTHKj+IFAOVQkfQBSX8SEVdL+kdJn59/ku1B2xO2J6anp1NnBHASFC8AKIf9kvZHxJOt8cOaK2LHiIjRiKhFRK27uztpQAAnR/ECgBKIiBcl7bN9eevQzZL+LmMkFIDtJDpfJXcAAMApG5L0oO13SHpO0u9kzoNFFhELjlF+FC8AKImI2C6pljsHgDPHW40AAACJULwAAGgTy5cvX3CM8iuseNm+3/ZB2zuP89pnbIftC4uaHwCAsnn99dcXHKP8irzH6wFJfyxp89EHbV8qqU/S8wXODQCAJGnTpk2anJzMHeOMjYyM5I5wSnp7ezU0NJQ7RtsrbMUrIh6XdOg4L/1nSZ+VxEc1AADAkpL0U422Py5pKiJ+dLK9SWwPShqUpDVr1iRIBwDoRGVahfn0pz+tbdu2vT2+5ppr9NWvfjVjIiy2ZDfX214p6fclffFUzmf3ZQDAUvOFL3xhwTHKL+WnGn9W0nsl/cj2HkmXSHrK9kUJMwAA0Laq1apWrlwpaW61q1qtZk6ExZbsrcaI2CHpPW+NW+WrFhEvp8oAAEC76+np0d69e1nt6lBFbifxkKQnJF1ue7/tO4uaCwCATnHOOeeot7eX1a4OVdiKV0TcfpLXe4qaGwAAoB2xcz0AAEAiFC8AAIBEKF4AAACJULwAAAASoXgBAAAkQvECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJELxAgAASITiBQAAkAjFCwAAIJFK7gAAgFNje4+kVyXNSGpGRC1vIgCni+IFAOXyqxHxcu4QAM4MbzUCAAAkQvECgPIISY/a3mZ78Hgn2B60PWF7Ynp6OnE8ACdD8QKA8rg+Ij4g6dck/Z7tG+afEBGjEVGLiFp3d3f6hAAWRPECgJKIiAOt7wclfVvStXkTAThdFC8AKAHb59p+11s/S/pnknbmTQXgdBVWvGzfb/ug7Z1HHftPtnfb/lvb37b97qLmB4AOs0rSD2z/SNIPJf2PiBjLnAnAaSpyxesBSf3zjo1LuiIifknS/5F0V4HzA0DHiIjnIuLK1tcvRsQf5s4E4PQVVrwi4nFJh+YdezQimq3h30i6pKj5AQAA2k3Oe7x+V9J3T/QiH4kGAACdJkvxsv37kpqSHjzROXwkGgAAdJrkjwyyPSDpFkk3R0Sknh8AACCXpMXLdr+kz0m6MSIOp5wbAAAgtyK3k3hI0hOSLre93/adkv5Y0rskjdvebvu/FDU/AABAuynyU423R8TFEXFORFwSEfdFRG9EXBoRV7W+/nVR86M9NBoNDQ8Pq9Fo5I4CAEB27FyPQtXrde3YsUObN2/OHQUAgOxOWrxs/1vb56cIg87SaDQ0NjamiNDY2BirXgCAJe9UVrwukvS/bX/Ddr9tFx0KnaFer2t2dlaSNDMzw6oX0GL7MdsfnXdsNFceAOmctHhFxH+Q9HOS7pP0CUnP2v6S7Z8tOBtKbuvWrWo25x5U0Gw2NT4+njkR0DbeK+lztu8+6lgtVxgA6ZzSPV6t/bZebH01JZ0v6WHbXykwG0pu7dq1qlTmdiypVCrq6+vLnAhoGz+WdLOkVbb/u+2fzpwHQCKnco/XsO1tkr4i6X9Jen9EfFLSNZL+ZcH5UGIDAwNatmzuj1hXV5fWrVuXORHQNhwRzYj4N5K+JekHkt6TOROABE5lxetCSf8iIv55RHwzIo5IUkTMam4HeuC4qtWq+vv7ZVv9/f2qVqu5IwHt4u09DCPiAc3dxvForjAA0jnpzvUR8cUFXtu1uHHQaQYGBrRnzx5Wu4CjRMR/nTfeJul3M8UBkFDyZzViaalWq9q4cWPuGAAAtAU2UAUAAEiE4gUAAJAIxQsAACARihcAAEAiFC8UqtFoaHh4mOc0AgAgihcKVq/XtWPHDp7TCACAKF4oUKPR0NjYmCJCY2NjrHoBAJY8ihcKU6/XNTs7K0mamZlh1QsAsORRvFCYrVu3qtlsSpKazabGx8czJwIAIC+KFwqzdu1aVSpzD0eoVCrq6+vLnAgAgLwoXijMwMCAbEuSli1bxvMagUVgu8v207a/kzsLgNPHsxpRmGq1qosuukj79u3TqlWrVK1Wc0cCOsGIpF2Szssx+aZNmzQ5OZlj6iXjrd/fkZGRzEk6X29vr4aGhpLOSfFCYRqNhqampiRJU1NTajQalC/gLNi+RNKvS/pDSf8uR4bJyUlt37lLMysvyDH9krDszZAkbXvupcxJOlvX4UNZ5qV4oTCjo6Nvf6pxdnZWo6OjuuuuuzKnAkrtXkmflfSuE51ge1DSoCStWbOmkBAzKy/Qa+/7aCH/bSCVFbu3ZJm3sHu8bN9v+6DtnUcdu8D2uO1nW9/PL2p+5PfYY48tOAZw6mzfIulgRGxb6LyIGI2IWkTUuru7E6UDcKqKvLn+AUn98459XtJjEfFzkh5rjdGhImLBMYDTcr2kj9veI+nrkm6y/Wd5IwE4XYUVr4h4XNL8N1BvlVRv/VyX9BtFzY/8PvjBDx4zvvbaazMlAcovIu6KiEsiokfSbZL+MiLuyBwLwGlKvZ3Eqoh4QZJa399zohNtD9qesD0xPT2dLCAWz/PPP3/MeO/evZmSAADQHtp2Hy/uUyi/AwcOLDgGcGYi4nsRcUvuHABOX+ri9ZLtiyWp9f1g4vkBAACySV28/kLSQOvnAUl/nnh+JNTV1bXgGACApabI7SQekvSEpMtt77d9p6QvS+qz/aykvtYYHWr+zfTcXA8AWOoK20A1Im4/wUs3FzUn2su+ffuOGe/fvz9TEgAA2kPb3lyP8ptftOYXMQAAlhqKFwrDPV4AAByL4oXCzMzMLDgGAGCpoXgBAAAkQvECAABIpLBPNWLxbdq0SZOTk7ljnJWRkZHcEU6qt7dXQ0NDuWMAADoQxQuFsa2IOGYMoNympqbUdfgnWrF7S+4owFnpOtzQ1FQz+bwUrxIp2yrMxMSEPvOZz7w9vueee3TNNddkTAQAQF4ULxSmVqu9ver1zne+k9IFdIDVq1frxTcqeu19H80dBTgrK3Zv0erVq5LPy831KFRPT48k6Utf+lLeIAAAtAGKFwp13nnn6corr2S1CwAAUbwAAACSoXgBAAAkQvECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJELxAgAASCRL8bK93vYztnfafsj28hw5AKAsbC+3/UPbP2pdP/8gdyYApy958bK9WtKwpFpEXCGpS9JtqXMAQMm8IemmiLhS0lWS+m1flzcSgNNVyTjvCttHJK2UdCBTDgAohYgISf+vNTyn9RX5EgE4E8lXvCJiStI9kp6X9IKkn0TEo6lzAEDZ2O6yvV3SQUnjEfFk5kgATlOOtxrPl3SrpPdK+hlJ59q+4zjnDdqesD0xPT2dOiYAtJ2ImImIqyRdIula21fMP4drJ9Dectxcv1bSP0TEdEQckfSIpA/PPykiRiOiFhG17u7u5CEBoF1FxI8lfU9S/3Fe49oJtLEc93g9L+k62yslvSbpZkkTGXIAQGnY7pZ0JCJ+bHuF5v4R+x9zZOk6fEgrdm/JMfWSsOz1VyRJs8vPy5yks3UdPiRpVfJ5kxeviHjS9sOSnpLUlPS0pNHUOQCgZC6WVLfdpbl3K74REd9JHaK3tzf1lEvO5OSrkqTey9KXgqVlVZY/z1k+1RgRd0u6O8fcAFBGEfG3kq7OnWNoaCh3hI43MjIiSdqwYUPmJCgCO9cDAAAkQvECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJELxAgAASITiBQAAkAjFCwAAIBGKFwAAQCIULwAAgEQoXgAAAIlQvAAAABKheAEAACRSyR0gl02bNmlycjJ3jI731u/xyMhI5iSdrbe3V0NDQ7ljAABOYskWr8nJSW3fuUszKy/IHaWjLXszJEnbnnspc5LO1XX4UO4IAIBTtGSLlyTNrLxAr73vo7ljAGdlxe4tuSMAAE4R93gBAAAkQvECAABIhOIFAACQSJbiZfvdth+2vdv2Ltu/nCMHAABASrlurt8gaSwiftP2OyStzJQDAAAgmeTFy/Z5km6Q9AlJiog3Jb2ZOgcAAEBqOd5qvEzStKQ/tf207a/ZPnf+SbYHbU/Ynpienk6fEgAAYJHlKF4VSR+Q9CcRcbWkf5T0+fknRcRoRNQiotbd3Z06IwC0FduX2v6r1n2xz9jmcRBACeUoXvsl7Y+IJ1vjhzVXxAAAJ9aU9OmI+AVJ10n6Pdv/NHMmAKcpefGKiBcl7bN9eevQzZL+LnUOACiTiHghIp5q/fyqpF2SVudNBeB05fpU45CkB1ufaHxO0u+kDjA1NaWuwz/hcSsova7DDU1NNXPHQEK2eyRdLenJ47w2KGlQktasWZM2GICTylK8ImK7pFqOuQGgzGz/lKRvSfpURLwy//WIGJU0Kkm1Wi0SxwNwEkv2IdmrV6/Wi29UeEg2Sm/F7i1avXpV7hhIwPY5mitdD0bEI7nzADh9PDIIAErAtiXdJ2lXRPxR7jwAzgzFCwDK4XpJvy3pJtvbW18s2QMls2TfagSAMomIH0hy7hwAzg4rXgAAAIlQvAAAABKheAEAACRC8QIAAEiE4gUAAJAIxQsAACARihcAAEAiFC8AAIBEKF4AAACJLOmd67sOH9KK3Vtyx+hoy15/RZI0u/y8zEk6V9fhQ5J4SDYAlMGSLV69vb25IywJk5OvSpJ6L6MYFGcVf54BoCSWbPEaGhrKHWFJGBkZkSRt2LAhcxIAAPLjHi8AAIBEKF4AAACJLNm3GgEAS8OmTZs0OTmZO8YpeyvrW7dqlEVvby+38ZwCihcAAG1kxYoVuSOgQBQvAEBHYxUG7YR7vAAAABKheAEAACSSrXjZ7rL9tO3v5MoAAACQUs4VrxFJuzLODwAAkFSW4mX7Ekm/LulrOeYHAADIIdeK172SPitp9kQn2B60PWF7Ynp6OlkwAACAoiQvXrZvkXQwIrYtdF5EjEZELSJq3d3didIBAJBXo9HQ8PCwGo1G7igoQI4Vr+slfdz2Hklfl3ST7T/LkAMAgLZTr9e1Y8cObd68OXcUFCB58YqIuyLikojokXSbpL+MiDtS5wAAoN00Gg2NjY0pIjQ2NsaqVwdi5/oSKdvzxqRyPnOM542hHdm+X9Jbt2pckTsPilGv1zU7O3f788zMjDZv3qz169dnToXFlHUD1Yj4XkTckjMDirVixQqeOwYsjgck9ecOgWJt3bpVzWZTktRsNjU+Pp45ERYbK14lwioMsHRFxOO2e3LnQLHWrl2rLVu2qNlsqlKpqK+vL3ckLDIeGQQAHYSteMptYGBAy5bN/dXc1dWldevWZU6ExUbxAoAOwlY85VatVtXf3y/b6u/vV7VazR0Ji4y3GgEAaCMDAwPas2cPq10diuIFAEAbqVar2rhxY+4YKAhvNQJACdh+SNITki63vd/2nbkzATh9rHgBQAlExO25MwA4e6x4AQAAJELxAgAASITiBQAAkAjFCwAAIBFHRO4MJ2V7WtLe3Dlwxi6U9HLuEDgj/yQi2IWzpLh2lhrXzXI74bWzFMUL5WZ7IiJquXMAQFlw3excvNUIAACQCMULAAAgEYoXUhjNHQAASobrZofiHi8AAIBEWPECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARP4/pLxTq5yNR/gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Run this code\n",
    "fig, ax = plt.subplots(1,2, figsize=(10, 5))\n",
    "sns.boxplot(y=diamonds.y, ax=ax[0])\n",
    "sns.boxplot(y=diamonds.z, ax=ax[1])\n",
    "plt.subplots_adjust(wspace=0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can see that all the values are within an acceptable range, but we have 2 big outliers in y and 1 in z. Now we know that our max values for y should be around 10 and the values for z should be around 6, so let's filter our dataset to find values higher than 10 in it.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [Unnamed: 0, carat, cut, color, clarity, depth, table, price, x, y, z, clarity_scale, color_scale]\n",
       "Index: []"
      ]
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[(diamonds.z)>10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have found the outlier, let's use the function we defined earlier to correct this value. First, we need to change the value to 0 (because that's how we defined the function before) and then we will apply it.\n",
    "\n",
    "**Apply `calculate_z` for the row with the outlier**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds.at[48410,'z']=0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I tried to apply the calculate_z function in the following ways but none worked, so i did without the function\n",
    "\n",
    "# diamonds.at[48410,'z']= calculate_z (diamonds.at[48410,'z'], diamonds.at[48410,'depth'], diamonds.at[48410,'x'],diamonds.at[48410,'y']), diamonds.at[48410,'z']\n",
    "# diamonds.at[48410,'z']=calculate_z(diamonds.at[48410,'depth'][i]*((diamonds.at[48410,'x'][i])+(diamonds.at[48410,'y'][i]))/(2*100))\n",
    "# diamonds.at[48410,'z']=calculate_z(diamonds.at['depth'][48410]*((diamonds.at['x'][48410])+(diamonds.at['y'][48410]))/(2*100))\n",
    "# diamonds.at[48410,'z']=calculate_z(diamonds.apply(lambda x: calculate_z(diamonds.at[48410,'x'], diamonds.at[48410,'y'], diamonds.at[48410,'depth'])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds.at[48410,'z']=diamonds.at[48410,'depth']*((diamonds.at[48410,'x'])+(diamonds.at[48410,'y']))/(2*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check if we actually corrected the outlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed: 0           48411\n",
       "carat                 0.51\n",
       "cut              Very Good\n",
       "color                    E\n",
       "clarity                VS1\n",
       "depth                 61.8\n",
       "table                 54.7\n",
       "price                 1970\n",
       "x                     5.12\n",
       "y                     5.15\n",
       "z                  3.17343\n",
       "clarity_scale            4\n",
       "color_scale              1\n",
       "Name: 48410, dtype: object"
      ]
     },
     "execution_count": 150,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds.loc[48410]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cool! Now let's validate our new `z`. We will check if we obtain the same value of depth using our validate function. If the formula applies, this means could approximate the real value of `z`.\n",
    "\n",
    "**Apply `validate_z` to the row used earlier**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's do the same for `y`. First, let's filter the DataFrame to find the outliers. We said that the maximum values should be around 10, so let's check what are the values above 10.\n",
    "\n",
    "**Check the values greater than 10 in the `y` column** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>24067</th>\n",
       "      <td>24068</td>\n",
       "      <td>2.00</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI2</td>\n",
       "      <td>58.9</td>\n",
       "      <td>57.0</td>\n",
       "      <td>12210</td>\n",
       "      <td>8.09</td>\n",
       "      <td>19.278421</td>\n",
       "      <td>8.06</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25998</th>\n",
       "      <td>25999</td>\n",
       "      <td>4.01</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>I1</td>\n",
       "      <td>61.0</td>\n",
       "      <td>61.0</td>\n",
       "      <td>15223</td>\n",
       "      <td>10.14</td>\n",
       "      <td>10.089508</td>\n",
       "      <td>6.17</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27415</th>\n",
       "      <td>27416</td>\n",
       "      <td>5.01</td>\n",
       "      <td>Fair</td>\n",
       "      <td>J</td>\n",
       "      <td>I1</td>\n",
       "      <td>65.5</td>\n",
       "      <td>59.0</td>\n",
       "      <td>18018</td>\n",
       "      <td>10.74</td>\n",
       "      <td>10.572977</td>\n",
       "      <td>6.98</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27630</th>\n",
       "      <td>27631</td>\n",
       "      <td>4.50</td>\n",
       "      <td>Fair</td>\n",
       "      <td>J</td>\n",
       "      <td>I1</td>\n",
       "      <td>65.8</td>\n",
       "      <td>58.0</td>\n",
       "      <td>18531</td>\n",
       "      <td>10.23</td>\n",
       "      <td>10.195532</td>\n",
       "      <td>6.72</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49189</th>\n",
       "      <td>49190</td>\n",
       "      <td>0.51</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>E</td>\n",
       "      <td>VS1</td>\n",
       "      <td>61.8</td>\n",
       "      <td>55.0</td>\n",
       "      <td>2075</td>\n",
       "      <td>5.15</td>\n",
       "      <td>11.419579</td>\n",
       "      <td>5.12</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0  carat      cut color clarity  depth  table  price      x  \\\n",
       "24067       24068   2.00  Premium     H     SI2   58.9   57.0  12210   8.09   \n",
       "25998       25999   4.01  Premium     I      I1   61.0   61.0  15223  10.14   \n",
       "27415       27416   5.01     Fair     J      I1   65.5   59.0  18018  10.74   \n",
       "27630       27631   4.50     Fair     J      I1   65.8   58.0  18531  10.23   \n",
       "49189       49190   0.51    Ideal     E     VS1   61.8   55.0   2075   5.15   \n",
       "\n",
       "               y     z  clarity_scale  color_scale  \n",
       "24067  19.278421  8.06              1            4  \n",
       "25998  10.089508  6.17              0            5  \n",
       "27415  10.572977  6.98              0            6  \n",
       "27630  10.195532  6.72              0            6  \n",
       "49189  11.419579  5.12              4            1  "
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[(diamonds.y)>10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can clearly see that the 31.8 in row 49189 is an outlier for the y value. Also, we can see that the 58.9 value for `y` in row 24067 is actually its depth, so it was a mistake when they introduced the data. Let's create a function to fix these outliers.\n",
    "\n",
    "**Create a function named `calculate_y` to calculate `y` using `z` and `x` the same way you did above**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.069507908611599"
      ]
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def calculate_y(i):\n",
    "    y=((diamonds['z'][i]*(2*100))/diamonds['depth'][i])-diamonds['x'][i]\n",
    "    return y\n",
    "calculate_y(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "calculate_y() takes 1 positional argument but 4 were given",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-153-f9ef3caefd1a>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Here too, I couldn't implement the calculate_y function grrr, so I implemented the formula on each row index that appeared above\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mdiamonds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'y'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwhere\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdiamonds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'y'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m>\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcalculate_y\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdiamonds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'depth'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'x'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'z'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdiamonds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'y'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: calculate_y() takes 1 positional argument but 4 were given"
     ]
    }
   ],
   "source": [
    "# Here too, I couldn't implement the calculate_y function grrr, so I implemented the formula on each row index that appeared above\n",
    "\n",
    "diamonds['y']= np.where((diamonds['y']>10), calculate_y(diamonds, 'depth', 'x','z'), diamonds['y'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds.at[24067,'y']=((diamonds.at[24067,'z']*(2*100))/diamonds.at[24067,'depth'])-diamonds.at[24067,'x']\n",
    "diamonds.at[25998,'y']=((diamonds.at[25998,'z']*(2*100))/diamonds.at[25998,'depth'])-diamonds.at[25998,'x']\n",
    "diamonds.at[27415,'y']=((diamonds.at[27415,'z']*(2*100))/diamonds.at[27415,'depth'])-diamonds.at[27415,'x']\n",
    "diamonds.at[27630,'y']=((diamonds.at[27630,'z']*(2*100))/diamonds.at[27630,'depth'])-diamonds.at[27630,'x']\n",
    "diamonds.at[49189,'y']=((diamonds.at[49189,'z']*(2*100))/diamonds.at[49189,'depth'])-diamonds.at[49189,'x']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will check the rows that had an outlier in `y`, to check that the values were changed.\n",
    "\n",
    "**Check those rows (also validating with your function) and comment what you see**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>24067</th>\n",
       "      <td>24068</td>\n",
       "      <td>2.00</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI2</td>\n",
       "      <td>58.9</td>\n",
       "      <td>57.0</td>\n",
       "      <td>12210</td>\n",
       "      <td>8.09</td>\n",
       "      <td>19.278421</td>\n",
       "      <td>8.06</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25998</th>\n",
       "      <td>25999</td>\n",
       "      <td>4.01</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>I1</td>\n",
       "      <td>61.0</td>\n",
       "      <td>61.0</td>\n",
       "      <td>15223</td>\n",
       "      <td>10.14</td>\n",
       "      <td>10.089508</td>\n",
       "      <td>6.17</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27415</th>\n",
       "      <td>27416</td>\n",
       "      <td>5.01</td>\n",
       "      <td>Fair</td>\n",
       "      <td>J</td>\n",
       "      <td>I1</td>\n",
       "      <td>65.5</td>\n",
       "      <td>59.0</td>\n",
       "      <td>18018</td>\n",
       "      <td>10.74</td>\n",
       "      <td>10.572977</td>\n",
       "      <td>6.98</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27630</th>\n",
       "      <td>27631</td>\n",
       "      <td>4.50</td>\n",
       "      <td>Fair</td>\n",
       "      <td>J</td>\n",
       "      <td>I1</td>\n",
       "      <td>65.8</td>\n",
       "      <td>58.0</td>\n",
       "      <td>18531</td>\n",
       "      <td>10.23</td>\n",
       "      <td>10.195532</td>\n",
       "      <td>6.72</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49189</th>\n",
       "      <td>49190</td>\n",
       "      <td>0.51</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>E</td>\n",
       "      <td>VS1</td>\n",
       "      <td>61.8</td>\n",
       "      <td>55.0</td>\n",
       "      <td>2075</td>\n",
       "      <td>5.15</td>\n",
       "      <td>11.419579</td>\n",
       "      <td>5.12</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0  carat      cut color clarity  depth  table  price      x  \\\n",
       "24067       24068   2.00  Premium     H     SI2   58.9   57.0  12210   8.09   \n",
       "25998       25999   4.01  Premium     I      I1   61.0   61.0  15223  10.14   \n",
       "27415       27416   5.01     Fair     J      I1   65.5   59.0  18018  10.74   \n",
       "27630       27631   4.50     Fair     J      I1   65.8   58.0  18531  10.23   \n",
       "49189       49190   0.51    Ideal     E     VS1   61.8   55.0   2075   5.15   \n",
       "\n",
       "               y     z  clarity_scale  color_scale  \n",
       "24067  19.278421  8.06              1            4  \n",
       "25998  10.089508  6.17              0            5  \n",
       "27415  10.572977  6.98              0            6  \n",
       "27630  10.195532  6.72              0            6  \n",
       "49189  11.419579  5.12              4            1  "
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[(diamonds.y)>10]\n",
    "# all values have been altered as per my formula, however, the y values are still above 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have corrected or dropped all of our outliers, lets plot another box plot to double check."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAEkCAYAAAD+cD+8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAZyklEQVR4nO3df5Bd5X3f8c9He7ElERPDZS2IQF2TTXAaHMBcY2LGkIC23TjYZNpkBmaI1gnTnbrp7lq1axunYyZ/xOO6OEVSZtLuGMLVhMFjY9ykrrxlReJgdwjuCuRIRErZEglpBWi5Ghsa8UN399s/9sJIO9Lq157nuefu+zWzs/uce8TzGY04+ui55z7HESEAAAAUb1nuAAAAAEsFxQsAACARihcAAEAiFC8AAIBEKF4AAACJULwAAAASKax42b7U9l/Z3mX7GdsjreMX2B63/Wzr+/lFZQAAAGgnLmofL9sXS7o4Ip6y/S5J2yT9hqRPSDoUEV+2/XlJ50fE5woJAQAA0EYKW/GKiBci4qnWz69K2iVptaRbJdVbp9U1V8YWZHusoJgAFsD/ewCwuCopJrHdI+lqSU9KWhURL0hz5cz2e07wawYlDUrS8uXLr6nVamyxD6T3Su4AOHMXXnhh9PT05I4BLDnbtm17OSK6j/da4cXL9k9J+pakT0XEK7ZP6ddFxKikUUmq1WoxMTFRXEgAx2X72dwZcOZ6enrEtRNIz/beE71W6KcabZ+judL1YEQ80jr8Uuv+r7fuAztYZAYAAIB2UeSnGi3pPkm7IuKPjnrpLyQNtH4ekPTnRWUAAABoJ0W+1Xi9pN+WtMP29taxL0j6sqRv2L5T0vOSfqvADAAAAG2jsOIVET+QdKIbum4ual4AAIB2xc71AAAAiVC8AAAAEqF4oVCNRkPDw8NqNBq5owBAKXDd7GwULxSqXq9rx44d2rx5c+4oAFAKXDc7G8ULhWk0GhobG1NEaGxsjH+9AcBJcN3sfBQvFKZer2t2dlaSNDMzw7/egLNke73tZ2zvtP2Q7eW5M2Fxcd3sfBQvFGbr1q1qNpuSpGazqfHx8cyJgPKyvVrSsKRaRFwhqUvSbXlTYbFx3ex8FC8UZu3atapU5raKq1Qq6uvry5wIKL2KpBW2K5JWSjqQOQ8WGdfNzkfxQmEGBga0bNncH7Guri6tW7cucyKgvCJiStI9mnvixwuSfhIRj84/z/ag7QnbE9PT06lj4ixx3ex8FC8Uplqtqr+/X7bV39+varWaOxJQWrbPl3SrpPdK+hlJ59q+Y/55ETEaEbWIqHV3d6eOibPEdbPzUbxQqIGBAb3//e/nX23A2Vsr6R8iYjoijkh6RNKHM2dCAbhudrYiH5INqFqtauPGjbljAJ3geUnX2V4p6TXNPfN2Im8kFIHrZmdjxQsASiAinpT0sKSnJO3Q3PV7NGsoAKeNFS8AKImIuFvS3blzADhzrHgBAAAkQvECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJFJY8bJ9v+2Dtncedewq239je7vtCdvXFjU/AABAuylyxesBSf3zjn1F0h9ExFWSvtgaAwAALAmFFa+IeFzSofmHJZ3X+vmnJR0oan4AAIB2k/oh2Z+S9D9t36O50vfhE51oe1DSoCStWbMmSTgAAIAipb65/pOS1kfEpZLWS7rvRCdGxGhE1CKi1t3dnSwgAABAUVIXrwFJj7R+/qYkbq4HAABLRuridUDSja2fb5L0bOL5AQAAsinsHi/bD0n6FUkX2t4v6W5J/0rSBtsVSa+rdQ8XAADAUlBY8YqI20/w0jVFzQkAANDO2LkeAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJELxAgAASITiBQAAkAjFCwBKwPbltrcf9fWK7U/lzoXF12g0NDw8rEajkTsKCkDxAoASiIi/j4irIuIqzT0B5LCkb+dNhSLU63Xt2LFDmzdvzh0FBaB4AUD53Czp/0bE3txBsLgajYbGxsYUERobG2PVqwNRvACgfG6T9NDxXrA9aHvC9sT09HTiWDhb9Xpds7OzkqSZmRlWvToQxQsASsT2OyR9XNI3j/d6RIxGRC0iat3d3WnD4axt3bpVzWZTktRsNjU+Pp45ERYbxQsAyuXXJD0VES/lDoLFt3btWlUqFUlSpVJRX19f5kRYbBQvACiX23WCtxlRfgMDA1q2bO6v5q6uLq1bty5zIiw2ihcAlITtlZL6JD2SOwuKUa1W1d/fL9vq7+9XtVrNHQmLrJI7AADg1ETEYUn8TdzhBgYGtGfPHla7OhQrXgAAAIlQvAAAaCNsoNrZKF4AALQJNlDtfBQvAADaBBuodj6KFwAAbYINVDsfxQsAgDbBBqqdr7DiZft+2wdt75x3fMj239t+xvZXipofAICyYQPVzlfkitcDkvqPPmD7VyXdKumXIuIXJd1T4PwAAJQKG6h2vsI2UI2Ix233zDv8SUlfjog3WuccLGp+AADKiA1UO1vqe7x+XtJHbD9p+69tf/BEJ9oetD1he2J6ejphRAAA8qlWq9q4cSOrXR0qdfGqSDpf0nWS/r2kb9j28U6MiNGIqEVErbu7O2VGAACyaTQaGh4eZg+vDpW6eO2X9EjM+aGkWUkXJs4AAEDbYuf6zpa6eP03STdJku2fl/QOSS8nzgAAQFs6euf67373u6x6daAit5N4SNITki63vd/2nZLul3RZa4uJr0saiIgoKgMAAGVSr9d15MgRSdKRI0dY9epARX6q8fYTvHRHUXMCAFBm4+Pjems9IiL06KOPav369ZlTYTGxcz0AAG1i1apVC45RfhQvAADaxEsvvbTgGOVH8QIAoE185CMfOWZ8ww03ZEqColC8AABoE2+++eYx4zfeeCNTEhSF4gUAQJv4/ve/v+AY5UfxAgCgTczOzi44RvlRvAAAaBPzn6J3gqfqocQoXgAAtIkPfehDC45RfhQvAADaxJ49e44Z7927N08QFIbiBQBAm3jhhReOGR84cCBTEhSF4gUAQJvgHq/OR/ECgJKw/W7bD9vebXuX7V/OnQmL68Ybb1xwjPKjeAFAeWyQNBYR75N0paRdmfNgkd1yyy3HjD/2sY9lSoKiULwAoARsnyfpBkn3SVJEvBkRP84aCotuw4YNx4zvvffePEFQGIoXAJTDZZKmJf2p7adtf832ublDYXHt27dvwTHKj+IFAOVQkfQBSX8SEVdL+kdJn59/ku1B2xO2J6anp1NnBHASFC8AKIf9kvZHxJOt8cOaK2LHiIjRiKhFRK27uztpQAAnR/ECgBKIiBcl7bN9eevQzZL+LmMkFIDtJDpfJXcAAMApG5L0oO13SHpO0u9kzoNFFhELjlF+FC8AKImI2C6pljsHgDPHW40AAACJULwAAGgTy5cvX3CM8iuseNm+3/ZB2zuP89pnbIftC4uaHwCAsnn99dcXHKP8irzH6wFJfyxp89EHbV8qqU/S8wXODQCAJGnTpk2anJzMHeOMjYyM5I5wSnp7ezU0NJQ7RtsrbMUrIh6XdOg4L/1nSZ+VxEc1AADAkpL0U422Py5pKiJ+dLK9SWwPShqUpDVr1iRIBwDoRGVahfn0pz+tbdu2vT2+5ppr9NWvfjVjIiy2ZDfX214p6fclffFUzmf3ZQDAUvOFL3xhwTHKL+WnGn9W0nsl/cj2HkmXSHrK9kUJMwAA0Laq1apWrlwpaW61q1qtZk6ExZbsrcaI2CHpPW+NW+WrFhEvp8oAAEC76+np0d69e1nt6lBFbifxkKQnJF1ue7/tO4uaCwCATnHOOeeot7eX1a4OVdiKV0TcfpLXe4qaGwAAoB2xcz0AAEAiFC8AAIBEKF4AAACJULwAAAASoXgBAAAkQvECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJELxAgAASITiBQAAkAjFCwAAIJFK7gAAgFNje4+kVyXNSGpGRC1vIgCni+IFAOXyqxHxcu4QAM4MbzUCAAAkQvECgPIISY/a3mZ78Hgn2B60PWF7Ynp6OnE8ACdD8QKA8rg+Ij4g6dck/Z7tG+afEBGjEVGLiFp3d3f6hAAWRPECgJKIiAOt7wclfVvStXkTAThdFC8AKAHb59p+11s/S/pnknbmTQXgdBVWvGzfb/ug7Z1HHftPtnfb/lvb37b97qLmB4AOs0rSD2z/SNIPJf2PiBjLnAnAaSpyxesBSf3zjo1LuiIifknS/5F0V4HzA0DHiIjnIuLK1tcvRsQf5s4E4PQVVrwi4nFJh+YdezQimq3h30i6pKj5AQAA2k3Oe7x+V9J3T/QiH4kGAACdJkvxsv37kpqSHjzROXwkGgAAdJrkjwyyPSDpFkk3R0Sknh8AACCXpMXLdr+kz0m6MSIOp5wbAAAgtyK3k3hI0hOSLre93/adkv5Y0rskjdvebvu/FDU/AABAuynyU423R8TFEXFORFwSEfdFRG9EXBoRV7W+/nVR86M9NBoNDQ8Pq9Fo5I4CAEB27FyPQtXrde3YsUObN2/OHQUAgOxOWrxs/1vb56cIg87SaDQ0NjamiNDY2BirXgCAJe9UVrwukvS/bX/Ddr9tFx0KnaFer2t2dlaSNDMzw6oX0GL7MdsfnXdsNFceAOmctHhFxH+Q9HOS7pP0CUnP2v6S7Z8tOBtKbuvWrWo25x5U0Gw2NT4+njkR0DbeK+lztu8+6lgtVxgA6ZzSPV6t/bZebH01JZ0v6WHbXykwG0pu7dq1qlTmdiypVCrq6+vLnAhoGz+WdLOkVbb/u+2fzpwHQCKnco/XsO1tkr4i6X9Jen9EfFLSNZL+ZcH5UGIDAwNatmzuj1hXV5fWrVuXORHQNhwRzYj4N5K+JekHkt6TOROABE5lxetCSf8iIv55RHwzIo5IUkTMam4HeuC4qtWq+vv7ZVv9/f2qVqu5IwHt4u09DCPiAc3dxvForjAA0jnpzvUR8cUFXtu1uHHQaQYGBrRnzx5Wu4CjRMR/nTfeJul3M8UBkFDyZzViaalWq9q4cWPuGAAAtAU2UAUAAEiE4gUAAJAIxQsAACARihcAAEAiFC8UqtFoaHh4mOc0AgAgihcKVq/XtWPHDp7TCACAKF4oUKPR0NjYmCJCY2NjrHoBAJY8ihcKU6/XNTs7K0mamZlh1QsAsORRvFCYrVu3qtlsSpKazabGx8czJwIAIC+KFwqzdu1aVSpzD0eoVCrq6+vLnAgAgLwoXijMwMCAbEuSli1bxvMagUVgu8v207a/kzsLgNPHsxpRmGq1qosuukj79u3TqlWrVK1Wc0cCOsGIpF2Szssx+aZNmzQ5OZlj6iXjrd/fkZGRzEk6X29vr4aGhpLOSfFCYRqNhqampiRJU1NTajQalC/gLNi+RNKvS/pDSf8uR4bJyUlt37lLMysvyDH9krDszZAkbXvupcxJOlvX4UNZ5qV4oTCjo6Nvf6pxdnZWo6OjuuuuuzKnAkrtXkmflfSuE51ge1DSoCStWbOmkBAzKy/Qa+/7aCH/bSCVFbu3ZJm3sHu8bN9v+6DtnUcdu8D2uO1nW9/PL2p+5PfYY48tOAZw6mzfIulgRGxb6LyIGI2IWkTUuru7E6UDcKqKvLn+AUn98459XtJjEfFzkh5rjdGhImLBMYDTcr2kj9veI+nrkm6y/Wd5IwE4XYUVr4h4XNL8N1BvlVRv/VyX9BtFzY/8PvjBDx4zvvbaazMlAcovIu6KiEsiokfSbZL+MiLuyBwLwGlKvZ3Eqoh4QZJa399zohNtD9qesD0xPT2dLCAWz/PPP3/MeO/evZmSAADQHtp2Hy/uUyi/AwcOLDgGcGYi4nsRcUvuHABOX+ri9ZLtiyWp9f1g4vkBAACySV28/kLSQOvnAUl/nnh+JNTV1bXgGACApabI7SQekvSEpMtt77d9p6QvS+qz/aykvtYYHWr+zfTcXA8AWOoK20A1Im4/wUs3FzUn2su+ffuOGe/fvz9TEgAA2kPb3lyP8ptftOYXMQAAlhqKFwrDPV4AAByL4oXCzMzMLDgGAGCpoXgBAAAkQvECAABIpLBPNWLxbdq0SZOTk7ljnJWRkZHcEU6qt7dXQ0NDuWMAADoQxQuFsa2IOGYMoNympqbUdfgnWrF7S+4owFnpOtzQ1FQz+bwUrxIp2yrMxMSEPvOZz7w9vueee3TNNddkTAQAQF4ULxSmVqu9ver1zne+k9IFdIDVq1frxTcqeu19H80dBTgrK3Zv0erVq5LPy831KFRPT48k6Utf+lLeIAAAtAGKFwp13nnn6corr2S1CwAAUbwAAACSoXgBAAAkQvECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJELxAgAASCRL8bK93vYztnfafsj28hw5AKAsbC+3/UPbP2pdP/8gdyYApy958bK9WtKwpFpEXCGpS9JtqXMAQMm8IemmiLhS0lWS+m1flzcSgNNVyTjvCttHJK2UdCBTDgAohYgISf+vNTyn9RX5EgE4E8lXvCJiStI9kp6X9IKkn0TEo6lzAEDZ2O6yvV3SQUnjEfFk5kgATlOOtxrPl3SrpPdK+hlJ59q+4zjnDdqesD0xPT2dOiYAtJ2ImImIqyRdIula21fMP4drJ9Dectxcv1bSP0TEdEQckfSIpA/PPykiRiOiFhG17u7u5CEBoF1FxI8lfU9S/3Fe49oJtLEc93g9L+k62yslvSbpZkkTGXIAQGnY7pZ0JCJ+bHuF5v4R+x9zZOk6fEgrdm/JMfWSsOz1VyRJs8vPy5yks3UdPiRpVfJ5kxeviHjS9sOSnpLUlPS0pNHUOQCgZC6WVLfdpbl3K74REd9JHaK3tzf1lEvO5OSrkqTey9KXgqVlVZY/z1k+1RgRd0u6O8fcAFBGEfG3kq7OnWNoaCh3hI43MjIiSdqwYUPmJCgCO9cDAAAkQvECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARCheAAAAiVC8AAAAEqF4AQAAJELxAgAASITiBQAAkAjFCwAAIBGKFwAAQCIULwAAgEQoXgAAAIlQvAAAABKheAEAACRSyR0gl02bNmlycjJ3jI731u/xyMhI5iSdrbe3V0NDQ7ljAABOYskWr8nJSW3fuUszKy/IHaWjLXszJEnbnnspc5LO1XX4UO4IAIBTtGSLlyTNrLxAr73vo7ljAGdlxe4tuSMAAE4R93gBAAAkQvECAABIhOIFAACQSJbiZfvdth+2vdv2Ltu/nCMHAABASrlurt8gaSwiftP2OyStzJQDAAAgmeTFy/Z5km6Q9AlJiog3Jb2ZOgcAAEBqOd5qvEzStKQ/tf207a/ZPnf+SbYHbU/Ynpienk6fEgAAYJHlKF4VSR+Q9CcRcbWkf5T0+fknRcRoRNQiotbd3Z06IwC0FduX2v6r1n2xz9jmcRBACeUoXvsl7Y+IJ1vjhzVXxAAAJ9aU9OmI+AVJ10n6Pdv/NHMmAKcpefGKiBcl7bN9eevQzZL+LnUOACiTiHghIp5q/fyqpF2SVudNBeB05fpU45CkB1ufaHxO0u+kDjA1NaWuwz/hcSsova7DDU1NNXPHQEK2eyRdLenJ47w2KGlQktasWZM2GICTylK8ImK7pFqOuQGgzGz/lKRvSfpURLwy//WIGJU0Kkm1Wi0SxwNwEkv2IdmrV6/Wi29UeEg2Sm/F7i1avXpV7hhIwPY5mitdD0bEI7nzADh9PDIIAErAtiXdJ2lXRPxR7jwAzgzFCwDK4XpJvy3pJtvbW18s2QMls2TfagSAMomIH0hy7hwAzg4rXgAAAIlQvAAAABKheAEAACRC8QIAAEiE4gUAAJAIxQsAACARihcAAEAiFC8AAIBEKF4AAACJLOmd67sOH9KK3Vtyx+hoy15/RZI0u/y8zEk6V9fhQ5J4SDYAlMGSLV69vb25IywJk5OvSpJ6L6MYFGcVf54BoCSWbPEaGhrKHWFJGBkZkSRt2LAhcxIAAPLjHi8AAIBEKF4AAACJLNm3GgEAS8OmTZs0OTmZO8YpeyvrW7dqlEVvby+38ZwCihcAAG1kxYoVuSOgQBQvAEBHYxUG7YR7vAAAABKheAEAACSSrXjZ7rL9tO3v5MoAAACQUs4VrxFJuzLODwAAkFSW4mX7Ekm/LulrOeYHAADIIdeK172SPitp9kQn2B60PWF7Ynp6OlkwAACAoiQvXrZvkXQwIrYtdF5EjEZELSJq3d3didIBAJBXo9HQ8PCwGo1G7igoQI4Vr+slfdz2Hklfl3ST7T/LkAMAgLZTr9e1Y8cObd68OXcUFCB58YqIuyLikojokXSbpL+MiDtS5wAAoN00Gg2NjY0pIjQ2NsaqVwdi5/oSKdvzxqRyPnOM542hHdm+X9Jbt2pckTsPilGv1zU7O3f788zMjDZv3qz169dnToXFlHUD1Yj4XkTckjMDirVixQqeOwYsjgck9ecOgWJt3bpVzWZTktRsNjU+Pp45ERYbK14lwioMsHRFxOO2e3LnQLHWrl2rLVu2qNlsqlKpqK+vL3ckLDIeGQQAHYSteMptYGBAy5bN/dXc1dWldevWZU6ExUbxAoAOwlY85VatVtXf3y/b6u/vV7VazR0Ji4y3GgEAaCMDAwPas2cPq10diuIFAEAbqVar2rhxY+4YKAhvNQJACdh+SNITki63vd/2nbkzATh9rHgBQAlExO25MwA4e6x4AQAAJELxAgAASITiBQAAkAjFCwAAIBFHRO4MJ2V7WtLe3Dlwxi6U9HLuEDgj/yQi2IWzpLh2lhrXzXI74bWzFMUL5WZ7IiJquXMAQFlw3excvNUIAACQCMULAAAgEYoXUhjNHQAASobrZofiHi8AAIBEWPECAABIhOIFAACQCMULAAAgEYoXAABAIhQvAACARP4/pLxTq5yNR/gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Run this code\n",
    "fig, ax = plt.subplots(1,2, figsize=(10, 5))\n",
    "sns.boxplot(y=diamonds.y, ax=ax[0])\n",
    "sns.boxplot(y=diamonds.z, ax=ax[1])\n",
    "plt.subplots_adjust(wspace=0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>23644</th>\n",
       "      <td>23645</td>\n",
       "      <td>3.65</td>\n",
       "      <td>Fair</td>\n",
       "      <td>H</td>\n",
       "      <td>I1</td>\n",
       "      <td>67.1</td>\n",
       "      <td>53.0</td>\n",
       "      <td>11668</td>\n",
       "      <td>9.53</td>\n",
       "      <td>9.480000</td>\n",
       "      <td>6.38</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24067</th>\n",
       "      <td>24068</td>\n",
       "      <td>2.00</td>\n",
       "      <td>Premium</td>\n",
       "      <td>H</td>\n",
       "      <td>SI2</td>\n",
       "      <td>58.9</td>\n",
       "      <td>57.0</td>\n",
       "      <td>12210</td>\n",
       "      <td>8.09</td>\n",
       "      <td>19.278421</td>\n",
       "      <td>8.06</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24328</th>\n",
       "      <td>24329</td>\n",
       "      <td>3.50</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>H</td>\n",
       "      <td>I1</td>\n",
       "      <td>62.8</td>\n",
       "      <td>57.0</td>\n",
       "      <td>12587</td>\n",
       "      <td>9.65</td>\n",
       "      <td>9.590000</td>\n",
       "      <td>6.03</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25998</th>\n",
       "      <td>25999</td>\n",
       "      <td>4.01</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>I1</td>\n",
       "      <td>61.0</td>\n",
       "      <td>61.0</td>\n",
       "      <td>15223</td>\n",
       "      <td>10.14</td>\n",
       "      <td>10.089508</td>\n",
       "      <td>6.17</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25999</th>\n",
       "      <td>26000</td>\n",
       "      <td>4.01</td>\n",
       "      <td>Premium</td>\n",
       "      <td>J</td>\n",
       "      <td>I1</td>\n",
       "      <td>62.5</td>\n",
       "      <td>62.0</td>\n",
       "      <td>15223</td>\n",
       "      <td>10.02</td>\n",
       "      <td>9.940000</td>\n",
       "      <td>6.24</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26194</th>\n",
       "      <td>26195</td>\n",
       "      <td>2.01</td>\n",
       "      <td>Fair</td>\n",
       "      <td>G</td>\n",
       "      <td>SI2</td>\n",
       "      <td>65.6</td>\n",
       "      <td>56.0</td>\n",
       "      <td>15562</td>\n",
       "      <td>7.89</td>\n",
       "      <td>7.840000</td>\n",
       "      <td>6.16</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26431</th>\n",
       "      <td>26432</td>\n",
       "      <td>3.40</td>\n",
       "      <td>Fair</td>\n",
       "      <td>D</td>\n",
       "      <td>I1</td>\n",
       "      <td>66.8</td>\n",
       "      <td>52.0</td>\n",
       "      <td>15964</td>\n",
       "      <td>9.42</td>\n",
       "      <td>9.340000</td>\n",
       "      <td>6.27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26444</th>\n",
       "      <td>26445</td>\n",
       "      <td>4.00</td>\n",
       "      <td>Very Good</td>\n",
       "      <td>I</td>\n",
       "      <td>I1</td>\n",
       "      <td>63.3</td>\n",
       "      <td>58.0</td>\n",
       "      <td>15984</td>\n",
       "      <td>10.01</td>\n",
       "      <td>9.940000</td>\n",
       "      <td>6.31</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26534</th>\n",
       "      <td>26535</td>\n",
       "      <td>3.67</td>\n",
       "      <td>Premium</td>\n",
       "      <td>I</td>\n",
       "      <td>I1</td>\n",
       "      <td>62.4</td>\n",
       "      <td>56.0</td>\n",
       "      <td>16193</td>\n",
       "      <td>9.86</td>\n",
       "      <td>9.810000</td>\n",
       "      <td>6.13</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27130</th>\n",
       "      <td>27131</td>\n",
       "      <td>4.13</td>\n",
       "      <td>Fair</td>\n",
       "      <td>H</td>\n",
       "      <td>I1</td>\n",
       "      <td>64.8</td>\n",
       "      <td>61.0</td>\n",
       "      <td>17329</td>\n",
       "      <td>10.00</td>\n",
       "      <td>9.850000</td>\n",
       "      <td>6.43</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27415</th>\n",
       "      <td>27416</td>\n",
       "      <td>5.01</td>\n",
       "      <td>Fair</td>\n",
       "      <td>J</td>\n",
       "      <td>I1</td>\n",
       "      <td>65.5</td>\n",
       "      <td>59.0</td>\n",
       "      <td>18018</td>\n",
       "      <td>10.74</td>\n",
       "      <td>10.572977</td>\n",
       "      <td>6.98</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27630</th>\n",
       "      <td>27631</td>\n",
       "      <td>4.50</td>\n",
       "      <td>Fair</td>\n",
       "      <td>J</td>\n",
       "      <td>I1</td>\n",
       "      <td>65.8</td>\n",
       "      <td>58.0</td>\n",
       "      <td>18531</td>\n",
       "      <td>10.23</td>\n",
       "      <td>10.195532</td>\n",
       "      <td>6.72</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27679</th>\n",
       "      <td>27680</td>\n",
       "      <td>3.51</td>\n",
       "      <td>Premium</td>\n",
       "      <td>J</td>\n",
       "      <td>VS2</td>\n",
       "      <td>62.5</td>\n",
       "      <td>59.0</td>\n",
       "      <td>18701</td>\n",
       "      <td>9.66</td>\n",
       "      <td>9.630000</td>\n",
       "      <td>6.03</td>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0  carat        cut color clarity  depth  table  price      x  \\\n",
       "23644       23645   3.65       Fair     H      I1   67.1   53.0  11668   9.53   \n",
       "24067       24068   2.00    Premium     H     SI2   58.9   57.0  12210   8.09   \n",
       "24328       24329   3.50      Ideal     H      I1   62.8   57.0  12587   9.65   \n",
       "25998       25999   4.01    Premium     I      I1   61.0   61.0  15223  10.14   \n",
       "25999       26000   4.01    Premium     J      I1   62.5   62.0  15223  10.02   \n",
       "26194       26195   2.01       Fair     G     SI2   65.6   56.0  15562   7.89   \n",
       "26431       26432   3.40       Fair     D      I1   66.8   52.0  15964   9.42   \n",
       "26444       26445   4.00  Very Good     I      I1   63.3   58.0  15984  10.01   \n",
       "26534       26535   3.67    Premium     I      I1   62.4   56.0  16193   9.86   \n",
       "27130       27131   4.13       Fair     H      I1   64.8   61.0  17329  10.00   \n",
       "27415       27416   5.01       Fair     J      I1   65.5   59.0  18018  10.74   \n",
       "27630       27631   4.50       Fair     J      I1   65.8   58.0  18531  10.23   \n",
       "27679       27680   3.51    Premium     J     VS2   62.5   59.0  18701   9.66   \n",
       "\n",
       "               y     z  clarity_scale  color_scale  \n",
       "23644   9.480000  6.38              0            4  \n",
       "24067  19.278421  8.06              1            4  \n",
       "24328   9.590000  6.03              0            4  \n",
       "25998  10.089508  6.17              0            5  \n",
       "25999   9.940000  6.24              0            6  \n",
       "26194   7.840000  6.16              1            3  \n",
       "26431   9.340000  6.27              0            0  \n",
       "26444   9.940000  6.31              0            5  \n",
       "26534   9.810000  6.13              0            5  \n",
       "27130   9.850000  6.43              0            4  \n",
       "27415  10.572977  6.98              0            6  \n",
       "27630  10.195532  6.72              0            6  \n",
       "27679   9.630000  6.03              3            6  "
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[(diamonds.z)>6]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>carat</th>\n",
       "      <th>cut</th>\n",
       "      <th>color</th>\n",
       "      <th>clarity</th>\n",
       "      <th>depth</th>\n",
       "      <th>table</th>\n",
       "      <th>price</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "      <th>clarity_scale</th>\n",
       "      <th>color_scale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>14635</th>\n",
       "      <td>14636</td>\n",
       "      <td>1.07</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>F</td>\n",
       "      <td>SI1</td>\n",
       "      <td>60.6</td>\n",
       "      <td>57.0</td>\n",
       "      <td>5909</td>\n",
       "      <td>6.62</td>\n",
       "      <td>6.67</td>\n",
       "      <td>1.07</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20694</th>\n",
       "      <td>20695</td>\n",
       "      <td>1.53</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>I</td>\n",
       "      <td>SI1</td>\n",
       "      <td>61.9</td>\n",
       "      <td>54.0</td>\n",
       "      <td>8971</td>\n",
       "      <td>7.43</td>\n",
       "      <td>7.50</td>\n",
       "      <td>1.53</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21654</th>\n",
       "      <td>21655</td>\n",
       "      <td>1.41</td>\n",
       "      <td>Ideal</td>\n",
       "      <td>H</td>\n",
       "      <td>VS1</td>\n",
       "      <td>60.7</td>\n",
       "      <td>56.0</td>\n",
       "      <td>9752</td>\n",
       "      <td>7.31</td>\n",
       "      <td>7.22</td>\n",
       "      <td>1.41</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Unnamed: 0  carat    cut color clarity  depth  table  price     x  \\\n",
       "14635       14636   1.07  Ideal     F     SI1   60.6   57.0   5909  6.62   \n",
       "20694       20695   1.53  Ideal     I     SI1   61.9   54.0   8971  7.43   \n",
       "21654       21655   1.41  Ideal     H     VS1   60.7   56.0   9752  7.31   \n",
       "\n",
       "          y     z  clarity_scale  color_scale  \n",
       "14635  6.67  1.07              2            2  \n",
       "20694  7.50  1.53              2            5  \n",
       "21654  7.22  1.41              4            4  "
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diamonds[(diamonds.z)<2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What do you think? Are these values more reasonable?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#your thoughts here\n",
    "\n",
    "y values are more reasonable than before. \n",
    "We still have one major outier in row 24067 and the remaing 4 to which we did the calculation formula here above\n",
    "In the plot on the right, we have 13 values to which z is larger than 6 (4 out the 5 we dealt with to recalculate the y appeared above.\n",
    "we also have few z values below 2 (3 records)\n",
    "Considering that the above are very few compared the full dataset (53940 values in the dataset), we can see that they will affect it. Something to consider is to remove these values but best to check first with the client/ management"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Once you are happy with your cleaning, save the cleaned data and continue to csv. Your new csv should be named ``diamonds_clean``**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds.to_csv('diamonds_clean.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}