ML-Fairness/fairness_income.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "_uuid": "30d019538f36d75ea323459e22311888a10bd68a",
        "id": "cFXe1zJDeRIj"
      },
      "source": [
        "# Applying fairnes metrics on the Adult Census Dataset\n",
        "\n",
        "<a href=\"https://colab.research.google.com/drive/10BcFbOU2RsKVF-o1wz40K-JNfhADE043\" target=\"_blank\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\">\n",
        "</a>\n",
        "\n",
        "Return to the [castle](https://github.com/Nkluge-correa/TeenyTinyCastle).\n",
        "\n",
        "\n",
        "A _machine Learning bias_, also sometimes called _algorithm bias_ (NOT TO BE CONFUSED WITH A MATH BIAS), is a phenomenon that occurs when an algorithm produces results that are systemically prejudiced due to erroneous assumptions in the machine learning process.\n",
        "\n",
        "Machine Learning Fairness is an area of research that seeks to uncover the roots of such biases and how we can mitigate them. Addressing algorithmic biases is a crucial issue in ML, given that machine learning models are increasingly used in fields where their decisions can impact people's lives, such as finance, employment, and criminal justice.\n",
        "\n",
        "<img src=\"https://miro.medium.com/max/1060/1*cc8OWxqKFXje4d_1eYrQkg.jpeg\" width=\"600\"/>\n",
        "\n",
        "Source: _[Moritz Hardt](https://towardsdatascience.com/a-tutorial-on-fairness-in-machine-learning-3ff8ba1040cb/)._\n",
        "\n",
        "In this notebook, we will be using the [`adult census dataset`](https://www.kaggle.com/datasets/uciml/adult-census-income), created by Barry Becker from the 1994 Census database (USA), to explore biases in ML algorithms. The prediction task of this dataset is to determine whether a person makes over 50K a year.\n",
        "\n",
        "The eatures and values that can be found in the `adult census dataset` are the following:\n",
        "\n",
        "- `Income`: '>50K', '<=50K'.\n",
        "- `Age`: continuous.\n",
        "- `Workclass`: Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked.  \n",
        "- `fnlwgt`: continuous.  \n",
        "- `Education`: Bachelors, Some-college, 11th, HS-grad, Prof-school, Assoc-acdm, Assoc-voc, 9th, 7th-8th, 12th, Masters, 1st-4th, 10th, Doctorate, 5th-6th, Preschool.  \n",
        "- `Education-num`: continuous.  \n",
        "- `Marital-status`: Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse.\n",
        "- `Occupation`: Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspect, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces.  \n",
        "- `Relationship`: Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried.  \n",
        "- `Race`: White, Asian-Pac-Islander, Amer-Indian--Eskimo, Other, Black.  \n",
        "- `Sex`: Female, Male.  \n",
        "- `Capital-gain`: continuous.  \n",
        "- `Capital-loss`: continuous.  \n",
        "- `Hours-per-week`: continuous.  \n",
        "- `Native-country`: United States, Cambodia, England, Puerto Rico, Canada, Germany, Outlying-US(Guam-USVI-etc), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad&Tobago, Peru, Hong, Holand-Netherlands.\n",
        "\n",
        "This notebook will consider `Age`, `Race`, and `Sex` as sensitive attributes. The dataset can be found on the Hugging Face Hub! 🤗\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 600
        },
        "id": "_MtBjNOGV7QI",
        "outputId": "e5be6173-c0dd-4dfb-94a5-5f3f4a22d2d1"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-bc7a7715-0f19-40d2-a1e3-01f42fa51df7\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>workclass</th>\n",
              "      <th>fnlwgt</th>\n",
              "      <th>education</th>\n",
              "      <th>education.num</th>\n",
              "      <th>marital.status</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>race</th>\n",
              "      <th>sex</th>\n",
              "      <th>capital.gain</th>\n",
              "      <th>capital.loss</th>\n",
              "      <th>hours.per.week</th>\n",
              "      <th>native.country</th>\n",
              "      <th>income</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>90</td>\n",
              "      <td>?</td>\n",
              "      <td>77053</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>9</td>\n",
              "      <td>Widowed</td>\n",
              "      <td>?</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>4356</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>82</td>\n",
              "      <td>Private</td>\n",
              "      <td>132870</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>9</td>\n",
              "      <td>Widowed</td>\n",
              "      <td>Exec-managerial</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>4356</td>\n",
              "      <td>18</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>66</td>\n",
              "      <td>?</td>\n",
              "      <td>186061</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>10</td>\n",
              "      <td>Widowed</td>\n",
              "      <td>?</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>Black</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>4356</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>54</td>\n",
              "      <td>Private</td>\n",
              "      <td>140359</td>\n",
              "      <td>7th-8th</td>\n",
              "      <td>4</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Machine-op-inspct</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>3900</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>41</td>\n",
              "      <td>Private</td>\n",
              "      <td>264663</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>10</td>\n",
              "      <td>Separated</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>3900</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32556</th>\n",
              "      <td>22</td>\n",
              "      <td>Private</td>\n",
              "      <td>310152</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>10</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Protective-serv</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32557</th>\n",
              "      <td>27</td>\n",
              "      <td>Private</td>\n",
              "      <td>257302</td>\n",
              "      <td>Assoc-acdm</td>\n",
              "      <td>12</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Tech-support</td>\n",
              "      <td>Wife</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>38</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32558</th>\n",
              "      <td>40</td>\n",
              "      <td>Private</td>\n",
              "      <td>154374</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>9</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Machine-op-inspct</td>\n",
              "      <td>Husband</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&gt;50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32559</th>\n",
              "      <td>58</td>\n",
              "      <td>Private</td>\n",
              "      <td>151910</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>9</td>\n",
              "      <td>Widowed</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32560</th>\n",
              "      <td>22</td>\n",
              "      <td>Private</td>\n",
              "      <td>201490</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>9</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>20</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>32561 rows × 15 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-bc7a7715-0f19-40d2-a1e3-01f42fa51df7')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-bc7a7715-0f19-40d2-a1e3-01f42fa51df7 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-bc7a7715-0f19-40d2-a1e3-01f42fa51df7');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-8a79240d-2fbd-4154-84ad-6ee3d86d5631\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-8a79240d-2fbd-4154-84ad-6ee3d86d5631')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-8a79240d-2fbd-4154-84ad-6ee3d86d5631 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "       age workclass  fnlwgt     education  education.num      marital.status  \\\n",
              "0       90         ?   77053       HS-grad              9             Widowed   \n",
              "1       82   Private  132870       HS-grad              9             Widowed   \n",
              "2       66         ?  186061  Some-college             10             Widowed   \n",
              "3       54   Private  140359       7th-8th              4            Divorced   \n",
              "4       41   Private  264663  Some-college             10           Separated   \n",
              "...    ...       ...     ...           ...            ...                 ...   \n",
              "32556   22   Private  310152  Some-college             10       Never-married   \n",
              "32557   27   Private  257302    Assoc-acdm             12  Married-civ-spouse   \n",
              "32558   40   Private  154374       HS-grad              9  Married-civ-spouse   \n",
              "32559   58   Private  151910       HS-grad              9             Widowed   \n",
              "32560   22   Private  201490       HS-grad              9       Never-married   \n",
              "\n",
              "              occupation   relationship   race     sex  capital.gain  \\\n",
              "0                      ?  Not-in-family  White  Female             0   \n",
              "1        Exec-managerial  Not-in-family  White  Female             0   \n",
              "2                      ?      Unmarried  Black  Female             0   \n",
              "3      Machine-op-inspct      Unmarried  White  Female             0   \n",
              "4         Prof-specialty      Own-child  White  Female             0   \n",
              "...                  ...            ...    ...     ...           ...   \n",
              "32556    Protective-serv  Not-in-family  White    Male             0   \n",
              "32557       Tech-support           Wife  White  Female             0   \n",
              "32558  Machine-op-inspct        Husband  White    Male             0   \n",
              "32559       Adm-clerical      Unmarried  White  Female             0   \n",
              "32560       Adm-clerical      Own-child  White    Male             0   \n",
              "\n",
              "       capital.loss  hours.per.week native.country income  \n",
              "0              4356              40  United-States  <=50K  \n",
              "1              4356              18  United-States  <=50K  \n",
              "2              4356              40  United-States  <=50K  \n",
              "3              3900              40  United-States  <=50K  \n",
              "4              3900              40  United-States  <=50K  \n",
              "...             ...             ...            ...    ...  \n",
              "32556             0              40  United-States  <=50K  \n",
              "32557             0              38  United-States  <=50K  \n",
              "32558             0              40  United-States   >50K  \n",
              "32559             0              40  United-States  <=50K  \n",
              "32560             0              20  United-States  <=50K  \n",
              "\n",
              "[32561 rows x 15 columns]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "!pip install datasets -q\n",
        "from datasets import load_dataset\n",
        "\n",
        "# load the datasets from the hub\n",
        "data = load_dataset('AiresPucrs/adult-census-income', split='train')\n",
        "\n",
        "# turn the datasets into a pandas.DataFrame\n",
        "df = data.to_pandas()\n",
        "display(df)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PUomj9j3eRIp"
      },
      "source": [
        "Below, we perform some simple pre-processing (e.g., excluding rows with `?` values and grouping the age category into buckets) to help the training of our model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 313
        },
        "id": "rp2UcXiDeRIp",
        "outputId": "3cafb2bf-774e-496d-8175-ead77278e84e"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-55c34cab-d6aa-4c9a-a77d-aac934bf1233\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>workclass</th>\n",
              "      <th>fnlwgt</th>\n",
              "      <th>education</th>\n",
              "      <th>education.num</th>\n",
              "      <th>marital.status</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>race</th>\n",
              "      <th>sex</th>\n",
              "      <th>capital.gain</th>\n",
              "      <th>capital.loss</th>\n",
              "      <th>hours.per.week</th>\n",
              "      <th>native.country</th>\n",
              "      <th>income</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>77-87</td>\n",
              "      <td>Private</td>\n",
              "      <td>132870</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>9</td>\n",
              "      <td>Widowed</td>\n",
              "      <td>Exec-managerial</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>4356</td>\n",
              "      <td>18</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>47-57</td>\n",
              "      <td>Private</td>\n",
              "      <td>140359</td>\n",
              "      <td>7th-8th</td>\n",
              "      <td>4</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Machine-op-inspct</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>3900</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>37-47</td>\n",
              "      <td>Private</td>\n",
              "      <td>264663</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>10</td>\n",
              "      <td>Separated</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>3900</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>27-37</td>\n",
              "      <td>Private</td>\n",
              "      <td>216864</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>9</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Other-service</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>3770</td>\n",
              "      <td>45</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>37-47</td>\n",
              "      <td>Private</td>\n",
              "      <td>150601</td>\n",
              "      <td>10th</td>\n",
              "      <td>6</td>\n",
              "      <td>Separated</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>3770</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-55c34cab-d6aa-4c9a-a77d-aac934bf1233')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-55c34cab-d6aa-4c9a-a77d-aac934bf1233 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-55c34cab-d6aa-4c9a-a77d-aac934bf1233');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-ebcbf422-5fa1-4fcd-8e95-fea988136cd7\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-ebcbf422-5fa1-4fcd-8e95-fea988136cd7')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-ebcbf422-5fa1-4fcd-8e95-fea988136cd7 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "     age workclass  fnlwgt     education  education.num marital.status  \\\n",
              "1  77-87   Private  132870       HS-grad              9        Widowed   \n",
              "3  47-57   Private  140359       7th-8th              4       Divorced   \n",
              "4  37-47   Private  264663  Some-college             10      Separated   \n",
              "5  27-37   Private  216864       HS-grad              9       Divorced   \n",
              "6  37-47   Private  150601          10th              6      Separated   \n",
              "\n",
              "          occupation   relationship   race     sex  capital.gain  \\\n",
              "1    Exec-managerial  Not-in-family  White  Female             0   \n",
              "3  Machine-op-inspct      Unmarried  White  Female             0   \n",
              "4     Prof-specialty      Own-child  White  Female             0   \n",
              "5      Other-service      Unmarried  White  Female             0   \n",
              "6       Adm-clerical      Unmarried  White    Male             0   \n",
              "\n",
              "   capital.loss  hours.per.week native.country income  \n",
              "1          4356              18  United-States  <=50K  \n",
              "3          3900              40  United-States  <=50K  \n",
              "4          3900              40  United-States  <=50K  \n",
              "5          3770              45  United-States  <=50K  \n",
              "6          3770              40  United-States  <=50K  "
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "df = df.replace('?', np.nan)\n",
        "df = df.dropna(axis=0).copy()\n",
        "\n",
        "bins = [17, 27, 37, 47, 57, 67, 77, 87, 97]\n",
        "labels = ['17-27','27-37','37-47','47-57','57-67','67-77','77-87','87-97']\n",
        "\n",
        "df.loc[:, 'age'] = pd.cut(df['age'], bins=bins, labels=labels)\n",
        "\n",
        "\n",
        "display(df.head())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x9yKBEqHeRIq"
      },
      "source": [
        "Now, let us try to see how our target (`income`) is related to the sensitive attributes of our dataset (`Age`, `Race`, and `Sex`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "2vKDSZNweRIq",
        "outputId": "430c9cb3-a165-4996-f4e2-2ab55df10782"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"aea555aa-603f-48d1-aea8-622c7b57242d\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"aea555aa-603f-48d1-aea8-622c7b57242d\")) {                    Plotly.newPlot(                        \"aea555aa-603f-48d1-aea8-622c7b57242d\",                        [{\"name\":\"\\u003c=50K\",\"x\":[\"77-87\",\"47-57\",\"37-47\",\"27-37\",\"67-77\",\"57-67\",\"17-27\",\"87-97\"],\"y\":[65,2780,4676,6269,395,1483,6627,31],\"type\":\"bar\"},{\"name\":\"\\u003e50K\",\"x\":[\"77-87\",\"47-57\",\"37-47\",\"27-37\",\"67-77\",\"57-67\",\"17-27\",\"87-97\"],\"y\":[17,1807,2723,1947,103,657,247,7],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"barmode\":\"group\",\"xaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eAge-Buckets\\u003c\\u002fb\\u003e\"}},\"yaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eIncome by Age Buckets\\u003c\\u002fb\\u003e\"}},\"title\":{\"text\":\"Distribution of \\u003ci\\u003eIncome\\u003c\\u002fi\\u003e by \\\"Age-Buckets\\\"\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('aea555aa-603f-48d1-aea8-622c7b57242d');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"e91e0432-5412-45b5-b149-06ca38ed87a1\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"e91e0432-5412-45b5-b149-06ca38ed87a1\")) {                    Plotly.newPlot(                        \"e91e0432-5412-45b5-b149-06ca38ed87a1\",                        [{\"name\":\"\\u003c=50K\",\"x\":[\"Female\",\"Male\"],\"y\":[8670,13984],\"type\":\"bar\"},{\"name\":\"\\u003e50K\",\"x\":[\"Female\",\"Male\"],\"y\":[1112,6396],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"barmode\":\"group\",\"xaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eSex\\u003c\\u002fb\\u003e\"}},\"yaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eIncome by Sex\\u003c\\u002fb\\u003e\"}},\"title\":{\"text\":\"Distribution of \\u003ci\\u003eIncome\\u003c\\u002fi\\u003e by \\\"Sex\\\"\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('e91e0432-5412-45b5-b149-06ca38ed87a1');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"fcbf1207-9924-4b52-82f3-19d538743f9b\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"fcbf1207-9924-4b52-82f3-19d538743f9b\")) {                    Plotly.newPlot(                        \"fcbf1207-9924-4b52-82f3-19d538743f9b\",                        [{\"name\":\"\\u003c=50K\",\"x\":[\"White\",\"Black\",\"Asian-Pac-Islander\",\"Other\",\"Amer-Indian-Eskimo\"],\"y\":[19094,2451,647,210,252],\"type\":\"bar\"},{\"name\":\"\\u003e50K\",\"x\":[\"White\",\"Black\",\"Asian-Pac-Islander\",\"Other\",\"Amer-Indian-Eskimo\"],\"y\":[6839,366,248,21,34],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"barmode\":\"group\",\"xaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eRace\\u003c\\u002fb\\u003e\"}},\"yaxis\":{\"title\":{\"text\":\"\\u003cb\\u003eIncome by Race\\u003c\\u002fb\\u003e\"}},\"title\":{\"text\":\"Distribution of \\u003ci\\u003eIncome\\u003c\\u002fi\\u003e by \\\"Race\\\"\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('fcbf1207-9924-4b52-82f3-19d538743f9b');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "less_50_list = []\n",
        "more_50_list = []\n",
        "\n",
        "for element in list(df['age'].unique().dropna()):\n",
        "    less_50 = df[df['age'] == element]['income'].value_counts()['<=50K']\n",
        "    more_50 = df[df['age'] == element]['income'].value_counts()['>50K']\n",
        "    less_50_list.append(less_50)\n",
        "    more_50_list.append(more_50)\n",
        "\n",
        "import plotly.graph_objects as go\n",
        "\n",
        "fig = go.Figure(data=[\n",
        "    go.Bar(name='<=50K', x=list(df['age'].unique().dropna()), y=less_50_list),\n",
        "    go.Bar(name='>50K', x=list(df['age'].unique().dropna()), y=more_50_list)\n",
        "])\n",
        "\n",
        "fig.update_layout(\n",
        "    barmode='group',\n",
        "    template='plotly_dark',\n",
        "    xaxis_title=\"<b>Age-Buckets</b>\",\n",
        "    yaxis_title=\"<b>Income by Age Buckets</b>\",\n",
        "    title='Distribution of <i>Income</i> by \"Age-Buckets\"',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    )\n",
        "fig.show()\n",
        "\n",
        "less_50_list = []\n",
        "more_50_list = []\n",
        "\n",
        "for element in list(df['sex'].unique()):\n",
        "    less_50 = df[df['sex'] == element]['income'].value_counts()['<=50K']\n",
        "    more_50 = df[df['sex'] == element]['income'].value_counts()['>50K']\n",
        "    less_50_list.append(less_50)\n",
        "    more_50_list.append(more_50)\n",
        "\n",
        "fig = go.Figure(data=[\n",
        "    go.Bar(name='<=50K', x=list(df['sex'].unique()), y=less_50_list),\n",
        "    go.Bar(name='>50K', x=list(df['sex'].unique()), y=more_50_list)\n",
        "])\n",
        "\n",
        "fig.update_layout(\n",
        "    barmode='group',\n",
        "    template='plotly_dark',\n",
        "    xaxis_title=\"<b>Sex</b>\",\n",
        "    yaxis_title=\"<b>Income by Sex</b>\",\n",
        "    title='Distribution of <i>Income</i> by \"Sex\"',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    )\n",
        "fig.show()\n",
        "\n",
        "less_50_list = []\n",
        "more_50_list = []\n",
        "\n",
        "for element in list(df['race'].unique()):\n",
        "    less_50 = df[df['race'] == element]['income'].value_counts()['<=50K']\n",
        "    more_50 = df[df['race'] == element]['income'].value_counts()['>50K']\n",
        "    less_50_list.append(less_50)\n",
        "    more_50_list.append(more_50)\n",
        "\n",
        "fig = go.Figure(data=[\n",
        "    go.Bar(name='<=50K', x=list(df['race'].unique()), y=less_50_list),\n",
        "    go.Bar(name='>50K', x=list(df['race'].unique()), y=more_50_list)\n",
        "])\n",
        "\n",
        "fig.update_layout(\n",
        "    barmode='group',\n",
        "    template='plotly_dark',\n",
        "    xaxis_title=\"<b>Race</b>\",\n",
        "    yaxis_title=\"<b>Income by Race</b>\",\n",
        "    title='Distribution of <i>Income</i> by \"Race\"',\n",
        "    paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    plot_bgcolor='rgba(0, 0, 0, 0)',\n",
        "    )\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RgnNKPDveRIr"
      },
      "source": [
        "Let us see if our sensitive attributes correlate with `Income`. If they are, this is already a sign that our future model could inherit these biases against a specific unprivileged group.\n",
        "\n",
        "To be able to calculate correlations, let us transform all categorical values into numbers."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "fXe6lxF6eRIr",
        "outputId": "60f3aec3-704f-4cab-dcbf-257f5c4dc335"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"242f2991-1950-416f-a243-3f2879f9cb0b\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"242f2991-1950-416f-a243-3f2879f9cb0b\")) {                    Plotly.newPlot(                        \"242f2991-1950-416f-a243-3f2879f9cb0b\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"texttemplate\":\"%{z}\",\"x\":[\"age\",\"workclass\",\"fnlwgt\",\"education\",\"education.num\",\"marital.status\",\"occupation\",\"relationship\",\"race\",\"sex\",\"capital.gain\",\"capital.loss\",\"hours.per.week\",\"native.country\",\"income\"],\"y\":[\"age\",\"workclass\",\"fnlwgt\",\"education\",\"education.num\",\"marital.status\",\"occupation\",\"relationship\",\"race\",\"sex\",\"capital.gain\",\"capital.loss\",\"hours.per.week\",\"native.country\",\"income\"],\"z\":[[1.0,0.060539842254470885,-0.0756596567936613,-0.15074074476796984,-0.055707215274160025,-0.18176280330572497,0.014672907605201234,-0.15291069612521005,0.03127520345967774,0.051480464619591375,0.06182735050526237,0.04680032600031118,-0.03040825677204806,0.007247000470261807,0.17604239479738906],[0.060539842254470885,1.0,-0.03249303938878188,0.017854900946348964,0.037832888786361304,-0.034240585769457536,0.015572229033043365,-0.06741675248616581,0.044730901817788854,0.0749731452883596,0.03534962402431612,0.007204021642490369,0.0507235711915347,0.007668264533886477,0.018043983433917464],[-0.0756596567936613,-0.03249303938878188,1.0,-0.027101745506657293,-0.044991742148081854,0.03216289239048977,0.0002041302722694447,0.009297558067912881,-0.023895349244312112,0.02536227134747545,0.0004215673835380509,-0.009749527811310938,-0.022885751617396227,-0.06671681046525728,-0.008957423359172337],[-0.15074074476796984,0.017854900946348964,-0.027101745506657293,1.0,0.34540951353395005,-0.04066406891636458,-0.038212117560863956,-0.012716505547690417,0.011153832965611453,-0.027888152402635917,0.03057544553404851,0.015027822155991181,0.059887395694669714,0.07879032750155776,0.07898745800459364],[-0.055707215274160025,0.037832888786361304,-0.044991742148081854,0.34540951353395005,1.0,-0.06341915959075793,0.08771714880594378,-0.09193528850240937,0.032804759831595975,0.006156852107851387,0.12441599532692403,0.07964641032298414,0.15252207096065878,0.09155502958739214,0.33528619675264226],[-0.18176280330572497,-0.034240585769457536,0.03216289239048977,-0.04066406891636458,-0.06341915959075793,1.0,0.022654545758652687,0.17796370075081627,-0.06862736764804918,-0.11981295741908478,-0.04241831347082254,-0.03520345334395935,-0.18900322054635946,-0.02590228363160914,-0.19351839380525632],[0.014672907605201234,0.015572229033043365,0.0002041302722694447,-0.038212117560863956,0.08771714880594378,0.022654545758652687,1.0,-0.05372724999356476,0.0007174904647550344,0.06231324002243865,0.022162418788689586,0.014606678741697125,0.01836524419284244,-0.003483338999629863,0.05157686310141771],[-0.15291069612521005,-0.06741675248616581,0.009297558067912881,-0.012716505547690417,-0.09193528850240937,0.17796370075081627,-0.05372724999356476,1.0,-0.11714325524208041,-0.5848764167491509,-0.05825870949497737,-0.06356652676316195,-0.25785012876392255,-0.010808695782368344,-0.25100282131714535],[0.03127520345967774,0.044730901817788854,-0.023895349244312112,0.011153832965611453,0.032804759831595975,-0.06862736764804918,0.0007174904647550344,-0.11714325524208041,1.0,0.0891855657041708,0.014353174222793131,0.023516855101107195,0.04853189656558753,0.12451370418131098,0.07165760495137209],[0.051480464619591375,0.0749731452883596,0.02536227134747545,-0.027888152402635917,0.006156852107851387,-0.11981295741908478,0.06231324002243865,-0.5848764167491509,0.0891855657041708,1.0,0.04881405374066122,0.047011239811242576,0.2312681271910294,0.0006180198534297345,0.2166986810755833],[0.06182735050526237,0.03534962402431612,0.0004215673835380509,0.03057544553404851,0.12441599532692403,-0.04241831347082254,0.022162418788689586,-0.05825870949497737,0.014353174222793131,0.04881405374066122,1.0,-0.03222932650311955,0.08043180069849806,0.008530428384064393,0.22119621454805358],[0.04680032600031118,0.007204021642490369,-0.009749527811310938,0.015027822155991181,0.07964641032298414,-0.03520345334395935,0.014606678741697125,-0.06356652676316195,0.023516855101107195,0.047011239811242576,-0.03222932650311955,1.0,0.05241704932626748,0.009385797117355184,0.15005330839729655],[-0.03040825677204806,0.0507235711915347,-0.022885751617396227,0.059887395694669714,0.15252207096065878,-0.18900322054635946,0.01836524419284244,-0.25785012876392255,0.04853189656558753,0.2312681271910294,0.08043180069849806,0.05241704932626748,1.0,0.008407559391231216,0.22948012988851088],[0.007247000470261807,0.007668264533886477,-0.06671681046525728,0.07879032750155776,0.09155502958739214,-0.02590228363160914,-0.003483338999629863,-0.010808695782368344,0.12451370418131098,0.0006180198534297345,0.008530428384064393,0.009385797117355184,0.008407559391231216,1.0,0.02326814043187281],[0.17604239479738906,0.018043983433917464,-0.008957423359172337,0.07898745800459364,0.33528619675264226,-0.19351839380525632,0.05157686310141771,-0.25100282131714535,0.07165760495137209,0.2166986810755833,0.22119621454805358,0.15005330839729655,0.22948012988851088,0.02326814043187281,1.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"Features: %{x}\\u003cbr\\u003eFeatures: %{y}\\u003cbr\\u003ecolor: %{z}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"Features\"},\"side\":\"top\"},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"Features\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"showscale\":false},\"margin\":{\"t\":60},\"title\":{\"text\":\"Correlation Matrix\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('242f2991-1950-416f-a243-3f2879f9cb0b');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "corr_df = df.copy()\n",
        "\n",
        "from sklearn.preprocessing import LabelEncoder\n",
        "\n",
        "le = LabelEncoder()\n",
        "\n",
        "for column in list(set(df.columns) - set(df._get_numeric_data().columns)):\n",
        "    corr_df[column] = le.fit_transform(corr_df[column])\n",
        "\n",
        "import plotly.express as px\n",
        "\n",
        "fig = px.imshow(corr_df.corr(numeric_only=True).values,\n",
        "                labels=dict(x=\"Features\", y=\"Features\"),\n",
        "                x=list(corr_df.columns),\n",
        "                y=list(corr_df.columns),\n",
        "                text_auto=True\n",
        "                )\n",
        "fig.update_xaxes(side='top')\n",
        "fig.update_layout(template='plotly_dark',\n",
        "                  title='Correlation Matrix',\n",
        "                  coloraxis_showscale=False,\n",
        "                  paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "                  plot_bgcolor='rgba(0, 0, 0, 0)')\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5J4KxbpceRIs"
      },
      "source": [
        "According to the correlation scores, `Sex` ($0.21$) and `Age` ($0.17$) have a slight positive correlation with `Income` (race has a correlation that verges 0). Hence, it is safe to assume that these features might have some influence on the classification of a system trained on this dataset.\n",
        "\n",
        "To test this claim, let us write a simple classifier (a `dense neural network`) and evaluate it against [statistical fairness metrics](https://arxiv.org/abs/2001.07864)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "R0lLyPBUeRIt",
        "outputId": "1fc51a44-8de1-43b4-8585-e8914fb5d53d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " flatten_1 (Flatten)         (None, 14)                0         \n",
            "                                                                 \n",
            " dense_3 (Dense)             (None, 64)                960       \n",
            "                                                                 \n",
            " dense_4 (Dense)             (None, 32)                2080      \n",
            "                                                                 \n",
            " dense_5 (Dense)             (None, 2)                 66        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 3106 (12.13 KB)\n",
            "Trainable params: 3106 (12.13 KB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n",
            "Version:  2.15.0\n",
            "Eager mode:  True\n",
            "GPU is NOT AVAILABLE\n",
            "Epoch 1/5\n",
            "151/151 [==============================] - 3s 7ms/step - loss: 574.8478 - accuracy: 0.6748 - val_loss: 60.8769 - val_accuracy: 0.7795\n",
            "Epoch 2/5\n",
            "151/151 [==============================] - 1s 6ms/step - loss: 294.3786 - accuracy: 0.6694 - val_loss: 34.2294 - val_accuracy: 0.7804\n",
            "Epoch 3/5\n",
            "151/151 [==============================] - 1s 5ms/step - loss: 223.5877 - accuracy: 0.6700 - val_loss: 47.5994 - val_accuracy: 0.7812\n",
            "Epoch 4/5\n",
            "151/151 [==============================] - 1s 6ms/step - loss: 251.2510 - accuracy: 0.6778 - val_loss: 295.9254 - val_accuracy: 0.7924\n",
            "Epoch 5/5\n",
            "151/151 [==============================] - 1s 5ms/step - loss: 265.2267 - accuracy: 0.6685 - val_loss: 557.9918 - val_accuracy: 0.7712\n",
            "189/189 [==============================] - 1s 3ms/step - loss: 561.1011 - accuracy: 0.7630\n",
            "Final Loss: 561.10.\n",
            "Final Performance: 76.30 %.\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"97bcf989-fb28-4301-9ccb-5e272f95e7a0\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"97bcf989-fb28-4301-9ccb-5e272f95e7a0\")) {                    Plotly.newPlot(                        \"97bcf989-fb28-4301-9ccb-5e272f95e7a0\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"texttemplate\":\"%{z}\",\"x\":[\"\\u003c=50K\",\"\\u003e50K\"],\"y\":[\"\\u003c=50K\",\"\\u003e50K\"],\"z\":[[4516,17],[1413,87]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"Predicted: %{x}\\u003cbr\\u003eTrue label: %{y}\\u003cbr\\u003ecolor: %{z}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"#f2f5fa\"},\"error_y\":{\"color\":\"#f2f5fa\"},\"marker\":{\"line\":{\"color\":\"rgb(17,17,17)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"baxis\":{\"endlinecolor\":\"#A2B1C6\",\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"minorgridcolor\":\"#506784\",\"startlinecolor\":\"#A2B1C6\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"marker\":{\"line\":{\"color\":\"#283442\"}},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#506784\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"header\":{\"fill\":{\"color\":\"#2a3f5f\"},\"line\":{\"color\":\"rgb(17,17,17)\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#f2f5fa\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#f2f5fa\"},\"geo\":{\"bgcolor\":\"rgb(17,17,17)\",\"lakecolor\":\"rgb(17,17,17)\",\"landcolor\":\"rgb(17,17,17)\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"#506784\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"dark\"},\"paper_bgcolor\":\"rgb(17,17,17)\",\"plot_bgcolor\":\"rgb(17,17,17)\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"radialaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"yaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"},\"zaxis\":{\"backgroundcolor\":\"rgb(17,17,17)\",\"gridcolor\":\"#506784\",\"gridwidth\":2,\"linecolor\":\"#506784\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"#C8D4E3\"}},\"shapedefaults\":{\"line\":{\"color\":\"#f2f5fa\"}},\"sliderdefaults\":{\"bgcolor\":\"#C8D4E3\",\"bordercolor\":\"rgb(17,17,17)\",\"borderwidth\":1,\"tickwidth\":0},\"ternary\":{\"aaxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"},\"bgcolor\":\"rgb(17,17,17)\",\"caxis\":{\"gridcolor\":\"#506784\",\"linecolor\":\"#506784\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"updatemenudefaults\":{\"bgcolor\":\"#506784\",\"borderwidth\":0},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"#283442\",\"linecolor\":\"#506784\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"#283442\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\",\"title\":{\"text\":\"Predicted\"},\"side\":\"top\"},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\",\"title\":{\"text\":\"True label\"}},\"coloraxis\":{\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"showscale\":false},\"margin\":{\"t\":60},\"title\":{\"text\":\"Confusion Matrix\"},\"paper_bgcolor\":\"rgba(0, 0, 0, 0)\",\"plot_bgcolor\":\"rgba(0, 0, 0, 0)\"},                        {\"responsive\": true}                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('97bcf989-fb28-4301-9ccb-5e272f95e7a0');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })                };                            </script>        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import tensorflow as tf\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "seed = 42\n",
        "x_train, x_test, y_train, y_test = train_test_split(corr_df.drop('income', axis=1).values,\\\n",
        "     tf.one_hot(corr_df['income'].values, 2).numpy(), test_size=0.2, random_state=seed)\n",
        "\n",
        "regularizer = tf.keras.regularizers.l1_l2(l1=0.001, l2=0.001)\n",
        "\n",
        "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(input_shape=(x_train.shape[1],)),\n",
        "                                    tf.keras.layers.Dense(\n",
        "                                        64, activation='relu', kernel_regularizer=regularizer),\n",
        "                                    tf.keras.layers.Dense(\n",
        "                                        32, activation='relu', kernel_regularizer=regularizer),\n",
        "                                    tf.keras.layers.Dense(2, activation='softmax')])\n",
        "\n",
        "opt = tf.keras.optimizers.Adam(learning_rate=0.001)\n",
        "model.compile(optimizer=opt,\n",
        "              loss='categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "model.summary()\n",
        "print(\"Version: \", tf.__version__)\n",
        "print(\"Eager mode: \", tf.executing_eagerly())\n",
        "print(\"GPU is\", \"available\" if tf.config.list_physical_devices('GPU') else \"NOT AVAILABLE\")\n",
        "\n",
        "model.fit(x_train, y_train, epochs=5,\n",
        "                    batch_size=128, validation_split=0.2,\n",
        "                    verbose=1)\n",
        "\n",
        "test_loss_score, test_acc_score = model.evaluate(x_test, y_test)\n",
        "\n",
        "print(f'Final Loss: {test_loss_score:.2f}.')\n",
        "print(f'Final Performance: {test_acc_score * 100:.2f} %.')\n",
        "\n",
        "predictions = model.predict(x_test, verbose=0)\n",
        "\n",
        "from sklearn.metrics import confusion_matrix\n",
        "\n",
        "matrix = confusion_matrix(y_test.argmax(axis=1), predictions.argmax(axis=1))\n",
        "\n",
        "fig = px.imshow(matrix,\n",
        "                labels=dict(x=\"Predicted\", y=\"True label\"),\n",
        "                x=['<=50K', '>50K'],\n",
        "                y=['<=50K', '>50K'],\n",
        "                text_auto=True\n",
        "                )\n",
        "fig.update_xaxes(side='top')\n",
        "fig.update_layout(template='plotly_dark',\n",
        "                  title='Confusion Matrix',\n",
        "                  coloraxis_showscale=False,\n",
        "                  paper_bgcolor='rgba(0, 0, 0, 0)',\n",
        "                  plot_bgcolor='rgba(0, 0, 0, 0)')\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u4TYP21qeRIt"
      },
      "source": [
        "The network tends to get an accuracy of $\\approx$ 70%. Let us see how this will change when we consider the sensitive attributes present in the dataset.\n",
        "\n",
        "### Statistical Fairness Metrics\n",
        "\n",
        "_Fairness metrics are a set of measures that enable you to detect the presence of bias in your model._ For a full review of the most prominent definitions of fairness in algorithmic classification, we recommend [_Fairness Definitions Explained_](https://fairware.cs.umass.edu/papers/Verma.pdf). Below, you will see how to apply these definitions.\n",
        "\n",
        "Since all these definitons are a way to compare the rates of correct/incorrect predictions, it is worth to remember what is a confusion matrix. A confusion matrix is a table often used to describe the performance of a classification model (or \"classifier\") on a set of test data for which the actual values are known.\n",
        "\n",
        "![confusion-matrix](https://upload.wikimedia.org/wikipedia/commons/3/32/Binary_confusion_matrix.jpg)\n",
        "\n",
        "Where:\n",
        "\n",
        "- $TP$: True Positive (_correct positive classification_, e.g., you make >50K and you are classified as `>50K`)\n",
        "- $FP$: False Positive (_incorrect positive classification_, e.g., you dont make >50K an you are classified as `>50K`)\n",
        "- $TN$: True Negative (_correct negative classification_, e.g., you make <=50K and you are classified as `<=50K`)\n",
        "- $FN$: False Negative (_incorrect negative classification_, e.g., you dont make <=50K and you are classified as `<=50K`)\n",
        "\n",
        "With these values, there are several ways to assess how biased an algorithm may be by comparing how certain combinations of these scores change about specific groups.\n",
        "\n",
        "> **Note: In the literature, it is common to refer to these groups, in the context of a fairness analysis, as `privileged` or `unprivileged`.**\n",
        "\n",
        "### Common Statistical Fairness Metrics\n",
        "\n",
        "- **Statistical Parity Ratio:** Statistical Parity Ratio compares the proportion of members of a given group that were classified for the positive class (i.e., correctly or not, a.k.a., TP and FP) to another group (privileged versus unprivileged).\n",
        "\n",
        "$$\\frac{\\text{Statistical Parity}_{\\text{ unprivileged}}}{\\text{Statistical Parity}_{\\text{ privileged}}} = \\frac{(\\frac{TP  +  FP}{TP  +  FP  +  TN  +  FN})_{\\text{unprivileged}}}{(\\frac{TP  +  FP}{TP  +  FP  +  TN  +  FN})_{\\text{privileged}}}$$\n",
        "\n",
        "- **Equal Opportunity Ratio:** Equal Opportunity ratio compares the true positive rate (i.e., TPR, a.k.a., _Sensitivity/Recall_) of different groups (privileged versus unprivileged).\n",
        "\n",
        "$$\\frac{\\text{TPR}_{\\text{ unprivileged}}}{\\text{TPR}_{\\text{ privileged}}} = \\frac{(\\frac{TP}{TP+FN})_{\\text{unprivileged}}}{(\\frac{TP}{TP+FN})_{\\text{privileged}}}$$\n",
        "\n",
        "- **Predictive Parity Ratio:** Predictive Parity Ratio compares the positive predictive value (i.e., PPV, a.k.a., _Precision_) of different groups (privileged versus unprivileged).\n",
        "\n",
        "$$\\frac{\\text{PPV}_{\\text{ unprivileged}}}{\\text{PPV}_{\\text{ privileged}}} = \\frac{(\\frac{TP}{TP+FP})_{\\text{unprivileged}}}{(\\frac{TP}{TP+FP})_{\\text{privileged}}}$$\n",
        "\n",
        "- **Predictive Equality Ratio:** Predictive Equality Ratio compares the false positive rate (FPR, a.k.a., _fall-out/false alarm ratio_) of different groups (privileged versus unprivileged).\n",
        "\n",
        "$$\\frac{\\text{FPR}_{\\text{ unprivileged}}}{\\text{FPR}_{\\text{ privileged}}} = \\frac{(\\frac{FP}{FP+TN})_{\\text{unprivileged}}}{(\\frac{FP}{FP+TN})_{\\text{privileged}}}$$\n",
        "\n",
        "- **Accuracy Equality Ratio:** Accuracy Equality Ratio compares the proportion of members of a given group that were correctly classified (i.e., _accuracy_) to another group (privileged versus unprivileged).\n",
        "\n",
        "$$\\frac{\\text{Accuracy}_{\\text{ unprivileged}}}{\\text{Accuracy}_{\\text{ privileged}}} = \\frac{(\\frac{TP  +  TN}{TP  +  FP  +  TN  +  FN})_{\\text{unprivileged}}}{(\\frac{TP  +  TN}{TP  +  FP  +  TN  +  FN})_{\\text{privileged}}}$$\n",
        "\n",
        "- **Equalized Odds:** Equalized Odds it is the most restrictive concept of ML Fairness. This criteria is only satisfied if both groups (privileged versus unprivileged) have equal TPR and FPR.\n",
        "\n",
        "$$\\frac{\\text{TPR}_{\\text{ unprivileged}}}{\\text{TPR}_{\\text{ privileged}}} = 1 \\;\\land\\;\\frac{\\text{FPR}_{\\text{ unprivileged}}}{\\text{FPR}_{\\text{ privileged}}} = 1$$\n",
        "\n",
        "Let us now  implement a function that calculates all these metrics."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "WWnB3WcMeRIt"
      },
      "outputs": [],
      "source": [
        "\n",
        "def calc_fair(model, DataFrame, protected_atributte, group_priv, group_unpriv, label):\n",
        "    \"\"\"\n",
        "    The function calc_fair computes several fairness metrics for a given machine\n",
        "    learning model on a test set DataFrame. The fairness metrics calculated include\n",
        "    statistical parity ratio, true positive rate, positive predictive value,\n",
        "    false positive rate, accuracy, equal opportunity ratio, predictive parity\n",
        "    ratio, predictive equality ratio, and accuracy equality ratio. The function\n",
        "    takes in the following arguments:\n",
        "\n",
        "    Args:\n",
        "    --------\n",
        "        - model: The trained machine learning model to evaluate fairness on.\n",
        "        - DataFrame: The test set data used to evaluate the model.\n",
        "        - protected_attribute: The name of the protected attribute in the DataFrame.\n",
        "        - group_priv: The value of the protected attribute for the privileged group.\n",
        "        - group_unpriv: The value of the protected attribute for the unprivileged group.\n",
        "        - label: The name of the column in the DataFrame that contains the ground truth labels.\n",
        "\n",
        "    Returns:\n",
        "    --------\n",
        "    The function returns a dictionary containing the fairness metric names and their\n",
        "    corresponding scores. The scores are rounded to two decimal places. Additionally,\n",
        "    the function returns the equalized odds as a string.\n",
        "    \"\"\"\n",
        "    test_set = DataFrame.copy()\n",
        "\n",
        "    for column in list(set(test_set.columns) - set(test_set._get_numeric_data().columns)):\n",
        "        test_set[column] = le.fit_transform(test_set[column])\n",
        "\n",
        "    test_set_priv_labels, test_set_priv = tf.one_hot(list(test_set[test_set[protected_atributte] == group_priv][label]), 2).numpy(), test_set[test_set[protected_atributte] == group_priv].drop(label, axis = 1).values\n",
        "    test_set_unpriv_labels, test_set_unpriv = tf.one_hot(list(test_set[test_set[protected_atributte] == group_unpriv][label]), 2).numpy(), test_set[test_set[protected_atributte] == group_unpriv].drop(label, axis = 1).values\n",
        "\n",
        "    preds_priv = model.predict(test_set_priv, verbose=0)\n",
        "    preds_unpriv = model.predict(test_set_unpriv, verbose=0)\n",
        "\n",
        "    TN_PV, FP_PV, FN_PV, TP_PV = confusion_matrix(test_set_priv_labels.argmax(axis=1), preds_priv.argmax(axis=1)).ravel()\n",
        "    TN_UPV, FP_UPV, FN_UPV, TP_UPV = confusion_matrix(test_set_unpriv_labels.argmax(axis=1), preds_unpriv.argmax(axis=1)).ravel()\n",
        "\n",
        "    statistical_parity_priv = (TP_PV + FP_PV)/(TP_PV + FP_PV + TN_PV + FN_PV)  # STATISTICAL PARITY RATIO\n",
        "    statistical_parity_unpriv = (TP_UPV + FP_UPV)/(TP_UPV + FP_UPV + TN_UPV + FN_UPV)  # STATISTICAL PARITY RATIO\n",
        "    equal_oportunity_priv = TP_PV / (TP_PV+FN_PV)  # TRUE POSITIVE RATIO\n",
        "    equal_oportunity_unpriv = TP_UPV / (TP_UPV+FN_UPV)  # TRUE POSITIVE RATIO\n",
        "    predictive_parity_priv = TP_PV/(TP_PV + FP_PV)  # POSITIVE PREDICTIVE VALUE\n",
        "    predictive_parity_unpriv = TP_UPV/(TP_UPV + FP_UPV)  # POSITIVE PREDICTIVE VALUE\n",
        "    predictive_equality_priv = FP_PV / (FP_PV+TN_PV)  # FALSE POSITIVE RATE\n",
        "    predictive_equality_unpriv = FP_UPV / (FP_UPV+TN_UPV)  # FALSE POSITIVE RATE\n",
        "    accuracy_equality_priv = (TP_PV + TN_PV)/(TP_PV + FP_PV + TN_PV + FN_PV)  # ACCURACY EQUALITY RATIO\n",
        "    accuracy_equality_unpriv = (TP_UPV + TN_UPV)/(TP_UPV + FP_UPV + TN_UPV + FN_UPV)  # ACCURACY EQUALITY RATIO\n",
        "\n",
        "    if statistical_parity_priv >= statistical_parity_unpriv:\n",
        "        statistical_parity_ratio = statistical_parity_unpriv/statistical_parity_priv\n",
        "    elif statistical_parity_priv < statistical_parity_unpriv:\n",
        "        statistical_parity_ratio = statistical_parity_priv/statistical_parity_unpriv\n",
        "\n",
        "    if equal_oportunity_priv >= equal_oportunity_unpriv:\n",
        "        equal_oportunity_ratio = equal_oportunity_unpriv/equal_oportunity_priv\n",
        "    elif equal_oportunity_priv < equal_oportunity_unpriv:\n",
        "        equal_oportunity_ratio = equal_oportunity_priv/equal_oportunity_unpriv\n",
        "\n",
        "    if predictive_parity_priv >= predictive_parity_unpriv:\n",
        "        predictive_parity_ratio = predictive_parity_unpriv/predictive_parity_priv\n",
        "    elif predictive_parity_priv < predictive_parity_unpriv:\n",
        "        predictive_parity_ratio = predictive_parity_priv/predictive_parity_unpriv\n",
        "\n",
        "    if predictive_equality_priv >= predictive_equality_unpriv:\n",
        "        predictive_equality_ratio = predictive_equality_unpriv/predictive_equality_priv\n",
        "    elif predictive_equality_priv < predictive_equality_unpriv:\n",
        "        predictive_equality_ratio = predictive_equality_priv/predictive_equality_unpriv\n",
        "\n",
        "    if accuracy_equality_priv >= accuracy_equality_unpriv:\n",
        "        accuracy_equality_ratio = accuracy_equality_unpriv/accuracy_equality_priv\n",
        "    elif accuracy_equality_priv < accuracy_equality_unpriv:\n",
        "        accuracy_equality_ratio = accuracy_equality_priv/accuracy_equality_unpriv\n",
        "\n",
        "    data = {'Fairness Metrics': ['Chance of receiving the positive class - privileged',\n",
        "                                'Chance of receiving the positive class - unprivileged',\n",
        "                                'Statistical Parity Ratio (SPR)',\n",
        "                                'True Positive Rate - privileged',\n",
        "                                'True Positive Rate - unprivileged',\n",
        "                                'Equal Opportunity Ratio (EOR)',\n",
        "                                'Positive Predictive Value - privileged',\n",
        "                                'Positive Predictive Value - unprivileged',\n",
        "                                'Predictive Parity Ratio (PPR)',\n",
        "                                'False Positive Rate - privileged',\n",
        "                                'False Positive Rate - unprivileged',\n",
        "                                'Predictive Equality Ratio (PER)',\n",
        "                                'Accuracy - privileged',\n",
        "                                'Accuracy - unprivileged',\n",
        "                                'Accuracy Equality Ratio (AER)',\n",
        "                                'Equalized Odds'],\n",
        "            'Scores': [round(statistical_parity_priv, 2),\n",
        "                        round(statistical_parity_unpriv, 2),\n",
        "                        round(statistical_parity_ratio,2),\n",
        "                        round(equal_oportunity_priv, 2),\n",
        "                        round(equal_oportunity_unpriv, 2),\n",
        "                        round(equal_oportunity_ratio, 2),\n",
        "                        round(predictive_parity_priv,2),\n",
        "                        round(predictive_parity_unpriv,2),\n",
        "                        round(predictive_parity_ratio,2),\n",
        "                        round(predictive_equality_priv,2),\n",
        "                        round(predictive_equality_unpriv,2),\n",
        "                        round(predictive_equality_ratio,2),\n",
        "                        round(accuracy_equality_priv,2),\n",
        "                        round(accuracy_equality_unpriv,2),\n",
        "                        round(accuracy_equality_ratio,2),\n",
        "                        f'TPR: {round(equal_oportunity_priv, 2)} vs {round(equal_oportunity_unpriv, 2)}. FPR: {round(predictive_equality_priv,2)} vs {round(predictive_equality_unpriv,2)}']\n",
        "            }\n",
        "    return pd.DataFrame(data).set_index('Fairness Metrics')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tha8ZanneRIt"
      },
      "source": [
        "Let us use the feature `race` as an example. Remember that, after encoding, `race` feature values are:\n",
        "\n",
        "- $4$ = `White`\n",
        "- $3$ = `Other`\n",
        "- $2$ = `Black`\n",
        "- $1$ = `Asian-Pac-Islander`\n",
        "- $0$ = `Amer-Indian-Eskimo`\n",
        "\n",
        "Let us define $4$, the group with more samples, as the privileged, and $2$ as the unprivileged group."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 551
        },
        "id": "Tu0ZBddUeRIt",
        "outputId": "5d56bcc4-4867-4054-cb6d-f3132b3945cc"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "| Fairness Metrics                                      | Scores                             |\n",
              "|:------------------------------------------------------|:-----------------------------------|\n",
              "| Chance of receiving the positive class - privileged   | 0.02                               |\n",
              "| Chance of receiving the positive class - unprivileged | 0.01                               |\n",
              "| Statistical Parity Ratio (SPR)                        | 0.49                               |\n",
              "| True Positive Rate - privileged                       | 0.06                               |\n",
              "| True Positive Rate - unprivileged                     | 0.07                               |\n",
              "| Equal Opportunity Ratio (EOR)                         | 0.95                               |\n",
              "| Positive Predictive Value - privileged                | 0.83                               |\n",
              "| Positive Predictive Value - unprivileged              | 0.89                               |\n",
              "| Predictive Parity Ratio (PPR)                         | 0.94                               |\n",
              "| False Positive Rate - privileged                      | 0.0                                |\n",
              "| False Positive Rate - unprivileged                    | 0.0                                |\n",
              "| Predictive Equality Ratio (PER)                       | 0.27                               |\n",
              "| Accuracy - privileged                                 | 0.75                               |\n",
              "| Accuracy - unprivileged                               | 0.88                               |\n",
              "| Accuracy Equality Ratio (AER)                         | 0.85                               |\n",
              "| Equalized Odds                                        | TPR: 0.06 vs 0.07. FPR: 0.0 vs 0.0 |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from IPython.display import Markdown\n",
        "fairness_df = calc_fair(model, df, 'race', 4, 2, 'income')\n",
        "\n",
        "display(Markdown(fairness_df.to_markdown()))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eb4LC7ReeRIu"
      },
      "source": [
        "Depending on the fairness metric we choose, we see different results. For example, with a **Statistical Parity Ratio** of $0.55$, the privileged class is almost $2$ times as likely to be classified as the positive class (if the positive class was `>50k`). Also, even though **Accuracy Equality Ratio** (AER) is above $80\\%$, the model is $10\\%$ less accurate against the unprivileged group. **Equal Opportunity Ratio** (EOR), **Predictive Parity Ratio** (PPR), and **Equalized Odds** are inside \"_acceptable ranges_\" (i.e., above 80%).\n",
        "\n",
        "> **Note: The results may vary depending on your classifier and how you have trained it.**\n",
        "\n",
        "Interestingly enough, these results do not appear when we, for example, measure group $4$ (`White`) against group $1$ (`Asian-Pac-Islander`). All metrics are well above acceptable thresholds.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 551
        },
        "id": "nIs_L2iWeRIu",
        "outputId": "df8752ab-f50a-4f89-d3a7-01c467157f3f"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "| Fairness Metrics                                      | Scores                             |\n",
              "|:------------------------------------------------------|:-----------------------------------|\n",
              "| Chance of receiving the positive class - privileged   | 0.02                               |\n",
              "| Chance of receiving the positive class - unprivileged | 0.02                               |\n",
              "| Statistical Parity Ratio (SPR)                        | 0.88                               |\n",
              "| True Positive Rate - privileged                       | 0.06                               |\n",
              "| True Positive Rate - unprivileged                     | 0.08                               |\n",
              "| Equal Opportunity Ratio (EOR)                         | 0.77                               |\n",
              "| Positive Predictive Value - privileged                | 0.83                               |\n",
              "| Positive Predictive Value - unprivileged              | 1.0                                |\n",
              "| Predictive Parity Ratio (PPR)                         | 0.83                               |\n",
              "| False Positive Rate - privileged                      | 0.0                                |\n",
              "| False Positive Rate - unprivileged                    | 0.0                                |\n",
              "| Predictive Equality Ratio (PER)                       | 0.0                                |\n",
              "| Accuracy - privileged                                 | 0.75                               |\n",
              "| Accuracy - unprivileged                               | 0.75                               |\n",
              "| Accuracy Equality Ratio (AER)                         | 0.99                               |\n",
              "| Equalized Odds                                        | TPR: 0.06 vs 0.08. FPR: 0.0 vs 0.0 |"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from IPython.display import Markdown\n",
        "fairness_df = calc_fair(model, df, 'race', 4, 1, 'income')\n",
        "\n",
        "display(Markdown(fairness_df.to_markdown()))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3VIOFW_eeRIu"
      },
      "source": [
        "Fairness metrics can help us determine if our model discriminates according to any particular definition of fairness. However, as an [impossibility theorem](https://arxiv.org/abs/2007.06024), some Fairness metrics (_Statistical Parity_, _Equalized Odds_, _Predictive Parity_) are incompatible and cannot be completely satisfied simultaneously. Thus, the choice of which metric to use must be made according to the context of an application (i.e., benefit awarding, medical diagnosis, etc.).\n",
        "\n",
        "---\n",
        "\n",
        "Return to the [castle](https://github.com/Nkluge-correa/TeenyTinyCastle)."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    },
    "vscode": {
      "interpreter": {
        "hash": "aca09746cf57686f00a55ae76e987247ecfb5dd0b3b2e2474d8dbbf0c5e3377e"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}