04 - Quarto dia.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introdução ao aprendizado de máquina utilizando Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Professor: Luiz Antonio de Sousa Ferreira"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Contato: luiz.ferreira@outlook.com | [@luizsferreira](http://instagram.com/luizsferreira) (IG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Acesso: [bit.ly/pucpcaldas](http://bit.ly/pucpcaldas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__________"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Na aula de hoje, discutiremos o pré-processamento, a análise e a visualização de dados. Além disso, neste pré-processamento de dados em machine learning em Python, veremos o reescalonamento, a padronização, a normalização e a binarização dos dados. Além disso, veremos diferentes etapas na Análise e Visualização de Dados.\n",
    "\n",
    "Então, vamos começar o aprendizado de máquina com o pré-processamento de dados do Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://d2h0cx97tjks2p.cloudfront.net/blogs/wp-content/uploads/Python-Machine-Learning-01.jpg)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data pre-processing\n",
    "\n",
    "O uso de pré-processamento de dados é um passo necessário em uma análise de dados, uma vez que se trata de um processo de converter ou mapear dados da forma bruta inicial para outro formato, a fim de preparar os dados para uma análise mais aprofundada.\n",
    "\n",
    "Podemos encontrar o pré-processamento como `\"Data Cleaning\"` ou `\"Data Wranging\"`.\n",
    "\n",
    "### Nessa aula teremos como objetivo:\n",
    "\n",
    "- Identificar e tratar dados ausêntes em nossa base\n",
    "- Adequar formatos de dados, que possuem diferentes unidades\n",
    "- Normalização de dados (centering / scaling) (trabalhar com o range de aplicação dos dados)\n",
    "- Data binning, criação de categorias de dados\n",
    "- Tornando dados categóricos em numéricos\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data missing\n",
    "\n",
    "Quando não possuímos nenhum valor relacionado à coluna de dados em que esta está presente, dizemos que temos um dado ausente. Podemos encontrar diversas variações para esse fenômeno, a célula pode conter o símbolo de interrogação, um valor nulo ou apenas estar em branco. Em nosso caso, os campos estarão marcados com o mneunômico `\"NaN\"`.\n",
    "\n",
    "Temos diversas formas de trabalhar com dados ausentes, sendo que a abordagem de todas seria impossível em um curto período de tempo, mas podemos dizer que esses métodos podem ser aplicados utilizando Python ou R. Consideremos então algumas opções de uso comum, são elas:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Remover os valores ausentes\n",
    "- remover a variável\n",
    "- remover a entrada do dado\n",
    "\n",
    "Essa primeira opção trata a presença desse registro como prejudicial à nossa análise e que o mesmo pode ser descartado. Essa remoção pode vir tanto por um único valor, como pode ser o registro todo (linha do banco).\n",
    "\n",
    "Essa solução é empregada, principalmente, quando a quantidade de dados ausentes é pequena, pois isso acarretará em impactos em nossa base de dados, o que não é interessante, mas devido ao problema gerado por um dado inexistente, o preço da remoção pode sair mais em conta."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Substituir os valores ausentes\n",
    "- substituir com a média \n",
    "- substituir pela frequência\n",
    "- substituir baseado em outras funções\n",
    "\n",
    "A substituição de dados é a melhor alternativa, pois preserva características da base original, focando em apenas estipular o valor ausente, o que trará um menor impacto dentro de nossa base (comparado à remoção).\n",
    "\n",
    "Um padrão utilizado em massa é a substituição dos valores ausentes pelo valor médio da variável inteira. Como exemplo, imaginemos que tenhamos algumas entradas com valores omissos para a coluna de perdas normalizadas e a média da coluna para as entradas seja 4500. Embora não seja possível obter uma estimativa realmente eficiente, a média trará um valor de menor impacto, pois é estimado em conformidade com os dados que a base já possui. \n",
    "\n",
    "Caso nosso dado ausente não seja numérico, a média é um recurso que não possuímos, porém, podemos estipular os valores através do uso de ferramentas de frequência, utilizando os mais comuns de acontecer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Remover dados ausentes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Imaginemos a seguinte sequência de dados:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>highway-mpg</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20</td>\n",
       "      <td>23875.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>22</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>29</td>\n",
       "      <td>16430.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   highway-mpg    price\n",
       "0           20  23875.0\n",
       "1           22      NaN\n",
       "2           29  16430.0"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "data = pd.DataFrame(data=[ 20, 22, 29], columns=[\"highway-mpg\"])\n",
    "data[\"price\"] = np.array([23875,np.nan, 16430])\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Aqui temos uma microbase, onde inserimos três linhas de valores para as colunas `highway-mpg` e `price`. \n",
    "\n",
    "Passamos então para a remoção de dados ausentes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>highway-mpg</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20</td>\n",
       "      <td>23875.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>29</td>\n",
       "      <td>16430.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   highway-mpg    price\n",
       "0           20  23875.0\n",
       "2           29  16430.0"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data.dropna(subset=[\"price\"],axis=0, inplace=True)\n",
    "\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "O uso correto do `df.dropna()`:\n",
    "\n",
    "- Devemos sempre especificar qual coluna devemos utilizar, através do parâmetro `subset`\n",
    "- Informamos qual a forma de remoção, se é por linha ou coluna para isso utilizaremos `axis=0` para linha e `axis=1` para coluna\n",
    "- O último parâmetro pode ser interpretado como modificação por substituição, afinal, ele representa o mesmo que:\n",
    "    - data = data.dropna(subset=[\"price\"], axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Substituição de valores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A biblioteca `Pandas` irá nos auxiliar também nesse caso, utilizaremos uma função chamada `replace` para que possamos fazer as substituições pelos novos valores calculados.\n",
    "\n",
    "Vejamos uma outra fonte de dados:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>normalized-losses</th>\n",
       "      <th>make</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>166.0</td>\n",
       "      <td>audi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>164.0</td>\n",
       "      <td>audi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>audi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>158.0</td>\n",
       "      <td>audi</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   normalized-losses  make\n",
       "0              166.0  audi\n",
       "1              164.0  audi\n",
       "2                NaN  audi\n",
       "3              158.0  audi"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = pd.DataFrame(data=[166,164,np.nan,158], columns=[\"normalized-losses\"])\n",
    "data[\"make\"] = np.array([\"audi\",\"audi\",\"audi\",\"audi\"])\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notamos que o valor de id 2 está com uma ausência de dados. Devemos então substituir o valor nulo pela média dos valores daquela coluna."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Valor médio:  166.0 \n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>normalized-losses</th>\n",
       "      <th>make</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>166.0</td>\n",
       "      <td>audi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>164.0</td>\n",
       "      <td>audi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>166.0</td>\n",
       "      <td>audi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>158.0</td>\n",
       "      <td>audi</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   normalized-losses  make\n",
       "0              166.0  audi\n",
       "1              164.0  audi\n",
       "2              166.0  audi\n",
       "3              158.0  audi"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mean = data[\"normalized-losses\"].max()\n",
    "print(\"Valor médio: \", mean,\"\\n\\n\")\n",
    "data[\"normalized-losses\"].replace(np.nan, mean, inplace=True)\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note que a média calculada foi inserida na base.\n",
    "\n",
    "Existem diversas técnicas que podemos utilizar, porém, devido ao curto período de tempo, essa técnica já tratará nossas informações de forma adequada."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data formatting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "No mundo real, geralmente encontraremos bases de dados coletadas de diferentes lugares e armazenadas de diferentes formas, preservando as características necessárias para o estabelecimento que as possui.\n",
    "\n",
    "Fazer com que os dados possuam uma padronização nos permitirá utilizar comparações realísticas de como trabalhar essas informações. Como parte de uma limpeza de uma base de dados, o processo de formatação de dados visa fazer com que tenhamos uma base consistente e de fácil compreensão."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exemplo:\n",
    "\n",
    "Costumamos tratar siglas de cidades de diferentes formas, pensemos na cidade de New York. Podemos ter as representações como:\n",
    "\n",
    "- New York\n",
    "- NY\n",
    "- N.Y.\n",
    "- NY.\n",
    "- N.Y\n",
    "\n",
    "Quando não formatado, temos uma situação confusa, de difícil compreensão (uma vez que pode se tratar de ambiguidades), difícil de ser somada ou comparada com outra variável."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Imaginemos outra situação.\n",
    "\n",
    "Temos uma base de dados onde o dataset de carros possui o consumo dentro de uma cidade para cada modelo, porém, essa mesma encontra-se em milhas por galão, um padrão não muito fácil de compreender."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>auto</th>\n",
       "      <th>city-mpg</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>cruze</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>audi</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>porsche</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      auto  city-mpg\n",
       "0    cruze        21\n",
       "1     audi        21\n",
       "2  porsche        19"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = pd.DataFrame(data=[\"cruze\",\"audi\",\"porsche\"], columns=[\"auto\"])\n",
    "data[\"city-mpg\"] = np.array([21,21,19])\n",
    "\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos realizar conversões de unidade com apenas uma linha de código em Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>auto</th>\n",
       "      <th>city-L/100km</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>cruze</td>\n",
       "      <td>11.190476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>audi</td>\n",
       "      <td>11.190476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>porsche</td>\n",
       "      <td>12.368421</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      auto  city-L/100km\n",
       "0    cruze     11.190476\n",
       "1     audi     11.190476\n",
       "2  porsche     12.368421"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[\"city-mpg\"] = 235/data[\"city-mpg\"]\n",
    "data.rename(columns={\"city-mpg\":\"city-L/100km\"}, inplace=True)\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Muitas vezes, problemas podem ocorrer durante uma importação de dados para o Python, fazendo com que o tipo do dado esteja incorreto, tornando assim, os cálculos não interpretáveis de maneira correta. \n",
    "\n",
    "Imaginemos a seguinte situação:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>highway-mpg</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20</td>\n",
       "      <td>23875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>22</td>\n",
       "      <td>23587</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>27</td>\n",
       "      <td>16430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>29</td>\n",
       "      <td>45500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   highway-mpg  price\n",
       "0           20  23875\n",
       "1           22  23587\n",
       "2           27  16430\n",
       "3           29  45500"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = pd.DataFrame(data=[ 20, 22, 27, 29], columns=[\"highway-mpg\"])\n",
    "data[\"price\"] = np.array([23875,23587, 16430, 45500])\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos obter o tipo de cada coluna dentro de um DataFrame em `pandas` utilizando o comando .dtypes()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('int32')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"price\"].dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Observemos que o tipo retornado para a variável \"price\" foi de int32, mas como sabemos, por se tratar de preços, essa informação deveria ser do tipo `float`, portanto, devemos alterar o tipo dessa informação:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "data[\"price\"] = data[\"price\"].astype(\"float\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "O que nos retornará:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"price\"].dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data normalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Essa técnica é utilizada para uniformizar dados para que os mesmos possam interagir entre assim, afim de obtermos resultados melhor elaborados. Observemos o seguinte DataFrame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>length</th>\n",
       "      <th>width</th>\n",
       "      <th>height</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>171.2</td>\n",
       "      <td>65.5</td>\n",
       "      <td>52.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>176.6</td>\n",
       "      <td>66.2</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>176.6</td>\n",
       "      <td>66.4</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>177.3</td>\n",
       "      <td>66.3</td>\n",
       "      <td>53.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>192.7</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>192.7</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>197.7</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   length  width  height\n",
       "0   168.8   64.1    48.8\n",
       "1   168.8   64.1    48.8\n",
       "2   171.2   65.5    52.4\n",
       "3   176.6   66.2    54.3\n",
       "4   176.6   66.4    54.3\n",
       "5   177.3   66.3    53.1\n",
       "6   192.7   71.4    55.7\n",
       "7   192.7   71.4    55.7\n",
       "8   197.7   71.4    55.9"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = pd.DataFrame(data=[168.8,168.8,171.2,176.6,176.6,177.3,192.7,192.7,197.7], columns=[\"length\"])\n",
    "data[\"width\"] = np.array([64.1,64.1,65.5,66.2,66.4,66.3,71.4,71.4,71.4])\n",
    "data[\"height\"] = np.array([48.8,48.8,52.4,54.3,54.3,53.1,55.7,55.7,55.9])\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fomos comunicados que nossos dados na coluna `\"lenght\"` possuem os limites entre 150 e 250 e os dados das colunas `\"width\"` e `\"height\"` dentro do intervalo entre 50 a 100.\n",
    "\n",
    "Com uma breve análise, foi nos passado que diferenças entre os valores não possuem tamanha relevância pra a solução de nosso problema, apenas são formas de classificar cada dado sobre veículo.\n",
    "\n",
    "Sabendo disso, aplicaremos a normalização como forma de categorizar cada veículo conforme segue:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tbody>\n",
       "<tr><td>Scale </td><td>[150-250]</td><td>[50-100]</td><td>[50-100]</td></tr>\n",
       "<tr><td>impact</td><td>large    </td><td>small   </td><td>small   </td></tr>\n",
       "</tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import HTML, display\n",
    "import tabulate\n",
    "table = [[\"Scale\",\"[150-250]\",\"[50-100]\",\"[50-100]\"],\n",
    "         [\"impact\",\"large\",\"small\",\"small\"]]\n",
    "display(HTML(tabulate.tabulate(table, tablefmt='html')))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Utilizaremos a normalização de dados para fazer com que nosso modelo possa ter valores minimizados, porém, com a mesma influência em nossa base como um todo.\n",
    "\n",
    "Para isso, possuímos três métodos principais utilizando o Python, são eles:\n",
    "\n",
    "- Simple feature scaling\n",
    "\n",
    "$$X_{new}= \\frac {X_{old}}{X_{max}}$$\n",
    "- Min-Max\n",
    "\n",
    "$$X_{new}= \\frac {X_{old}-X_{min}}{X_{max}-X_{min}}$$\n",
    "\n",
    "- z-Score\n",
    "\n",
    "$$X_{new}= \\frac {X_{old}-\\mu}{\\sigma}$$\n",
    "\n",
    "    - Para: \n",
    "        - valor médio = mu\n",
    "        - desvio padrão = sigma"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seguindo nosso exemplo anterior, temos:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>length</th>\n",
       "      <th>width</th>\n",
       "      <th>height</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>171.2</td>\n",
       "      <td>65.5</td>\n",
       "      <td>52.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>176.6</td>\n",
       "      <td>66.2</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>176.6</td>\n",
       "      <td>66.4</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>177.3</td>\n",
       "      <td>66.3</td>\n",
       "      <td>53.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>192.7</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>192.7</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>197.7</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   length  width  height\n",
       "0   168.8   64.1    48.8\n",
       "1   168.8   64.1    48.8\n",
       "2   171.2   65.5    52.4\n",
       "3   176.6   66.2    54.3\n",
       "4   176.6   66.4    54.3\n",
       "5   177.3   66.3    53.1\n",
       "6   192.7   71.4    55.7\n",
       "7   192.7   71.4    55.7\n",
       "8   197.7   71.4    55.9"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Passamos a executar a normalização utilizando o primeiro método:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>length</th>\n",
       "      <th>width</th>\n",
       "      <th>height</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.853819</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.853819</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.865959</td>\n",
       "      <td>65.5</td>\n",
       "      <td>52.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.893273</td>\n",
       "      <td>66.2</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.893273</td>\n",
       "      <td>66.4</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.896813</td>\n",
       "      <td>66.3</td>\n",
       "      <td>53.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.974709</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.974709</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     length  width  height\n",
       "0  0.853819   64.1    48.8\n",
       "1  0.853819   64.1    48.8\n",
       "2  0.865959   65.5    52.4\n",
       "3  0.893273   66.2    54.3\n",
       "4  0.893273   66.4    54.3\n",
       "5  0.896813   66.3    53.1\n",
       "6  0.974709   71.4    55.7\n",
       "7  0.974709   71.4    55.7\n",
       "8  1.000000   71.4    55.9"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[\"length\"] = data[\"length\"]/data[\"length\"].max()\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Já utilizando o método min-max:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>length</th>\n",
       "      <th>width</th>\n",
       "      <th>height</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.083045</td>\n",
       "      <td>65.5</td>\n",
       "      <td>52.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.269896</td>\n",
       "      <td>66.2</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.269896</td>\n",
       "      <td>66.4</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.294118</td>\n",
       "      <td>66.3</td>\n",
       "      <td>53.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.826990</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.826990</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     length  width  height\n",
       "0  0.000000   64.1    48.8\n",
       "1  0.000000   64.1    48.8\n",
       "2  0.083045   65.5    52.4\n",
       "3  0.269896   66.2    54.3\n",
       "4  0.269896   66.4    54.3\n",
       "5  0.294118   66.3    53.1\n",
       "6  0.826990   71.4    55.7\n",
       "7  0.826990   71.4    55.7\n",
       "8  1.000000   71.4    55.9"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[\"length\"] = (data[\"length\"]-data[\"length\"].min())/(data[\"length\"].max()-data[\"length\"].min())\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Por último faremos com os valores zScore:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>length</th>\n",
       "      <th>width</th>\n",
       "      <th>height</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-1.028721</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1.028721</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.813408</td>\n",
       "      <td>65.5</td>\n",
       "      <td>52.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.328952</td>\n",
       "      <td>66.2</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.328952</td>\n",
       "      <td>66.4</td>\n",
       "      <td>54.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.266152</td>\n",
       "      <td>66.3</td>\n",
       "      <td>53.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1.115445</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1.115445</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1.564015</td>\n",
       "      <td>71.4</td>\n",
       "      <td>55.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     length  width  height\n",
       "0 -1.028721   64.1    48.8\n",
       "1 -1.028721   64.1    48.8\n",
       "2 -0.813408   65.5    52.4\n",
       "3 -0.328952   66.2    54.3\n",
       "4 -0.328952   66.4    54.3\n",
       "5 -0.266152   66.3    53.1\n",
       "6  1.115445   71.4    55.7\n",
       "7  1.115445   71.4    55.7\n",
       "8  1.564015   71.4    55.9"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[\"length\"] = (data[\"length\"]-data[\"length\"].mean())/data[\"length\"].std()\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Binning\n",
    "\n",
    "Esse processo define a categorização de nossos dados em detrimento dos valores, utilizamos então uma base de dados de preço:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>13495</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>16500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>18920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>41315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6295</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   price\n",
       "0  13495\n",
       "1  16500\n",
       "2  18920\n",
       "3  41315\n",
       "4   5151\n",
       "5   6295"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = pd.DataFrame(data=[13495,16500,18920,41315,5151,6295], columns=[\"price\"])\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos então criar três categorias:\n",
    "\n",
    "- baixo = para valores entre 5000 e 12000\n",
    "- médio = para valores entre 30000,31000\n",
    "- alto = para valores entre 39000 e 44500"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>price-binned</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>13495</td>\n",
       "      <td>baixo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>16500</td>\n",
       "      <td>médio</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>18920</td>\n",
       "      <td>médio</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>41315</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5151</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6295</td>\n",
       "      <td>baixo</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   price price-binned\n",
       "0  13495        baixo\n",
       "1  16500        médio\n",
       "2  18920        médio\n",
       "3  41315          NaN\n",
       "4   5151          NaN\n",
       "5   6295        baixo"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "binwidth = int((max(data[\"price\"])-min(data[\"price\"]))/4)\n",
    "\n",
    "bins = range(min(data[\"price\"]), max(data[\"price\"]), binwidth)\n",
    "\n",
    "group_names = [\"baixo\", \"médio\", \"alto\"]\n",
    "\n",
    "data[\"price-binned\"] = pd.cut(data[\"price\"], bins, labels=group_names)\n",
    "\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Categórico em quantitativo\n",
    "\n",
    "Diversas vezes teremos de traduzir para nossa base os valores categorizados em quantitativos, observemos nossa base:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>car</th>\n",
       "      <th>fuel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B</td>\n",
       "      <td>diesel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>C</td>\n",
       "      <td>gas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>D</td>\n",
       "      <td>gas</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  car    fuel\n",
       "0   A     gas\n",
       "1   B  diesel\n",
       "2   C     gas\n",
       "3   D     gas"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = pd.DataFrame(data=[\"A\",\"B\",\"C\",\"D\"], columns=[\"car\"])\n",
    "data[\"fuel\"] = np.array([\"gas\",\"diesel\",\"gas\",\"gas\"])\n",
    "display(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Em python podemos utilizar o método `dummies` para extrair essa informação:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>diesel</th>\n",
       "      <th>gas</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   diesel  gas\n",
       "0       0    1\n",
       "1       1    0\n",
       "2       0    1\n",
       "3       0    1"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.get_dummies(data[\"fuel\"])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}