01-notebook/01-polars_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0fbd46c7-ab7b-4e8d-b67e-f04d7053b6bf",
   "metadata": {},
   "source": [
    "# **Polars: um simples mas prático tutorial**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f010993-e6a8-41f5-8d06-3c3239a02a86",
   "metadata": {},
   "source": [
    "## **0. Setup e considerações iniciais** <a class=\"anchor\" id=\"header0\"></a>\n",
    "\n",
    "\n",
    "- Este \"tutorial\" não têm o objetivo de te transformar em um hard-user do Polars e sim mostrar algumas das funções e manuseio de dados mais comuns que existem no Polars\n",
    "- Se você quiser estudar um pouco da teoria do Polars, existem algumas referências na [última seção](https://nbviewer.org/github/barbosarafael/polars_python_test/blob/main/01-notebook/01-polars_notebook.ipynb#header20) deste notebook\n",
    "- A API/funções do Polars é bem parecido com a do PySpark. Eu optei por fazer esse tutorial parecido com o que já trabalhei anteriormente.\n",
    "    - Em termos práticos, enquanto em algum tutorial você deva ver `pl.col()`, aqui você verá apenas `col()`\n",
    "    - Fora que eu escrevo com as identações com as \"/\"s e alguns Enters a cada comando"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c9a732ca-e201-4725-b58f-dd77f8dbe1aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "from polars import *\n",
    "from datetime import date\n",
    "import json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "881ac4b9-fe6d-4406-a589-2379f19a662c",
   "metadata": {},
   "source": [
    "## **1. Como carregar os dados <a class=\"anchor\" id=\"header1\"></a>**\n",
    "\n",
    "Nada muito longe do que já é feito no Pandas ou no próprio R. Inclusive, usa exatamente a mesma síntaxe que a do Pandas :) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62cdf56e-852e-4dfe-b38c-f241aed3f2f5",
   "metadata": {},
   "source": [
    "### **1.1. CSV**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "01acacab-1463-4655-8097-da4452a3f196",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (1309, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mr. H...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "135\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mrs. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Anderson, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "48.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;19952&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;E12&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;3&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Andrews, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "63.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;13502&quot;\n",
       "</td>\n",
       "<td>\n",
       "77.9583\n",
       "</td>\n",
       "<td>\n",
       "&quot;D7&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;10&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Hudson, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Andrews, Mr. T...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "39.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;112050&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;A36&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Belfast, NI&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Appleton, Mrs....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "53.0\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;11769&quot;\n",
       "</td>\n",
       "<td>\n",
       "51.4792\n",
       "</td>\n",
       "<td>\n",
       "&quot;C101&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;D&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bayside, Queen...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Artagaveytia, ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "71.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17609&quot;\n",
       "</td>\n",
       "<td>\n",
       "49.5042\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "22\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montevideo, Ur...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Astor, Col. Jo...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "47.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17757&quot;\n",
       "</td>\n",
       "<td>\n",
       "227.525\n",
       "</td>\n",
       "<td>\n",
       "&quot;C62 C64&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "124\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Astor, Mrs. Jo...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "18.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17757&quot;\n",
       "</td>\n",
       "<td>\n",
       "227.525\n",
       "</td>\n",
       "<td>\n",
       "&quot;C62 C64&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;4&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Wiseman, Mr. P...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;A/4. 34244&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.25\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Wittevrongel, ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "36.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;345771&quot;\n",
       "</td>\n",
       "<td>\n",
       "9.5\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Yasbeck, Mr. A...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "27.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2659&quot;\n",
       "</td>\n",
       "<td>\n",
       "14.4542\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Yasbeck, Mrs. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "15.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2659&quot;\n",
       "</td>\n",
       "<td>\n",
       "14.4542\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Youseff, Mr. G...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "45.5\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2628&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.225\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "312\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Yousif, Mr. Wa...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2647&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.225\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Yousseff, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2627&quot;\n",
       "</td>\n",
       "<td>\n",
       "14.4583\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zabour, Miss. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "14.5\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2665&quot;\n",
       "</td>\n",
       "<td>\n",
       "14.4542\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "328\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zabour, Miss. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2665&quot;\n",
       "</td>\n",
       "<td>\n",
       "14.4542\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zakarian, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.5\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2656&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.225\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "304\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zakarian, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "27.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2670&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.225\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zimmerman, Mr....\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;315082&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.875\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (1309, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Mr.      ┆ male   ┆ ... ┆ S        ┆ null ┆ 135  ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Joshua     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│        ┆          ┆ Creig...          ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ ...    ┆ ...      ┆ ...               ┆ ...    ┆ ... ┆ ...      ┆ ...  ┆ ...  ┆ ...              │\n",
       "│ 3      ┆ 0        ┆ Zabour, Miss.     ┆ female ┆ ... ┆ C        ┆ null ┆ null ┆ null             │\n",
       "│        ┆          ┆ Thamine           ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 3      ┆ 0        ┆ Zakarian, Mr.     ┆ male   ┆ ... ┆ C        ┆ null ┆ 304  ┆ null             │\n",
       "│        ┆          ┆ Mapriededer       ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 3      ┆ 0        ┆ Zakarian, Mr.     ┆ male   ┆ ... ┆ C        ┆ null ┆ null ┆ null             │\n",
       "│        ┆          ┆ Ortin             ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 3      ┆ 0        ┆ Zimmerman, Mr.    ┆ male   ┆ ... ┆ S        ┆ null ┆ null ┆ null             │\n",
       "│        ┆          ┆ Leo               ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados = read_csv(file = '../02-data/titanic.csv')\n",
    "\n",
    "dados"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a802781-e180-4979-b183-3138de34e8e9",
   "metadata": {},
   "source": [
    "### **1.2. Excel (xlsx)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a3bef784-eecf-4679-8b68-7c8f4ea5be70",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados = read_excel(file = '../02-data/titanic.xlsx')\n",
    "\n",
    "dados.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92a98f5e-b0c3-45d7-81b3-f64449b50741",
   "metadata": {},
   "source": [
    "## **2. Informações iniciais do dataframe <a class=\"anchor\" id=\"header2\"></a>**\n",
    "\n",
    "Esse tópico também não foge muito se você já trabalhou com o Pandas. Apenas a inclusão de um novo método do dataframe que é bem parecido com o que têm no Pyspark: o `.limit()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "dc6b69a0-c8a3-4396-a909-9d5e378c3d9c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "73fc7e03-a41d-46ba-84c2-dacce54169e1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados.limit(3) #---- alô usuários do PySpark"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ef40bbc6-be05-4fa5-9a71-71d9d606fe58",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zakarian, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.5\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2656&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.225\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "304\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zakarian, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "27.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2670&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.225\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zimmerman, Mr....\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;315082&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.875\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────────────┬──────┬─────┬──────────┬──────┬──────┬───────────┐\n",
       "│ pclass ┆ survived ┆ name                      ┆ sex  ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest │\n",
       "│ ---    ┆ ---      ┆ ---                       ┆ ---  ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---       │\n",
       "│ i64    ┆ i64      ┆ str                       ┆ str  ┆     ┆ str      ┆ str  ┆ i64  ┆ str       │\n",
       "╞════════╪══════════╪═══════════════════════════╪══════╪═════╪══════════╪══════╪══════╪═══════════╡\n",
       "│ 3      ┆ 0        ┆ Zakarian, Mr. Mapriededer ┆ male ┆ ... ┆ C        ┆ null ┆ 304  ┆ null      │\n",
       "│ 3      ┆ 0        ┆ Zakarian, Mr. Ortin       ┆ male ┆ ... ┆ C        ┆ null ┆ null ┆ null      │\n",
       "│ 3      ┆ 0        ┆ Zimmerman, Mr. Leo        ┆ male ┆ ... ┆ S        ┆ null ┆ null ┆ null      │\n",
       "└────────┴──────────┴───────────────────────────┴──────┴─────┴──────────┴──────┴──────┴───────────┘"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados.tail(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "34d9fc26-96f5-479b-ad27-898b91ce6473",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1309, 14)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "20788097-6efd-4a3e-80b4-1382b11783c9",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (7, 15)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "describe\n",
       "</th>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;count&quot;\n",
       "</td>\n",
       "<td>\n",
       "1309.0\n",
       "</td>\n",
       "<td>\n",
       "1309.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;1309&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;1309&quot;\n",
       "</td>\n",
       "<td>\n",
       "1309.0\n",
       "</td>\n",
       "<td>\n",
       "1309.0\n",
       "</td>\n",
       "<td>\n",
       "1309.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;1309&quot;\n",
       "</td>\n",
       "<td>\n",
       "1309.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;1309&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;1309&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;1309&quot;\n",
       "</td>\n",
       "<td>\n",
       "1309.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;1309&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;null_count&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;0&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;0&quot;\n",
       "</td>\n",
       "<td>\n",
       "263.0\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;0&quot;\n",
       "</td>\n",
       "<td>\n",
       "1.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;1014&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;823&quot;\n",
       "</td>\n",
       "<td>\n",
       "1188.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;564&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;mean&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.294882\n",
       "</td>\n",
       "<td>\n",
       "0.381971\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "29.881138\n",
       "</td>\n",
       "<td>\n",
       "0.498854\n",
       "</td>\n",
       "<td>\n",
       "0.385027\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "33.295479\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "160.809917\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;std&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.837836\n",
       "</td>\n",
       "<td>\n",
       "0.486055\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "14.413493\n",
       "</td>\n",
       "<td>\n",
       "1.041658\n",
       "</td>\n",
       "<td>\n",
       "0.86556\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "51.758668\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "97.696922\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;min&quot;\n",
       "</td>\n",
       "<td>\n",
       "1.0\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Abbing, Mr. An...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.17\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;110152&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;A10&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;1&quot;\n",
       "</td>\n",
       "<td>\n",
       "1.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;?Havana, Cuba&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;max&quot;\n",
       "</td>\n",
       "<td>\n",
       "3.0\n",
       "</td>\n",
       "<td>\n",
       "1.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;van Melkebeke,...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "80.0\n",
       "</td>\n",
       "<td>\n",
       "8.0\n",
       "</td>\n",
       "<td>\n",
       "9.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;WE/P 5735&quot;\n",
       "</td>\n",
       "<td>\n",
       "512.3292\n",
       "</td>\n",
       "<td>\n",
       "&quot;T&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;D&quot;\n",
       "</td>\n",
       "<td>\n",
       "328.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zurich, Switze...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;median&quot;\n",
       "</td>\n",
       "<td>\n",
       "3.0\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "28.0\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "14.4542\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "155.0\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (7, 15)\n",
       "┌────────────┬──────────┬──────────┬─────────────┬─────┬──────────┬──────┬────────────┬────────────┐\n",
       "│ describe   ┆ pclass   ┆ survived ┆ name        ┆ ... ┆ embarked ┆ boat ┆ body       ┆ home.dest  │\n",
       "│ ---        ┆ ---      ┆ ---      ┆ ---         ┆     ┆ ---      ┆ ---  ┆ ---        ┆ ---        │\n",
       "│ str        ┆ f64      ┆ f64      ┆ str         ┆     ┆ str      ┆ str  ┆ f64        ┆ str        │\n",
       "╞════════════╪══════════╪══════════╪═════════════╪═════╪══════════╪══════╪════════════╪════════════╡\n",
       "│ count      ┆ 1309.0   ┆ 1309.0   ┆ 1309        ┆ ... ┆ 1309     ┆ 1309 ┆ 1309.0     ┆ 1309       │\n",
       "│ null_count ┆ 0.0      ┆ 0.0      ┆ 0           ┆ ... ┆ 2        ┆ 823  ┆ 1188.0     ┆ 564        │\n",
       "│ mean       ┆ 2.294882 ┆ 0.381971 ┆ null        ┆ ... ┆ null     ┆ null ┆ 160.809917 ┆ null       │\n",
       "│ std        ┆ 0.837836 ┆ 0.486055 ┆ null        ┆ ... ┆ null     ┆ null ┆ 97.696922  ┆ null       │\n",
       "│ min        ┆ 1.0      ┆ 0.0      ┆ Abbing, Mr. ┆ ... ┆ C        ┆ 1    ┆ 1.0        ┆ ?Havana,   │\n",
       "│            ┆          ┆          ┆ Anthony     ┆     ┆          ┆      ┆            ┆ Cuba       │\n",
       "│ max        ┆ 3.0      ┆ 1.0      ┆ van         ┆ ... ┆ S        ┆ D    ┆ 328.0      ┆ Zurich,    │\n",
       "│            ┆          ┆          ┆ Melkebeke,  ┆     ┆          ┆      ┆            ┆ Switzerlan │\n",
       "│            ┆          ┆          ┆ Mr.         ┆     ┆          ┆      ┆            ┆ d          │\n",
       "│            ┆          ┆          ┆ Philemon    ┆     ┆          ┆      ┆            ┆            │\n",
       "│ median     ┆ 3.0      ┆ 0.0      ┆ null        ┆ ... ┆ null     ┆ null ┆ 155.0      ┆ null       │\n",
       "└────────────┴──────────┴──────────┴─────────────┴─────┴──────────┴──────┴────────────┴────────────┘"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d121fac8-8be3-4de8-b3d2-d7531f6b6753",
   "metadata": {},
   "source": [
    "## **3. Selecionando colunas específicas <a class=\"anchor\" id=\"header3\"></a>**\n",
    "\n",
    "Aqui temos algumas particularidades do **Polars**, que também são bem parecidas com o que já existe no Pandas e no PySpark."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b8cc776-1b74-4c82-8132-abeec4308179",
   "metadata": {},
   "source": [
    "### **3.1. Por nome**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "25c169a9-2718-4003-9af8-15b5bc2edd74",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 1)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 1)\n",
       "┌──────┐\n",
       "│ age  │\n",
       "│ ---  │\n",
       "│ f64  │\n",
       "╞══════╡\n",
       "│ 29.0 │\n",
       "│ 0.92 │\n",
       "│ 2.0  │\n",
       "└──────┘"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .select(col('age'))\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "081374e7-9f72-4cfa-869c-ba1a03578eae",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 3)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 3)\n",
       "┌────────┬──────────┬────────────────────────────────┐\n",
       "│ pclass ┆ survived ┆ name                           │\n",
       "│ ---    ┆ ---      ┆ ---                            │\n",
       "│ i64    ┆ i64      ┆ str                            │\n",
       "╞════════╪══════════╪════════════════════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss. Elisabeth Walton  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master. Hudson Trevor │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss. Helen Loraine   │\n",
       "└────────┴──────────┴────────────────────────────────┘"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .select(col(['pclass', 'survived', 'name']))\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ef7c8b5-870d-4f00-b8cf-1273bc94c1d5",
   "metadata": {},
   "source": [
    "### **3.2. Por index/posição**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ada0a5f4-dfff-4871-bd92-0b97b1cfd908",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3,)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3,)\n",
       "Series: 'pclass' [i64]\n",
       "[\n",
       "\t1\n",
       "\t1\n",
       "\t1\n",
       "]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados[:, 0]\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3421fe3a-5f9e-41ed-b25d-7089bab9f6aa",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 3)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 3)\n",
       "┌────────┬──────────┬────────────────────────────────┐\n",
       "│ pclass ┆ survived ┆ name                           │\n",
       "│ ---    ┆ ---      ┆ ---                            │\n",
       "│ i64    ┆ i64      ┆ str                            │\n",
       "╞════════╪══════════╪════════════════════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss. Elisabeth Walton  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master. Hudson Trevor │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss. Helen Loraine   │\n",
       "└────────┴──────────┴────────────────────────────────┘"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados[:, 0:3]\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87c06a9b-349e-4d34-9221-b42e73403104",
   "metadata": {},
   "source": [
    "### **3.3. Bônus (1): Selecionando todas as colunas**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c862dd44-1441-4057-a0b6-63375c60eaef",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .select(\\\n",
    "           col('*')\n",
    "           )\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11de14b4-9306-436c-a44a-8b624abee4f6",
   "metadata": {},
   "source": [
    "### **3.4. Bônus (2): Excluindo colunas**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "31bdad63-55e0-447b-9a59-b7d0a1bd8b17",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .select(\\\n",
    "           exclude('PassengerId')\n",
    "           )\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "1fa7a406-a757-4e72-b4bb-f84b66cdc0c3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .select(\\\n",
    "           exclude(['PassengerId', 'Survived'])\n",
    "           )\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f7a6e41-7d13-4d87-84bf-a87ff1d26f06",
   "metadata": {},
   "source": [
    "## **4. Operações entre colunas <a class=\"anchor\" id=\"header4\"></a>**\n",
    "\n",
    "\n",
    "Existem duas funções específicas para este fim: a `with_column` e a `with_columns`. \n",
    "\n",
    "Qual a melhor usar? Vai na `with_columns`, pois ela permite criar várias colunas de uma vez só, bem parecido com o que `dplyr::mutate()` faz no R.\n",
    "\n",
    "- **Notem que para adicionar mais de uma coluna, precisa estar dentro de uma lista.**\n",
    "- **No método 2:**\n",
    "    - Tem uma mudança do tipo da variável explicitamente na mesma linha da criação da coluna\n",
    "    - Preenchimento de valores nulos (NAs) diretamente também"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e909731-e875-4a8d-a219-b513fcc2ca2d",
   "metadata": {},
   "source": [
    "### **4.1. Criando apenas 1 coluna**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "37c36cca-a982-436a-b592-05bcd0ed4c4c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 15)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "1.84\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "4.0\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 15)\n",
       "┌────────┬──────────┬─────────────────┬────────┬─────┬──────┬──────┬─────────────────┬─────────────┐\n",
       "│ pclass ┆ survived ┆ name            ┆ sex    ┆ ... ┆ boat ┆ body ┆ home.dest       ┆ nova_coluna │\n",
       "│ ---    ┆ ---      ┆ ---             ┆ ---    ┆     ┆ ---  ┆ ---  ┆ ---             ┆ ---         │\n",
       "│ i64    ┆ i64      ┆ str             ┆ str    ┆     ┆ str  ┆ i64  ┆ str             ┆ f64         │\n",
       "╞════════╪══════════╪═════════════════╪════════╪═════╪══════╪══════╪═════════════════╪═════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.    ┆ female ┆ ... ┆ 2    ┆ null ┆ St Louis, MO    ┆ 0.0         │\n",
       "│        ┆          ┆ Elisabeth       ┆        ┆     ┆      ┆      ┆                 ┆             │\n",
       "│        ┆          ┆ Walton          ┆        ┆     ┆      ┆      ┆                 ┆             │\n",
       "│ 1      ┆ 1        ┆ Allison,        ┆ male   ┆ ... ┆ 11   ┆ null ┆ Montreal, PQ /  ┆ 1.84        │\n",
       "│        ┆          ┆ Master. Hudson  ┆        ┆     ┆      ┆      ┆ Chesterville,   ┆             │\n",
       "│        ┆          ┆ Trevor          ┆        ┆     ┆      ┆      ┆ ON              ┆             │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.  ┆ female ┆ ... ┆ null ┆ null ┆ Montreal, PQ /  ┆ 4.0         │\n",
       "│        ┆          ┆ Helen Loraine   ┆        ┆     ┆      ┆      ┆ Chesterville,   ┆             │\n",
       "│        ┆          ┆                 ┆        ┆     ┆      ┆      ┆ ON              ┆             │\n",
       "└────────┴──────────┴─────────────────┴────────┴─────┴──────┴──────┴─────────────────┴─────────────┘"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .with_columns(\\\n",
    "                  (col('age') * col('parch')).alias('nova_coluna'),\n",
    "                 )\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f895655f-f7ff-4f12-9224-7571c05ad01f",
   "metadata": {},
   "source": [
    "### **4.2. Criando várias colunas (bem parecido com o dplyr::mutate)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "fb07012e-12e6-4c2b-b4b2-1513d3bb1a57",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 17)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna1\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna2\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "6128.7875\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "1.84\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "139.426\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "4.0\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "303.1\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 17)\n",
       "┌────────┬──────────┬────────────┬────────┬─────┬────────────┬───────────┬────────────┬────────────┐\n",
       "│ pclass ┆ survived ┆ name       ┆ sex    ┆ ... ┆ home.dest  ┆ nova_colu ┆ nova_colun ┆ nova_colun │\n",
       "│ ---    ┆ ---      ┆ ---        ┆ ---    ┆     ┆ ---        ┆ na        ┆ a1         ┆ a2         │\n",
       "│ i64    ┆ i64      ┆ str        ┆ str    ┆     ┆ str        ┆ ---       ┆ ---        ┆ ---        │\n",
       "│        ┆          ┆            ┆        ┆     ┆            ┆ f64       ┆ f64        ┆ f64        │\n",
       "╞════════╪══════════╪════════════╪════════╪═════╪════════════╪═══════════╪════════════╪════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen,     ┆ female ┆ ... ┆ St Louis,  ┆ 0.0       ┆ 0.0        ┆ 6128.7875  │\n",
       "│        ┆          ┆ Miss.      ┆        ┆     ┆ MO         ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Elisabeth  ┆        ┆     ┆            ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Walton     ┆        ┆     ┆            ┆           ┆            ┆            │\n",
       "│ 1      ┆ 1        ┆ Allison,   ┆ male   ┆ ... ┆ Montreal,  ┆ 1.84      ┆ 0.92       ┆ 139.426    │\n",
       "│        ┆          ┆ Master.    ┆        ┆     ┆ PQ / Chest ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Hudson     ┆        ┆     ┆ erville,   ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Trevor     ┆        ┆     ┆ ON         ┆           ┆            ┆            │\n",
       "│ 1      ┆ 0        ┆ Allison,   ┆ female ┆ ... ┆ Montreal,  ┆ 4.0       ┆ 2.0        ┆ 303.1      │\n",
       "│        ┆          ┆ Miss.      ┆        ┆     ┆ PQ / Chest ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Helen      ┆        ┆     ┆ erville,   ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Loraine    ┆        ┆     ┆ ON         ┆           ┆            ┆            │\n",
       "└────────┴──────────┴────────────┴────────┴─────┴────────────┴───────────┴────────────┴────────────┘"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Método 1:\n",
    "\n",
    "dados\\\n",
    "    .with_columns(\\\n",
    "                 nova_coluna = col('age') * col('parch'),\n",
    "                 nova_coluna1 = col('age') * col('sibsp'),\n",
    "                 nova_coluna2 = col('age') * col('fare')\n",
    "                 )\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "af7e93e6-a2d8-4b8d-b947-28c9f2001741",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 17)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna1\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna2\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "f32\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "6128.787598\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "1.84\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "139.425995\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "4.0\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "303.100006\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 17)\n",
       "┌────────┬──────────┬────────────┬────────┬─────┬────────────┬───────────┬────────────┬────────────┐\n",
       "│ pclass ┆ survived ┆ name       ┆ sex    ┆ ... ┆ home.dest  ┆ nova_colu ┆ nova_colun ┆ nova_colun │\n",
       "│ ---    ┆ ---      ┆ ---        ┆ ---    ┆     ┆ ---        ┆ na        ┆ a1         ┆ a2         │\n",
       "│ i64    ┆ i64      ┆ str        ┆ str    ┆     ┆ str        ┆ ---       ┆ ---        ┆ ---        │\n",
       "│        ┆          ┆            ┆        ┆     ┆            ┆ f64       ┆ i64        ┆ f32        │\n",
       "╞════════╪══════════╪════════════╪════════╪═════╪════════════╪═══════════╪════════════╪════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen,     ┆ female ┆ ... ┆ St Louis,  ┆ 0.0       ┆ 0          ┆ 6128.78759 │\n",
       "│        ┆          ┆ Miss.      ┆        ┆     ┆ MO         ┆           ┆            ┆ 8          │\n",
       "│        ┆          ┆ Elisabeth  ┆        ┆     ┆            ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Walton     ┆        ┆     ┆            ┆           ┆            ┆            │\n",
       "│ 1      ┆ 1        ┆ Allison,   ┆ male   ┆ ... ┆ Montreal,  ┆ 1.84      ┆ 0          ┆ 139.425995 │\n",
       "│        ┆          ┆ Master.    ┆        ┆     ┆ PQ / Chest ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Hudson     ┆        ┆     ┆ erville,   ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Trevor     ┆        ┆     ┆ ON         ┆           ┆            ┆            │\n",
       "│ 1      ┆ 0        ┆ Allison,   ┆ female ┆ ... ┆ Montreal,  ┆ 4.0       ┆ 2          ┆ 303.100006 │\n",
       "│        ┆          ┆ Miss.      ┆        ┆     ┆ PQ / Chest ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Helen      ┆        ┆     ┆ erville,   ┆           ┆            ┆            │\n",
       "│        ┆          ┆ Loraine    ┆        ┆     ┆ ON         ┆           ┆            ┆            │\n",
       "└────────┴──────────┴────────────┴────────┴─────┴────────────┴───────────┴────────────┴────────────┘"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Método 2:\n",
    "\n",
    "dados\\\n",
    "    .with_columns([\\\n",
    "                  (col('age') * col('parch')).alias('nova_coluna'),\n",
    "                  (col('age') * col('sibsp')).cast(Int64).alias('nova_coluna1'),\n",
    "                  (col('age') * col('fare')).cast(Float32).fill_nan(0).alias('nova_coluna2')\n",
    "                  ])\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ab4b780-52b5-4640-b4a6-7902717210c3",
   "metadata": {},
   "source": [
    "### **4.3. Bônus:**\n",
    "\n",
    "- `nova_coluna_constante`: Coluna com um valor único\n",
    "- `nova_coluna_soma`: Coluna com um valor único, que é a soma da variável Age\n",
    "- `nova_coluna_42`: Coluna com um valor somado (42) constante para a variável Age\n",
    "- `nova_coluna_case_when`: Coluna com uma ideia de case when\n",
    "- `nova_coluna_lista`: Coluna que concatena todas as variáveis em uma lista"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "60af2ae7-c6b9-4418-80d0-136e878452b3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 6)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna_constante\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna_soma\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna_42\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna_case_when\n",
       "</th>\n",
       "<th>\n",
       "nova_coluna_lista\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i32\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "list[str]\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "31255.67\n",
       "</td>\n",
       "<td>\n",
       "71.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;novo&quot;\n",
       "</td>\n",
       "<td>\n",
       "[&quot;29.0&quot;, &quot;1&quot;, ... &quot;novo&quot;]\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "31255.67\n",
       "</td>\n",
       "<td>\n",
       "42.92\n",
       "</td>\n",
       "<td>\n",
       "&quot;idoso&quot;\n",
       "</td>\n",
       "<td>\n",
       "[&quot;0.92&quot;, &quot;1&quot;, ... &quot;idoso&quot;]\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "31255.67\n",
       "</td>\n",
       "<td>\n",
       "44.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;idoso&quot;\n",
       "</td>\n",
       "<td>\n",
       "[&quot;2.0&quot;, &quot;1&quot;, ... &quot;idoso&quot;]\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 6)\n",
       "┌──────┬──────────────────┬──────────────────┬────────────────┬──────────────────┬─────────────────┐\n",
       "│ age  ┆ nova_coluna_cons ┆ nova_coluna_soma ┆ nova_coluna_42 ┆ nova_coluna_case ┆ nova_coluna_lis │\n",
       "│ ---  ┆ tante            ┆ ---              ┆ ---            ┆ _when            ┆ ta              │\n",
       "│ f64  ┆ ---              ┆ f64              ┆ f64            ┆ ---              ┆ ---             │\n",
       "│      ┆ i32              ┆                  ┆                ┆ str              ┆ list[str]       │\n",
       "╞══════╪══════════════════╪══════════════════╪════════════════╪══════════════════╪═════════════════╡\n",
       "│ 29.0 ┆ 1                ┆ 31255.67         ┆ 71.0           ┆ novo             ┆ [\"29.0\", \"1\",   │\n",
       "│      ┆                  ┆                  ┆                ┆                  ┆ ... \"novo\"]     │\n",
       "│ 0.92 ┆ 1                ┆ 31255.67         ┆ 42.92          ┆ idoso            ┆ [\"0.92\", \"1\",   │\n",
       "│      ┆                  ┆                  ┆                ┆                  ┆ ... \"idoso\"]    │\n",
       "│ 2.0  ┆ 1                ┆ 31255.67         ┆ 44.0           ┆ idoso            ┆ [\"2.0\", \"1\",    │\n",
       "│      ┆                  ┆                  ┆                ┆                  ┆ ... \"idoso\"]    │\n",
       "└──────┴──────────────────┴──────────────────┴────────────────┴──────────────────┴─────────────────┘"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .select(col('age'))\\\n",
    "    .with_columns(\\\n",
    "                 nova_coluna_constante = lit(1),\n",
    "                 nova_coluna_soma = col('age').sum(),\n",
    "                 nova_coluna_42 = col('age') + 42,\n",
    "                 nova_coluna_case_when = when(col('age') > 10).then(lit('novo')).otherwise(lit('idoso')) # ^.^\n",
    "                 )\\\n",
    "    .with_columns(\\\n",
    "                  nova_coluna_lista = concat_list(all()) # Problema aqui: tive que adicionar um novo with_columns, pois ele estava criando a lista com um único elemento, que era da coluna PassengerId quando adicionava no primeiro\n",
    "                 )\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ae810f1-da54-4ba1-ba3a-64aeacef6eb2",
   "metadata": {},
   "source": [
    "## **5. Renomeando colunas e printando as colunas <a class=\"anchor\" id=\"header5\"></a>**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40b4e480-bf0e-4532-bc3b-548cfa858b6d",
   "metadata": {},
   "source": [
    "Segue a mesma ideia do rename do Pandas. Irei trocar age -> Age."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "c48d8ce8-2341-4e28-b8a6-cd66286a1de5",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['pclass',\n",
       " 'survived',\n",
       " 'name',\n",
       " 'sex',\n",
       " 'Age',\n",
       " 'sibsp',\n",
       " 'parch',\n",
       " 'ticket',\n",
       " 'fare',\n",
       " 'cabin',\n",
       " 'embarked',\n",
       " 'boat',\n",
       " 'body',\n",
       " 'home.dest']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .rename(mapping = {'age': 'Age'})\\\n",
    "    .columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c6c82f2-8c45-4b4d-b711-3ff655377899",
   "metadata": {},
   "source": [
    "## **6. Filtros**\n",
    "\n",
    "Obviamente, alguns filtros podem ser aplicados tanto à variáveis quantitativas quanto qualitativas!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "feb21add-dbe5-4f0c-a462-cc0ce0caab79",
   "metadata": {},
   "source": [
    "### **6.1. Variáveis quantitativas <a class=\"anchor\" id=\"header6\"></a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "7af2438d-3a46-4349-851b-908d5a48a9e3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mr. H...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "135\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mrs. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 0        ┆ Allison, Mr.      ┆ male   ┆ ... ┆ S        ┆ null ┆ 135  ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Joshua     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│        ┆          ┆ Creig...          ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 0        ┆ Allison, Mrs.     ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson J C        ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│        ┆          ┆ (Bessie...        ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .filter(col('age') > 10)\\\n",
    "    .head(3)\n",
    "\n",
    "# age maior que 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "270fea11-386a-4d7a-8855-954a1cb50d72",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Dodge, Master....\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "4.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;33638&quot;\n",
       "</td>\n",
       "<td>\n",
       "81.8583\n",
       "</td>\n",
       "<td>\n",
       "&quot;A34&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;5&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;San Francisco,...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 1        ┆ Dodge, Master.    ┆ male   ┆ ... ┆ S        ┆ 5    ┆ null ┆ San Francisco,   │\n",
       "│        ┆          ┆ Washington        ┆        ┆     ┆          ┆      ┆      ┆ CA               │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .filter(col('age') <= 10)\\\n",
    "    .head(3)\n",
    "\n",
    "# age menor ou igual a 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "a9566604-c2b9-449e-87b4-ff44dabc35ba",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Astor, Mrs. Jo...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "18.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17757&quot;\n",
       "</td>\n",
       "<td>\n",
       "227.525\n",
       "</td>\n",
       "<td>\n",
       "&quot;C62 C64&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;4&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bishop, Mrs. D...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "19.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;11967&quot;\n",
       "</td>\n",
       "<td>\n",
       "91.0792\n",
       "</td>\n",
       "<td>\n",
       "&quot;B49&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;7&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Dowagiac, MI&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Carrau, Mr. Jo...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "17.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;113059&quot;\n",
       "</td>\n",
       "<td>\n",
       "47.1\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montevideo, Ur...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬────────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────┐\n",
       "│ pclass ┆ survived ┆ name               ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest    │\n",
       "│ ---    ┆ ---      ┆ ---                ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---          │\n",
       "│ i64    ┆ i64      ┆ str                ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str          │\n",
       "╞════════╪══════════╪════════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════╡\n",
       "│ 1      ┆ 1        ┆ Astor, Mrs. John   ┆ female ┆ ... ┆ C        ┆ 4    ┆ null ┆ New York, NY │\n",
       "│        ┆          ┆ Jacob (Madelein... ┆        ┆     ┆          ┆      ┆      ┆              │\n",
       "│ 1      ┆ 1        ┆ Bishop, Mrs.       ┆ female ┆ ... ┆ C        ┆ 7    ┆ null ┆ Dowagiac, MI │\n",
       "│        ┆          ┆ Dickinson H (Helen ┆        ┆     ┆          ┆      ┆      ┆              │\n",
       "│        ┆          ┆ ...                ┆        ┆     ┆          ┆      ┆      ┆              │\n",
       "│ 1      ┆ 0        ┆ Carrau, Mr. Jose   ┆ male   ┆ ... ┆ S        ┆ null ┆ null ┆ Montevideo,  │\n",
       "│        ┆          ┆ Pedro              ┆        ┆     ┆          ┆      ┆      ┆ Uruguay      │\n",
       "└────────┴──────────┴────────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────┘"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .filter(col('age').is_between(10, 20))\\\n",
    "    .head(3)\n",
    "\n",
    "# age entre 10 e 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "f80a42c7-2da0-4466-b717-c998c48ee7ea",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Anderson, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "48.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;19952&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;E12&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;3&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Andrews, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "63.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;13502&quot;\n",
       "</td>\n",
       "<td>\n",
       "77.9583\n",
       "</td>\n",
       "<td>\n",
       "&quot;D7&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;10&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Hudson, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────┐\n",
       "│ pclass ┆ survived ┆ name                  ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest    │\n",
       "│ ---    ┆ ---      ┆ ---                   ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---          │\n",
       "│ i64    ┆ i64      ┆ str                   ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str          │\n",
       "╞════════╪══════════╪═══════════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.          ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO │\n",
       "│        ┆          ┆ Elisabeth Walton      ┆        ┆     ┆          ┆      ┆      ┆              │\n",
       "│ 1      ┆ 1        ┆ Anderson, Mr. Harry   ┆ male   ┆ ... ┆ S        ┆ 3    ┆ null ┆ New York, NY │\n",
       "│ 1      ┆ 1        ┆ Andrews, Miss.        ┆ female ┆ ... ┆ S        ┆ 10   ┆ null ┆ Hudson, NY   │\n",
       "│        ┆          ┆ Kornelia Theodosi...  ┆        ┆     ┆          ┆      ┆      ┆              │\n",
       "└────────┴──────────┴───────────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────┘"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .filter((col('age') > 10) & (col('survived') == 1))\\\n",
    "    .head(3)\n",
    "\n",
    "# ge maior que 10 E survided igual a 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "fac4f19c-0e8c-42d1-9eee-7cb4d7a9b805",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mr. H...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "135\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Mr.      ┆ male   ┆ ... ┆ S        ┆ null ┆ 135  ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Joshua     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│        ┆          ┆ Creig...          ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .filter((col('age') > 10) | (col('survived') == 1))\\\n",
    "    .head(3)\n",
    "\n",
    "# age maior que 10 OU survided igual a 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d2dec7b-0883-4f08-9176-dfdcf20d0e6f",
   "metadata": {},
   "source": [
    "### **6.2. Variáveis qualitativas**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "5410a64e-59fb-4871-991f-38acbbe89413",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .filter(col('embarked') == 'S')\\\n",
    "    .head(3)\n",
    "\n",
    "# age maior que 10 OU survided igual a 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "2fbd3f57-81e8-4602-9736-8d4da1d2179d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Artagaveytia, ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "71.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17609&quot;\n",
       "</td>\n",
       "<td>\n",
       "49.5042\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "22\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montevideo, Ur...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Astor, Col. Jo...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "47.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17757&quot;\n",
       "</td>\n",
       "<td>\n",
       "227.525\n",
       "</td>\n",
       "<td>\n",
       "&quot;C62 C64&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "124\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Astor, Mrs. Jo...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "18.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17757&quot;\n",
       "</td>\n",
       "<td>\n",
       "227.525\n",
       "</td>\n",
       "<td>\n",
       "&quot;C62 C64&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;4&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬────────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────┐\n",
       "│ pclass ┆ survived ┆ name               ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest    │\n",
       "│ ---    ┆ ---      ┆ ---                ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---          │\n",
       "│ i64    ┆ i64      ┆ str                ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str          │\n",
       "╞════════╪══════════╪════════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════╡\n",
       "│ 1      ┆ 0        ┆ Artagaveytia, Mr.  ┆ male   ┆ ... ┆ C        ┆ null ┆ 22   ┆ Montevideo,  │\n",
       "│        ┆          ┆ Ramon              ┆        ┆     ┆          ┆      ┆      ┆ Uruguay      │\n",
       "│ 1      ┆ 0        ┆ Astor, Col. John   ┆ male   ┆ ... ┆ C        ┆ null ┆ 124  ┆ New York, NY │\n",
       "│        ┆          ┆ Jacob              ┆        ┆     ┆          ┆      ┆      ┆              │\n",
       "│ 1      ┆ 1        ┆ Astor, Mrs. John   ┆ female ┆ ... ┆ C        ┆ 4    ┆ null ┆ New York, NY │\n",
       "│        ┆          ┆ Jacob (Madelein... ┆        ┆     ┆          ┆      ┆      ┆              │\n",
       "└────────┴──────────┴────────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────┘"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .filter(col('embarked') != 'S')\\\n",
    "    .head(3)\n",
    "\n",
    "# Embarked diferente de 'S'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "4a2ae646-7ab7-475a-b4c5-f613fdfe49d1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .filter(col('embarked').is_in(['C', 'S']))\\\n",
    "    .head(3)\n",
    "\n",
    "# Embarked está na lista C ou S"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "4f922eda-b40a-4995-be69-1914123d7d1f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mr. H...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "135\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mrs. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Anderson, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "48.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;19952&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;E12&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;3&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name          ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---           ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str           ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 0        ┆ Allison, Mr.  ┆ male   ┆ ... ┆ S        ┆ null ┆ 135  ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Joshua ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│        ┆          ┆ Creig...      ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 0        ┆ Allison, Mrs. ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson J C    ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│        ┆          ┆ (Bessie...    ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Anderson, Mr. ┆ male   ┆ ... ┆ S        ┆ 3    ┆ null ┆ New York, NY     │\n",
       "│        ┆          ┆ Harry         ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴───────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Contém uma palavra\n",
    "\n",
    "dados\\\n",
    "    .filter(col('name').str.contains('Mr'))\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "01cbcc77-2713-4241-90a8-36e39e4be9a3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (4, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mr. H...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "135\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mrs. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (4, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Mr.      ┆ male   ┆ ... ┆ S        ┆ null ┆ 135  ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Joshua     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│        ┆          ┆ Creig...          ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 0        ┆ Allison, Mrs.     ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson J C        ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│        ┆          ┆ (Bessie...        ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Começa com uma palavra\n",
    "\n",
    "dados\\\n",
    "    .filter(col('name').str.starts_with('Allison'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "7e5b8321-36aa-4100-b5ce-8c162639f0b7",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (1, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (1, 14)\n",
       "┌────────┬──────────┬──────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────┐\n",
       "│ pclass ┆ survived ┆ name             ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest    │\n",
       "│ ---    ┆ ---      ┆ ---              ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---          │\n",
       "│ i64    ┆ i64      ┆ str              ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str          │\n",
       "╞════════╪══════════╪══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.     ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO │\n",
       "│        ┆          ┆ Elisabeth Walton ┆        ┆     ┆          ┆      ┆      ┆              │\n",
       "└────────┴──────────┴──────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────┘"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Finaliza com uma palavra\n",
    "\n",
    "dados\\\n",
    "    .filter(col('name').str.ends_with('Walton'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "468b5b25-b8e0-465f-9794-bc7bcba456ce",
   "metadata": {},
   "source": [
    "### **6.3. Variáveis de data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "dac1b050-4cfb-43b2-934d-ca54ee3b88cc",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (5, 1)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "drange\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "date\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-01\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-02\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-03\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-04\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-05\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (5, 1)\n",
       "┌────────────┐\n",
       "│ drange     │\n",
       "│ ---        │\n",
       "│ date       │\n",
       "╞════════════╡\n",
       "│ 2022-01-01 │\n",
       "│ 2022-01-02 │\n",
       "│ 2022-01-03 │\n",
       "│ 2022-01-04 │\n",
       "│ 2022-01-05 │\n",
       "└────────────┘"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates = DataFrame(date_range(date(2022, 1, 1), date(2022, 3, 1), '1d', name = 'drange'))\n",
    "\n",
    "dates.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "f26d5883-185a-4627-a066-3fca868e7c9b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (2, 1)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "drange\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "date\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-01\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-02\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (2, 1)\n",
       "┌────────────┐\n",
       "│ drange     │\n",
       "│ ---        │\n",
       "│ date       │\n",
       "╞════════════╡\n",
       "│ 2022-01-01 │\n",
       "│ 2022-01-02 │\n",
       "└────────────┘"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates\\\n",
    "    .filter(col('drange') <= datetime(2022, 1, 2))\n",
    "\n",
    "# drange menor ou igual à 02/01/2022"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "ff766bfd-3bb5-4bbf-be71-36465d2090ec",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 1)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "drange\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "date\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-01\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-02\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-03\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 1)\n",
       "┌────────────┐\n",
       "│ drange     │\n",
       "│ ---        │\n",
       "│ date       │\n",
       "╞════════════╡\n",
       "│ 2022-01-01 │\n",
       "│ 2022-01-02 │\n",
       "│ 2022-01-03 │\n",
       "└────────────┘"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates\\\n",
    "    .filter(col('drange').is_between(datetime(2022, 1, 1), datetime(2022, 1, 3)))\n",
    "\n",
    "# drange entre 01/01/2022 e 03/01/2022"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fbd60ce-4f05-405d-bd01-eb18563dfbba",
   "metadata": {},
   "source": [
    "## **7. Funções de agregação, sem agupamento <a class=\"anchor\" id=\"header7\"></a>**\n",
    "\n",
    "Aqui temos algumas particularidades do **Polars**. Pelo **Polars** ser relativamente novo, em comparação com um Pandas da vida, por exemplo, algumas das funções não são criadas tão facilmente (e.g. `value_counts()` ou `dplyr::count()`). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "22067b74-d871-4f9e-9e19-6bc40d18e7a9",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (2, 2)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "counts\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "809\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "500\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (2, 2)\n",
       "┌──────────┬────────┐\n",
       "│ survived ┆ counts │\n",
       "│ ---      ┆ ---    │\n",
       "│ i64      ┆ u32    │\n",
       "╞══════════╪════════╡\n",
       "│ 0        ┆ 809    │\n",
       "│ 1        ┆ 500    │\n",
       "└──────────┴────────┘"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- value counts da variável survived\n",
    "\n",
    "dados\\\n",
    "    .select([\\\n",
    "             col('survived').value_counts()\n",
    "            ])\\\n",
    "    .to_series().struct.unnest()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "e72accbf-dcf6-417c-baa3-ae0001718ea4",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (1, 1)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "80.0\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (1, 1)\n",
       "┌──────┐\n",
       "│ age  │\n",
       "│ ---  │\n",
       "│ f64  │\n",
       "╞══════╡\n",
       "│ 80.0 │\n",
       "└──────┘"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- agregação de uma única variável: máximo da var age\n",
    "\n",
    "dados\\\n",
    "    .select(col('age'))\\\n",
    "    .max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "770cbf86-47df-49fa-9bfe-a6fe83fd6379",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "80.0"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Para acessar a primeira linha e trazer o valor\n",
    "\n",
    "dados\\\n",
    "    .select(col('age'))\\\n",
    "    .max()\\\n",
    "    .row(0)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6b2c97c-abd0-457c-8e1f-cfd9d12ff83b",
   "metadata": {},
   "source": [
    "## **8. Funções de agregação, COM agupamento <a class=\"anchor\" id=\"header8\"></a>**\n",
    "\n",
    "Lembrem de usar colchete \"[]\", caso queiram adicionar mais de uma função de agregação. Atualmente, temos as seguintes funções de agregação, segundo a documentação do **Polars**.\n",
    "\n",
    "- mean\n",
    "- median\n",
    "- quantile\n",
    "- first\n",
    "- last\n",
    "- min\n",
    "- max\n",
    "- sum\n",
    "- n_unique\n",
    "- agg to series\n",
    "- return group indexes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "f0bcfea4-f2fb-4af1-b6cf-11ae12be2b34",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (2, 12)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "n\n",
       "</th>\n",
       "<th>\n",
       "mean_age\n",
       "</th>\n",
       "<th>\n",
       "median_age\n",
       "</th>\n",
       "<th>\n",
       "max_age\n",
       "</th>\n",
       "<th>\n",
       "min_age\n",
       "</th>\n",
       "<th>\n",
       "sum_fare\n",
       "</th>\n",
       "<th>\n",
       "q10_fare\n",
       "</th>\n",
       "<th>\n",
       "q90_fare\n",
       "</th>\n",
       "<th>\n",
       "first_name\n",
       "</th>\n",
       "<th>\n",
       "last_name\n",
       "</th>\n",
       "<th>\n",
       "unique_names\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "809\n",
       "</td>\n",
       "<td>\n",
       "30.545363\n",
       "</td>\n",
       "<td>\n",
       "28.0\n",
       "</td>\n",
       "<td>\n",
       "74.0\n",
       "</td>\n",
       "<td>\n",
       "0.33\n",
       "</td>\n",
       "<td>\n",
       "18869\n",
       "</td>\n",
       "<td>\n",
       "7.25\n",
       "</td>\n",
       "<td>\n",
       "50.4958\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zimmerman, Mr....\n",
       "</td>\n",
       "<td>\n",
       "808\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "500\n",
       "</td>\n",
       "<td>\n",
       "28.918244\n",
       "</td>\n",
       "<td>\n",
       "28.0\n",
       "</td>\n",
       "<td>\n",
       "80.0\n",
       "</td>\n",
       "<td>\n",
       "0.17\n",
       "</td>\n",
       "<td>\n",
       "24680\n",
       "</td>\n",
       "<td>\n",
       "7.75\n",
       "</td>\n",
       "<td>\n",
       "120.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Yasbeck, Mrs. ...\n",
       "</td>\n",
       "<td>\n",
       "500\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (2, 12)\n",
       "┌──────────┬─────┬───────────┬────────────┬─────┬──────────┬─────────────┬────────────┬────────────┐\n",
       "│ survived ┆ n   ┆ mean_age  ┆ median_age ┆ ... ┆ q90_fare ┆ first_name  ┆ last_name  ┆ unique_nam │\n",
       "│ ---      ┆ --- ┆ ---       ┆ ---        ┆     ┆ ---      ┆ ---         ┆ ---        ┆ es         │\n",
       "│ i64      ┆ u32 ┆ f64       ┆ f64        ┆     ┆ f64      ┆ str         ┆ str        ┆ ---        │\n",
       "│          ┆     ┆           ┆            ┆     ┆          ┆             ┆            ┆ u32        │\n",
       "╞══════════╪═════╪═══════════╪════════════╪═════╪══════════╪═════════════╪════════════╪════════════╡\n",
       "│ 0        ┆ 809 ┆ 30.545363 ┆ 28.0       ┆ ... ┆ 50.4958  ┆ Allison,    ┆ Zimmerman, ┆ 808        │\n",
       "│          ┆     ┆           ┆            ┆     ┆          ┆ Miss. Helen ┆ Mr. Leo    ┆            │\n",
       "│          ┆     ┆           ┆            ┆     ┆          ┆ Loraine     ┆            ┆            │\n",
       "│ 1        ┆ 500 ┆ 28.918244 ┆ 28.0       ┆ ... ┆ 120.0    ┆ Allen,      ┆ Yasbeck,   ┆ 500        │\n",
       "│          ┆     ┆           ┆            ┆     ┆          ┆ Miss.       ┆ Mrs.       ┆            │\n",
       "│          ┆     ┆           ┆            ┆     ┆          ┆ Elisabeth   ┆ Antoni     ┆            │\n",
       "│          ┆     ┆           ┆            ┆     ┆          ┆ Walton      ┆ (Selini    ┆            │\n",
       "│          ┆     ┆           ┆            ┆     ┆          ┆             ┆ Ale...     ┆            │\n",
       "└──────────┴─────┴───────────┴────────────┴─────┴──────────┴─────────────┴────────────┴────────────┘"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .groupby('survived')\\\n",
    "    .agg([\\\n",
    "          col('age').count().alias('n'), # tanto faz a variável, common kids\n",
    "          col('age').mean().alias('mean_age'),\n",
    "          col('age').median().alias('median_age'),\n",
    "          col('age').max().alias('max_age'),\n",
    "          col('age').min().alias('min_age'),\n",
    "          col('fare').sum().cast(Int64).alias('sum_fare'), # trocando o tipo da variável\n",
    "          col('fare').quantile(0.1).alias('q10_fare'),\n",
    "          col('fare').quantile(0.9).alias('q90_fare'),\n",
    "          col('name').first().alias('first_name'),\n",
    "          col('name').last().alias('last_name'),\n",
    "          col('name').n_unique().alias('unique_names')\n",
    "         ])\n",
    "\n",
    "# nunique"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "987f32a4-4627-4337-8590-78fed2591ba5",
   "metadata": {},
   "source": [
    "### **8.1. Bônus: Funções de agregações + agrupamentos + filtros (ao mesmo tempo)**\n",
    "\n",
    "Vamos considerar que você quer fazer saber somente a média da idade de quem sobreviveu ou não para os passageiros das classes 1 e 2. Podemos fazer o seguinte códiguin:\n",
    "\n",
    "**PS: Esse tipo de agregação é possível no PySpark, SQL e R, mas no Pandas não :/**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "000afb94-5c58-4517-aa31-1fbd5dd53365",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (2, 3)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "idade_media_classe_1_2\n",
       "</th>\n",
       "<th>\n",
       "idade_media_classe_2_3\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "32.181318\n",
       "</td>\n",
       "<td>\n",
       "23.094187\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "37.337349\n",
       "</td>\n",
       "<td>\n",
       "28.009845\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (2, 3)\n",
       "┌──────────┬────────────────────────┬────────────────────────┐\n",
       "│ survived ┆ idade_media_classe_1_2 ┆ idade_media_classe_2_3 │\n",
       "│ ---      ┆ ---                    ┆ ---                    │\n",
       "│ i64      ┆ f64                    ┆ f64                    │\n",
       "╞══════════╪════════════════════════╪════════════════════════╡\n",
       "│ 1        ┆ 32.181318              ┆ 23.094187              │\n",
       "│ 0        ┆ 37.337349              ┆ 28.009845              │\n",
       "└──────────┴────────────────────────┴────────────────────────┘"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .groupby('survived', maintain_order = True)\\\n",
    "    .agg([\\\n",
    "        col('age').filter(col('pclass').is_in([1, 2])).mean().alias('idade_media_classe_1_2'),\n",
    "        col('age').filter(col('pclass').is_in([2, 3])).mean().alias('idade_media_classe_2_3')\n",
    "        ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "0361f409-7eff-4ac2-ac8e-47ee69ed318d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (2, 2)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "mean_age\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "32.181318\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "37.337349\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (2, 2)\n",
       "┌──────────┬───────────┐\n",
       "│ survived ┆ mean_age  │\n",
       "│ ---      ┆ ---       │\n",
       "│ i64      ┆ f64       │\n",
       "╞══════════╪═══════════╡\n",
       "│ 1        ┆ 32.181318 │\n",
       "│ 0        ┆ 37.337349 │\n",
       "└──────────┴───────────┘"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Mesma coisa que o código acima\n",
    "\n",
    "dados\\\n",
    "    .filter(col('pclass').is_in([1, 2]))\\\n",
    "    .groupby('survived', maintain_order = True)\\\n",
    "    .agg(\\\n",
    "        col('age').mean().alias('mean_age')\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ca7237c-66cf-4a98-a277-f2970e9eb167",
   "metadata": {},
   "source": [
    "## **9. Ordenando as linhas <a class=\"anchor\" id=\"header9\"></a>**\n",
    "\n",
    "Aos mais chegados, é a mesma lógica de:\n",
    "\n",
    "- **Pandas**: sort_values\n",
    "- **R (dplyr)**: arrange\n",
    "- **SQL**: Order By"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "68f4869e-563a-4830-878d-cba6000f0c29",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Baumann, Mr. J...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17318&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.925\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bradley, Mr. G...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;111427&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;9&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Los Angeles, C...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Brewe, Dr. Art...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;112379&quot;\n",
       "</td>\n",
       "<td>\n",
       "39.6\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Philadelphia, ...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬─────────────────────┬──────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name                ┆ sex  ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---                 ┆ ---  ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str                 ┆ str  ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═════════════════════╪══════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 0        ┆ Baumann, Mr. John D ┆ male ┆ ... ┆ S        ┆ null ┆ null ┆ New York, NY     │\n",
       "│ 1      ┆ 1        ┆ Bradley, Mr. George ┆ male ┆ ... ┆ S        ┆ 9    ┆ null ┆ Los Angeles, CA  │\n",
       "│        ┆          ┆ (\"George Art...     ┆      ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 0        ┆ Brewe, Dr. Arthur   ┆ male ┆ ... ┆ C        ┆ null ┆ null ┆ Philadelphia, PA │\n",
       "│        ┆          ┆ Jackson             ┆      ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴─────────────────────┴──────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Ordem crescente da variável age\n",
    "\n",
    "dados\\\n",
    "    .sort(by = 'age', reverse = False)\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "57155ef2-59e3-40b8-a7d3-1962e04a311c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Barkworth, Mr....\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "80.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;27042&quot;\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;A23&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;B&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Hessle, Yorks&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Cavendish, Mrs...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "76.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;19877&quot;\n",
       "</td>\n",
       "<td>\n",
       "78.85\n",
       "</td>\n",
       "<td>\n",
       "&quot;C46&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;6&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Little Onn Hal...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Svensson, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "74.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;347060&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.775\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Barkworth, Mr.    ┆ male   ┆ ... ┆ S        ┆ B    ┆ null ┆ Hessle, Yorks    │\n",
       "│        ┆          ┆ Algernon Henry    ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│        ┆          ┆ Wi...             ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Cavendish, Mrs.   ┆ female ┆ ... ┆ S        ┆ 6    ┆ null ┆ Little Onn Hall, │\n",
       "│        ┆          ┆ Tyrell William    ┆        ┆     ┆          ┆      ┆      ┆ Staffs           │\n",
       "│        ┆          ┆ (...              ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 3      ┆ 0        ┆ Svensson, Mr.     ┆ male   ┆ ... ┆ S        ┆ null ┆ null ┆ null             │\n",
       "│        ┆          ┆ Johan             ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Ordem decrescente da variável age\n",
    "\n",
    "dados\\\n",
    "    .sort(by = 'age', reverse = True)\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "9a1b2ffa-eded-48ad-bc1c-56e4ed559435",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Barkworth, Mr....\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "80.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;27042&quot;\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;A23&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;B&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Hessle, Yorks&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Cavendish, Mrs...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "76.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;19877&quot;\n",
       "</td>\n",
       "<td>\n",
       "78.85\n",
       "</td>\n",
       "<td>\n",
       "&quot;C46&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;6&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Little Onn Hal...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Svensson, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "74.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;347060&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.775\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Barkworth, Mr.    ┆ male   ┆ ... ┆ S        ┆ B    ┆ null ┆ Hessle, Yorks    │\n",
       "│        ┆          ┆ Algernon Henry    ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│        ┆          ┆ Wi...             ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Cavendish, Mrs.   ┆ female ┆ ... ┆ S        ┆ 6    ┆ null ┆ Little Onn Hall, │\n",
       "│        ┆          ┆ Tyrell William    ┆        ┆     ┆          ┆      ┆      ┆ Staffs           │\n",
       "│        ┆          ┆ (...              ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 3      ┆ 0        ┆ Svensson, Mr.     ┆ male   ┆ ... ┆ S        ┆ null ┆ null ┆ null             │\n",
       "│        ┆          ┆ Johan             ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Ordem decrescente da variável age e crescente da variável fare\n",
    "\n",
    "dados\\\n",
    "    .sort(by = ['age', 'fare'], reverse = [True, False])\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1097aedc-c4ae-46c9-97ca-d758464637fe",
   "metadata": {},
   "source": [
    "## **10. Trabalhando com duplicatas <a class=\"anchor\" id=\"header10\"></a>**\n",
    "\n",
    "Funciona bem parecido com o `drop_duplicates` do Pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "db09a076-d3de-4ff2-8ae7-af724bdd9ff8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (4, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Artagaveytia, ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "71.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17609&quot;\n",
       "</td>\n",
       "<td>\n",
       "49.5042\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "22\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montevideo, Ur...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Icard, Miss. A...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "38.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;113572&quot;\n",
       "</td>\n",
       "<td>\n",
       "80.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;B28&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;6&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Minahan, Dr. W...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "44.0\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;19928&quot;\n",
       "</td>\n",
       "<td>\n",
       "90.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;C78&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Q&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "230\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fond du Lac, W...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (4, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 0        ┆ Artagaveytia, Mr. ┆ male   ┆ ... ┆ C        ┆ null ┆ 22   ┆ Montevideo,      │\n",
       "│        ┆          ┆ Ramon             ┆        ┆     ┆          ┆      ┆      ┆ Uruguay          │\n",
       "│ 1      ┆ 1        ┆ Icard, Miss.      ┆ female ┆ ... ┆ null     ┆ 6    ┆ null ┆ null             │\n",
       "│        ┆          ┆ Amelie            ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 0        ┆ Minahan, Dr.      ┆ male   ┆ ... ┆ Q        ┆ null ┆ 230  ┆ Fond du Lac, WI  │\n",
       "│        ┆          ┆ William Edward    ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .unique(subset = 'embarked', keep = 'first')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "5d5bc40b-7cb6-4b60-a5cd-d417b7c9e5bc",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (10, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Minahan, Mrs. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "37.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;19928&quot;\n",
       "</td>\n",
       "<td>\n",
       "90.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;C78&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Q&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;14&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fond du Lac, W...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Stone, Mrs. Ge...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "62.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;113572&quot;\n",
       "</td>\n",
       "<td>\n",
       "80.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;B28&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;6&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Cincinatti, OH...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Wright, Mr. Ge...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "62.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;113807&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Halifax, NS&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Young, Miss. M...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "36.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17760&quot;\n",
       "</td>\n",
       "<td>\n",
       "135.6333\n",
       "</td>\n",
       "<td>\n",
       "&quot;C32&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;8&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Slayter, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;234818&quot;\n",
       "</td>\n",
       "<td>\n",
       "12.35\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Q&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;13&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Halifax, NS&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Stanton, Mr. S...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "41.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;237734&quot;\n",
       "</td>\n",
       "<td>\n",
       "15.0458\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Yrois, Miss. H...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "24.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;248747&quot;\n",
       "</td>\n",
       "<td>\n",
       "13.0\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Paris&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Tobin, Mr. Rog...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;383121&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.75\n",
       "</td>\n",
       "<td>\n",
       "&quot;F38&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Q&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zakarian, Mr. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "27.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;2670&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.225\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Zimmerman, Mr....\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;315082&quot;\n",
       "</td>\n",
       "<td>\n",
       "7.875\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (10, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Minahan, Mrs.     ┆ female ┆ ... ┆ Q        ┆ 14   ┆ null ┆ Fond du Lac, WI  │\n",
       "│        ┆          ┆ William Edward    ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│        ┆          ┆ (Li...            ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Stone, Mrs.       ┆ female ┆ ... ┆ null     ┆ 6    ┆ null ┆ Cincinatti, OH   │\n",
       "│        ┆          ┆ George Nelson     ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│        ┆          ┆ (Marth...         ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 0        ┆ Wright, Mr.       ┆ male   ┆ ... ┆ S        ┆ null ┆ null ┆ Halifax, NS      │\n",
       "│        ┆          ┆ George            ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Young, Miss.      ┆ female ┆ ... ┆ C        ┆ 8    ┆ null ┆ New York, NY /   │\n",
       "│        ┆          ┆ Marie Grice       ┆        ┆     ┆          ┆      ┆      ┆ Washington, DC   │\n",
       "│ ...    ┆ ...      ┆ ...               ┆ ...    ┆ ... ┆ ...      ┆ ...  ┆ ...  ┆ ...              │\n",
       "│ 2      ┆ 0        ┆ Yrois, Miss.      ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Paris            │\n",
       "│        ┆          ┆ Henriette (\"Mrs   ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│        ┆          ┆ Har...            ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 3      ┆ 0        ┆ Tobin, Mr. Roger  ┆ male   ┆ ... ┆ Q        ┆ null ┆ null ┆ null             │\n",
       "│ 3      ┆ 0        ┆ Zakarian, Mr.     ┆ male   ┆ ... ┆ C        ┆ null ┆ null ┆ null             │\n",
       "│        ┆          ┆ Ortin             ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 3      ┆ 0        ┆ Zimmerman, Mr.    ┆ male   ┆ ... ┆ S        ┆ null ┆ null ┆ null             │\n",
       "│        ┆          ┆ Leo               ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .unique(subset = ['embarked', 'pclass'], maintain_order = True, keep = 'last')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fcd606a-1010-4c73-bdd8-7eb3dbabe0a8",
   "metadata": {},
   "source": [
    "## **11. Lidando com valores nulos<a class=\"anchor\" id=\"header11\"></a>**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ea44b73-ee13-4228-9535-be48e782851a",
   "metadata": {},
   "source": [
    "### **11.1. Identificando as variáveis com valores nulos**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "461f8fec-82e6-4626-8eef-79b03a106b04",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (1, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "263\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1014\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "823\n",
       "</td>\n",
       "<td>\n",
       "1188\n",
       "</td>\n",
       "<td>\n",
       "564\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (1, 14)\n",
       "┌────────┬──────────┬──────┬─────┬─────┬──────────┬──────┬──────┬───────────┐\n",
       "│ pclass ┆ survived ┆ name ┆ sex ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest │\n",
       "│ ---    ┆ ---      ┆ ---  ┆ --- ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---       │\n",
       "│ u32    ┆ u32      ┆ u32  ┆ u32 ┆     ┆ u32      ┆ u32  ┆ u32  ┆ u32       │\n",
       "╞════════╪══════════╪══════╪═════╪═════╪══════════╪══════╪══════╪═══════════╡\n",
       "│ 0      ┆ 0        ┆ 0    ┆ 0   ┆ ... ┆ 2        ┆ 823  ┆ 1188 ┆ 564       │\n",
       "└────────┴──────────┴──────┴─────┴─────┴──────────┴──────┴──────┴───────────┘"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados.null_count()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6c84ca3-4cdf-4f73-844a-e3d502b71764",
   "metadata": {},
   "source": [
    "### **11.2. Filtrando os nulos**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "e2a144bc-c1f9-4b59-a230-cba170f6c367",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Baumann, Mr. J...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17318&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.925\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bradley, Mr. G...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;111427&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;9&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Los Angeles, C...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Brewe, Dr. Art...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;112379&quot;\n",
       "</td>\n",
       "<td>\n",
       "39.6\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Philadelphia, ...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬─────────────────────┬──────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name                ┆ sex  ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---                 ┆ ---  ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str                 ┆ str  ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═════════════════════╪══════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 0        ┆ Baumann, Mr. John D ┆ male ┆ ... ┆ S        ┆ null ┆ null ┆ New York, NY     │\n",
       "│ 1      ┆ 1        ┆ Bradley, Mr. George ┆ male ┆ ... ┆ S        ┆ 9    ┆ null ┆ Los Angeles, CA  │\n",
       "│        ┆          ┆ (\"George Art...     ┆      ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 0        ┆ Brewe, Dr. Arthur   ┆ male ┆ ... ┆ C        ┆ null ┆ null ┆ Philadelphia, PA │\n",
       "│        ┆          ┆ Jackson             ┆      ┆     ┆          ┆      ┆      ┆                  │\n",
       "└────────┴──────────┴─────────────────────┴──────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Selecionando as linhas nulas da variável age\n",
    "\n",
    "dados\\\n",
    "    .filter(col('age').is_null())\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "885ad4d7-034e-441a-a810-1ea11c9b3d2d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 14)\n",
       "┌────────┬──────────┬───────────────────┬────────┬─────┬──────────┬──────┬──────┬──────────────────┐\n",
       "│ pclass ┆ survived ┆ name              ┆ sex    ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest        │\n",
       "│ ---    ┆ ---      ┆ ---               ┆ ---    ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---              │\n",
       "│ i64    ┆ i64      ┆ str               ┆ str    ┆     ┆ str      ┆ str  ┆ i64  ┆ str              │\n",
       "╞════════╪══════════╪═══════════════════╪════════╪═════╪══════════╪══════╪══════╪══════════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.      ┆ female ┆ ... ┆ S        ┆ 2    ┆ null ┆ St Louis, MO     │\n",
       "│        ┆          ┆ Elisabeth Walton  ┆        ┆     ┆          ┆      ┆      ┆                  │\n",
       "│ 1      ┆ 1        ┆ Allison, Master.  ┆ male   ┆ ... ┆ S        ┆ 11   ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Hudson Trevor     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "│ 1      ┆ 0        ┆ Allison, Miss.    ┆ female ┆ ... ┆ S        ┆ null ┆ null ┆ Montreal, PQ /   │\n",
       "│        ┆          ┆ Helen Loraine     ┆        ┆     ┆          ┆      ┆      ┆ Chesterville, ON │\n",
       "└────────┴──────────┴───────────────────┴────────┴─────┴──────────┴──────┴──────┴──────────────────┘"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Excluindo as linhas nulas da variável age\n",
    "\n",
    "dados\\\n",
    "    .filter(~ col('age').is_null())\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54b651a7-99db-4f01-91ad-230beb82e573",
   "metadata": {},
   "source": [
    "### **11.3. Substituindo os valores nulos por algum outro**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "ec89b8f8-6ffd-46ed-9962-5288e797275e",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 15)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "<th>\n",
       "age_fill_null\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Baumann, Mr. J...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17318&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.925\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bradley, Mr. G...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;111427&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;9&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Los Angeles, C...\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Brewe, Dr. Art...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;112379&quot;\n",
       "</td>\n",
       "<td>\n",
       "39.6\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Philadelphia, ...\n",
       "</td>\n",
       "<td>\n",
       "0.0\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 15)\n",
       "┌────────┬──────────┬─────────────────┬──────┬─────┬──────┬──────┬─────────────────┬───────────────┐\n",
       "│ pclass ┆ survived ┆ name            ┆ sex  ┆ ... ┆ boat ┆ body ┆ home.dest       ┆ age_fill_null │\n",
       "│ ---    ┆ ---      ┆ ---             ┆ ---  ┆     ┆ ---  ┆ ---  ┆ ---             ┆ ---           │\n",
       "│ i64    ┆ i64      ┆ str             ┆ str  ┆     ┆ str  ┆ i64  ┆ str             ┆ f64           │\n",
       "╞════════╪══════════╪═════════════════╪══════╪═════╪══════╪══════╪═════════════════╪═══════════════╡\n",
       "│ 1      ┆ 0        ┆ Baumann, Mr.    ┆ male ┆ ... ┆ null ┆ null ┆ New York, NY    ┆ 0.0           │\n",
       "│        ┆          ┆ John D          ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│ 1      ┆ 1        ┆ Bradley, Mr.    ┆ male ┆ ... ┆ 9    ┆ null ┆ Los Angeles, CA ┆ 0.0           │\n",
       "│        ┆          ┆ George (\"George ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│        ┆          ┆ Art...          ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│ 1      ┆ 0        ┆ Brewe, Dr.      ┆ male ┆ ... ┆ null ┆ null ┆ Philadelphia,   ┆ 0.0           │\n",
       "│        ┆          ┆ Arthur Jackson  ┆      ┆     ┆      ┆      ┆ PA              ┆               │\n",
       "└────────┴──────────┴─────────────────┴──────┴─────┴──────┴──────┴─────────────────┴───────────────┘"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .with_columns(\\\n",
    "                  (col('age').fill_null(lit(0))).alias('age_fill_null')\n",
    "                  )\\\n",
    "    .filter(col('age').is_null())\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06252748-34ae-4c5f-94ae-3a972b375aaf",
   "metadata": {},
   "source": [
    "### **11.4. Substituindo pela mediana**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "03d1172d-c8b6-449a-ae13-cece64ccab51",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 15)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "<th>\n",
       "age_fill_null\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Baumann, Mr. J...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17318&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.925\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "<td>\n",
       "28.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bradley, Mr. G...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;111427&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;9&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Los Angeles, C...\n",
       "</td>\n",
       "<td>\n",
       "28.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Brewe, Dr. Art...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;112379&quot;\n",
       "</td>\n",
       "<td>\n",
       "39.6\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Philadelphia, ...\n",
       "</td>\n",
       "<td>\n",
       "28.0\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 15)\n",
       "┌────────┬──────────┬─────────────────┬──────┬─────┬──────┬──────┬─────────────────┬───────────────┐\n",
       "│ pclass ┆ survived ┆ name            ┆ sex  ┆ ... ┆ boat ┆ body ┆ home.dest       ┆ age_fill_null │\n",
       "│ ---    ┆ ---      ┆ ---             ┆ ---  ┆     ┆ ---  ┆ ---  ┆ ---             ┆ ---           │\n",
       "│ i64    ┆ i64      ┆ str             ┆ str  ┆     ┆ str  ┆ i64  ┆ str             ┆ f64           │\n",
       "╞════════╪══════════╪═════════════════╪══════╪═════╪══════╪══════╪═════════════════╪═══════════════╡\n",
       "│ 1      ┆ 0        ┆ Baumann, Mr.    ┆ male ┆ ... ┆ null ┆ null ┆ New York, NY    ┆ 28.0          │\n",
       "│        ┆          ┆ John D          ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│ 1      ┆ 1        ┆ Bradley, Mr.    ┆ male ┆ ... ┆ 9    ┆ null ┆ Los Angeles, CA ┆ 28.0          │\n",
       "│        ┆          ┆ George (\"George ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│        ┆          ┆ Art...          ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│ 1      ┆ 0        ┆ Brewe, Dr.      ┆ male ┆ ... ┆ null ┆ null ┆ Philadelphia,   ┆ 28.0          │\n",
       "│        ┆          ┆ Arthur Jackson  ┆      ┆     ┆      ┆      ┆ PA              ┆               │\n",
       "└────────┴──────────┴─────────────────┴──────┴─────┴──────┴──────┴─────────────────┴───────────────┘"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .with_columns(\\\n",
    "                  (col('age').fill_null(col('age').median())).alias('age_fill_null')\n",
    "                  )\\\n",
    "    .filter(col('age').is_null())\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "277b1b49-66df-46a3-aa98-438c4ab1e947",
   "metadata": {},
   "source": [
    "### **11.5. Substituindo baseado na interpolação**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "e3d3a849-0af1-49fa-8d08-920feace8b0f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 15)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "<th>\n",
       "age_fill_null\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Baumann, Mr. J...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17318&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.925\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "<td>\n",
       "52.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bradley, Mr. G...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;111427&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;9&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Los Angeles, C...\n",
       "</td>\n",
       "<td>\n",
       "31.5\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Brewe, Dr. Art...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;112379&quot;\n",
       "</td>\n",
       "<td>\n",
       "39.6\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Philadelphia, ...\n",
       "</td>\n",
       "<td>\n",
       "46.0\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 15)\n",
       "┌────────┬──────────┬─────────────────┬──────┬─────┬──────┬──────┬─────────────────┬───────────────┐\n",
       "│ pclass ┆ survived ┆ name            ┆ sex  ┆ ... ┆ boat ┆ body ┆ home.dest       ┆ age_fill_null │\n",
       "│ ---    ┆ ---      ┆ ---             ┆ ---  ┆     ┆ ---  ┆ ---  ┆ ---             ┆ ---           │\n",
       "│ i64    ┆ i64      ┆ str             ┆ str  ┆     ┆ str  ┆ i64  ┆ str             ┆ f64           │\n",
       "╞════════╪══════════╪═════════════════╪══════╪═════╪══════╪══════╪═════════════════╪═══════════════╡\n",
       "│ 1      ┆ 0        ┆ Baumann, Mr.    ┆ male ┆ ... ┆ null ┆ null ┆ New York, NY    ┆ 52.0          │\n",
       "│        ┆          ┆ John D          ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│ 1      ┆ 1        ┆ Bradley, Mr.    ┆ male ┆ ... ┆ 9    ┆ null ┆ Los Angeles, CA ┆ 31.5          │\n",
       "│        ┆          ┆ George (\"George ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│        ┆          ┆ Art...          ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│ 1      ┆ 0        ┆ Brewe, Dr.      ┆ male ┆ ... ┆ null ┆ null ┆ Philadelphia,   ┆ 46.0          │\n",
       "│        ┆          ┆ Arthur Jackson  ┆      ┆     ┆      ┆      ┆ PA              ┆               │\n",
       "└────────┴──────────┴─────────────────┴──────┴─────┴──────┴──────┴─────────────────┴───────────────┘"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .with_columns(\\\n",
    "                  (col('age').interpolate()).alias('age_fill_null')\n",
    "                  )\\\n",
    "    .filter(col('age').is_null())\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "925bd9af-fb5c-4153-8cd5-b88012e0ce55",
   "metadata": {},
   "source": [
    "### **11.6. Substituindo baseado na moda**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "4069baa4-064f-432c-852a-f6db50e14261",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (2, 15)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "<th>\n",
       "embarked_fill_null\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Icard, Miss. A...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "38.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;113572&quot;\n",
       "</td>\n",
       "<td>\n",
       "80.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;B28&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;6&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Stone, Mrs. Ge...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "62.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;113572&quot;\n",
       "</td>\n",
       "<td>\n",
       "80.0\n",
       "</td>\n",
       "<td>\n",
       "&quot;B28&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;6&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Cincinatti, OH...\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (2, 15)\n",
       "┌────────┬──────────┬────────────────┬────────┬─────┬──────┬──────┬────────────────┬───────────────┐\n",
       "│ pclass ┆ survived ┆ name           ┆ sex    ┆ ... ┆ boat ┆ body ┆ home.dest      ┆ embarked_fill │\n",
       "│ ---    ┆ ---      ┆ ---            ┆ ---    ┆     ┆ ---  ┆ ---  ┆ ---            ┆ _null         │\n",
       "│ i64    ┆ i64      ┆ str            ┆ str    ┆     ┆ str  ┆ i64  ┆ str            ┆ ---           │\n",
       "│        ┆          ┆                ┆        ┆     ┆      ┆      ┆                ┆ str           │\n",
       "╞════════╪══════════╪════════════════╪════════╪═════╪══════╪══════╪════════════════╪═══════════════╡\n",
       "│ 1      ┆ 1        ┆ Icard, Miss.   ┆ female ┆ ... ┆ 6    ┆ null ┆ null           ┆ S             │\n",
       "│        ┆          ┆ Amelie         ┆        ┆     ┆      ┆      ┆                ┆               │\n",
       "│ 1      ┆ 1        ┆ Stone, Mrs.    ┆ female ┆ ... ┆ 6    ┆ null ┆ Cincinatti, OH ┆ S             │\n",
       "│        ┆          ┆ George Nelson  ┆        ┆     ┆      ┆      ┆                ┆               │\n",
       "│        ┆          ┆ (Marth...      ┆        ┆     ┆      ┆      ┆                ┆               │\n",
       "└────────┴──────────┴────────────────┴────────┴─────┴──────┴──────┴────────────────┴───────────────┘"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .with_columns(\\\n",
    "                  (col('embarked').fill_null(col('embarked').mode())).alias('embarked_fill_null')\n",
    "                  )\\\n",
    "    .filter(col('embarked').is_null())\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01cde052-8b8e-4009-875f-016c2035b534",
   "metadata": {},
   "source": [
    "### **11.7. Dropando todos os nulos de uma vez**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "0a331dad-b566-40b6-8fc4-749f5e76dfae",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (0, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (0, 14)\n",
       "┌────────┬──────────┬──────┬─────┬─────┬──────────┬──────┬──────┬───────────┐\n",
       "│ pclass ┆ survived ┆ name ┆ sex ┆ ... ┆ embarked ┆ boat ┆ body ┆ home.dest │\n",
       "│ ---    ┆ ---      ┆ ---  ┆ --- ┆     ┆ ---      ┆ ---  ┆ ---  ┆ ---       │\n",
       "│ i64    ┆ i64      ┆ str  ┆ str ┆     ┆ str      ┆ str  ┆ i64  ┆ str       │\n",
       "╞════════╪══════════╪══════╪═════╪═════╪══════════╪══════╪══════╪═══════════╡\n",
       "└────────┴──────────┴──────┴─────┴─────┴──────────┴──────┴──────┴───────────┘"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- A variável body tem basicamente só nulos\n",
    "\n",
    "dados\\\n",
    "    .drop_nulls()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17ab4b35-c1e5-469d-803d-f4fd47a74034",
   "metadata": {},
   "source": [
    "### **11.8. Criando uma varaiável/flag que identifica nulos de uma variável**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "d345c5f2-a9a0-4797-926f-e495b937cd02",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 15)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "<th>\n",
       "flag_age_null\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "bool\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Baumann, Mr. J...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;PC 17318&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.925\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;New York, NY&quot;\n",
       "</td>\n",
       "<td>\n",
       "true\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bradley, Mr. G...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;111427&quot;\n",
       "</td>\n",
       "<td>\n",
       "26.55\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;9&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Los Angeles, C...\n",
       "</td>\n",
       "<td>\n",
       "true\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Brewe, Dr. Art...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;112379&quot;\n",
       "</td>\n",
       "<td>\n",
       "39.6\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;C&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Philadelphia, ...\n",
       "</td>\n",
       "<td>\n",
       "true\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 15)\n",
       "┌────────┬──────────┬─────────────────┬──────┬─────┬──────┬──────┬─────────────────┬───────────────┐\n",
       "│ pclass ┆ survived ┆ name            ┆ sex  ┆ ... ┆ boat ┆ body ┆ home.dest       ┆ flag_age_null │\n",
       "│ ---    ┆ ---      ┆ ---             ┆ ---  ┆     ┆ ---  ┆ ---  ┆ ---             ┆ ---           │\n",
       "│ i64    ┆ i64      ┆ str             ┆ str  ┆     ┆ str  ┆ i64  ┆ str             ┆ bool          │\n",
       "╞════════╪══════════╪═════════════════╪══════╪═════╪══════╪══════╪═════════════════╪═══════════════╡\n",
       "│ 1      ┆ 0        ┆ Baumann, Mr.    ┆ male ┆ ... ┆ null ┆ null ┆ New York, NY    ┆ true          │\n",
       "│        ┆          ┆ John D          ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│ 1      ┆ 1        ┆ Bradley, Mr.    ┆ male ┆ ... ┆ 9    ┆ null ┆ Los Angeles, CA ┆ true          │\n",
       "│        ┆          ┆ George (\"George ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│        ┆          ┆ Art...          ┆      ┆     ┆      ┆      ┆                 ┆               │\n",
       "│ 1      ┆ 0        ┆ Brewe, Dr.      ┆ male ┆ ... ┆ null ┆ null ┆ Philadelphia,   ┆ true          │\n",
       "│        ┆          ┆ Arthur Jackson  ┆      ┆     ┆      ┆      ┆ PA              ┆               │\n",
       "└────────┴──────────┴─────────────────┴──────┴─────┴──────┴──────┴─────────────────┴───────────────┘"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .with_columns(\\\n",
    "                 col('age').is_null().alias('flag_age_null')\n",
    "                 )\\\n",
    "    .filter(col('age').is_null())\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "671aaa63-107c-4d11-ad7b-e4b170759620",
   "metadata": {},
   "source": [
    "## **12. Joins <a class=\"anchor\" id=\"header12\"></a>**\n",
    "\n",
    "Atualmente, segundo a documentação, existem os seguintes joins:\n",
    "\n",
    "- inner\n",
    "- left\n",
    "- outer\n",
    "- cross\n",
    "- asof\n",
    "- semi\n",
    "- anti\n",
    "\n",
    "**Exemplo copiado descaradamente da documentação, por motivos de não ter um dataset para fazer joins. No fim, basta trocar qual tipo de join você deseja utilizar, dentro da própria função.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "e39c7cca-cc56-4123-98bc-773379413872",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 2)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "customer_id\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Alice&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bob&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "&quot;Charlie&quot;\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 2)\n",
       "┌─────────────┬─────────┐\n",
       "│ customer_id ┆ name    │\n",
       "│ ---         ┆ ---     │\n",
       "│ i64         ┆ str     │\n",
       "╞═════════════╪═════════╡\n",
       "│ 1           ┆ Alice   │\n",
       "│ 2           ┆ Bob     │\n",
       "│ 3           ┆ Charlie │\n",
       "└─────────────┴─────────┘"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_customers = DataFrame(\n",
    "    {\n",
    "        \"customer_id\": [1, 2, 3],\n",
    "        \"name\": [\"Alice\", \"Bob\", \"Charlie\"],\n",
    "    }\n",
    ")\n",
    "\n",
    "df_customers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "5d01e60c-3a2c-461c-a8b7-c2f083da37e3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 3)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "order_id\n",
       "</th>\n",
       "<th>\n",
       "customer_id\n",
       "</th>\n",
       "<th>\n",
       "amount\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;a&quot;\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;b&quot;\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "200\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;c&quot;\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "300\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 3)\n",
       "┌──────────┬─────────────┬────────┐\n",
       "│ order_id ┆ customer_id ┆ amount │\n",
       "│ ---      ┆ ---         ┆ ---    │\n",
       "│ str      ┆ i64         ┆ i64    │\n",
       "╞══════════╪═════════════╪════════╡\n",
       "│ a        ┆ 1           ┆ 100    │\n",
       "│ b        ┆ 2           ┆ 200    │\n",
       "│ c        ┆ 2           ┆ 300    │\n",
       "└──────────┴─────────────┴────────┘"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_orders = DataFrame(\n",
    "    {\n",
    "        \"order_id\": [\"a\", \"b\", \"c\"],\n",
    "        \"customer_id\": [1, 2, 2],\n",
    "        \"amount\": [100, 200, 300],\n",
    "    }\n",
    ")\n",
    "df_orders"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "9ff8e760-057e-4019-9a4e-efac99d65a74",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 4)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "customer_id\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "order_id\n",
       "</th>\n",
       "<th>\n",
       "amount\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Alice&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;a&quot;\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bob&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;b&quot;\n",
       "</td>\n",
       "<td>\n",
       "200\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bob&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;c&quot;\n",
       "</td>\n",
       "<td>\n",
       "300\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 4)\n",
       "┌─────────────┬───────┬──────────┬────────┐\n",
       "│ customer_id ┆ name  ┆ order_id ┆ amount │\n",
       "│ ---         ┆ ---   ┆ ---      ┆ ---    │\n",
       "│ i64         ┆ str   ┆ str      ┆ i64    │\n",
       "╞═════════════╪═══════╪══════════╪════════╡\n",
       "│ 1           ┆ Alice ┆ a        ┆ 100    │\n",
       "│ 2           ┆ Bob   ┆ b        ┆ 200    │\n",
       "│ 2           ┆ Bob   ┆ c        ┆ 300    │\n",
       "└─────────────┴───────┴──────────┴────────┘"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_customers\\\n",
    "    .join(df_orders, on = 'customer_id', how = 'inner') # mudar o parâmetro do how para o join desejado"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb68e17f-2e6f-4a98-a30c-170ee08d80c1",
   "metadata": {},
   "source": [
    "## **13. Pivot e melt <a class=\"anchor\" id=\"header13\"></a>**\n",
    "\n",
    "Também temos no Polars essas funções, vou criar um exemplo somente com o pivot, pois os dados estão num formato que faz sentido. \n",
    "\n",
    "Você pode encontrar um exemplo do melt na [documentação](https://pola-rs.github.io/polars/py-polars/html/reference/dataframe/api/polars.DataFrame.melt.html#polars.DataFrame.melt) ou nesse [exemplo do Stack Overflow](https://stackoverflow.com/questions/71775175/convert-pandas-pivot-table-function-into-polars-pivot-function)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "e66e93e4-c141-49dd-83b8-f70ea1afd747",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 1308)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "Allen, Miss. Elisabeth Walton\n",
       "</th>\n",
       "<th>\n",
       "Allison, Master. Hudson Trevor\n",
       "</th>\n",
       "<th>\n",
       "Allison, Miss. Helen Loraine\n",
       "</th>\n",
       "<th>\n",
       "Allison, Mr. Hudson Joshua Creighton\n",
       "</th>\n",
       "<th>\n",
       "Allison, Mrs. Hudson J C (Bessie Waldo Daniels)\n",
       "</th>\n",
       "<th>\n",
       "Anderson, Mr. Harry\n",
       "</th>\n",
       "<th>\n",
       "Andrews, Miss. Kornelia Theodosia\n",
       "</th>\n",
       "<th>\n",
       "Andrews, Mr. Thomas Jr\n",
       "</th>\n",
       "<th>\n",
       "Appleton, Mrs. Edward Dale (Charlotte Lamson)\n",
       "</th>\n",
       "<th>\n",
       "Artagaveytia, Mr. Ramon\n",
       "</th>\n",
       "<th>\n",
       "Astor, Col. John Jacob\n",
       "</th>\n",
       "<th>\n",
       "Astor, Mrs. John Jacob (Madeleine Talmadge Force)\n",
       "</th>\n",
       "<th>\n",
       "Aubart, Mme. Leontine Pauline\n",
       "</th>\n",
       "<th>\n",
       "Barber, Miss. Ellen \"Nellie\"\n",
       "</th>\n",
       "<th>\n",
       "Barkworth, Mr. Algernon Henry Wilson\n",
       "</th>\n",
       "<th>\n",
       "Baumann, Mr. John D\n",
       "</th>\n",
       "<th>\n",
       "Baxter, Mr. Quigg Edmond\n",
       "</th>\n",
       "<th>\n",
       "Baxter, Mrs. James (Helene DeLaudeniere Chaput)\n",
       "</th>\n",
       "<th>\n",
       "Bazzani, Miss. Albina\n",
       "</th>\n",
       "<th>\n",
       "Beattie, Mr. Thomson\n",
       "</th>\n",
       "<th>\n",
       "Beckwith, Mr. Richard Leonard\n",
       "</th>\n",
       "<th>\n",
       "Beckwith, Mrs. Richard Leonard (Sallie Monypeny)\n",
       "</th>\n",
       "<th>\n",
       "Behr, Mr. Karl Howell\n",
       "</th>\n",
       "<th>\n",
       "Bidois, Miss. Rosalie\n",
       "</th>\n",
       "<th>\n",
       "Bird, Miss. Ellen\n",
       "</th>\n",
       "<th>\n",
       "Birnbaum, Mr. Jakob\n",
       "</th>\n",
       "<th>\n",
       "Bishop, Mr. Dickinson H\n",
       "</th>\n",
       "<th>\n",
       "Bishop, Mrs. Dickinson H (Helen Walton)\n",
       "</th>\n",
       "<th>\n",
       "Bissette, Miss. Amelia\n",
       "</th>\n",
       "<th>\n",
       "Bjornstrom-Steffansson, Mr. Mauritz Hakan\n",
       "</th>\n",
       "<th>\n",
       "Blackwell, Mr. Stephen Weart\n",
       "</th>\n",
       "<th>\n",
       "Blank, Mr. Henry\n",
       "</th>\n",
       "<th>\n",
       "Bonnell, Miss. Caroline\n",
       "</th>\n",
       "<th>\n",
       "Bonnell, Miss. Elizabeth\n",
       "</th>\n",
       "<th>\n",
       "Borebank, Mr. John James\n",
       "</th>\n",
       "<th>\n",
       "Bowen, Miss. Grace Scott\n",
       "</th>\n",
       "<th>\n",
       "...\n",
       "</th>\n",
       "<th>\n",
       "Vander Cruyssen, Mr. Victor\n",
       "</th>\n",
       "<th>\n",
       "Vander Planke, Miss. Augusta Maria\n",
       "</th>\n",
       "<th>\n",
       "Vander Planke, Mr. Julius\n",
       "</th>\n",
       "<th>\n",
       "Vander Planke, Mr. Leo Edmondus\n",
       "</th>\n",
       "<th>\n",
       "Vander Planke, Mrs. Julius (Emelia Maria Vandemoortele)\n",
       "</th>\n",
       "<th>\n",
       "Vartanian, Mr. David\n",
       "</th>\n",
       "<th>\n",
       "Vendel, Mr. Olof Edvin\n",
       "</th>\n",
       "<th>\n",
       "Vestrom, Miss. Hulda Amanda Adolfina\n",
       "</th>\n",
       "<th>\n",
       "Vovk, Mr. Janko\n",
       "</th>\n",
       "<th>\n",
       "Waelens, Mr. Achille\n",
       "</th>\n",
       "<th>\n",
       "Ware, Mr. Frederick\n",
       "</th>\n",
       "<th>\n",
       "Warren, Mr. Charles William\n",
       "</th>\n",
       "<th>\n",
       "Webber, Mr. James\n",
       "</th>\n",
       "<th>\n",
       "Wenzel, Mr. Linhart\n",
       "</th>\n",
       "<th>\n",
       "Whabee, Mrs. George Joseph (Shawneene Abi-Saab)\n",
       "</th>\n",
       "<th>\n",
       "Widegren, Mr. Carl/Charles Peter\n",
       "</th>\n",
       "<th>\n",
       "Wiklund, Mr. Jakob Alfred\n",
       "</th>\n",
       "<th>\n",
       "Wiklund, Mr. Karl Johan\n",
       "</th>\n",
       "<th>\n",
       "Wilkes, Mrs. James (Ellen Needs)\n",
       "</th>\n",
       "<th>\n",
       "Willer, Mr. Aaron (\"Abi Weller\")\n",
       "</th>\n",
       "<th>\n",
       "Willey, Mr. Edward\n",
       "</th>\n",
       "<th>\n",
       "Williams, Mr. Howard Hugh \"Harry\"\n",
       "</th>\n",
       "<th>\n",
       "Williams, Mr. Leslie\n",
       "</th>\n",
       "<th>\n",
       "Windelov, Mr. Einar\n",
       "</th>\n",
       "<th>\n",
       "Wirz, Mr. Albert\n",
       "</th>\n",
       "<th>\n",
       "Wiseman, Mr. Phillippe\n",
       "</th>\n",
       "<th>\n",
       "Wittevrongel, Mr. Camille\n",
       "</th>\n",
       "<th>\n",
       "Yasbeck, Mr. Antoni\n",
       "</th>\n",
       "<th>\n",
       "Yasbeck, Mrs. Antoni (Selini Alexander)\n",
       "</th>\n",
       "<th>\n",
       "Youseff, Mr. Gerious\n",
       "</th>\n",
       "<th>\n",
       "Yousif, Mr. Wazli\n",
       "</th>\n",
       "<th>\n",
       "Yousseff, Mr. Gerious\n",
       "</th>\n",
       "<th>\n",
       "Zabour, Miss. Hileni\n",
       "</th>\n",
       "<th>\n",
       "Zabour, Miss. Thamine\n",
       "</th>\n",
       "<th>\n",
       "Zakarian, Mr. Mapriededer\n",
       "</th>\n",
       "<th>\n",
       "Zakarian, Mr. Ortin\n",
       "</th>\n",
       "<th>\n",
       "Zimmerman, Mr. Leo\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "25.0\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;E12&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "48.0\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "...\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 1308)\n",
       "┌─────────┬────────────┬────────────┬────────────┬─────┬────────────┬────────────┬────────────┬────────────┐\n",
       "│ cabin   ┆ Allen,     ┆ Allison,   ┆ Allison,   ┆ ... ┆ Zabour,    ┆ Zakarian,  ┆ Zakarian,  ┆ Zimmerman, │\n",
       "│ ---     ┆ Miss.      ┆ Master.    ┆ Miss.      ┆     ┆ Miss.      ┆ Mr. Maprie ┆ Mr. Ortin  ┆ Mr. Leo    │\n",
       "│ str     ┆ Elisabeth  ┆ Hudson     ┆ Helen      ┆     ┆ Thamine    ┆ deder      ┆ ---        ┆ ---        │\n",
       "│         ┆ Walton     ┆ Trevor     ┆ Loraine    ┆     ┆ ---        ┆ ---        ┆ f64        ┆ f64        │\n",
       "│         ┆ ---        ┆ ---        ┆ ---        ┆     ┆ f64        ┆ f64        ┆            ┆            │\n",
       "│         ┆ f64        ┆ f64        ┆ f64        ┆     ┆            ┆            ┆            ┆            │\n",
       "╞═════════╪════════════╪════════════╪════════════╪═════╪════════════╪════════════╪════════════╪════════════╡\n",
       "│ B5      ┆ 29.0       ┆ null       ┆ null       ┆ ... ┆ null       ┆ null       ┆ null       ┆ null       │\n",
       "│ C22 C26 ┆ null       ┆ 0.92       ┆ 2.0        ┆ ... ┆ null       ┆ null       ┆ null       ┆ null       │\n",
       "│ E12     ┆ null       ┆ null       ┆ null       ┆ ... ┆ null       ┆ null       ┆ null       ┆ null       │\n",
       "└─────────┴────────────┴────────────┴────────────┴─────┴────────────┴────────────┴────────────┴────────────┘"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .select([col('name'), col('cabin'), col('age')])\\\n",
    "    .pivot(values = 'age', index = 'cabin', columns = 'name')\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01020878-420b-4bcc-90ab-b87a59c5ad11",
   "metadata": {},
   "source": [
    "## **14. Operações com datas <a class=\"anchor\" id=\"header14\"></a>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "2b1c7666-4609-4b3c-ba67-7d86c4c0a3ff",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 1)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "drange\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "date\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-01\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-02\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-03\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 1)\n",
       "┌────────────┐\n",
       "│ drange     │\n",
       "│ ---        │\n",
       "│ date       │\n",
       "╞════════════╡\n",
       "│ 2022-01-01 │\n",
       "│ 2022-01-02 │\n",
       "│ 2022-01-03 │\n",
       "└────────────┘"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2f0ef81-b66b-4618-a01b-028eb16b754d",
   "metadata": {},
   "source": [
    "### **14.1. Agregações**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "4ba2ffce-076b-442a-a853-1e065bf67d3a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (1, 1)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "drange\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "date\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "2022-03-01\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (1, 1)\n",
       "┌────────────┐\n",
       "│ drange     │\n",
       "│ ---        │\n",
       "│ date       │\n",
       "╞════════════╡\n",
       "│ 2022-03-01 │\n",
       "└────────────┘"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates\\\n",
    "    .select(col('drange'))\\\n",
    "    .max()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5ec231d-6684-497a-9c90-9737acd651c4",
   "metadata": {},
   "source": [
    "### **14.2. Transformando para string e fazendo o inverso também**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "aace9ed8-063e-49f3-a1ee-02934b359e0d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 3)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "drange\n",
       "</th>\n",
       "<th>\n",
       "drange_string\n",
       "</th>\n",
       "<th>\n",
       "drange_new\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "date\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "date\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-01\n",
       "</td>\n",
       "<td>\n",
       "&quot;2022-01-01&quot;\n",
       "</td>\n",
       "<td>\n",
       "2022-01-01\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-02\n",
       "</td>\n",
       "<td>\n",
       "&quot;2022-01-02&quot;\n",
       "</td>\n",
       "<td>\n",
       "2022-01-02\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-01-03\n",
       "</td>\n",
       "<td>\n",
       "&quot;2022-01-03&quot;\n",
       "</td>\n",
       "<td>\n",
       "2022-01-03\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 3)\n",
       "┌────────────┬───────────────┬────────────┐\n",
       "│ drange     ┆ drange_string ┆ drange_new │\n",
       "│ ---        ┆ ---           ┆ ---        │\n",
       "│ date       ┆ str           ┆ date       │\n",
       "╞════════════╪═══════════════╪════════════╡\n",
       "│ 2022-01-01 ┆ 2022-01-01    ┆ 2022-01-01 │\n",
       "│ 2022-01-02 ┆ 2022-01-02    ┆ 2022-01-02 │\n",
       "│ 2022-01-03 ┆ 2022-01-03    ┆ 2022-01-03 │\n",
       "└────────────┴───────────────┴────────────┘"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates\\\n",
    "    .with_columns([\\\n",
    "                   (col('drange').cast(str)).alias('drange_string')\n",
    "                  ])\\\n",
    "    .with_columns((col('drange_string').str.strptime(Date, fmt = '%Y-%m-%d', strict = False).cast(Date)).alias('drange_new'))\\\n",
    "    .head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e19c9c5-a6c1-47bb-acd9-002249a5f048",
   "metadata": {},
   "source": [
    "### **14.3. Criando novas colunas de datas**\n",
    "\n",
    "**Para outras funções/métodos, ver esse [link](https://pola-rs.github.io/polars/py-polars/html/reference/series/timeseries.html).**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "a74d7a9d-00a2-43b2-920d-61bb84a17e9d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 7)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "drange\n",
       "</th>\n",
       "<th>\n",
       "day\n",
       "</th>\n",
       "<th>\n",
       "month\n",
       "</th>\n",
       "<th>\n",
       "year\n",
       "</th>\n",
       "<th>\n",
       "quarter\n",
       "</th>\n",
       "<th>\n",
       "week\n",
       "</th>\n",
       "<th>\n",
       "weekday\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "date\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "i32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "2022-02-27\n",
       "</td>\n",
       "<td>\n",
       "27\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "2022\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "8\n",
       "</td>\n",
       "<td>\n",
       "7\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-02-28\n",
       "</td>\n",
       "<td>\n",
       "28\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "2022\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "9\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2022-03-01\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "2022\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "9\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 7)\n",
       "┌────────────┬─────┬───────┬──────┬─────────┬──────┬─────────┐\n",
       "│ drange     ┆ day ┆ month ┆ year ┆ quarter ┆ week ┆ weekday │\n",
       "│ ---        ┆ --- ┆ ---   ┆ ---  ┆ ---     ┆ ---  ┆ ---     │\n",
       "│ date       ┆ u32 ┆ u32   ┆ i32  ┆ u32     ┆ u32  ┆ u32     │\n",
       "╞════════════╪═════╪═══════╪══════╪═════════╪══════╪═════════╡\n",
       "│ 2022-02-27 ┆ 27  ┆ 2     ┆ 2022 ┆ 1       ┆ 8    ┆ 7       │\n",
       "│ 2022-02-28 ┆ 28  ┆ 2     ┆ 2022 ┆ 1       ┆ 9    ┆ 1       │\n",
       "│ 2022-03-01 ┆ 1   ┆ 3     ┆ 2022 ┆ 1       ┆ 9    ┆ 2       │\n",
       "└────────────┴─────┴───────┴──────┴─────────┴──────┴─────────┘"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates\\\n",
    "    .with_columns([\\\n",
    "                   col('drange').dt.day().alias('day'),                   \n",
    "                   col('drange').dt.month().alias('month'),\n",
    "                   col('drange').dt.year().alias('year'),\n",
    "                   col('drange').dt.quarter().alias('quarter'),\n",
    "                   col('drange').dt.week().alias('week'),\n",
    "                   col('drange').dt.weekday().alias('weekday')\n",
    "                  ])\\\n",
    "    .tail(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56d2ffe3-ad1c-41fa-9dfc-0cf1d95799d8",
   "metadata": {},
   "source": [
    "## **15. Exportando seus dados <a class=\"anchor\" id=\"header15\"></a>**\n",
    "\n",
    "Não foge muito do que temos no Pandas também."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c192cfc-0ba5-4781-83c8-b9e8ee185515",
   "metadata": {},
   "source": [
    "### **15.1. CSV**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "026ad455-833a-420f-9387-fde31cd1f8f5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dados.write_csv('../02-data/output_csv.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99311e6f-92e6-4468-b810-fc6e0a5d21e6",
   "metadata": {},
   "source": [
    "### **15.2. XLSX**\n",
    "\n",
    "**Ainda** não existe uma função nativa para exportar para o formato xlsx. Então, o melhor jeito é transformar o dataframe Polars para Pandas e só aí fazer a exportação."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "6c854555-5527-4b89-8145-b0fd2cbbe72f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dados.to_pandas().to_excel('../02-data/output_xlsx.xlsx')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b86610cc-4791-4cbd-a8cb-c93c36c064bc",
   "metadata": {},
   "source": [
    "### **15.3. Parquet**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "6c105e10-3dc3-4787-b55e-e4a7ec45cab5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dados.write_parquet('../02-data/output_parquet.parquet')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdbc61da-ffcf-4953-bc71-8107aad24a77",
   "metadata": {
    "toc-hr-collapsed": true
   },
   "source": [
    "## **16. Criando e aplicando funções personalizadas <a class=\"anchor\" id=\"header16\"></a>**\n",
    "\n",
    "Vamos criar uma função personalizada que traz **somente a primeira palavra da variável name**. Assim como no `Pandas`, o nosso combo `.apply()` + `lambda(x)` irá nos salvar, só que de um jeito diferente."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "8ef8e95c-59dd-4421-881c-895c7a8488b5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_first_word(x):\n",
    "    \n",
    "    result = x.split(',', 1)[0]\n",
    "    \n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "592b8ea0-7b55-47b1-9e35-359e0a2cc13e",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (5, 16)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "pclass\n",
       "</th>\n",
       "<th>\n",
       "survived\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "sex\n",
       "</th>\n",
       "<th>\n",
       "age\n",
       "</th>\n",
       "<th>\n",
       "sibsp\n",
       "</th>\n",
       "<th>\n",
       "parch\n",
       "</th>\n",
       "<th>\n",
       "ticket\n",
       "</th>\n",
       "<th>\n",
       "fare\n",
       "</th>\n",
       "<th>\n",
       "cabin\n",
       "</th>\n",
       "<th>\n",
       "embarked\n",
       "</th>\n",
       "<th>\n",
       "boat\n",
       "</th>\n",
       "<th>\n",
       "body\n",
       "</th>\n",
       "<th>\n",
       "home.dest\n",
       "</th>\n",
       "<th>\n",
       "first_name\n",
       "</th>\n",
       "<th>\n",
       "first_word_home.dest\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen, Miss. E...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;24160&quot;\n",
       "</td>\n",
       "<td>\n",
       "211.3375\n",
       "</td>\n",
       "<td>\n",
       "&quot;B5&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis, MO&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allen&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;St Louis&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Maste...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "0.92\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;11&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Miss....\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "2.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mr. H...\n",
       "</td>\n",
       "<td>\n",
       "&quot;male&quot;\n",
       "</td>\n",
       "<td>\n",
       "30.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "135\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison, Mrs. ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;female&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.0\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;113781&quot;\n",
       "</td>\n",
       "<td>\n",
       "151.55\n",
       "</td>\n",
       "<td>\n",
       "&quot;C22 C26&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;S&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal, PQ /...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Allison&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Montreal&quot;\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (5, 16)\n",
       "┌────────┬──────────┬───────────────┬────────┬─────┬──────┬──────────────┬────────────┬────────────┐\n",
       "│ pclass ┆ survived ┆ name          ┆ sex    ┆ ... ┆ body ┆ home.dest    ┆ first_name ┆ first_word │\n",
       "│ ---    ┆ ---      ┆ ---           ┆ ---    ┆     ┆ ---  ┆ ---          ┆ ---        ┆ _home.dest │\n",
       "│ i64    ┆ i64      ┆ str           ┆ str    ┆     ┆ i64  ┆ str          ┆ str        ┆ ---        │\n",
       "│        ┆          ┆               ┆        ┆     ┆      ┆              ┆            ┆ str        │\n",
       "╞════════╪══════════╪═══════════════╪════════╪═════╪══════╪══════════════╪════════════╪════════════╡\n",
       "│ 1      ┆ 1        ┆ Allen, Miss.  ┆ female ┆ ... ┆ null ┆ St Louis, MO ┆ Allen      ┆ St Louis   │\n",
       "│        ┆          ┆ Elisabeth     ┆        ┆     ┆      ┆              ┆            ┆            │\n",
       "│        ┆          ┆ Walton        ┆        ┆     ┆      ┆              ┆            ┆            │\n",
       "│ 1      ┆ 1        ┆ Allison,      ┆ male   ┆ ... ┆ null ┆ Montreal, PQ ┆ Allison    ┆ Montreal   │\n",
       "│        ┆          ┆ Master.       ┆        ┆     ┆      ┆ / Chestervil ┆            ┆            │\n",
       "│        ┆          ┆ Hudson Trevor ┆        ┆     ┆      ┆ le, ON       ┆            ┆            │\n",
       "│ 1      ┆ 0        ┆ Allison,      ┆ female ┆ ... ┆ null ┆ Montreal, PQ ┆ Allison    ┆ Montreal   │\n",
       "│        ┆          ┆ Miss. Helen   ┆        ┆     ┆      ┆ / Chestervil ┆            ┆            │\n",
       "│        ┆          ┆ Loraine       ┆        ┆     ┆      ┆ le, ON       ┆            ┆            │\n",
       "│ 1      ┆ 0        ┆ Allison, Mr.  ┆ male   ┆ ... ┆ 135  ┆ Montreal, PQ ┆ Allison    ┆ Montreal   │\n",
       "│        ┆          ┆ Hudson Joshua ┆        ┆     ┆      ┆ / Chestervil ┆            ┆            │\n",
       "│        ┆          ┆ Creig...      ┆        ┆     ┆      ┆ le, ON       ┆            ┆            │\n",
       "│ 1      ┆ 0        ┆ Allison, Mrs. ┆ female ┆ ... ┆ null ┆ Montreal, PQ ┆ Allison    ┆ Montreal   │\n",
       "│        ┆          ┆ Hudson J C    ┆        ┆     ┆      ┆ / Chestervil ┆            ┆            │\n",
       "│        ┆          ┆ (Bessie...    ┆        ┆     ┆      ┆ le, ON       ┆            ┆            │\n",
       "└────────┴──────────┴───────────────┴────────┴─────┴──────┴──────────────┴────────────┴────────────┘"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dados\\\n",
    "    .with_columns([\\\n",
    "                   (col('name').apply(lambda x: get_first_word(x))).alias('first_name'),\n",
    "                   (col('home.dest').apply(lambda x: get_first_word(x))).alias('first_word_home.dest')\n",
    "                 ])\\\n",
    "    .head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21c0d906-abde-43d5-88b4-a13e6241fbba",
   "metadata": {},
   "source": [
    "## **17. Trabalhando com JSONs <a class=\"anchor\" id=\"header17\"></a>**\n",
    "\n",
    "Exemplo retirado **descaradamente** do [Stack Overflow](https://stackoverflow.com/questions/73120642/in-python-polars-convert-a-json-string-column-to-dict-for-filtering). E também ainda não existe uma função parecida com a `json_normalize` do Pandas 😢"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "3a2ac38f-f711-4458-b808-452167c64fda",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "json_list = [\n",
    "    \"\"\"{\"name\": \"Maria\",\n",
    "        \"position\": \"developer\",\n",
    "        \"office\": \"Seattle\"}\"\"\",\n",
    "    \"\"\"{\"name\": \"Josh\",\n",
    "        \"position\": \"analyst\",\n",
    "        \"termination_date\": \"2020-01-01\"}\"\"\",\n",
    "    \"\"\"{\"name\": \"Jorge\",\n",
    "        \"position\": \"architect\",\n",
    "        \"office\": \"\",\n",
    "        \"manager_st_dt\": \"2020-01-01\"}\"\"\",\n",
    "]\n",
    "\n",
    "df = DataFrame(\n",
    "    {\n",
    "        'tags': json_list,\n",
    "    }\n",
    ").with_row_count('id', 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "651ad762-e62f-46ef-939e-23787c687dc3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 5)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "id\n",
       "</th>\n",
       "<th>\n",
       "tags\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "location\n",
       "</th>\n",
       "<th>\n",
       "manager_start_date\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;{&quot;name&quot;: &quot;Mari...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Maria&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Seattle&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;{&quot;name&quot;: &quot;Josh...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Josh&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "&quot;{&quot;name&quot;: &quot;Jorg...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Jorge&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2020-01-01&quot;\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 5)\n",
       "┌─────┬────────────────────┬───────┬──────────┬────────────────────┐\n",
       "│ id  ┆ tags               ┆ name  ┆ location ┆ manager_start_date │\n",
       "│ --- ┆ ---                ┆ ---   ┆ ---      ┆ ---                │\n",
       "│ u32 ┆ str                ┆ str   ┆ str      ┆ str                │\n",
       "╞═════╪════════════════════╪═══════╪══════════╪════════════════════╡\n",
       "│ 1   ┆ {\"name\": \"Maria\",  ┆ Maria ┆ Seattle  ┆ null               │\n",
       "│     ┆         \"posit...  ┆       ┆          ┆                    │\n",
       "│ 2   ┆ {\"name\": \"Josh\",   ┆ Josh  ┆ null     ┆ null               │\n",
       "│     ┆         \"positi... ┆       ┆          ┆                    │\n",
       "│ 3   ┆ {\"name\": \"Jorge\",  ┆ Jorge ┆          ┆ 2020-01-01         │\n",
       "│     ┆         \"posit...  ┆       ┆          ┆                    │\n",
       "└─────┴────────────────────┴───────┴──────────┴────────────────────┘"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Abrindo o JSON em cada uma das chaves\n",
    "\n",
    "df.with_columns([\n",
    "    col('tags').str.json_path_match(r'$.name').alias('name'),\n",
    "    col('tags').str.json_path_match(r'$.office').alias('location'),\n",
    "    col('tags').str.json_path_match(r'$.manager_st_dt').alias('manager_start_date'),\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "587a469a-c999-406c-97a3-420a94bc9fb1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 5)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "manager_st_dt\n",
       "</th>\n",
       "<th>\n",
       "name\n",
       "</th>\n",
       "<th>\n",
       "office\n",
       "</th>\n",
       "<th>\n",
       "position\n",
       "</th>\n",
       "<th>\n",
       "termination_date\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Maria&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Seattle&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;developer&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;Josh&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "&quot;analyst&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;2020-01-01&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "&quot;2020-01-01&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Jorge&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;architect&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 5)\n",
       "┌───────────────┬───────┬─────────┬───────────┬──────────────────┐\n",
       "│ manager_st_dt ┆ name  ┆ office  ┆ position  ┆ termination_date │\n",
       "│ ---           ┆ ---   ┆ ---     ┆ ---       ┆ ---              │\n",
       "│ str           ┆ str   ┆ str     ┆ str       ┆ str              │\n",
       "╞═══════════════╪═══════╪═════════╪═══════════╪══════════════════╡\n",
       "│ null          ┆ Maria ┆ Seattle ┆ developer ┆ null             │\n",
       "│ null          ┆ Josh  ┆ null    ┆ analyst   ┆ 2020-01-01       │\n",
       "│ 2020-01-01    ┆ Jorge ┆         ┆ architect ┆ null             │\n",
       "└───────────────┴───────┴─────────┴───────────┴──────────────────┘"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Abrindo todas as chaves do JSON de uma vez (não testado em JSON dentro de JSON)\n",
    "\n",
    "df.select(col('tags').apply(json.loads)).unnest('tags')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79ad68b0-4e5a-47fb-84e1-70b529e88b25",
   "metadata": {},
   "source": [
    "## **18. Window functions <a class=\"anchor\" id=\"header18\"></a>**\n",
    "\n",
    "Conheci essa ideia de Window Function dentro do SQL, com as funções mais básicas de `row_number` e `rank`. Se eu pudesse traduzi-la para quem não conhece, seria um `groupby` com mais funcionalidades. Como teoria, recomendo ler esse [link de como usar window functions em SQL](https://portosql.wordpress.com/2018/10/14/funcoes-de-janela-window-functions/).\n",
    "\n",
    "Vou utilizar como exemplo o dataset que está na [documentação do Polars](https://pola-rs.github.io/polars-book/user-guide/dsl/window_functions.html). Esse dataset está no contexto do desenho Pokémon e está na granularidade de cada um dos bichinhos (pokémons) junto às suas características, como velocidade, ataque e etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "3e5c2cf8-4042-4e2e-a940-cc6656380924",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (5, 13)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "#\n",
       "</th>\n",
       "<th>\n",
       "Name\n",
       "</th>\n",
       "<th>\n",
       "Type 1\n",
       "</th>\n",
       "<th>\n",
       "Type 2\n",
       "</th>\n",
       "<th>\n",
       "Total\n",
       "</th>\n",
       "<th>\n",
       "HP\n",
       "</th>\n",
       "<th>\n",
       "Attack\n",
       "</th>\n",
       "<th>\n",
       "Defense\n",
       "</th>\n",
       "<th>\n",
       "Sp. Atk\n",
       "</th>\n",
       "<th>\n",
       "Sp. Def\n",
       "</th>\n",
       "<th>\n",
       "Speed\n",
       "</th>\n",
       "<th>\n",
       "Generation\n",
       "</th>\n",
       "<th>\n",
       "Legendary\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "bool\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "&quot;Bulbasaur&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Grass&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Poison&quot;\n",
       "</td>\n",
       "<td>\n",
       "318\n",
       "</td>\n",
       "<td>\n",
       "45\n",
       "</td>\n",
       "<td>\n",
       "49\n",
       "</td>\n",
       "<td>\n",
       "49\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "45\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "<td>\n",
       "&quot;Ivysaur&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Grass&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Poison&quot;\n",
       "</td>\n",
       "<td>\n",
       "405\n",
       "</td>\n",
       "<td>\n",
       "60\n",
       "</td>\n",
       "<td>\n",
       "62\n",
       "</td>\n",
       "<td>\n",
       "63\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "60\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "&quot;Venusaur&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Grass&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Poison&quot;\n",
       "</td>\n",
       "<td>\n",
       "525\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "82\n",
       "</td>\n",
       "<td>\n",
       "83\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "<td>\n",
       "&quot;VenusaurMega V...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Grass&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Poison&quot;\n",
       "</td>\n",
       "<td>\n",
       "625\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "123\n",
       "</td>\n",
       "<td>\n",
       "122\n",
       "</td>\n",
       "<td>\n",
       "120\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "4\n",
       "</td>\n",
       "<td>\n",
       "&quot;Charmander&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "309\n",
       "</td>\n",
       "<td>\n",
       "39\n",
       "</td>\n",
       "<td>\n",
       "52\n",
       "</td>\n",
       "<td>\n",
       "43\n",
       "</td>\n",
       "<td>\n",
       "60\n",
       "</td>\n",
       "<td>\n",
       "50\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (5, 13)\n",
       "┌─────┬───────────────────────┬────────┬────────┬─────┬─────────┬───────┬────────────┬───────────┐\n",
       "│ #   ┆ Name                  ┆ Type 1 ┆ Type 2 ┆ ... ┆ Sp. Def ┆ Speed ┆ Generation ┆ Legendary │\n",
       "│ --- ┆ ---                   ┆ ---    ┆ ---    ┆     ┆ ---     ┆ ---   ┆ ---        ┆ ---       │\n",
       "│ i64 ┆ str                   ┆ str    ┆ str    ┆     ┆ i64     ┆ i64   ┆ i64        ┆ bool      │\n",
       "╞═════╪═══════════════════════╪════════╪════════╪═════╪═════════╪═══════╪════════════╪═══════════╡\n",
       "│ 1   ┆ Bulbasaur             ┆ Grass  ┆ Poison ┆ ... ┆ 65      ┆ 45    ┆ 1          ┆ false     │\n",
       "│ 2   ┆ Ivysaur               ┆ Grass  ┆ Poison ┆ ... ┆ 80      ┆ 60    ┆ 1          ┆ false     │\n",
       "│ 3   ┆ Venusaur              ┆ Grass  ┆ Poison ┆ ... ┆ 100     ┆ 80    ┆ 1          ┆ false     │\n",
       "│ 3   ┆ VenusaurMega Venusaur ┆ Grass  ┆ Poison ┆ ... ┆ 120     ┆ 80    ┆ 1          ┆ false     │\n",
       "│ 4   ┆ Charmander            ┆ Fire   ┆ null   ┆ ... ┆ 50      ┆ 65    ┆ 1          ┆ false     │\n",
       "└─────┴───────────────────────┴────────┴────────┴─────┴─────────┴───────┴────────────┴───────────┘"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = read_csv(\n",
    "    \"https://gist.githubusercontent.com/ritchie46/cac6b337ea52281aa23c049250a4ff03/raw/89a957ff3919d90e6ef2d34235e6bf22304f3366/pokemon.csv\"\n",
    ")\n",
    "\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3b06078-8014-4034-b2e5-d8736969e79e",
   "metadata": {},
   "source": [
    "### **18.1. Row number <a class=\"anchor\" id=\"header18_1\"></a>**\n",
    "\n",
    "Vamos criar uma coluna que conta a quantidade de cada um dos **Type 1** disponíveis.\n",
    "\n",
    "Abaixo filtrei somente os pokémons onde o Type 1 é Fire. Notem que a coluna \"#\", que representa um ID da linha, mostra que a contagem das linhas continuou mesmo após um longo pulo das linhas. Em SQL, é equivalente a:\n",
    "\n",
    "> ROW_NUMBER() OVER (PARTITION BY Type1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "c9be54e1-6302-44e7-89f9-5c0e908044a7",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (14, 4)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "#\n",
       "</th>\n",
       "<th>\n",
       "Name\n",
       "</th>\n",
       "<th>\n",
       "Type 1\n",
       "</th>\n",
       "<th>\n",
       "row_number_type1\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i32\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "4\n",
       "</td>\n",
       "<td>\n",
       "&quot;Charmander&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "5\n",
       "</td>\n",
       "<td>\n",
       "&quot;Charmeleon&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "6\n",
       "</td>\n",
       "<td>\n",
       "&quot;Charizard&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "6\n",
       "</td>\n",
       "<td>\n",
       "&quot;CharizardMega ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "4\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "6\n",
       "</td>\n",
       "<td>\n",
       "&quot;CharizardMega ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "5\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "37\n",
       "</td>\n",
       "<td>\n",
       "&quot;Vulpix&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "6\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "38\n",
       "</td>\n",
       "<td>\n",
       "&quot;Ninetales&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "7\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "58\n",
       "</td>\n",
       "<td>\n",
       "&quot;Growlithe&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "8\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "59\n",
       "</td>\n",
       "<td>\n",
       "&quot;Arcanine&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "9\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "77\n",
       "</td>\n",
       "<td>\n",
       "&quot;Ponyta&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "10\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "78\n",
       "</td>\n",
       "<td>\n",
       "&quot;Rapidash&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "11\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "126\n",
       "</td>\n",
       "<td>\n",
       "&quot;Magmar&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "12\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "136\n",
       "</td>\n",
       "<td>\n",
       "&quot;Flareon&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "13\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "146\n",
       "</td>\n",
       "<td>\n",
       "&quot;Moltres&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "14\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (14, 4)\n",
       "┌─────┬───────────────────────────┬────────┬──────────────────┐\n",
       "│ #   ┆ Name                      ┆ Type 1 ┆ row_number_type1 │\n",
       "│ --- ┆ ---                       ┆ ---    ┆ ---              │\n",
       "│ i64 ┆ str                       ┆ str    ┆ i32              │\n",
       "╞═════╪═══════════════════════════╪════════╪══════════════════╡\n",
       "│ 4   ┆ Charmander                ┆ Fire   ┆ 1                │\n",
       "│ 5   ┆ Charmeleon                ┆ Fire   ┆ 2                │\n",
       "│ 6   ┆ Charizard                 ┆ Fire   ┆ 3                │\n",
       "│ 6   ┆ CharizardMega Charizard X ┆ Fire   ┆ 4                │\n",
       "│ ... ┆ ...                       ┆ ...    ┆ ...              │\n",
       "│ 78  ┆ Rapidash                  ┆ Fire   ┆ 11               │\n",
       "│ 126 ┆ Magmar                    ┆ Fire   ┆ 12               │\n",
       "│ 136 ┆ Flareon                   ┆ Fire   ┆ 13               │\n",
       "│ 146 ┆ Moltres                   ┆ Fire   ┆ 14               │\n",
       "└─────┴───────────────────────────┴────────┴──────────────────┘"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df\\\n",
    "    .with_columns(\\\n",
    "                  lit(1).alias('constante')\n",
    "                  )\\\n",
    "    .select([\\\n",
    "             col('#'),\n",
    "             col('Name'),\n",
    "             col('Type 1'),\n",
    "             col('constante').cumsum().over('Type 1').alias('row_number_type1') # Criação efetiva do row_number\n",
    "           ])\\\n",
    "    .filter(col('Type 1') == 'Fire')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72654fc7-5b67-4900-ad98-4f2d83935aea",
   "metadata": {},
   "source": [
    "### **18.2. Rank <a class=\"anchor\" id=\"header18_2\"></a>**\n",
    "\n",
    "A ideia agora é **rankear** os tipos de Pokémons (Type 1) por seu ataque (Attack). Para isso, vamos criar uma coluna que faz essa indicação de qual a linha possui qual rank.\n",
    "\n",
    "> RANK(Attack) OVER (PARTITION BY Type1 ORDER BY Attack ASC)\n",
    "\n",
    "Notem que foi criado uma nova coluna chamada `rank_attack_by_type1` que mostra qual o rank daquela linha baseado no ataque e \"agrupada\" para cada um dos `Type 1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "b4f1a4dc-b229-49a2-b36d-d4ce9809e886",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (14, 14)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "#\n",
       "</th>\n",
       "<th>\n",
       "Name\n",
       "</th>\n",
       "<th>\n",
       "Type 1\n",
       "</th>\n",
       "<th>\n",
       "Type 2\n",
       "</th>\n",
       "<th>\n",
       "Total\n",
       "</th>\n",
       "<th>\n",
       "HP\n",
       "</th>\n",
       "<th>\n",
       "Attack\n",
       "</th>\n",
       "<th>\n",
       "Defense\n",
       "</th>\n",
       "<th>\n",
       "Sp. Atk\n",
       "</th>\n",
       "<th>\n",
       "Sp. Def\n",
       "</th>\n",
       "<th>\n",
       "Speed\n",
       "</th>\n",
       "<th>\n",
       "Generation\n",
       "</th>\n",
       "<th>\n",
       "Legendary\n",
       "</th>\n",
       "<th>\n",
       "rank_attack_by_type1\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "i64\n",
       "</td>\n",
       "<td>\n",
       "bool\n",
       "</td>\n",
       "<td>\n",
       "u32\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "4\n",
       "</td>\n",
       "<td>\n",
       "&quot;Charmander&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "309\n",
       "</td>\n",
       "<td>\n",
       "39\n",
       "</td>\n",
       "<td>\n",
       "52\n",
       "</td>\n",
       "<td>\n",
       "43\n",
       "</td>\n",
       "<td>\n",
       "60\n",
       "</td>\n",
       "<td>\n",
       "50\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "2\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "5\n",
       "</td>\n",
       "<td>\n",
       "&quot;Charmeleon&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "405\n",
       "</td>\n",
       "<td>\n",
       "58\n",
       "</td>\n",
       "<td>\n",
       "64\n",
       "</td>\n",
       "<td>\n",
       "58\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "3\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "6\n",
       "</td>\n",
       "<td>\n",
       "&quot;Charizard&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Flying&quot;\n",
       "</td>\n",
       "<td>\n",
       "534\n",
       "</td>\n",
       "<td>\n",
       "78\n",
       "</td>\n",
       "<td>\n",
       "84\n",
       "</td>\n",
       "<td>\n",
       "78\n",
       "</td>\n",
       "<td>\n",
       "109\n",
       "</td>\n",
       "<td>\n",
       "85\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "6\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "6\n",
       "</td>\n",
       "<td>\n",
       "&quot;CharizardMega ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Dragon&quot;\n",
       "</td>\n",
       "<td>\n",
       "634\n",
       "</td>\n",
       "<td>\n",
       "78\n",
       "</td>\n",
       "<td>\n",
       "130\n",
       "</td>\n",
       "<td>\n",
       "111\n",
       "</td>\n",
       "<td>\n",
       "130\n",
       "</td>\n",
       "<td>\n",
       "85\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "12\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "6\n",
       "</td>\n",
       "<td>\n",
       "&quot;CharizardMega ...\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Flying&quot;\n",
       "</td>\n",
       "<td>\n",
       "634\n",
       "</td>\n",
       "<td>\n",
       "78\n",
       "</td>\n",
       "<td>\n",
       "104\n",
       "</td>\n",
       "<td>\n",
       "78\n",
       "</td>\n",
       "<td>\n",
       "159\n",
       "</td>\n",
       "<td>\n",
       "115\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "10\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "37\n",
       "</td>\n",
       "<td>\n",
       "&quot;Vulpix&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "299\n",
       "</td>\n",
       "<td>\n",
       "38\n",
       "</td>\n",
       "<td>\n",
       "41\n",
       "</td>\n",
       "<td>\n",
       "40\n",
       "</td>\n",
       "<td>\n",
       "50\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "38\n",
       "</td>\n",
       "<td>\n",
       "&quot;Ninetales&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "505\n",
       "</td>\n",
       "<td>\n",
       "73\n",
       "</td>\n",
       "<td>\n",
       "76\n",
       "</td>\n",
       "<td>\n",
       "75\n",
       "</td>\n",
       "<td>\n",
       "81\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "5\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "58\n",
       "</td>\n",
       "<td>\n",
       "&quot;Growlithe&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "350\n",
       "</td>\n",
       "<td>\n",
       "55\n",
       "</td>\n",
       "<td>\n",
       "70\n",
       "</td>\n",
       "<td>\n",
       "45\n",
       "</td>\n",
       "<td>\n",
       "70\n",
       "</td>\n",
       "<td>\n",
       "50\n",
       "</td>\n",
       "<td>\n",
       "60\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "4\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "59\n",
       "</td>\n",
       "<td>\n",
       "&quot;Arcanine&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "555\n",
       "</td>\n",
       "<td>\n",
       "90\n",
       "</td>\n",
       "<td>\n",
       "110\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "95\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "11\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "77\n",
       "</td>\n",
       "<td>\n",
       "&quot;Ponyta&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "410\n",
       "</td>\n",
       "<td>\n",
       "50\n",
       "</td>\n",
       "<td>\n",
       "85\n",
       "</td>\n",
       "<td>\n",
       "55\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "90\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "7\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "78\n",
       "</td>\n",
       "<td>\n",
       "&quot;Rapidash&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "500\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "70\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "80\n",
       "</td>\n",
       "<td>\n",
       "105\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "9\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "126\n",
       "</td>\n",
       "<td>\n",
       "&quot;Magmar&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "495\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "95\n",
       "</td>\n",
       "<td>\n",
       "57\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "85\n",
       "</td>\n",
       "<td>\n",
       "93\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "8\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "136\n",
       "</td>\n",
       "<td>\n",
       "&quot;Flareon&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "null\n",
       "</td>\n",
       "<td>\n",
       "525\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "130\n",
       "</td>\n",
       "<td>\n",
       "60\n",
       "</td>\n",
       "<td>\n",
       "95\n",
       "</td>\n",
       "<td>\n",
       "110\n",
       "</td>\n",
       "<td>\n",
       "65\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "false\n",
       "</td>\n",
       "<td>\n",
       "12\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "146\n",
       "</td>\n",
       "<td>\n",
       "&quot;Moltres&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Fire&quot;\n",
       "</td>\n",
       "<td>\n",
       "&quot;Flying&quot;\n",
       "</td>\n",
       "<td>\n",
       "580\n",
       "</td>\n",
       "<td>\n",
       "90\n",
       "</td>\n",
       "<td>\n",
       "100\n",
       "</td>\n",
       "<td>\n",
       "90\n",
       "</td>\n",
       "<td>\n",
       "125\n",
       "</td>\n",
       "<td>\n",
       "85\n",
       "</td>\n",
       "<td>\n",
       "90\n",
       "</td>\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "true\n",
       "</td>\n",
       "<td>\n",
       "9\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (14, 14)\n",
       "┌─────┬─────────────────┬────────┬────────┬─────┬───────┬────────────┬───────────┬─────────────────┐\n",
       "│ #   ┆ Name            ┆ Type 1 ┆ Type 2 ┆ ... ┆ Speed ┆ Generation ┆ Legendary ┆ rank_attack_by_ │\n",
       "│ --- ┆ ---             ┆ ---    ┆ ---    ┆     ┆ ---   ┆ ---        ┆ ---       ┆ type1           │\n",
       "│ i64 ┆ str             ┆ str    ┆ str    ┆     ┆ i64   ┆ i64        ┆ bool      ┆ ---             │\n",
       "│     ┆                 ┆        ┆        ┆     ┆       ┆            ┆           ┆ u32             │\n",
       "╞═════╪═════════════════╪════════╪════════╪═════╪═══════╪════════════╪═══════════╪═════════════════╡\n",
       "│ 4   ┆ Charmander      ┆ Fire   ┆ null   ┆ ... ┆ 65    ┆ 1          ┆ false     ┆ 2               │\n",
       "│ 5   ┆ Charmeleon      ┆ Fire   ┆ null   ┆ ... ┆ 80    ┆ 1          ┆ false     ┆ 3               │\n",
       "│ 6   ┆ Charizard       ┆ Fire   ┆ Flying ┆ ... ┆ 100   ┆ 1          ┆ false     ┆ 6               │\n",
       "│ 6   ┆ CharizardMega   ┆ Fire   ┆ Dragon ┆ ... ┆ 100   ┆ 1          ┆ false     ┆ 12              │\n",
       "│     ┆ Charizard X     ┆        ┆        ┆     ┆       ┆            ┆           ┆                 │\n",
       "│ ... ┆ ...             ┆ ...    ┆ ...    ┆ ... ┆ ...   ┆ ...        ┆ ...       ┆ ...             │\n",
       "│ 78  ┆ Rapidash        ┆ Fire   ┆ null   ┆ ... ┆ 105   ┆ 1          ┆ false     ┆ 9               │\n",
       "│ 126 ┆ Magmar          ┆ Fire   ┆ null   ┆ ... ┆ 93    ┆ 1          ┆ false     ┆ 8               │\n",
       "│ 136 ┆ Flareon         ┆ Fire   ┆ null   ┆ ... ┆ 65    ┆ 1          ┆ false     ┆ 12              │\n",
       "│ 146 ┆ Moltres         ┆ Fire   ┆ Flying ┆ ... ┆ 90    ┆ 1          ┆ true      ┆ 9               │\n",
       "└─────┴─────────────────┴────────┴────────┴─────┴───────┴────────────┴───────────┴─────────────────┘"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df\\\n",
    "    .with_columns(\\\n",
    "                 (col('Attack').rank('dense').over('Type 1')).alias('rank_attack_by_type1')\n",
    "                 )\\\n",
    "    .filter(col('Type 1') == 'Fire')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f76c4b1-c6cd-4974-90f4-062e4fa76525",
   "metadata": {
    "tags": []
   },
   "source": [
    "## **19. Um pouco de teoria <a class=\"anchor\" id=\"header19\"></a>**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ac75674-9c73-41fa-b846-e0232e3b4255",
   "metadata": {},
   "source": [
    "### **19.1. Alguns pontos importantes <a class=\"anchor\" id=\"header19_1\"></a>**\n",
    "\n",
    "- **Polars** é escrito em Rust + Arrow para otimizar o código. Diferente do Pandas que é escrito em C. Pelos boatos que o pessoal comenta por aí, Rust poderá ser uma alternativa ao Python para Machine Learning no futuro\n",
    "- Otimizar o código, utilizar todos cores do seu CPU (não tenho info sobre GPU) e fazer com você consiga trabalhar com aquele dataset que é maior que a sua memória RAM são os objetivos principais do **Polars**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "164e4a40-4f0f-4e0a-a816-f52e6965eee0",
   "metadata": {},
   "source": [
    "### **19.2. Entendendo o Lazy Evaluation <a class=\"anchor\" id=\"header19_2\"></a>**\n",
    "\n",
    "Usuários de Spark já conhecem.\n",
    "\n",
    "- **Eager evaluation**: quando você salva na memória os dados e todas suas manipulações/modificações. Algo bem parecido com o que o Pandas faz atualmente\n",
    "- **Lazy evaluation**: quando você aplica todas suas manipulações/modificações nos dados mas essa execução só é feita quando você desejar. Ex.: você cria novas colunas e aplica filtros, mas você só executa TUDO isso quando você desejar (no final). É bom e ruim ao mesmo tempo. E bem chatinho de explicar, portanto, sigam esse [link para mais detalhes](https://towardsdatascience.com/3-reasons-why-sparks-lazy-evaluation-is-useful-ed06e27360c4#:~:text=Lazy%20Evaluation%20is%20an%20evaluation,used%20in%20most%20programming%20languages.)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb4921e3-db33-4249-aaba-c117f0518bbe",
   "metadata": {},
   "source": [
    "**Abaixo como podemos deixar o nosso dataframe como um objetivo **lazy** dentro do Polars, adicionando apenas uma linha de comando.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "351c89c6-f151-4ccb-b049-667eb36fe902",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (3, 5)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "sepal_length\n",
       "</th>\n",
       "<th>\n",
       "sepal_width\n",
       "</th>\n",
       "<th>\n",
       "petal_length\n",
       "</th>\n",
       "<th>\n",
       "petal_width\n",
       "</th>\n",
       "<th>\n",
       "species\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "5.1\n",
       "</td>\n",
       "<td>\n",
       "3.5\n",
       "</td>\n",
       "<td>\n",
       "1.4\n",
       "</td>\n",
       "<td>\n",
       "0.2\n",
       "</td>\n",
       "<td>\n",
       "&quot;setosa&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "4.9\n",
       "</td>\n",
       "<td>\n",
       "3.0\n",
       "</td>\n",
       "<td>\n",
       "1.4\n",
       "</td>\n",
       "<td>\n",
       "0.2\n",
       "</td>\n",
       "<td>\n",
       "&quot;setosa&quot;\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "4.7\n",
       "</td>\n",
       "<td>\n",
       "3.2\n",
       "</td>\n",
       "<td>\n",
       "1.3\n",
       "</td>\n",
       "<td>\n",
       "0.2\n",
       "</td>\n",
       "<td>\n",
       "&quot;setosa&quot;\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (3, 5)\n",
       "┌──────────────┬─────────────┬──────────────┬─────────────┬─────────┐\n",
       "│ sepal_length ┆ sepal_width ┆ petal_length ┆ petal_width ┆ species │\n",
       "│ ---          ┆ ---         ┆ ---          ┆ ---         ┆ ---     │\n",
       "│ f64          ┆ f64         ┆ f64          ┆ f64         ┆ str     │\n",
       "╞══════════════╪═════════════╪══════════════╪═════════════╪═════════╡\n",
       "│ 5.1          ┆ 3.5         ┆ 1.4          ┆ 0.2         ┆ setosa  │\n",
       "│ 4.9          ┆ 3.0         ┆ 1.4          ┆ 0.2         ┆ setosa  │\n",
       "│ 4.7          ┆ 3.2         ┆ 1.3          ┆ 0.2         ┆ setosa  │\n",
       "└──────────────┴─────────────┴──────────────┴─────────────┴─────────┘"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iris = read_csv(\"https://j.mp/iriscsv\")\n",
    "\n",
    "iris.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "c53ef5ab-4a39-47a7-b165-8bab32cd71b9",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<i>naive plan: (run <b>LazyFrame.describe_optimized_plan()</b> to see the optimized plan)</i>\n",
       "    <p></p>\n",
       "    <div>  DF [\"sepal_length\", \"sepal_width\", \"petal_length\", \"petal_width\"]; PROJECT */5 COLUMNS; SELECTION: \"None\"<p></p></div>"
      ],
      "text/plain": [
       "<polars.LazyFrame object at 0x7F6C033D3040>"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Notem que ele foi para um \"Naive Plan\"\n",
    "\n",
    "iris\\\n",
    "    .lazy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "a3486379-2cc6-4df9-a460-4209e7e348ad",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<i>naive plan: (run <b>LazyFrame.describe_optimized_plan()</b> to see the optimized plan)</i>\n",
       "    <p></p>\n",
       "    <div>   WITH_COLUMNS:<p></p>   [[(col(\"sepal_length\")) * (5i32)].alias(\"new_sepal_length\")]<p></p>    DF [\"sepal_length\", \"sepal_width\", \"petal_length\", \"petal_width\"]; PROJECT */5 COLUMNS; SELECTION: \"None\"<p></p></div>"
      ],
      "text/plain": [
       "<polars.LazyFrame object at 0x7F6C033D1DE0>"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Criei uma coluna nova e mesmo assim ele não me mostrou o output\n",
    "\n",
    "iris\\\n",
    "    .lazy()\\\n",
    "    .with_columns(\\\n",
    "                 (col('sepal_length') * 5).alias('new_sepal_length')\n",
    "                 )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9745b149-565b-4d2c-a110-5050db4b08e8",
   "metadata": {},
   "source": [
    "**Para recuperar o seu dataframe após as manipulações/modificações nos dados, basta você utilizar um `.collect()`.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "1742e2ee-af6e-4f65-95b4-2ec4e9a9175d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       ".pl-dataframe > thead > tr > th {\n",
       "  text-align: right;\n",
       "}\n",
       "</style>\n",
       "\n",
       "<table border=\"1\" class=\"pl-dataframe\">\n",
       "<small>shape: (5, 6)</small>\n",
       "<thead>\n",
       "<tr>\n",
       "<th>\n",
       "sepal_length\n",
       "</th>\n",
       "<th>\n",
       "sepal_width\n",
       "</th>\n",
       "<th>\n",
       "petal_length\n",
       "</th>\n",
       "<th>\n",
       "petal_width\n",
       "</th>\n",
       "<th>\n",
       "species\n",
       "</th>\n",
       "<th>\n",
       "new_sepal_length\n",
       "</th>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "<td>\n",
       "str\n",
       "</td>\n",
       "<td>\n",
       "f64\n",
       "</td>\n",
       "</tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr>\n",
       "<td>\n",
       "5.1\n",
       "</td>\n",
       "<td>\n",
       "3.5\n",
       "</td>\n",
       "<td>\n",
       "1.4\n",
       "</td>\n",
       "<td>\n",
       "0.2\n",
       "</td>\n",
       "<td>\n",
       "&quot;setosa&quot;\n",
       "</td>\n",
       "<td>\n",
       "25.5\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "5.4\n",
       "</td>\n",
       "<td>\n",
       "3.9\n",
       "</td>\n",
       "<td>\n",
       "1.7\n",
       "</td>\n",
       "<td>\n",
       "0.4\n",
       "</td>\n",
       "<td>\n",
       "&quot;setosa&quot;\n",
       "</td>\n",
       "<td>\n",
       "27.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "5.4\n",
       "</td>\n",
       "<td>\n",
       "3.7\n",
       "</td>\n",
       "<td>\n",
       "1.5\n",
       "</td>\n",
       "<td>\n",
       "0.2\n",
       "</td>\n",
       "<td>\n",
       "&quot;setosa&quot;\n",
       "</td>\n",
       "<td>\n",
       "27.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "5.8\n",
       "</td>\n",
       "<td>\n",
       "4.0\n",
       "</td>\n",
       "<td>\n",
       "1.2\n",
       "</td>\n",
       "<td>\n",
       "0.2\n",
       "</td>\n",
       "<td>\n",
       "&quot;setosa&quot;\n",
       "</td>\n",
       "<td>\n",
       "29.0\n",
       "</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td>\n",
       "5.7\n",
       "</td>\n",
       "<td>\n",
       "4.4\n",
       "</td>\n",
       "<td>\n",
       "1.5\n",
       "</td>\n",
       "<td>\n",
       "0.4\n",
       "</td>\n",
       "<td>\n",
       "&quot;setosa&quot;\n",
       "</td>\n",
       "<td>\n",
       "28.5\n",
       "</td>\n",
       "</tr>\n",
       "</tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "shape: (5, 6)\n",
       "┌──────────────┬─────────────┬──────────────┬─────────────┬─────────┬──────────────────┐\n",
       "│ sepal_length ┆ sepal_width ┆ petal_length ┆ petal_width ┆ species ┆ new_sepal_length │\n",
       "│ ---          ┆ ---         ┆ ---          ┆ ---         ┆ ---     ┆ ---              │\n",
       "│ f64          ┆ f64         ┆ f64          ┆ f64         ┆ str     ┆ f64              │\n",
       "╞══════════════╪═════════════╪══════════════╪═════════════╪═════════╪══════════════════╡\n",
       "│ 5.1          ┆ 3.5         ┆ 1.4          ┆ 0.2         ┆ setosa  ┆ 25.5             │\n",
       "│ 5.4          ┆ 3.9         ┆ 1.7          ┆ 0.4         ┆ setosa  ┆ 27.0             │\n",
       "│ 5.4          ┆ 3.7         ┆ 1.5          ┆ 0.2         ┆ setosa  ┆ 27.0             │\n",
       "│ 5.8          ┆ 4.0         ┆ 1.2          ┆ 0.2         ┆ setosa  ┆ 29.0             │\n",
       "│ 5.7          ┆ 4.4         ┆ 1.5          ┆ 0.4         ┆ setosa  ┆ 28.5             │\n",
       "└──────────────┴─────────────┴──────────────┴─────────────┴─────────┴──────────────────┘"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#---- Utilizando o collect para recuperar o dataframe\n",
    "\n",
    "iris\\\n",
    "    .lazy()\\\n",
    "    .with_columns(\\\n",
    "                 (col('sepal_length') * 5).alias('new_sepal_length')\n",
    "                 )\\\n",
    "    .filter(col('new_sepal_length') > 25)\\\n",
    "    .collect()\\\n",
    "    .head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c418a45e-508b-4cbd-a120-247c207eeda4",
   "metadata": {
    "tags": []
   },
   "source": [
    "## **20. Bônus e referências <a class=\"anchor\" id=\"header20\"></a>**\n",
    "\n",
    "Dica da [Cenobita](https://twitter.com/Inferente3) que recentemente precisou testar o Polars para uma tarefa de tratar dados, segundo ela, enormes. \n",
    "\n",
    "Mesmo que seus dados não caibam na sua memória, você consegue aplicar toda sua manipulação, filtros e tudo mais nesses dados e salvar o resultado em um arquivo `parquet`. E tudo isso é possível graças a combinação do `lazy()` com o `sink_parquet()`.\n",
    "\n",
    "Para mais detalhes, veja este link sobre [\"Sinking larger-than-memory Parquet files\"](https://www.rhosignal.com/posts/sink-parquet-files/).\n",
    "\n",
    "Na prática, como ele funciona:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "a248b85b-061e-4957-8498-858ec52206ef",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "read_csv(\"https://j.mp/iriscsv\")\\\n",
    "    .lazy()\\\n",
    "    .with_columns(\\\n",
    "                 (col('sepal_length') * 5).alias('new_sepal_length')\n",
    "                 )\\\n",
    "    .filter(col('new_sepal_length') > 25)\\\n",
    "    .sink_parquet('../02-data/sink_parquet_output.parquet')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0ffba5a-4fa4-46f5-9a73-85ee1b38fa7d",
   "metadata": {},
   "source": [
    "### **Referências**:\n",
    "\n",
    "- **User guide (referência mais genérica e completa):** https://pola-rs.github.io/polars-book/user-guide/index.html\n",
    "- **Data Cleansing in Polars:** https://towardsdatascience.com/data-cleansing-in-polars-f9314ea04a8e\n",
    "- **Getting Started with the Polars DataFrame Library:** https://towardsdatascience.com/getting-started-with-the-polars-dataframe-library-6f9e1c014c5c\n",
    "- **Using the Polars DataFrame Library:** https://www.codemag.com/Article/2212051/Using-the-Polars-DataFrame-Library\n",
    "- **Calmcode - polars: introduction:** https://calmcode.io/polars/introduction.html\n",
    "- **Pandas vs. Polars: A Syntax and Speed Comparison:** https://towardsdatascience.com/pandas-vs-polars-a-syntax-and-speed-comparison-5aa54e27497e#:~:text=The%20main%20advantage%20of%20Polars,switch%20from%20Pandas%20to%20Polars.\n",
    "- **Sinking larger-than-memory Parquet files:** https://www.rhosignal.com/posts/sink-parquet-files/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of C",
   "title_sidebar": "Table of C",
   "toc_cell": true,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "275.6px"
   },
   "toc_section_display": true,
   "toc_window_display": false
  },
  "toc-autonumbering": false,
  "toc-showcode": false,
  "toc-showmarkdowntxt": false,
  "toc-showtags": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}