guias/Guia_Naive_Bayes.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Guia-Naive_Bayes.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jumafernandez/BDM/blob/master/Guias/Guia_Naive_Bayes.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R7t49mkzKf-9",
        "colab_type": "text"
      },
      "source": [
        "# Naive Bayes con Scikit Learn\n",
        "\n",
        "## Introducción\n",
        "En esta guía vamos a experimientar cómo trabajar con el algoritmo de _naive Bayes_. Se trata de una técnica supervisada de aprendizaje probabilístico que se aplica a tareas de clasificación. Vamos a utilizar el dataset _iris_, con el que ya hemos trabajado en varias guías de laboratorio y TP's.\n",
        "<br />\n",
        "A lo largo de esta guía veremos:\n",
        "- Cómo preprocesar el dataset y prepararlo para trabajar con Naive Bayes (NB) en Scikit Learn,\n",
        "- Cómo entrenar un clasificador NB con la librería Scikit Learn,\n",
        "- Cómo predecir la clase ante la aparición de nuevas instancias utilizando el modelo y la probabilidad aparejada,\n",
        "- Cuáles son los hiper-parámetros más importantes del algoritmo,\n",
        "- Cómo generar nuevas métricas de evaluación de los modelos.\n",
        "\n",
        "## Trabajando con datos en Scikit Learn (Python)\n",
        "Para iniciar vamos a cargar los datos desde Scikit-Learn, que es una de las modalidades que ya veníamos trabajando. Como sabemos, se va a instanciar un objeto bunch que es básicamente un diccionario:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YfsIoe-wKfwc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Importamos el dataset iris de la librerías sklearn.datasets\n",
        "from sklearn.datasets import load_iris\n",
        "\n",
        "# Iris será un objeto \"bunch\" que es básicamente un diccionario de Python con un conjunto de claves\n",
        "iris = load_iris()\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nlJ3Fl0DK6Oi",
        "colab_type": "text"
      },
      "source": [
        "### Preprarando los datos para trabajar con NB en Scikit Learn\n",
        "\n",
        "Al igual que hicimos con árboles de clasificación, vamos a tener que transformar los atributos discretos a numéricos previo a entrenar el clasificador. En este caso, el dataset no va a sufrir modificaciones puesto que todos sus _features_ son continuos, pero el código es facilmente trasladable a datasets con atributos cualitativos."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_XSz4YSeK7D-",
        "colab_type": "code",
        "outputId": "3a407d40-9bab-424f-b644-0654da1cf2fc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "from sklearn import preprocessing\n",
        "\n",
        "# Cantidad de columnas\n",
        "ncols=iris.data.shape[1]\n",
        "\n",
        "# Creamos el objeto labelEncoder\n",
        "le = preprocessing.LabelEncoder()\n",
        "\n",
        "# Numerizamos todos los atributos que no sean numéricos\n",
        "for column in range(0, ncols-1):\n",
        "  if (iris.data[:,column].dtype == object):\n",
        "    iris.data[:,column] = le.fit_transform(iris.data[:,column])\n",
        "\n",
        "iris.data[0:6]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[5.1, 3.5, 1.4, 0.2],\n",
              "       [4.9, 3. , 1.4, 0.2],\n",
              "       [4.7, 3.2, 1.3, 0.2],\n",
              "       [4.6, 3.1, 1.5, 0.2],\n",
              "       [5. , 3.6, 1.4, 0.2],\n",
              "       [5.4, 3.9, 1.7, 0.4]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IbXAwWlDV2mr",
        "colab_type": "text"
      },
      "source": [
        "Y generamos nuestro conjunto de _train_ y de _test_ como venimos haciendo:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7U24RukFV85X",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Separo en 70-30 entrenamiento y testeo\n",
        "x_train, x_test, y_train, y_test = train_test_split(iris.data, iris.target, random_state=0, test_size=0.2)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "weNFL-cbSWRV",
        "colab_type": "text"
      },
      "source": [
        "## Clasificador Naive Bayes con Scikit Learn\n",
        "\n",
        "Scikit Learn provee tres clases para clasificación con Naive Bayes:\n",
        "- __GaussianNB:__ Éste clasificador supone que la distribución de los datos es gaussiana (__warning!__).\n",
        "-  __MultinomialNB:__ En éste caso, el caso, el clasificador asume un distribución multinomial de los datos.\n",
        "- __ComplementNB:__ Ésta es una tercera implementación del clasificador que posee optimizaciones para trabajar con datasets desbalanceados.\n",
        "<br /> \n",
        "\n",
        "En ésta guía vamos a limitarnos a trabajar de manera indistinta con las clases para realizar diferentes pruebas puesto que resulta muy sencillo cambiar el clasificador ya que las tres clases poseen métodos y parámetros muy similares.\n",
        "\n",
        "### Entrenando un clasificador NB\n",
        "A continuación, vamos a explorar como entrenar clasificador Naive Bayes, analizar el modelo entrenado y evaluarlo:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Pn5tRAMSm74",
        "colab_type": "code",
        "outputId": "6fa784e8-abd4-4dc5-c8df-d187c2997a0f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Importamos el módulo GaussianNB de scikit-learn\n",
        "from sklearn.naive_bayes import GaussianNB, MultinomialNB, ComplementNB\n",
        "\n",
        "#modelo = GaussianNB()\n",
        "#modelo = MultinomialNB()\n",
        "modelo = ComplementNB()\n",
        "\n",
        "modelo"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "ComplementNB(alpha=1.0, class_prior=None, fit_prior=True, norm=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ira9JnK1Tawp",
        "colab_type": "text"
      },
      "source": [
        "Ahora vamos a entrenar el modelo, para ello le pasamos los datos de los _features_ y el _target_:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5b96OJUFToz9",
        "colab_type": "code",
        "outputId": "fd7ac6a8-ec86-48e2-f8ff-ada6bae466c2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Ahora vamos a entrenar el dato con el método fit -le pasamos los X e Y\n",
        "# X e Y (X=atributos e Y=target)\n",
        "modelo.fit(x_train, y_train)\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "ComplementNB(alpha=1.0, class_prior=None, fit_prior=True, norm=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wKnV2NIZYlCM",
        "colab_type": "code",
        "outputId": "217b59bb-df82-488f-c725-d38b83bdca2b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# Número de muestras encontradas para cada clase durante el ajuste. \n",
        "# (Este valor es ponderado por el peso de la muestra cuando se proporciona.)\n",
        "print(\"Número de muestras encontradas para cada clase durante el ajuste:\")\n",
        "print(str(iris.target_names[0]) + \": \" + str(modelo.class_count_[0]))\n",
        "print(str(iris.target_names[1]) + \": \" + str(modelo.class_count_[1]))\n",
        "print(str(iris.target_names[2]) + \": \" + str(modelo.class_count_[2]))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Número de muestras encontradas para cada clase durante el ajuste:\n",
            "setosa: 39.0\n",
            "versicolor: 37.0\n",
            "virginica: 44.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gwXAImZRaRYF",
        "colab_type": "code",
        "outputId": "51338d5c-70ef-4c89-a4b8-bf211c2c7866",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "modelo.feature_count_"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[195.8, 132.7,  57. ,   9.4],\n",
              "       [217.8, 102.2, 156. ,  49. ],\n",
              "       [292.1, 131.5, 244.9,  89.4]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CM1vBakhble5",
        "colab_type": "code",
        "outputId": "5fa85100-f9a8-4fd1-d55b-106c1efc412e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "print(\"Probabilidad empírica de las features x dada la clase y: P(x_i|y)\")\n",
        "print(\"x = \" + str(iris.feature_names))\n",
        "print(\"y = \" + str(iris.target_names)+\"\\n\")\n",
        "\n",
        "modelo.feature_log_prob_"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Probabilidad empírica de las features x dada la clase y: P(x_i|y)\n",
            "x = ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']\n",
            "y = ['setosa' 'versicolor' 'virginica']\n",
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.92381763, 1.7016834 , 1.1637882 , 2.22264401],\n",
              "       [0.86125488, 1.47292859, 1.34001013, 2.45024467],\n",
              "       [0.80128964, 1.36519585, 1.46262783, 2.74430961]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qXRnUoigcvM9",
        "colab_type": "code",
        "outputId": "dafbb9ec-322d-4f3e-e58c-50e6a4d5556d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "print(\"Coeficientes del clasificador:\")\n",
        "modelo.coef_"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Coeficientes del clasificador:\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.92381763, 1.7016834 , 1.1637882 , 2.22264401],\n",
              "       [0.86125488, 1.47292859, 1.34001013, 2.45024467],\n",
              "       [0.80128964, 1.36519585, 1.46262783, 2.74430961]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MocGh8LtdoJM",
        "colab_type": "code",
        "outputId": "8955a595-85fc-476c-df7d-071313df03f4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "modelo.intercept_"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([-1.1239301 , -1.17657383, -1.00330211])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sU14aoOZDU7u",
        "colab_type": "text"
      },
      "source": [
        "## Utilizando el modelo para clasificar/predecir casos nuevos\n",
        "\n",
        "A partir del modelo generado, podríamos darle un nuevo caso y el clasificador debería clasificarlo y devolver la proba asociada a cada clase:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "we0OSJ_vDVjW",
        "colab_type": "code",
        "outputId": "d4cf3355-2055-4e38-87c3-746267ad92bf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Haciendo nuevas predicciones a partir de los datos\n",
        "pred_ejemplo1 = int(modelo.predict([[5, 7, 1, 3]]))\n",
        "pred_ejemplo2 = int(modelo.predict([[1, 1, 1, 1]]))\n",
        "pred_ejemplo3 = int(modelo.predict([[15, 17, 11, 13]]))\n",
        "\n",
        "proba_ejemplo1 = modelo.predict_proba([[5, 7, 1, 3]])\n",
        "\n",
        "print(\"La probabilidad asociada es \" +str(proba_ejemplo1)+ \"para las clases\" + str(iris.target_names) + \" respectivamente.\")\n",
        "print(f\"En función de las probabilidades, este ejemplo corresponde a una flor {iris.target_names[pred_ejemplo1]}.\\n\")\n",
        "\n",
        "print(f\"Este ejemplo corresponde a una flor {iris.target_names[pred_ejemplo2]}.\")\n",
        "print(f\"Este ejemplo corresponde a una flor {iris.target_names[pred_ejemplo3]}.\")\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "La probabilidad asociada es [[0.5953666  0.20728691 0.19734649]]para las clases['setosa' 'versicolor' 'virginica'] respectivamente.\n",
            "En función de las probabilidades, este ejemplo corresponde a una flor setosa.\n",
            "\n",
            "Este ejemplo corresponde a una flor virginica.\n",
            "Este ejemplo corresponde a una flor virginica.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hj2YSiHcAXXZ",
        "colab_type": "text"
      },
      "source": [
        "## Parámetros de NB\n",
        "\n",
        "Como la mayoría de los algoritmos de aprendizaje automático, NB posee hiper-parámetros que podemos ajustar."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bnMb4k-WA06c",
        "colab_type": "code",
        "outputId": "fe655f71-2122-4c52-8cf4-e221292b68ec",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "from sklearn import metrics\n",
        "\n",
        "# En este caso, instancio, los parametros a ajustar son:\n",
        "# GuassianNB:\n",
        "# priors: Probabilidades previas de las clases.\n",
        "# var_smoothing: Parte de la mayor variación de todas las características que se agrega a las variaciones para la estabilidad del cálculo.\n",
        "\n",
        "# MultinomialNB:\n",
        "# alpha: Parámetro de suavizado aditivo (Laplace/Lidstone)\n",
        "# class_prior: Idem priors de GaussianNB.\n",
        "\n",
        "# ComplementNB:\n",
        "# A los parámetros del Multinomial se le agrega norm.\n",
        "# norm: Si se realiza o no una normalización de los pesos.\n",
        "modelo_parametrizado = GaussianNB()\n",
        "\n",
        "# Entreno el modelo con el mismo muestreo generado antes (70-30 %)\n",
        "modelo_parametrizado = modelo_parametrizado.fit(x_train,y_train)\n",
        "\n",
        "# Realizo la predicción de y con el x_test\n",
        "y_pred = modelo_parametrizado.predict(x_test)\n",
        "\n",
        "print(\"Accuracy:\",metrics.accuracy_score(y_test, y_pred))\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy: 0.9666666666666667\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JRIJ_PRZ-Xbo",
        "colab_type": "text"
      },
      "source": [
        "## Métricas de evaluación de modelos\n",
        "\n",
        "Como vimos en teoría, luego debemos evaluar el modelo generado, para saber cuán preciso es. Vamos a incluir F-score al set que venimos utilizando, la curva ROC y el heatmap para dar un enfoque más gráfico a la matriz de confusión.\n",
        "\n",
        "Comenzamos con la curva ROC:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gh12t28--ZPn",
        "colab_type": "code",
        "outputId": "545df2df-95e2-4840-af92-41262fba381c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        }
      },
      "source": [
        "from sklearn.metrics import roc_curve\n",
        "from sklearn.metrics import roc_auc_score\n",
        "from matplotlib import pyplot\n",
        "from sklearn.preprocessing import label_binarize\n",
        "\n",
        "# Para la versión 0.22 de scikit-learn se incorpora el método plot_roc_curve\n",
        "# https://scikit-learn.org/dev/modules/generated/sklearn.metrics.plot_roc_curve.html\n",
        "# Por ahora, lo graficamos a mano con roc_auc_score\n",
        "\n",
        "# Seteo la clase a graficar (para iris entre 0-2)\n",
        "clase_elegida = 1\n",
        "\n",
        "# Me quedo con la clase elegida\n",
        "y = label_binarize(y_test, classes=[0, 1, 2])\n",
        "y_class = y[:,clase_elegida]\n",
        "\n",
        "# Calculo las probas para el test\n",
        "modelo_probas = modelo.predict_proba(x_test)\n",
        "\n",
        "# Me quedo con la proba para la clase elegida\n",
        "positivos_probas = modelo_probas[:,clase_elegida]\n",
        "\n",
        "# calculate scores\n",
        "lr_auc = roc_auc_score(y_class, positivos_probas)\n",
        "\n",
        "# summarize scores\n",
        "print('NB: ROC AUC=%.3f' % (lr_auc))\n",
        "\n",
        "# Calcula la curva ROC\n",
        "lr_fpr, lr_tpr, _ = roc_curve(y_class, positivos_probas)\n",
        "\n",
        "# Plotea\n",
        "pyplot.plot(lr_fpr, lr_tpr, marker='.', label='NB ' + iris.target_names[clase_elegida])\n",
        "\n",
        "# Nombres de los ejes\n",
        "pyplot.xlabel('False Positive Rate')\n",
        "pyplot.ylabel('True Positive Rate')\n",
        "\n",
        "# Muestra las leyendas\n",
        "pyplot.legend()\n",
        "# Plotea\n",
        "pyplot.show()\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "NB: ROC AUC=0.991\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAeHklEQVR4nO3de5gV1Z3u8e/b0NJeUBHJqCA3b4gK\nKA0yw5yjxFzQOJDxBg4xMiMSr/Gao3OSkzjOmBMTo4mGqCiJiUHxMsYQh9ETDRFHRYERREETRNHW\nJCKiEmJrN/07f1RBNk1fdtNde9Nd7+d5+smuqlW1f9WY/XbVqr2WIgIzM8uvinIXYGZm5eUgMDPL\nOQeBmVnOOQjMzHLOQWBmlnPdy11AW+29994xcODAcpdhZtapLFmy5J2I6NPUtk4XBAMHDmTx4sXl\nLsPMrFORtKa5bb41ZGaWcw4CM7OccxCYmeWcg8DMLOccBGZmOZdZEEj6kaS3Jb3QzHZJulHSKknP\nSzoqq1rMzKx5WV4R3AGMb2H78cBB6c904OYMa+kwS9asZ8b8VSxZs77cpZhZjmT52ZPZ9wgiYoGk\ngS00mQj8NJJxsBdK2lPSvhHx+6xqaq8la9YzeebT1G0KKgRD9ulJz6rKcpdlZl3chto6XvrDBiKg\nR2UFs6eNYeSAXh12/HL2EfQF3ihYrknXbUPSdEmLJS1eu3ZtSYprysLV66jblMzf0BDwQW192Wox\ns/z4oLaehoAA6uobWLh6XYcev1N8szgiZgIzAaqrq8s2k86Ywb2pUBICVZUVfH/ykR2aymZmTVmy\nZj1Tbl9IXX0Dld0rGDO4d4cev5xB8Cawf8Fyv3TdDmvkgF4M2acnH9TWOwTMrGRGDujF7GljWLh6\nHWMG9+7wz55yBsFc4AJJc4Cjgfd35P6BzXpWVdKzqtIhYGYlNXJAr8w+dzILAkl3A8cCe0uqAb4B\nVAJExC3APOAEYBXwZ+Afs6rFzMyal+VTQ6e3sj2A87N6fzMzK46/WWxmlnMOAjOznHMQmJnlnIPA\nzCznHARmZjnnIDAzyzkHgZlZzjkIzMxyzkFgZpZzDgIzs5xzEJiZ5ZyDwMws5xwEZmY55yAwM8s5\nB4GZWc45CMzMcs5BYGaWcw4CM7OccxCYmeWcg8DMLOccBGZmOecgMDPLOQeBmVnO5SYIlqxZz4z5\nq1iyZn27jrOhto433/uw3ccxM9tRdC93AaWwZM16Js98mrpNQYVgyD496VlV2ebjbKitY8XvNwAw\n5faFzJ42hpEDenV0uWZmJZWLK4KFq9dRtykAaAj4oLZ+u45TuF9dfQMLV6/rkPrMzMopF1cEYwb3\npkJJCFRVVvD9yUdu11/yS9asZ8rtC6mrb6CyewVjBvfOoFozs9LKRRCMHNCLIfv05IPa+u0Ogc3H\nmT1tDAtXr2PM4N6+LWRmXUIuggCgZ1UlPasq2/3hPXJALweAmXUpuegjMDOz5jkIzMxyLtMgkDRe\n0suSVkm6sont/SXNl/ScpOclnZBlPWZmtq3MgkBSN2AGcDwwFDhd0tBGzb4G3BsRRwKTgR9mVY+Z\nmTUtyyuC0cCqiFgdER8Dc4CJjdoEsHv6eg/grQzrMTOzJmQZBH2BNwqWa9J1ha4CviCpBpgHXNjU\ngSRNl7RY0uK1a9dmUauZWW6Vu7P4dOCOiOgHnADcKWmbmiJiZkRUR0R1nz59Sl6kmVlXlmUQvAns\nX7DcL11X6CzgXoCIeBqoAvbOsCYzM2skyyBYBBwkaZCknUg6g+c2avM6cByApENJgsD3fszMSiiz\nIIiIeuAC4BFgJcnTQS9KulrShLTZZcDZkpYBdwNTIyKyqsnMzLaV6RATETGPpBO4cN3XC16vAMZm\nWYOZmbWs3J3FZmZWZg4CM7OccxCYmeWcg8DMLOccBGZmOecgMDPLOQeBmVnOOQjMzHLOQWBmlnNF\nBYGknSQdmHUxZmZWeq0GgaTPAcuBX6XLIyT9POvCzMysNIq5IrgaOBp4DyAilgK+OjAz6yKKCYK6\niHiv0TqPEGpm1kUUM/roSkmnARWSBgFfBhZmW5aZmZVKMVcEFwAjgQbgAeAj4KIsizIzs9Ip5org\nsxFxBXDF5hWSTiIJBTMz6+SKuSL4WhPrvtrRhZiZWXk0e0Ug6bPAeKCvpOsLNu1OcpvIzMy6gJZu\nDb0NvADUAi8WrN8AXJllUWZmVjrNBkFEPAc8J2l2RNSWsCYzMyuhYjqL+0q6BhgKVG1eGREHZ1aV\nmZmVTDGdxXcAPwYEHA/cC9yTYU1mZlZCxQTBLhHxCEBEvBIRXyMJBDMz6wKKuTX0kaQK4BVJ5wBv\nAj2zLcvMzEqlmCC4BNiVZGiJa4A9gH/KsigzMyudVoMgIp5JX24AzgCQ1DfLoszMrHRa7COQNErS\n5yXtnS4fJumnwDMt7WdmZp1Hs0Eg6f8Cs4EpwMOSrgLmA8sAPzpqZtZFtHRraCIwPCI+lLQX8AZw\nRESsLk1pZmZWCi3dGqqNiA8BIuJd4LcOATOzrqelK4LBkjYPNS1gUMEyEXFSaweXNB74PtANuD0i\nvtVEm9OAq0hmPVsWEf9QfPlmZtZeLQXByY2Wf9CWA0vqBswAPg3UAIskzY2IFQVtDgL+GRgbEesl\nfaIt72FmZu3X0qBzj7Xz2KOBVZtvJ0maQ9LvsKKgzdnAjIhYn77n2+18TzMza6NihpjYXn1JOpg3\nq0nXFToYOFjSk5IWpreStiFpuqTFkhavXbs2o3LNzPIpyyAoRnfgIOBY4HTgNkl7Nm4UETMjojoi\nqvv06VPiEs3Muraig0BSjzYe+01g/4Llfum6QjXA3Iioi4hXgd+SBIOZmZVIq0EgabSk5cDv0uXh\nkm4q4tiLgIMkDZK0EzAZmNuozYMkVwOk314+GPAjqmZmJVTMFcGNwInAOoCIWAaMa22niKgHLgAe\nAVYC90bEi5KuljQhbfYIsE7SCpJvLX8lIta1/TTMzGx7FTP6aEVErJFUuG5TMQePiHnAvEbrvl7w\nOoBL0x8zMyuDYoLgDUmjgUi/G3Ahyb18MzPrAoq5NXQuyV/s/YE/AmPSdWZm1gUUc0VQHxGTM6/E\nzMzKopgrgkWS5kk6U5KnqDQz62JaDYKIOAD4N2AksFzSg5J8hWBm1kUU9YWyiHgqIr4MHAV8QDJh\njZmZdQHFfKFsN0lTJP0SeBZYC/xN5pWZmVlJFNNZ/ALwS+DbEfFExvWYmVmJFRMEgyOiIfNKzMys\nLJoNAknfjYjLgH+XFI23FzNDmZmZ7fhauiK4J/3fNs1MZmZmnUtLM5Q9m748NCK2CgNJFwDtncHM\nzMx2AMU8PvpPTaw7q6MLMTOz8mipj2ASyRwCgyQ9ULCpJ/Be1oWZmVlptNRH8CzJHAT9gBkF6zcA\nz2VZlJmZlU5LfQSvAq8Cj5auHDMzK7WWbg09HhHHSFoPFD4+KpI5ZfbKvDozM8tcS7eGNk9HuXcp\nCjEzs/Jo9qmhgm8T7w90i4hNwF8DXwJ2LUFtZmZWAsU8PvogyTSVBwA/Bg4C7sq0KjMzK5ligqAh\nIuqAk4CbIuISoG+2ZZmZWakUEwT1kk4FzgAeStdVZleSmZmVUrHfLB5HMgz1akmDgLuzLcvMzEql\n1WGoI+IFSV8GDpQ0BFgVEddkX5qZmZVCq0Eg6X8AdwJvknyHYB9JZ0TEk1kXZ2Zm2StmYpobgBMi\nYgWApENJgqE6y8LMzKw0iukj2GlzCABExEpgp+xKMjOzUirmiuC/Jd0C/CxdnoIHnTMz6zKKCYJz\ngC8D/ytdfgK4KbOKzMyspFoMAklHAAcAP4+Ib5emJDMzK6Vm+wgk/W+S4SWmAL+S1NRMZWZm1sm1\n1Fk8BRgWEacCo4Bz23pwSeMlvSxplaQrW2h3sqSQ5CeRzMxKrKUg+CgiNgJExNpW2m5DUjeSmc2O\nB4YCp0sa2kS7nsBFwDNtOb6ZmXWMlvoIBhfMVSzggMK5iyPipFaOPZrkW8irASTNASYCKxq1+1fg\nWuArbSnczMw6RktBcHKj5R+08dh9gTcKlmuAowsbSDoK2D8i/kNSs0EgaTowHaB///5tLMPMzFrS\n0pzFj2X5xpIqgOuBqa21jYiZwEyA6urqaKW5mZm1QZvu+7fRmySzm23WL123WU/gcOA3kl4DxgBz\n3WFsZlZaWQbBIuAgSYMk7QRMBuZu3hgR70fE3hExMCIGAguBCRGxOMOazMyskaKDQFKPthw4IuqB\nC4BHgJXAvRHxoqSrJU1oW5lmZpaVYoahHg3MAvYA+ksaDkyLiAtb2zci5gHzGq37ejNtjy2mYDMz\n61jFXBHcCJwIrAOIiGUkM5aZmVkXUEwQVETEmkbrNmVRjJmZlV4xo4++kd4eivTbwhcCv822LDMz\nK5VirgjOBS4F+gN/JHnMs83jDpmZ2Y6pmMnr3yZ59NPMzLqgYp4aug3Y5tu8ETE9k4rMzKykiukj\neLTgdRXw92w9hpCZmXVixdwauqdwWdKdwH9lVpGZmZXU9gwxMQj4q44uxMzMyqOYPoL1/KWPoAJ4\nF2h2tjEzM+tcWpu8XsBw/jJqaENEeBhoM7MupMVbQ+mH/ryI2JT+OATMzLqYYvoIlko6MvNKzMys\nLJq9NSSpezqU9JHAIkmvABtJ5i+OiDiqRDWamVmGWuojeBY4CvDcAWZmXVhLQSCAiHilRLWYmVkZ\ntBQEfSRd2tzGiLg+g3rMzKzEWgqCbsBupFcGZmbWNbUUBL+PiKtLVomZmZVFS4+P+krAzCwHWgqC\n40pWhZmZlU2zQRAR75ayEDMzK4/tGX3UzMy6EAeBmVnOOQjMzHLOQWBmlnMOAjOznHMQmJnlnIPA\nzCznHARmZjmXaRBIGi/pZUmrJG0z4b2kSyWtkPS8pMckDciyHjMz21ZmQSCpGzADOB4YCpwuaWij\nZs8B1RExDLgf+HZW9ZiZWdOyvCIYDayKiNUR8TEwB5hY2CAi5kfEn9PFhUC/DOsxM7MmZBkEfYE3\nCpZr0nXNOQv4z6Y2SJouabGkxWvXru3AEs3MbIfoLJb0BaAa+E5T2yNiZkRUR0R1nz59SlucmVkX\n19LENO31JrB/wXK/dN1WJH0K+CpwTER8lGE9ZmbWhCyvCBYBB0kaJGknYDIwt7CBpCOBW4EJEfF2\nhrWYmVkzMguCiKgHLgAeAVYC90bEi5KuljQhbfYdknmR75O0VNLcZg5nZmYZyfLWEBExD5jXaN3X\nC15/Ksv3NzOz1u0QncVmZlY+DgIzs5xzEJiZ5ZyDwMws5xwEZmY55yAwM8s5B4GZWc45CMzMcs5B\nYGaWcw4CM7OccxCYmeWcg8DMLOccBGZmOecgMDPLOQeBmVnOOQjMzHLOQWBmlnO5CYINtXW8+d6H\nLFmzvtylmJntUHIRBEvWrOelP2ygZv2HTLl9ocPAzKxALoJg4ep1NETyuq6+gYWr15W3IDOzHUgu\ngmDM4N5UKHld2b2CMYN7l7cgM7MdSPdyF1AKIwf0Ysg+Pfmgtp7vTz6SkQN6lbsksy6hrq6Ompoa\namtry12KpaqqqujXrx+VlZVF75OLIADoWVVJz6pKh4BZB6qpqaFnz54MHDgQSeUuJ/cignXr1lFT\nU8OgQYOK3i8Xt4bMLBu1tbX07t3bIbCDkETv3r3bfIXmIDCzdnEI7Fi259/DQWBmlnMOAjPr1CRx\n2WWXbVm+7rrruOqqqwC46qqr6Nu3LyNGjGDIkCGce+65NDQ0lKSut956i1NOOWW79j322GNZvHhx\nB1fUPAeBmZXUkjXrmTF/VYd9sbNHjx488MADvPPOO01uv+SSS1i6dCkrVqxg+fLlPP744x3yvpvV\n19c3uX6//fbj/vvv79D3as6mTZvatX9unhoys2z9yy9fZMVbH7TYZkNtHS/9YQMNARWCIfv0pGdV\n8485Dt1vd77xd4e1eMzu3bszffp0brjhBq655ppm23388cfU1tbSq9fWTw6+//77DBs2jFdffZWK\nigo2btzIkCFDWL16Na+//jrnn38+a9euZZddduG2225jyJAhTJ06laqqKp577jnGjh3LxIkTueii\ni4DkCmXBggWsW7eOE088kRdeeIFNmzZxxRVX8PDDD1NRUcHZZ5/NhRdeyGOPPcbll19OfX09o0aN\n4uabb6ZHjx5b1Xf33XfzzW9+k4jgc5/7HNdeey0Au+22G1/60pd49NFHmTFjBn/7t3/b4u+pJb4i\nMLOS+aC2fsu3/BsiWe4I559/PrNnz+b999/fZtsNN9zAiBEj2HfffTn44IMZMWLEVtv32GMPRowY\nseVK4aGHHuKzn/0slZWVTJ8+nZtuuoklS5Zw3XXXcd55523Zr6amhqeeeorrr7+e6667jhkzZrB0\n6VKeeOIJdt55563eY+bMmbz22mssXbqU559/nilTplBbW8vUqVO55557WL58OfX19dx8881b7ffW\nW29xxRVX8Otf/5qlS5eyaNEiHnzwQQA2btzI0UcfzbJly9oVAuArAjPrIK395Q7JbaEpty+krr6B\nyu4VHfYFz913350vfvGL3Hjjjdt8CF9yySVcfvnl1NXVccoppzBnzhwmT568VZtJkyZxzz33MG7c\nOObMmcN5553Hn/70J5566ilOPfXULe0++uijLa9PPfVUunXrBsDYsWO59NJLmTJlCieddBL9+vXb\n6viPPvoo55xzDt27Jx+5e+21F8uWLWPQoEEcfPDBAJx55pnMmDGDiy++eMt+ixYt4thjj6VPnz4A\nTJkyhQULFvD5z3+ebt26cfLJJ7f3VwdkfEUgabyklyWtknRlE9t7SLon3f6MpIFZ1mNm5TVyQC9m\nTxvDpZ85hNnTxnToFzwvvvhiZs2axcaNG5vcXllZyfjx41mwYME22yZMmMDDDz/Mu+++y5IlS/jk\nJz9JQ0MDe+65J0uXLt3ys3Llyi377LrrrlteX3nlldx+++18+OGHjB07lpdeeqnDzqs5VVVVW4Ko\nvTILAkndgBnA8cBQ4HRJQxs1OwtYHxEHAjcA12ZVj4ehNtsxjBzQi/PHHdjh3/Lfa6+9OO2005g1\na1aT2yOCJ598kgMOOGCbbbvtthujRo3ioosu4sQTT6Rbt27svvvuDBo0iPvuu2/L/suWLWvy2K+8\n8gpHHHEEV1xxBaNGjdomCD796U9z6623bulYfvfddznkkEN47bXXWLVqFQB33nknxxxzzFb7jR49\nmscff5x33nmHTZs2cffdd2/TpiNkeUUwGlgVEasj4mNgDjCxUZuJwE/S1/cDxymDb6d4GGqzfLjs\nssu2eXpocx/B4YcfzqZNm7a6z19o0qRJ/OxnP2PSpElb1s2ePZtZs2YxfPhwDjvsMH7xi180ue/3\nvvc9Dj/8cIYNG0ZlZSXHH3/8VtunTZtG//79GTZsGMOHD+euu+6iqqqKH//4x5x66qkcccQRVFRU\ncM4552y137777su3vvUtxo0bx/Dhwxk5ciQTJzb+GG0/RUSHHxRA0inA+IiYli6fARwdERcUtHkh\nbVOTLr+Stnmn0bGmA9MB+vfvP3LNmjVtqmXG/FVc98jLBNBNcOlnDuH8cQe24+zMDGDlypUceuih\n5S7DGmnq30XSkoiobqp9p3hqKCJmRkR1RFRv7jRpizGDe9OjsoJu8jDUZmaNZfnU0JvA/gXL/dJ1\nTbWpkdQd2APo8FljNndQLVy9jjGDe3sEUjOzAlkGwSLgIEmDSD7wJwP/0KjNXOBM4GngFODXkdG9\nqpEDejkAzDIQER54bgeyPR+hmd0aioh64ALgEWAlcG9EvCjpakkT0mazgN6SVgGXAts8YmpmO66q\nqirWrVu3XR8+1vE2z0dQVVXVpv0y6yzOSnV1dZRyMCYza55nKNvxNDdDWUudxf5msZltt8rKyjbN\nhGU7pk7x1JCZmWXHQWBmlnMOAjOznOt0ncWS1gJt+2rxX+wNND17Rdflc84Hn3M+tOecB0REk9/I\n7XRB0B6SFjfXa95V+ZzzweecD1mds28NmZnlnIPAzCzn8hYEM8tdQBn4nPPB55wPmZxzrvoIzMxs\nW3m7IjAzs0YcBGZmOdclg0DSeEkvS1olaZsRTSX1kHRPuv0ZSQNLX2XHKuKcL5W0QtLzkh6TNKAc\ndXak1s65oN3JkkJSp3/UsJhzlnRa+m/9oqS7Sl1jRyviv+3+kuZLei797/uEctTZUST9SNLb6QyO\nTW2XpBvT38fzko5q95tGRJf6AboBrwCDgZ2AZcDQRm3OA25JX08G7il33SU453HALunrc/Nwzmm7\nnsACYCFQXe66S/DvfBDwHNArXf5EuesuwTnPBM5NXw8FXit33e085/8JHAW80Mz2E4D/BASMAZ5p\n73t2xSuC0cCqiFgdER8Dc4DGsz1PBH6Svr4fOE6de2aNVs85IuZHxJ/TxYUkM8Z1ZsX8OwP8K3At\n0BXGSS7mnM8GZkTEeoCIeLvENXa0Ys45gN3T13sAb5Wwvg4XEQuAd1toMhH4aSQWAntK2rc979kV\ng6Av8EbBck26rsk2kUyg8z7QmScyLuacC51F8hdFZ9bqOaeXzPtHxH+UsrAMFfPvfDBwsKQnJS2U\nNL5k1WWjmHO+CviCpBpgHnBhaUorm7b+/71Vno8gZyR9AagGjil3LVmSVAFcD0wtcyml1p3k9tCx\nJFd9CyQdERHvlbWqbJ0O3BER35X018Cdkg6PiIZyF9ZZdMUrgjeB/QuW+6XrmmwjqTvJ5eS6klSX\njWLOGUmfAr4KTIiIj0pUW1ZaO+eewOHAbyS9RnIvdW4n7zAu5t+5BpgbEXUR8SrwW5Jg6KyKOeez\ngHsBIuJpoIpkcLauqqj/v7dFVwyCRcBBkgZJ2omkM3huozZzgTPT16cAv460F6aTavWcJR0J3EoS\nAp39vjG0cs4R8X5E7B0RAyNiIEm/yISI6MzznBbz3/aDJFcDSNqb5FbR6lIW2cGKOefXgeMAJB1K\nEgRrS1plac0Fvpg+PTQGeD8ift+eA3a5W0MRUS/pAuARkicOfhQRL0q6GlgcEXOBWSSXj6tIOmUm\nl6/i9ivynL8D7Abcl/aLvx4RE8pWdDsVec5dSpHn/AjwGUkrgE3AVyKi017tFnnOlwG3SbqEpON4\namf+w07S3SRhvnfa7/ENoBIgIm4h6Qc5AVgF/Bn4x3a/Zyf+fZmZWQfoireGzMysDRwEZmY55yAw\nM8s5B4GZWc45CMzMcs5BYDscSZskLS34GdhC24HNjdLYxvf8TTrC5bJ0eIZDtuMY50j6Yvp6qqT9\nCrbdLmloB9e5SNKIIva5WNIu7X1v67ocBLYj+jAiRhT8vFai950SEcNJBiT8Tlt3johbIuKn6eJU\nYL+CbdMiYkWHVPmXOn9IcXVeDDgIrFkOAusU0r/8n5D03+nP3zTR5jBJz6ZXEc9LOihd/4WC9bdK\n6tbK2y0ADkz3PS4d5355Ok58j3T9t/SX+R2uS9ddJelySaeQjOc0O33PndO/5KvTq4YtH97plcMP\ntrPOpykYbEzSzZIWK5mH4F/SdV8mCaT5kuan6z4j6en093ifpN1aeR/r4hwEtiPaueC20M/TdW8D\nn46Io4BJwI1N7HcO8P2IGEHyQVyTDjkwCRibrt8ETGnl/f8OWC6pCrgDmBQRR5B8E/9cSb2BvwcO\ni4hhwL8V7hwR9wOLSf5yHxERHxZs/vd0380mAXO2s87xJENKbPbViKgGhgHHSBoWETeSDMs8LiLG\npcNOfA34VPq7XAxc2sr7WBfX5YaYsC7hw/TDsFAl8IP0nvgmkjF0Gnsa+KqkfsADEfE7SccBI4FF\n6dAaO5OESlNmS/oQeI1kKONDgFcj4rfp9p8A5wM/IJnfYJakh4CHij2xiFgraXU6RszvgCHAk+lx\n21LnTiRDhhT+nk6TNJ3k/9f7kkzS8nyjfcek659M32cnkt+b5ZiDwDqLS4A/AsNJrmS3mWgmIu6S\n9AzwOWCepC+RzOL0k4j45yLeY0rhoHSS9mqqUTr+zWiSgc5OAS4APtmGc5kDnAa8BPw8IkLJp3LR\ndQJLSPoHbgJOkjQIuBwYFRHrJd1BMvhaYwJ+FRGnt6Fe6+J8a8g6iz2A36djzJ9BMgDZViQNBlan\nt0N+QXKL5DHgFEmfSNvspeLna34ZGCjpwHT5DODx9J76HhExjySghjex7waSobCb8nOSWaZOJwkF\n2lpnOqja/wHGSBpCMkPXRuB9SX8FHN9MLQuBsZvPSdKukpq6urIccRBYZ/FD4ExJy0hup2xsos1p\nwAuSlpLMRfDT9EmdrwH/T9LzwK9Ibpu0KiJqSUZ2vE/ScqABuIXkQ/Wh9Hj/RdP32O8AbtncWdzo\nuOuBlcCAiHg2XdfmOtO+h++SjDC6jGSu4peAu0huN202E3hY0vyIWEvyRNPd6fs8TfL7tBzz6KNm\nZjnnKwIzs5xzEJiZ5ZyDwMws5xwEZmY55yAwM8s5B4GZWc45CMzMcu7/A59ISV6rDiB9AAAAAElF\nTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BGEF_w-eZF1J",
        "colab_type": "text"
      },
      "source": [
        " Ahora calculamos F-score:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a8w0xy4YZFTo",
        "colab_type": "code",
        "outputId": "35e2ae56-76bd-4854-9ac7-044fc960da20",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "from sklearn.metrics import fbeta_score\n",
        "\n",
        "# Parametro average: se utiliza en multiclase, si no está seteado devuelve la métrica para cada clase\n",
        "# Si se setea se devuelve una métrica única para el modelo (por ejemplo el promedio del indicador para todas las clases)\n",
        "\n",
        "fbeta_score(y_test, y_pred, average='micro', beta=0.5)\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9666666666666666"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-Qw6bnd5aXFH",
        "colab_type": "text"
      },
      "source": [
        "Por último, vamos a realizar un heatmap de la matriz de confusión del modelo:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pHgEEn_QFkZx",
        "colab_type": "code",
        "outputId": "60d42ef0-f1c8-4958-d1a5-33e21722e20d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        }
      },
      "source": [
        "import numpy as np\n",
        "import seaborn as sns; sns.set()\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "species = np.array(y_test)\n",
        "predictions = np.array(y_pred)\n",
        "\n",
        "mat = metrics.confusion_matrix(species, predictions)\n",
        "\n",
        "sns.heatmap(mat.T, square=True, annot=True, fmt='d', cbar=False,\n",
        "            xticklabels=iris.target_names, yticklabels=iris.target_names)\n",
        "\n",
        "plt.xlabel('true label')\n",
        "plt.ylabel('predicted label');\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEZCAYAAABy91VnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1gUZf8/8PeyQIJiiiEqKpoIaJYn\nSFFCWXk84KWAp0czw8yvRnl4yLP4S0pTgTSUykP6aJ5TAzUUjynmidCnB0iD9UBiGFjigfMuu/v9\ng6/7ExOdBXbWnX2/rqvrYmd25v6wk29m77nnHplOp9OBiIgkycrUBRARkfEw5ImIJIwhT0QkYQx5\nIiIJY8gTEUkYQ56ISMIY8kREEsaQF9nWrVsxbNgwdOrUCXPnztUvV6lUmDZtGhQKBTw8PJCSkmLC\nKolIKhjyImvatCnef/99DB8+/G/runXrhujoaDg5OZmgMiKSImtTF2Bp+vfvDwDIyMhAfn6+frmt\nrS3Gjx8PALCy4t9eIqobTBMiIgkT9Uy+sLAQ2dnZKC8v1y/z9vYWswQiIosiWsgfPHgQUVFRePDg\nAZo2bYqcnBx4enoiISHBoP2U7os2UoXiqlBeQMX9kif+PrqyQpSf2YnSvGQTVFa3HEauNHUJdapL\nl1dw6uQ+NGzkpl8W/q/J6NPHB8Eh401XGD2RpRyvClVutetE665Zs2YN4uPj4erqisOHD2P9+vV4\n9dVXxWqeqE4olddhbS2Hm1tb/bLXXuuIy5ezTFgVVYfHS8SQt7a2RpMmTaDRaAAAvXv3RkZGhljN\nPzcqNFqUqyug1emg1VX+XKHRAgBUFRqUqysAAGpN5c+cCfr5UlJSioS9SYhcOBP29nbo5eOFoUP6\nY+u270xdGj0Bj5eI3TW2trbQ6XRwdXXFli1b4OLigpKSErGaf258ffy/WHvsZ/3rA/+5hskBXRHW\nvxuCYvbgj7tFAID31x+uXD93FFwcHUxSKz3ZlKnzsf7r5fgjNx137tzFB1Pn4fJlpanLompY+vGS\nifXQkHPnzqFTp064c+cOIiMj8eDBA8ycORO9evUyaD9S6ZO3FFLrkyd6Hj2tT160kK8rDHnzwpAn\nMr7n4sLrxo0bUVhYCACYPXs2Bg4ciNOnT4vVPBGRRRIt5OPj4+Hg4IDz58/jzp07WLJkCVasWCFW\n80REFkm0kJfL5QCAlJQUDBkyBN26dePIESIiIxMt5OvVq4d169bhwIED6N27N3Q6HdRqtVjNExFZ\nJNFCfunSpfjzzz8xc+ZMODk54ebNmxgyZIhYzRMRWSTRR9c8HBtvb29fo+05usa8cHQNkfE9F6Nr\ncnJyMGrUKPTo0QM9e/bE6NGjcfPmTbGaJyKySKKF/MKFCzFq1Cikp6cjLS0NI0eOxEcffSRW80RE\nFkm0kC8oKMCIESMgk8kgk8kwfPhwFBQUiNU8EZFFEi3krayscP36df3r7Oxs/bBKIiIyDtEmKAsP\nD8fYsWPRoUMHAEBmZiaio3kRlYjImEQLeT8/Pxw4cABpaWkAgM6dO8PR0VGs5omILJJo3TWffvop\nHB0d4e/vD39/fzg6OuLTTz8Vq3kiIoskWshfuHDhb8tSU1PFap6IyCIZvbsmKSkJSUlJyM3NxfTp\n0/XLi4qKUK9ePWM3T0Rk0Ywe8m3btkXfvn2RkZGBvn376pc3aNAAPj4+xm6eiMiiGT3kPT094enp\nCYVCgUaNGhm7OSIieoRoffL37t3DmDFjoFAoAACXLl1CXFycWM0TEVkk0UI+MjISYWFhcHCofCh1\nhw4dcOjQIbGaJyKySKKFfGFhIfz8/CCTySobtrKCjY2NWM0TEVkkUZ8MpVar9SGfn58PKyvRmici\nskiipeybb76JKVOm4O7du4iLi8OYMWMwYcIEsZonIrJIok1rEBwcjJYtW+LEiRMoKytDdHQ0vLy8\nxGqeiMgiiXYmHx4eDg8PD0ydOhXHjh3D+++/jw0bNojVPBGRRRIt5LOzs+Hg4ICTJ0+iZ8+eSE5O\nxt69e8VqnojIIokW8hUVFQAq56vx8/ODnZ0dL7wSERmZaCnbrl07TJw4ESdOnICPjw/KysrEapqI\nyGKJduE1KioKp0+fhoeHB+zt7ZGfn48ZM2aI1TwRkUUSLeTr1auHgIAA/WtnZ2c4OzuL1TwRkUVi\npzgRkYQx5ImIJIwhT0QkYQx5IiIJY8gTEUkYQ56ISMIY8kREEsaQJyKSMIY8EZGEMeSJiCSMIU9E\nJGEMeSIiCWPIExFJGEOeiEjCGPJERBLGkCcikjCGPBGRhDHkiYgkjCFPRCRhMp1OpzN1EYawtnUx\ndQlkgNJbP5q6BDJQO/cgU5dABsopyKh2Hc/kiYgkjCFPRCRhDHkiIgljyBMRSRhDnohIwhjyREQS\nxpAnIpIwhjwRkYQx5ImIJIwhT0QkYQx5IiIJs65uxblz5wTtwMfHp86KISKiulVtyEdERDxzY5lM\nhuPHj9dpQUREVHeqDfkffvhBzDqIiMgIBPfJq9VqXLhwAQcPHgQAlJSUoKSkxGiFERFR7VV7Jv+o\nrKwshIWFwdbWFvn5+QgMDERqaioSEhIQGxtr7BqJiKiGBJ3JR0ZGYtq0aTh06BCsrSv/Lnh7e+Pi\nxYtGLY6IiGpHUMhfvXoVQUGVT4uRyWQAAHt7e5SXlxuvMiIiqjVBIe/i4oJffvmlyrL09HS0bt3a\nKEUREVHdENQnP336dEyePBmjR4+GWq3G2rVrsXPnTixatMjY9RERUS0IOpP39/fH+vXrUVBQAG9v\nb+Tm5iIuLg6+vr7Gro+IiGpBptPpdKYuwhDWti6mLoEMUHrrR1OXQAZq5x5k6hLIQDkFGdWuE9Rd\no1KpsHr1ahw4cAC3b99G06ZNERgYiLCwMLzwwgt1VigREdUtQSEfGRmJ7OxsREREwMXFBbm5uVi7\ndi3y8/OxdOlSY9dIREQ1JCjkjx8/jqNHj6Jhw4YAADc3N3Tu3Bn9+/c3anFERFQ7gi68vvTSSygt\nLa2yrLy8HE5OTkYpioiI6oagqYaDgoIwceJEjBs3Ds7OzsjLy8O2bdv0N0gREdHzqdrRNQqF4tkb\nm2CqYY6uMS8cXWN+OLrG/NRodA2nGiYiMn98/B8RkYQJGl1TVFSEuLg4pKam4u7du3i0h+fkyZPG\nqo2IiGpJ8FTDly9fxvvvv4979+5hwYIFaN68OcaPH2/k8oiIqDYEncmfOXMGBw8eROPGjSGXyxEQ\nEIBXX30V7733HoOeiOg5JuhMXqvVwsHBAUDlPPKFhYVwcnLCjRs3jFocERHVjqAzeU9PT6SmpsLH\nxwdeXl6IjIxE/fr10aZNGyOXR0REtSHoTH7x4sVwcakcnx4REYF69erhwYMHiI6ONmpxRERUO5xq\nmIyKN0OZH94MZX5qdDPUnj17BO18xIgRhldERESiqDbk9+3b98yNZTIZQ56I6DlWbchv2bJFzDqI\niMgIOK0BEZGEMeSJiCSMIU9EJGEMeSIiCav2wuvNmzcF7aBVq1Z1VgwREdWtakP+H//4B2QyGXQ6\nHWQymX75469//fVX41ZIREQ1Vm3IZ2Zm6n/+7rvvcPbsWUydOhUtWrTArVu38OWXX8LHx0eUIomI\nqGYETWvg5+eHI0eOoF69evplpaWlGDBgAE6dOmXUAh/HaQ3MC6c1MD+c1sD8PG1aA8FTDefm5lZZ\nduvWLWi12tpVRkRERiUo5MePH4/Q0FCsWLEC27dvx4oVKxAaGorQ0FBj1yd5jRs3wp7d63H/7hVc\nu5KC0aODTV0SPWL7nv0YNWEauvYdgojFy/XLr2XfwKgJ09Br4Ej0GjgSE6fPw7VsPl/heRQ6cQwS\nj+/ElT8uYvkXi01djugEzSc/ceJEuLu749ChQ7h8+TKcnJywZMkS+Pn5Gbs+yYtb9SlUKjVatOyM\nLp1fwf59m5GefhmXLytNXRoBcHqpCSaPH40zKRdRXq6qsvzzTyPQollTaLVa7IhPxMyFy5CwebUJ\nq6Unyc+7jVXL16GPoleVLmdLISjkgcp+eYZ63bK3t8OwkEB07toPxcUlOHM2Fd8nHsVbY4djfsRS\nU5dHAP7RtzcA4FLmFeTf/ku/vKFDAzR0aAAA0OkAuZUVbv7+h0lqpKc7lHgcAPBal1fQvAVD/olU\nKhW+/PJLJCYm4t69e7h48SJOnz6N3377DW+99dYzt9dqtVAqlfD09Kx1wVLi7v4yKio0uHLlun5Z\nevol+Plx1JK58BkwAiWlpdBqdZgycZypyyH6G0F98kuWLIFSqcRnn32mHyPfvn177NixQ1gjVlaY\nNWtWzauUqAb16+PBg8Iqy+7fL4RDg/omqogMde7wHpw7/B0iPnwfnu7tTF0O0d8IOpM/duwYjhw5\nAnt7e1hZVf5dcHZ2Rn5+vuCGXF1d8fvvv6Nly5Y1q1SCioqL0bChQ5VlDRs6oLCo2EQVUU3Y29XD\nqOBAvDF4NPZvX4cmjRuZuiQiPUEhb2NjA41GU2VZQUEBGjUS/j9zcXExhg4diu7du8Pe3l6/fOXK\nlYL3ITVK5XVYW8vh5tYWV69mAwBee60jLl/OMnFlZCitVoeysnLc/vMvhjw9VwSF/MCBAzFnzhzM\nmzcPAHD79m0sWbIEgwcPFtzQ0KFDMXTo0JpVKVElJaVI2JuEyIUzMWnyTHTp/AqGDumPN/rwZpTn\nRUWFBhqNBhqNFhqtFuXlKsjlcvz0nzQ0btQQ7u3aorSsDKvWbUZDhwZ42bW1qUumx8jlclhbyyGX\nW0Eut8ILL9jqj6slEHTHq0qlwmeffYbdu3ejtLQUdnZ2GDlyJGbOnAlbW1sx6tST2h2vjRs3wvqv\nlyOgnx/u3LmL+QuWYOfOvaYuq86Y+x2vX27YitX/3lZlWdiEsXBr64ovvt6MvD//Qr0XbNGpgwf+\n9d478HBra6JK647U7ngNnxOG8DnvV1n2edRX+DxKOsNdn3bHq6CQf1RBQQEaN25cZZIyodstWrQI\n586dAwD07t0bERERcHR0NGg/Ugt5qTP3kLdEUgt5S1DraQ1ef/11/c+Ojo76gDdkgrKFCxeiTZs2\n2LdvH/bu3QtXV1d89NFHgrcnIiLDCQp5tVr9xGWGzF2Tk5OD6dOnw9nZGc2aNcO0adMEz1lPREQ1\n89QLr2+++SZkMhlUKhXGjh1bZV1eXh66du0quCGtVos7d+6gSZMmAIA7d+5wgjMiIiN7asiPHDkS\nOp0OGRkZGDFihH65TCZDkyZN0LNnT8ENvfvuuwgODkbfvn0BAMnJyZgxY0bNqiYiIkEEXXi9du0a\n2rWr/d18SqUSP/30EwCgR48eaN++vcH74IVX88ILr+aHF17Nz9MuvAoaJ79jxw4EBgaiW7du+mX/\n+c9/kJSUhIiICMGFuLu7w93dXfD7iYiodgSdyffs2ROnTp2qMiZepVKhT58++iGR1Rk+fPhTh1vu\n2bPHgHJ5Jm9ueCZvfngmb35qfSb/8IHej9JoNIIunM6ZM0dIE0REZASCQt7LywuxsbGYNWsWrKys\noNVqERcXBy8vr2du++gYewAoKSkBgCrz1xARkXEICvmIiAhMnjwZvr6+aNGiBf744w84OTlhzZo1\nghu6efMmZsyYgV9//RUymQwdO3ZETEwMWrVqVePiiYjo6QRPa6DVapGWloa8vDw0b94cr732mn7a\nYSHeeecdDB48GMOHDwcAxMfHIzExERs3bjSoYPbJmxf2yZsf9smbn1r3yQOVD/4w5OanxxUUFFQZ\naz98+HBs3ry5xvsjIqJnqzbkBw0ahKSkJABAnz59qh0hc/LkSUENWVlZ4fr163j55ZcBANnZ2ZDL\n5QaWS0REhqg25BctWqT/OSYmptYNhYeHY+zYsejQoQMAIDMzE9HR0bXeLxERVc/gqYZro6CgAGlp\naQCAzp07GzzNMMA+eXPDPnnzwz5581OjPnmhj+WbPn26oPcplUq0bNkS/v7+ACofB3jlypUaTW1A\nRETCVBvyeXl5+p/Ly8tx5MgRdOrUCS4uLrh16xYyMjLQv39/wQ3NnTsX3377rf61jY0N5syZg/j4\n+BqWTkREz1JtyC9dulT/c3h4OJYvX44BAwbolx05cgSHDh0S3JBGo4GNjY3+ta2trcU8Y5GIyFQE\nDXQ/deoUAgICqixTKBRITk4W3JC1tXWVh4Tk5ORwdA0RkZEJGifv6uqKbdu24e2339Yv27FjB1q3\nFv5k+ilTpmDMmDHo06cPgMr55BcvXmxguUREZAhBo2suX76MKVOmoKKiAs7OzsjPz4e1tTXi4uLw\nyiuvCG4sOzsbZ8+eBQD4+vrC1dXV4II5usa8cHSN+eHoGvPztNE1godQqtVqpKWl4fbt23ByckKX\nLl2q9LGLhSFvXhjy5ochb37qZFqDR3l7e6OkpARqtfqZs0nOmjULMTEx1c4rb+h88kREJJygkM/K\nykJYWBhsbW2Rn5+PwMBApKamIiEhAbGxsU/dNjQ0FADnlSciMgVB3TVjxozBP//5TwQHB8Pb2xup\nqakoKSnBgAED8OOPhn8dV6lUuH//PpycnAzelt015oXdNeaH3TXm52ndNYKGUF69ehVBQZUH/mGX\ni729PcrLywUXER4ejsLCQpSVlWHIkCEYPHgwNmzYIHh7IiIynKCQd3FxwS+//FJlWXp6ukFDKLOz\ns+Hg4ICTJ0+iR48eSE5Oxt69ew2rloiIDCKoT3769OmYPHkyRo8eDbVajbVr12Lnzp1VZqp8loqK\nCgBAamoq+vTpAzs7O4MeOkJERIYTlLL+/v5Yv349CgoK4O3tjdzcXMTFxcHX11dwQ+3atcPEiRNx\n4sQJ+Pj4oKysrMZFExGRMM+88KrRaDB//nwsWrQItra2NW6orKwMp0+fhoeHB1q1aoX8/HxkZWXB\nz8/PoP3wwqt54YVX88MLr+anVuPk5XI5zpw5U+2ToYTQaDQYM2YMEhIS9MucnZ3h7Oxc430SEdGz\nCequCQ0NRVxcHNRqdY0akcvlBo/GISKi2hN04XXr1q3466+/sHHjRjg6OlY5qxf6jNe2bdti7Nix\nGDBgQJW7ZMeOHWtYxUREJJigkK+LZ7xqNBq0b98e169fr/W+iIhIGFGf8VoXeOHVvPDCq/nhhVfz\nU+s7XlUqFVauXIn+/fujS5cu6N+/P2JjYw3qYy8tLUVsbCxmzJgBALh27RqOHTsmeHsiIjKcoO6a\nyMhIZGdnIyIiAi4uLsjNzcXatWuRn59f5TGBz9qHk5MTMjMzAQDNmjXDjBkz/vbEKSIiqjuCQv74\n8eM4evQoGjZsCABwc3ND586dDXqQd1ZWFqKionD69GkAQP369aHVamtQMhERCSWou+all15CaWlp\nlWXl5eUGzSL5+I1U5eXlMLPLAUREZkfQmXxQUBAmTpyIcePGwdnZGXl5edi2bRuCgoJw7tw5/ft8\nfHyq3YeXlxfWrFkDlUqFlJQUbNy4EQqFova/ARERVUvQ6BohYSyTyXD8+PFq1xcVFWHLli344Ycf\noNPpoFAoMGnSJFhbG/ZwKo6uMS8cXWN+OLrG/NT68X8//PBDrYsICAhAv379MGfOHHh5edV6f0RE\n9GyizfV76NAhdOjQAUuWLMGAAQOwZs0a5OXlidU8EZFFMsnNUEqlEhs3bsT+/ftx6dIlg7Zld415\nYXeN+WF3jfmpdXdNXdFqtUhOTkZCQgJSU1MREhIiZvNERBZHtJBfunQpDh48iPbt2yM4OBjR0dGo\nV6+eWM0TEVkk0UK+UaNG2LVrF5o3by5Wk0REFk+0kA8LCxOrKSIi+j98kjYRkYSJeuGVLI9dizdM\nXQIZaHVTf1OXQHWIZ/JERBLGkCcikjCGPBGRhDHkiYgkjCFPRCRhDHkiIgljyBMRSRhDnohIwhjy\nREQSxpAnIpIwhjwRkYQx5ImIJIwhT0QkYQx5IiIJY8gTEUkYQ56ISMIY8kREEsaQJyKSMIY8EZGE\nMeSJiCSMIU9EJGEMeSIiCWPIExFJGEOeiEjCGPJERBLGkCcikjCGPBGRhDHkiYgkjCFPRCRhDHki\nIgljyBMRSRhDnohIwhjyREQSxpAnIpIwhjwRkYQx5ImIJIwhT0QkYQx5IiIJY8gTEUkYQ56ISMIY\n8kREEsaQJyKSMIY8EZGEMeSJiCSMIU9EJGEMeSIiCWPIExFJGEOeiEjCGPJERBLGkDexxo0bYc/u\n9bh/9wquXUnB6NHBpi6JnoHHzPwM3h2Bd67+G+Oz1mN81nqMTI4xdUmisTZ1AZYubtWnUKnUaNGy\nM7p0fgX7921GevplXL6sNHVpVA0eM/N09v9tRtaOk6YuQ3Q8kzche3s7DAsJxMLIGBQXl+DM2VR8\nn3gUb40dburSqBo8ZmRuGPIm5O7+MioqNLhy5bp+WXr6JXTs6GHCquhpeMzMl/fcURiXvhpDEj5C\nc58Opi5HNDKdTqcTq7Hr168jMzMTKpVKvyw4mP2ZRETGIlqf/ObNm/Htt9/izz//xKuvvooLFy7A\n29ubIU9EZESiddfs2rULu3fvRvPmzbFhwwbs3r0b9evXF6t5IiKLJFrI29rawt7eHlqtFjqdDu7u\n7vjtt9/Eap6IyCKJ1l1jZ2cHtVoNT09PxMTEoHnz5tBqtWI1T0RkkUS78KpUKtGyZUuUlpZixYoV\nKCwsRFhYGDp0sJyr3EREYhN1dA0REYlLtD75qVOn4t69e/rXd+/exfTp08VqnojIIokW8jdv3kSj\nRo30rxs3boycnByxmiciskiihbxGo4FGo9G/VqvVVW6KIiKiuifa6BpfX1+Eh4fj7bffBlB5c9Qb\nb7whVvNERBZJtDP5Dz/8EO7u7li2bBmWLVsGd3d3fPjhh2I1/1z59ddfcfDgQVOXQQbKyMjAjBkz\narx9SkoKhg0bVocVWY4dO3Zg06ZNNdo2IiICFy5ceOb7Vq5cKcl/lxxdYwLx8fE4efIkVq1aZepS\n6DEVFRWwtjbOF9yUlBRERUUhPj7eoO00Gg3kcrlRapICfj5PZ/TumqSkJAwaNAjbtm174vqxY8ca\nuwSjKy0txZw5c3D16lVYW1ujbdu2WLlyJRISErB9+3ZoNBo0aNAAkZGRaNy4MVatWoWioiIEBQXB\n29sbCxYswKlTp7BixQpoNBo4Ojrik08+gaurK65fv4558+ahtLQUWq0WISEhePfdd3Hu3DnExsai\nvLwcGo0G7733HgYPHmzqj8LkvvrqK9y7dw/z588HUDmKa+DAgTh27Bi++uorpKamQqVSwcPDA5GR\nkahfvz7mzp0LuVyO7OxsFBcXY+fOnU88no+H9IkTJxAXF4eKigpYWVlh2bJl8PT0rPZYPm7v3r3Y\nsGEDAKB169b45JNP0KRJE8THx2P//v2oX78+bty4gZiYGIu5n6S64/fwG9CcOXOe+PnY2Njo/514\nenoiJycHYWFh8Pf3x7hx4zBhwgT4+/tj7ty5sLW1xW+//Ya8vDx06dIFUVFRkMlkmDt3Ljp16oS3\n3noLKpUKn3/+OX788UdYWVmhVatW+PLLL5GVlYWPP/4YpaWlKC8vx6hRozB+/HgTfmLPZvSQv3Ll\nCgYNGoRffvnF2E2ZzOnTp1FcXKz/qnf//n1cuHABSUlJ2LZtG2xtbZGcnIz58+dj586dmDZtWpUz\n+Tt37mD27NnYunUr3NzcsHv3bsycORO7d+/G9u3boVAoMHnyZP2+AaBjx47Yvn075HI5/vrrLwwb\nNgy+vr548cUXTfMhPCeCg4MxatQozJ49G9bW1khMTIRCocCWLVvg4OCAPXv2AABiYmKwbt06hIeH\nA6jsQtu6dSvs7e1x9OjRvx3Px2VnZ2PBggXYtm0b2rRpA5VKBZVK9dRj+SilUonPPvsM8fHxaNq0\nKWJjY7Fo0SLExsYCANLS0rBv3z60bt3amB/Xc6e642dvb4+SkhL9+x7/fIYNG4bQ0FAEBQUhIyMD\no0aNqraNK1euYNOmTZDJZAgJCcHZs2fRu3fvKu9Zt24dbt68ifj4eNja2qKgoAAA4OLigk2bNsHW\n1hbFxcUYOXIk3njjDbRr184In0bdMHrIT5s2DQCwdOlSYzdlMp6enrh27Ro+/vhjvP766+jbty9+\n+OEHZGZmYuTIkQAAnU6HBw8ePHH7tLQ0eHp6ws3NDQAwfPhwfPzxxygqKoK3tzdiYmJQWlqKHj16\noGfPngCAgoICzJ8/Hzdu3IBcLsf9+/eRnZ2NLl26iPNLP6datGgBNzc3JCcno1+/fkhISMC8efMQ\nFRWFoqIiHD58GACgUqng6emp327gwIGwt7cH8OTj+bizZ8/Cz88Pbdq0AVA5N5OtrS1++umnao/l\no1JSUtCnTx80bdoUADB69GgEBQXp13fr1s3iAh6o/vidP3++yvse/XyKioqgVCoxZMgQAMCrr74K\nD4/q5/cPCAjACy+8AKDyZCknJ+dvIX/ixAn9WT8AODo6AgDKysoQGRmJrKwsyGQy3L59G5mZmZYd\n8o86d+4ccnJyUFFRoV8mhe6aVq1aITExEefPn8epU6fw+eefo1+/fhg+fHitb/gaMGAAunTpgjNn\nzuDrr7/Gd999h88++wyRkZFQKBT44osvIJPJMGDAAJSXl9fRb2TeQkJCsHfvXrRs2RKFhYXw8vKC\nTqfDwoUL4ePj88RtHgY88OTj+f3334tVPgBY9AytTzp+j4f8kz4fmUwmaP8PAx4A5HJ5laHdz7Ji\nxQo4OTlh2bJlsLa2xoQJE577f3eija6ZM2cOPv30U1y8eBG//PKL/j8pyMvLg1wuR0BAAObNm4eC\nggIoFArs27cPeXl5ACovDj38fRs0aIDCwkL99l26dEFmZiauXbsGAEhISEDHjh3RoEED3LhxA05O\nThg2bBg++OADZGRkAAAKCwvh4uICmUyGM2fO4MaNGyL/1s+v/v37IzU1FRs3bkRISAhkMhkUCgU2\nbdqEsrIyAJVnfw8/78c96Xg+erc2APTu3RunTp3Sz6SqUqlQVFT01GP5qB49eiA5ORl//vkngMqp\nuHv16lWXH4PZetLxe5oGDRqgffv2SExMBABcunQJSmXtnrfr7++Pb775Rn8vz8PumsLCQjRr1gzW\n1tZQKpWCRu2Ymmhn8v/972EUaG8AAAb7SURBVH+RmJgIGxsbsZoUTVZWFpYvXw4A0Gq1mDRpEry9\nvfGvf/0LYWFh0Gg0UKvVGDhwIDp16gQfHx/8+9//xtChQ/H6669jwYIFiI6OxsyZM1FRUQFHR0fE\nxFQ+TT4pKQnff/89bGxsIJPJ9BekZsyYgY8//hhxcXHP/Hpqaezs7NCvXz/Ex8fj+PHjAIBJkybh\niy++wIgRIyCTySCTyTBlypQnfs1+0vF0dnauMjV2mzZtsGjRIoSHh+tHdyxbtgweHh7VHstHubu7\nY+bMmZgwYQKAym8Pn3zyiRE+DfPzpOP3LFFRUZg/fz7WrVsHd3d3uLu7w8HBocY1TJo0CcuXL0dw\ncDBsbGzg6uqKVatWISwsDLNnz8aePXvQtm1beHt717gNsYg2hDI0NBTr16+XZMgTkWkVFxfD3t4e\nMpkMV69exbhx43Do0CGLH4gAiHgm36ZNG4wfPx4BAQH6ixmANPrkici0fv75Z0RHR+PhOeuiRYsY\n8P9HtJBXqVRo3bp1rfvKiIge5+vrC19fX1OX8VziHa9ERBJm9DP5ixcvonv37khOTn7i+j59+hi7\nBCIii2X0kE9ISED37t2xfv36v62TyWQMeSIiI2J3DVENpaSkYNasWTh16tQz3xsfH4/du3djx44d\nBrdTm22JRLvw+qTumgYNGtR6PCvRQwqFAosXL+ZNRUSPEC3kv/rqK2RkZOhv2lEqlfDw8EB+fj4W\nL14Mf39/sUohC2XMaYSJnleiTWvQunVr7Nq1CwkJCUhISMCuXbvw8ssvY/PmzfqZ94hqatasWbh1\n6xbee+89dO3aFV9//TV+//13eHh4YPfu3ejbty9CQ0ORkpICPz+/KtsqFAqcPXsWQOUdruvWrUNA\nQAB69OiB6dOn/21Kg+o83K5r164IDAzE0aNHq6zX6XT45JNP0L17dwwcOBDnzp3TryssLMT8+fPh\n6+uLN954A59//rlBc6oQVUe0kM/MzESnTp30r1955RUolUq0a9cOvCxAtRUTE4MWLVpgzZo1+Pnn\nn/E///M/+nWpqak4ePCgfu72p9myZQuOHTuGrVu34scff8SLL74oeLqBVq1aYdu2bbh48SKmTJmC\nWbNm4fbt2/r16enpaN26Nc6fP49p06ZhypQp+j8gc+fOhbW1NY4cOYK9e/fizJkzf5uemKgmRAt5\nOzs7/QRCAJCYmIh69eoBED57HFFNTJ06Ffb29vr/355m586dCA8PR7NmzWBra4spU6bg8OHDVWZO\nrc6gQYPg7OwMKysrBAYGwtXVFenp6fr1jo6OCA0NhY2NDQIDA9G2bVucPHkSf/31l/55A/b29mjS\npAnGjx+PAwcO1Or3JgJE7JNfunQpZs2apZ9gy83NDdHR0SgpKcHs2bPFKoMsULNmzQS/99atW/jg\ngw9gZfX/z3+srKxw584dODs7P3XbvXv3YuPGjcjNzQUAlJSU4O7du/r1zs7OVU5oWrRogdu3b+PW\nrVuoqKiocsemVqtF8+bNBddNVB1RQl6r1UKtViM+Pl7/8IRHp159fMJ+orr0aLDa2dnppxsGKqeA\nfjiNLFD5B2HJkiXo3r27QW3k5uZiwYIF2LRpE7p27Qq5XF7lISAAkJ+fD51Op6/njz/+gEKh0H9r\nOH/+PC8MU50TpbvGysoKs2bNAlAZ7o/PrU1UF1566SXcvHnzqe9p27YtysvLcfLkSajVaqxevVo/\nZzgAjBkzBrGxsfqz8YKCAhw7duyZbZeWlkImk+mfIPTdd9/hypUrVd5TUFCAzZs3Q61WIykpCdeu\nXdM/Hap3795YtmwZioqKoNVqkZOTg59++snQj4Dob0Trk3d1dcXvv/8uVnNkgSZNmoTVq1fDy8ur\n2ousDg4OWLhwIRYsWAA/Pz/Y2dlV6c55++23oVAoMGHCBHTt2hWjRo2q0q9eHTc3N0yYMAGjR49G\nr169oFQq0a1btyrvee2113Djxg307NkTsbGxWLVqFRo3bgwAiI6OhlqtRmBgILy9vTFt2jT9A0WI\nakO0O17feecdpKWloXv37lUetbZy5UoxmiciskiidQAOHToUQ4cOFas5IiIC564hIpI0o5/Jf/PN\nNwgNDUVUVNQTx8Nz+CQRkfEYPeRfeOEFAJXD2B7tiyciIuMTrbumZ8+e6NevH0JCQuDl5SVGk0RE\nFk+0kL937x4SExMRHx+P4uJihISEIDg42KC7EYmIyDAmufCqVCqxceNG7N+/H5cuXRK7eSIiiyHq\nPdRarRbJyclISEhAamoqQkJCxGyeiMjiiHYmv3TpUhw8eBDt27dHcHAw+vfvL2hWQCIiqjnRzuQb\nNWqEXbt2cWY9IiIR8WYoIiIJE22CMiIiEh9DnohIwhjyREQSxpAnIpKw/wUX05Fb1tvEbQAAAABJ\nRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ulWtSbCI5-tG",
        "colab_type": "text"
      },
      "source": [
        "## Referencias & Recursos Web\n",
        "\n",
        "- https://scikit-learn.org/stable/modules/naive_bayes.html\n",
        "\n",
        "- https://www.datacamp.com/community/tutorials/naive-bayes-scikit-learn\n",
        "\n",
        "- https://jakevdp.github.io/PythonDataScienceHandbook/05.05-naive-bayes.html\n",
        "\n",
        "- https://www.sicara.ai/blog/2018-02-28-naive-bayes-classification-sklearn\n",
        "\n",
        "- https://machinelearningmastery.com/roc-curves-and-precision-recall-curves-for-classification-in-python/\n"
      ]
    }
  ]
}