step_2_model_research.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fe1db50a-ca05-4dfa-a549-2e2a32c2c984",
   "metadata": {
    "tags": []
   },
   "source": [
    " # \"СберАвтоподписка\": разработка модели, предсказывающей целевое событие  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ecfba788-3fce-4777-aca2-cb2f57fb4292",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**Цель проекта**: разработать модель предсказания совершения одного из целевых действий (\"Заказать звонок\", \"Оставить заявку\") для сессий по введенным атрибутам типа utm_*, device_*, geo_* и упаковать модель в сервис.\n",
    "\n",
    "\n",
    "\n",
    "**Целевая метрика**: ориентировочное значение roc-auc > 0.65 — факт совершения\n",
    "пользователем целевого действия.\n",
    "\n",
    "**Формат вывода ответа** - 0/1\n",
    "\n",
    "**Скорость ответа сервиса** - не более 3 секунд\n",
    "\n",
    "**Сервис** - это должен быть (минимум) - py-скрипт с инструкцией по запуску, (максимум) - localhost web app.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d8e0393-5a37-4de4-a585-48fcc324c69b",
   "metadata": {},
   "source": [
    "В данной работе я буду использовать библиотеку [Feature-engine](https://feature-engine.trainindata.com/en/latest/). Мы не рассматривали ее в настоящем курсе, однако, в процессе самостоятельного изучения аспектов создания моделей предсказания, я нашел ее очень интересным инструментом и решил изучить и использовать его на практике.\n",
    "\n",
    "Вот небольшая характеристика Feature-engine:\n",
    "\n",
    "* Feature-engine содержит наиболее исчерпывающую коллекцию преобразований для разработки функций.\n",
    "\n",
    "* Feature-engine может преобразовывать определенную группу переменных в фрейм данных.\n",
    "\n",
    "* Feature-engine возвращает кадры данных, поэтому подходит для исследования данных и развертывания модели.\n",
    "\n",
    "* Feature-engine совместим с конвейером Scikit-learn, грид- и случайным поиском и перекрестной проверкой.\n",
    "\n",
    "* Feature-engine автоматически распознает числовые, категориальные переменные и переменные даты и времени.\n",
    "\n",
    "* Feature-engine предупреждает вас, если преобразование невозможно, например, если применяется логарифмирование к отрицательным переменным или деление на 0.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e50f8b1-e0c0-4d4d-8a7c-bd51ded7776e",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Импорт библиотек"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f794e5f7-3937-4fb1-9cd6-098492d72b2c",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Перед импортом библиотек предварительно устанавливается:\n",
    "* `pip install xgboost` согласно [документации](https://xgboost.readthedocs.io/en/stable/install.html#python)\n",
    "* `pip install lightgbm` согласно [документации](https://github.com/microsoft/LightGBM/tree/master/python-package)\n",
    "* `pip install bayesian-optimization` согласно [документации](https://github.com/fmfn/BayesianOptimization)\n",
    "* `pip install feature-engine`\n",
    "* `pip install bayesian-optimization`\n",
    "* `pip install colorama`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "id": "899a0201-7553-4a0d-a3ba-4de62f616f50",
   "metadata": {
    "id": "s45aMMmq98og"
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "from datetime import datetime\n",
    "import warnings\n",
    "from pathlib import Path\n",
    "from typing import Union\n",
    "from functools import partial\n",
    "\n",
    "\n",
    "import dill\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# препроцессинг и метрики\n",
    "from sklearn.preprocessing import FunctionTransformer, StandardScaler\n",
    "from sklearn.model_selection import (\n",
    "    train_test_split, GridSearchCV, StratifiedKFold)\n",
    "from sklearn.metrics import (\n",
    "    roc_auc_score, accuracy_score, confusion_matrix, precision_score, \n",
    "    recall_score, f1_score, make_scorer, roc_curve)\n",
    "from sklearn.pipeline import Pipeline\n",
    "from feature_engine.encoding import RareLabelEncoder, OneHotEncoder\n",
    "from feature_engine.wrappers import SklearnTransformerWrapper\n",
    "from feature_engine.outliers import Winsorizer\n",
    "from feature_engine.selection import (\n",
    "    DropDuplicateFeatures, DropConstantFeatures, \n",
    "    DropCorrelatedFeatures, DropFeatures)\n",
    "from feature_engine.transformation import YeoJohnsonTransformer\n",
    "from bayes_opt import BayesianOptimization\n",
    "\n",
    "# модели\n",
    "from sklearn.base import BaseEstimator\n",
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import (\n",
    "    RandomForestClassifier, HistGradientBoostingClassifier)\n",
    "from catboost import CatBoostClassifier\n",
    "from xgboost import XGBClassifier\n",
    "from lightgbm import LGBMClassifier\n",
    "\n",
    "# дополнительные данные\n",
    "from additional_data import (\n",
    "    get_distance_from_moscow, target_events, missing_values, organic_mediums, \n",
    "    social_media_sources, moscow_region_cities, big_cities, russian_holidays)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b11d547d-2ab6-4bb8-810a-a6916c831d2a",
   "metadata": {
    "id": "30fUVLzuiQEa",
    "tags": []
   },
   "source": [
    "### Создание констант"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "id": "050373e7-66db-4777-954b-8921c0bc4656",
   "metadata": {
    "id": "2BJ2wLSSWCH8"
   },
   "outputs": [],
   "source": [
    "# необходимо указать путь к папкам с данными и моделями\n",
    "\n",
    "data_folder = Path('..', 'data')\n",
    "models_folder = Path('.', 'models')\n",
    "\n",
    "sessions_filename = 'ga_sessions.csv'\n",
    "hits_filename = 'ga_hits.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "id": "892b0225-6cb6-4cae-bdcc-d925a3713635",
   "metadata": {
    "id": "s1ljYcGeWGTs"
   },
   "outputs": [],
   "source": [
    "TEST_SIZE = 200_000\n",
    "RANDOM_SEED = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05138d53-7927-4851-81fc-0ceb1586161c",
   "metadata": {
    "id": "W4r8kBnGh3xP",
    "tags": []
   },
   "source": [
    "### Настройка ноутбука"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "id": "58604011-229f-414d-98c9-64bb4a26a841",
   "metadata": {
    "id": "XWlAoE1QLlkl"
   },
   "outputs": [],
   "source": [
    "pd.set_option('display.max_columns', 100)\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9a5f3f7-dd54-4793-bade-5d06166086f8",
   "metadata": {
    "id": "_e6pxmqQi9Zv",
    "tags": []
   },
   "source": [
    "### Загрузка данных\n",
    "\n",
    "Для файла `ga_hits.csv` нужно загрузить только колонки 'session_id' и 'event_action', так как остальные не используются."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "id": "7c5cf862-04a8-409c-b7a6-55f04d7ddace",
   "metadata": {
    "id": "phTdIgg9MQkL",
    "tags": []
   },
   "outputs": [],
   "source": [
    "sessions = pd.read_csv(data_folder / sessions_filename)\n",
    "hits = pd.read_csv(data_folder / hits_filename, \n",
    "                   usecols=['session_id', 'event_action'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "id": "d1dc8b05-f505-40f5-a44a-08f98d7e384c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1860042 entries, 0 to 1860041\n",
      "Data columns (total 18 columns):\n",
      " #   Column                    Dtype \n",
      "---  ------                    ----- \n",
      " 0   session_id                object\n",
      " 1   client_id                 object\n",
      " 2   visit_date                object\n",
      " 3   visit_time                object\n",
      " 4   visit_number              int64 \n",
      " 5   utm_source                object\n",
      " 6   utm_medium                object\n",
      " 7   utm_campaign              object\n",
      " 8   utm_adcontent             object\n",
      " 9   utm_keyword               object\n",
      " 10  device_category           object\n",
      " 11  device_os                 object\n",
      " 12  device_brand              object\n",
      " 13  device_model              object\n",
      " 14  device_screen_resolution  object\n",
      " 15  device_browser            object\n",
      " 16  geo_country               object\n",
      " 17  geo_city                  object\n",
      "dtypes: int64(1), object(17)\n",
      "memory usage: 1.8 GB\n"
     ]
    }
   ],
   "source": [
    "sessions.info(memory_usage='deep')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "id": "3dd7bc93-60f0-489b-ae76-88e28f76e088",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 15726470 entries, 0 to 15726469\n",
      "Data columns (total 2 columns):\n",
      " #   Column        Dtype \n",
      "---  ------        ----- \n",
      " 0   session_id    object\n",
      " 1   event_action  object\n",
      "dtypes: object(2)\n",
      "memory usage: 2.5 GB\n"
     ]
    }
   ],
   "source": [
    "hits.info(memory_usage='deep')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75f12ae8-4979-4e37-829a-050369289f90",
   "metadata": {
    "id": "sG0_hMFGB9dn",
    "tags": []
   },
   "source": [
    "## Подготовка данных"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65d9f3d5-3c37-4e3a-a6f2-3eb625a0539e",
   "metadata": {
    "id": "O1OIZL9lr4tl",
    "tags": []
   },
   "source": [
    "### Целевая переменная  \n",
    "\n",
    "Целевая переменная считается положительной, если для сессии из `sessions` есть хотя бы одно целевое событие в `hits`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "id": "ac8efd10-a8b8-44ac-944a-a91f4bf68a52",
   "metadata": {
    "id": "M5fWRJzNvsg_"
   },
   "outputs": [],
   "source": [
    "hits['target'] = hits['event_action'].isin(target_events) # заполняет столбец hits['target'] 'True', если hits['event_action'] содержит целевую\n",
    "is_target_event = hits.groupby('session_id')['target'].any().astype(int) # target группирует по id (session_id) и по столбцу 'target', то есть все одинаковые session_id собираются"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "id": "813eaf62-dd8c-4f82-98fd-67d83b956035",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0           False\n",
       "1           False\n",
       "2           False\n",
       "3           False\n",
       "4           False\n",
       "            ...  \n",
       "15726465    False\n",
       "15726466    False\n",
       "15726467    False\n",
       "15726468    False\n",
       "15726469    False\n",
       "Name: target, Length: 15726470, dtype: bool"
      ]
     },
     "execution_count": 202,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hits['target']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "id": "77ce0eb7-3c1e-4dc7-9ce6-ab66605dd125",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "session_id\n",
       "1000009318903347362.1632663668.1632663668    0\n",
       "1000010177899156286.1635013443.1635013443    0\n",
       "1000013386240115915.1635402956.1635402956    0\n",
       "1000017303238376207.1623489300.1623489300    0\n",
       "1000020580299877109.1624943350.1624943350    0\n",
       "                                            ..\n",
       "999960188766601545.1626816843.1626816843     0\n",
       "99996598443387715.1626811203.1626811203      0\n",
       "999966717128502952.1638428330.1638428330     0\n",
       "999988617151873171.1623556243.1623556243     0\n",
       "999989480451054428.1634311006.1634311006     0\n",
       "Name: target, Length: 1734610, dtype: int32"
      ]
     },
     "execution_count": 203,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "is_target_event"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "id": "81a98655-8ca1-4d63-a35c-8262618b5e67",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "BqV9-6p7wgmj",
    "outputId": "754192e6-374d-4c7b-81ce-a931a9eceb56"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0    0.97295\n",
       "1.0    0.02705\n",
       "Name: target, dtype: float64"
      ]
     },
     "execution_count": 204,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target = pd.Series(is_target_event, index=sessions['session_id']).fillna(0.0) # Заполним пропуски 0,0\n",
    "target.value_counts(dropna=False, normalize=True) # посчитаем значения, так и есть - целевых значений  2,7%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "id": "06b77c4c-57d4-4fe4-bb1a-98fed888b1eb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "session_id\n",
       "9055434745589932991.1637753792.1637753792    0.0\n",
       "905544597018549464.1636867290.1636867290     0.0\n",
       "9055446045651783499.1640648526.1640648526    0.0\n",
       "9055447046360770272.1622255328.1622255328    0.0\n",
       "9055447046360770272.1622255345.1622255345    0.0\n",
       "                                            ... \n",
       "9055415581448263752.1640159305.1640159305    0.0\n",
       "9055421130527858185.1622007305.1622007305    0.0\n",
       "9055422955903931195.1636979515.1636979515    0.0\n",
       "905543020766873816.1638189404.1638189404     0.0\n",
       "9055430416266113553.1640968742.1640968742    0.0\n",
       "Name: target, Length: 1860042, dtype: float64"
      ]
     },
     "execution_count": 205,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "id": "b98b535d-1538-4633-823e-3f190b550d88",
   "metadata": {
    "id": "VuTc3J5CkhkT"
   },
   "outputs": [],
   "source": [
    "del hits # удалим датафрейм за ненадобностью чтобы не занимал много памяти"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b65898b4-acba-46a9-9100-7db91e2c8828",
   "metadata": {
    "id": "r5K3svqb1tjM",
    "tags": []
   },
   "source": [
    "### Заполнение пропусков  \n",
    "\n",
    "Пропуски в колонке `device_screen_resolution` заполняем самым частым значением.  \n",
    "Все остальные пропуски в колонках заполняем значением '(nan)'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "id": "6484af97-8486-41b6-b6dc-b4cc590a9f24",
   "metadata": {
    "id": "hPrvYDhY1QfY"
   },
   "outputs": [],
   "source": [
    "def fill_missings(data: pd.DataFrame) -> pd.DataFrame:\n",
    "    \"\"\"Заполняет пропущенные значения:\n",
    "    * самым частым значением для `device_screen_resolution`;\n",
    "    * значением '(nan)' во всех остальных случаях.\n",
    "    \"\"\"\n",
    "\n",
    "    data = data.copy()\n",
    "\n",
    "    if 'device_screen_resolution' in data.columns:\n",
    "        # '414x896' - самое частое значение в 'device_screen_resolution'\n",
    "        # согласно предварительному анализу данных\n",
    "        data['device_screen_resolution'] = \\\n",
    "            data['device_screen_resolution'].replace(missing_values, '414x896')\n",
    "    \n",
    "    return data.fillna('(nan)')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b396b3da-b7e4-4b39-aa04-8ab9c874f710",
   "metadata": {
    "id": "4jwtfKo29Ad7",
    "tags": []
   },
   "source": [
    "### Генерация признаков\n",
    "\n",
    "Создаётся множество дополнительных переменных: день недели и день месяца, является ли день выходным, час и минута посещения, ночью ли посещение, ширина, высота, площадь и соотношение экрана.\n",
    "\n",
    "В том числе с дополнительными данными: является ли день празничным, является ли трафик органическим, из социальных ли сетей он, находится ли пользователь в большом городе или в московской области, расстояние до Москвы как численно, так и в виде категорий."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "id": "cd8c23e3-75c5-4d31-b726-efd114bdfa14",
   "metadata": {
    "id": "RmS25hxv_s8V"
   },
   "outputs": [],
   "source": [
    "def distance_category(distance: float) -> str:\n",
    "    \"\"\"Возвращает категорию расстояния до Москвы.\"\"\"\n",
    "\n",
    "    if distance == -1: return 'no distance'\n",
    "    elif distance == 0: return 'moscow'\n",
    "    elif distance < 100: return '< 100 km'\n",
    "    elif distance < 500: return '100-500 km'\n",
    "    elif distance < 1000: return '500-1000 km'\n",
    "    elif distance < 3000: return '1000-3000 km'\n",
    "    else: return '>= 3000 km'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "id": "ef8cebbc-1316-44c0-bda8-216e74bd03e1",
   "metadata": {
    "id": "MXGGgtUq2frS"
   },
   "outputs": [],
   "source": [
    "def create_features(data: pd.DataFrame) -> pd.DataFrame:\n",
    "    \"\"\"Создаёт новые признаки из существующих.\"\"\"\n",
    "\n",
    "    data = data.copy()\n",
    "    \n",
    "    # visit_date признаки \n",
    "    if 'visit_date' in data.columns:\n",
    "        data['visit_date'] = data['visit_date'].astype('datetime64[ns]')\n",
    "        data['visit_date_added_holiday'] = \\\n",
    "            data['visit_date'].isin(russian_holidays)\n",
    "        # числовые признаки сделаем строго положительными \n",
    "        # для лучшей обработки на шаге с YeoJohnsonTransformer\n",
    "        data['visit_date_weekday'] = data['visit_date'].dt.weekday + 1\n",
    "        data['visit_date_weekend'] = data['visit_date'].dt.weekday > 4\n",
    "        data['visit_date_day'] = data['visit_date'].dt.day + 1\n",
    "\n",
    "    # visit_time признаки\n",
    "    if 'visit_time' in data.columns:\n",
    "        data['visit_time'] = data['visit_time'].astype('datetime64[ns]')\n",
    "        data['visit_time_hour'] = data['visit_time'].dt.hour + 1\n",
    "        data['visit_time_minute'] = data['visit_time'].dt.minute + 1\n",
    "        data['visit_time_night'] = data['visit_time'].dt.hour < 9\n",
    "\n",
    "    # utm_* признаки\n",
    "    if 'utm_medium' in data.columns:\n",
    "        data['utm_medium_added_is_organic'] = \\\n",
    "            data['utm_medium'].isin(organic_mediums)\n",
    "    if 'utm_source' in data.columns: \n",
    "        data['utm_source_added_is_social'] = \\\n",
    "            data['utm_source'].isin(social_media_sources)\n",
    "    \n",
    "    # device_screen признаки\n",
    "    if 'device_screen_resolution' in data.columns:\n",
    "        name = 'device_screen_resolution'\n",
    "        data[[name + '_width', name + '_height']] = \\\n",
    "            data[name].str.split('x', expand=True).astype(float)\n",
    "        data[name + '_area'] = data[name + '_width'] * data[name + '_height']\n",
    "        data[name + '_ratio'] = data[name + '_width'] / data[name + '_height']\n",
    "        data[name + '_ratio_greater_1'] = data[name + '_ratio'] > 1\n",
    "\n",
    "    # geo_city признаки \n",
    "    if 'geo_city' in data.columns:\n",
    "        data['geo_city_added_is_moscow_region'] = \\\n",
    "            data['geo_city'].isin(moscow_region_cities)\n",
    "        data['geo_city_added_is_big'] = data['geo_city'].isin(big_cities)\n",
    "        data['geo_city_is_big_or_in_moscow_region'] = \\\n",
    "            data['geo_city_added_is_moscow_region'] \\\n",
    "            | data['geo_city_added_is_big']\n",
    "        data['geo_city_added_distance_from_moscow'] = \\\n",
    "            data['geo_city'].apply(get_distance_from_moscow)\n",
    "        data['geo_city_added_distance_from_moscow_category'] = \\\n",
    "            data['geo_city_added_distance_from_moscow'].apply(distance_category)\n",
    "\n",
    "    return data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e99efb5a-34d3-46f5-a36c-71b6c8a6d795",
   "metadata": {
    "id": "SdmCT5bFkTpy",
    "tags": []
   },
   "source": [
    "### Дополнительно"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "id": "22ab52f0-7dd6-456c-a118-b47e175fa8d7",
   "metadata": {
    "id": "-h_M6P4-5Gag"
   },
   "outputs": [],
   "source": [
    "def set_index(data: pd.DataFrame, column: str = 'session_id') -> pd.DataFrame:\n",
    "    \"\"\"Устанавливает в качестве индекса датафрейма колонку `column`.\"\"\"\n",
    "    \n",
    "    data = data.copy()\n",
    "\n",
    "    if column in data.columns:\n",
    "        data = data.set_index(column)\n",
    "    \n",
    "    return data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "id": "829caacf-c302-45a5-b206-b58d740213f7",
   "metadata": {
    "id": "XVHwnlccAUWB"
   },
   "outputs": [],
   "source": [
    "def converse_types(data: pd.DataFrame) -> pd.DataFrame:\n",
    "    \"\"\"Приводит типы переменных к float. В первую очередь \n",
    "    необходимо для преобразования bool значений.\n",
    "    \"\"\"\n",
    "\n",
    "    return data.astype(float)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25b92a17-5a2e-4ae3-a5f8-512f7577646f",
   "metadata": {
    "id": "I4Owup725D1u",
    "tags": []
   },
   "source": [
    "### Собираем пайплайн  \n",
    "\n",
    "Пайплайн по подготовке данных состоит из 4 частей:  \n",
    "1. Создание дополнительных признаков\n",
    "2. Преобразование численных переменных\n",
    "3. Преобразование категориальных переменных\n",
    "4. Удаление лишних признаков"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "id": "3827e122-8c55-4067-8267-b33464c453d9",
   "metadata": {
    "id": "4aBifLGa14rl"
   },
   "outputs": [],
   "source": [
    "preprocessor = Pipeline(steps=[\n",
    "\n",
    "    # Создание дополнительных признаков и\n",
    "    # Приведение датафрейма к удобному виду \n",
    "    ('indexer', FunctionTransformer(set_index)), \n",
    "    ('imputer', FunctionTransformer(fill_missings)), \n",
    "    ('engineer', FunctionTransformer(create_features)), \n",
    "    ('dropper', DropFeatures(['client_id', 'visit_date', 'visit_time', \n",
    "                              'device_screen_resolution'])), \n",
    "\n",
    "    # Преобразования численных переменных\n",
    "    ('normalization', YeoJohnsonTransformer()), \n",
    "    ('outlier_remover', Winsorizer()), \n",
    "    ('scaler', SklearnTransformerWrapper(StandardScaler())), \n",
    "\n",
    "    # Преобразования категориальных признаков\n",
    "    ('rare_encoder', RareLabelEncoder(tol=0.05, replace_with='rare')), \n",
    "    ('onehot_encoder', OneHotEncoder(drop_last_binary=True)), \n",
    "    ('bool_converter', FunctionTransformer(converse_types)), \n",
    "\n",
    "    # Удаление дубликатов и коррелируемых признаков\n",
    "    ('constant_dropper', DropConstantFeatures(tol=0.99)), \n",
    "    ('duplicated_dropper', DropDuplicateFeatures()), \n",
    "    ('correlated_dropper', DropCorrelatedFeatures(threshold=0.8)), \n",
    "\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a434954f-a68a-472a-b8f0-8efffb72157e",
   "metadata": {
    "id": "Vapx7kE14bXl",
    "tags": []
   },
   "source": [
    "## Моделирование"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c5a3f14-7b2d-4284-929f-b8665f57f151",
   "metadata": {
    "id": "p7no5DMn5KQA",
    "tags": []
   },
   "source": [
    "### Разделение данных  \n",
    "\n",
    "Разделим данные на тренировочную, валидационную и тестовую выборки.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "id": "c5968ea4-77c4-4486-83f6-90fd4b85eabc",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IbEwzDFl5SBC",
    "outputId": "fe8d6058-adff-403e-e6bb-9c3303eac4d1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train shapes: (1460042, 18) (1460042,)\n",
      "valid shapes: (200000, 18) (200000,)\n",
      "test  shapes: (200000, 18) (200000,)\n"
     ]
    }
   ],
   "source": [
    "X, X_test, y, y_test = train_test_split(\n",
    "    sessions, target, test_size=TEST_SIZE, \n",
    "    stratify=target, random_state=RANDOM_SEED)\n",
    "\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(\n",
    "    X, y, test_size=TEST_SIZE, \n",
    "    stratify=y, random_state=RANDOM_SEED)\n",
    "\n",
    "print(f'train shapes: {X_train.shape} {y_train.shape}')\n",
    "print(f'valid shapes: {X_valid.shape} {y_valid.shape}')\n",
    "print(f'test  shapes: {X_test.shape} {y_test.shape}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3d98cba-ece3-4253-9aa3-accb19aad0c5",
   "metadata": {
    "id": "faISJrub1s56",
    "tags": []
   },
   "source": [
    "### Препроцессинг данных  \n",
    "\n",
    "Преобразуем тренировочные и валидационные данные с помощью построенного конвейера."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 654,
   "id": "567f0ae0-c1d8-4bab-860d-c378063ecf3a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>session_id</th>\n",
       "      <th>client_id</th>\n",
       "      <th>visit_date</th>\n",
       "      <th>visit_time</th>\n",
       "      <th>visit_number</th>\n",
       "      <th>utm_source</th>\n",
       "      <th>utm_medium</th>\n",
       "      <th>utm_campaign</th>\n",
       "      <th>utm_adcontent</th>\n",
       "      <th>utm_keyword</th>\n",
       "      <th>device_category</th>\n",
       "      <th>device_os</th>\n",
       "      <th>device_brand</th>\n",
       "      <th>device_model</th>\n",
       "      <th>device_screen_resolution</th>\n",
       "      <th>device_browser</th>\n",
       "      <th>geo_country</th>\n",
       "      <th>geo_city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1056663</th>\n",
       "      <td>547332070992313735.1638727047.1638727047</td>\n",
       "      <td>127435678.163873</td>\n",
       "      <td>2021-12-05</td>\n",
       "      <td>20:57:27</td>\n",
       "      <td>1</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>TmThBvoCcwkCZZUWACYq</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>mobile</td>\n",
       "      <td>iOS</td>\n",
       "      <td>Apple</td>\n",
       "      <td>NaN</td>\n",
       "      <td>428x926</td>\n",
       "      <td>Safari</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Novotroitsk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1409550</th>\n",
       "      <td>7049980726847853060.1629809978.1629809978</td>\n",
       "      <td>1641451550.162934</td>\n",
       "      <td>2021-08-24</td>\n",
       "      <td>15:00:00</td>\n",
       "      <td>5</td>\n",
       "      <td>MvfHsxITijuriZxsqZqt</td>\n",
       "      <td>cpm</td>\n",
       "      <td>FTjNLDyTrXaWYgZymFkV</td>\n",
       "      <td>PkybGvWbaqORmxjNunqZ</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Xiaomi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>393x851</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Balashikha</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>739873</th>\n",
       "      <td>4064566220882386809.1639930788.1639930788</td>\n",
       "      <td>946355569.163992</td>\n",
       "      <td>2021-12-19</td>\n",
       "      <td>19:19:48</td>\n",
       "      <td>2</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>LEoPHuyFvzoNfnzGgfcd</td>\n",
       "      <td>vCIpmpaGBnIQhyYNkXqp</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>mobile</td>\n",
       "      <td>Android</td>\n",
       "      <td>Samsung</td>\n",
       "      <td>NaN</td>\n",
       "      <td>339x753</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Almetyevsk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62902</th>\n",
       "      <td>1033849280328168809.1631647080.1631647080</td>\n",
       "      <td>240711793.163165</td>\n",
       "      <td>2021-09-14</td>\n",
       "      <td>22:00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>gecBYcKZCPMcVYdSSzKP</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Apple</td>\n",
       "      <td>NaN</td>\n",
       "      <td>414x896</td>\n",
       "      <td>Safari</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Saint Petersburg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1297234</th>\n",
       "      <td>6544287746962620171.1638268683.1638268683</td>\n",
       "      <td>1523710728.163827</td>\n",
       "      <td>2021-11-30</td>\n",
       "      <td>13:38:03</td>\n",
       "      <td>1</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>LEoPHuyFvzoNfnzGgfcd</td>\n",
       "      <td>vCIpmpaGBnIQhyYNkXqp</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>mobile</td>\n",
       "      <td>Android</td>\n",
       "      <td>Xiaomi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>393x851</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Moscow</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1793000</th>\n",
       "      <td>8755082352522037809.1633552946.1633552946</td>\n",
       "      <td>2038451459.163355</td>\n",
       "      <td>2021-10-06</td>\n",
       "      <td>23:00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>fDLlAcSmythWSCVMvqvL</td>\n",
       "      <td>(none)</td>\n",
       "      <td>LTuZkdKfxRGVceoWkVyg</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Samsung</td>\n",
       "      <td>NaN</td>\n",
       "      <td>360x800</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Moscow</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>207261</th>\n",
       "      <td>1682284231298311525.1640424805.1640424805</td>\n",
       "      <td>391687320.1640424805</td>\n",
       "      <td>2021-12-25</td>\n",
       "      <td>12:33:25</td>\n",
       "      <td>1</td>\n",
       "      <td>fDLlAcSmythWSCVMvqvL</td>\n",
       "      <td>(none)</td>\n",
       "      <td>LTuZkdKfxRGVceoWkVyg</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>mobile</td>\n",
       "      <td>iOS</td>\n",
       "      <td>Apple</td>\n",
       "      <td>NaN</td>\n",
       "      <td>375x812</td>\n",
       "      <td>Safari</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Saint Petersburg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93412</th>\n",
       "      <td>1170303784354950884.1640345317.1640345317</td>\n",
       "      <td>272482583.164035</td>\n",
       "      <td>2021-12-24</td>\n",
       "      <td>14:28:37</td>\n",
       "      <td>1</td>\n",
       "      <td>fDLlAcSmythWSCVMvqvL</td>\n",
       "      <td>(none)</td>\n",
       "      <td>LTuZkdKfxRGVceoWkVyg</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>desktop</td>\n",
       "      <td>Windows</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1920x1080</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Moscow</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1568591</th>\n",
       "      <td>7753976912353269271.1634047522.1634047522</td>\n",
       "      <td>1805363435.163405</td>\n",
       "      <td>2021-10-12</td>\n",
       "      <td>17:00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>gecBYcKZCPMcVYdSSzKP</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Xiaomi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>393x851</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Novosibirsk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1119538</th>\n",
       "      <td>5753600408244704247.1626854393.1626854393</td>\n",
       "      <td>1339614486.162685</td>\n",
       "      <td>2021-07-21</td>\n",
       "      <td>10:00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>MvfHsxITijuriZxsqZqt</td>\n",
       "      <td>cpm</td>\n",
       "      <td>FTjNLDyTrXaWYgZymFkV</td>\n",
       "      <td>dUuXlWzvmhDSyclWRhNP</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Xiaomi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>393x851</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Saint Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1460042 rows × 18 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                        session_id             client_id  \\\n",
       "1056663   547332070992313735.1638727047.1638727047      127435678.163873   \n",
       "1409550  7049980726847853060.1629809978.1629809978     1641451550.162934   \n",
       "739873   4064566220882386809.1639930788.1639930788      946355569.163992   \n",
       "62902    1033849280328168809.1631647080.1631647080      240711793.163165   \n",
       "1297234  6544287746962620171.1638268683.1638268683     1523710728.163827   \n",
       "...                                            ...                   ...   \n",
       "1793000  8755082352522037809.1633552946.1633552946     2038451459.163355   \n",
       "207261   1682284231298311525.1640424805.1640424805  391687320.1640424805   \n",
       "93412    1170303784354950884.1640345317.1640345317      272482583.164035   \n",
       "1568591  7753976912353269271.1634047522.1634047522     1805363435.163405   \n",
       "1119538  5753600408244704247.1626854393.1626854393     1339614486.162685   \n",
       "\n",
       "         visit_date visit_time  visit_number            utm_source utm_medium  \\\n",
       "1056663  2021-12-05   20:57:27             1  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "1409550  2021-08-24   15:00:00             5  MvfHsxITijuriZxsqZqt        cpm   \n",
       "739873   2021-12-19   19:19:48             2  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "62902    2021-09-14   22:00:00             1  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "1297234  2021-11-30   13:38:03             1  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "...             ...        ...           ...                   ...        ...   \n",
       "1793000  2021-10-06   23:00:00             1  fDLlAcSmythWSCVMvqvL     (none)   \n",
       "207261   2021-12-25   12:33:25             1  fDLlAcSmythWSCVMvqvL     (none)   \n",
       "93412    2021-12-24   14:28:37             1  fDLlAcSmythWSCVMvqvL     (none)   \n",
       "1568591  2021-10-12   17:00:00             1  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "1119538  2021-07-21   10:00:00             1  MvfHsxITijuriZxsqZqt        cpm   \n",
       "\n",
       "                 utm_campaign         utm_adcontent           utm_keyword  \\\n",
       "1056663  TmThBvoCcwkCZZUWACYq  JNHcPlZPxEMWDnRiyoBf  puhZPIYqKXeFPaUviSjo   \n",
       "1409550  FTjNLDyTrXaWYgZymFkV  PkybGvWbaqORmxjNunqZ                   NaN   \n",
       "739873   LEoPHuyFvzoNfnzGgfcd  vCIpmpaGBnIQhyYNkXqp  puhZPIYqKXeFPaUviSjo   \n",
       "62902    gecBYcKZCPMcVYdSSzKP  JNHcPlZPxEMWDnRiyoBf                   NaN   \n",
       "1297234  LEoPHuyFvzoNfnzGgfcd  vCIpmpaGBnIQhyYNkXqp  puhZPIYqKXeFPaUviSjo   \n",
       "...                       ...                   ...                   ...   \n",
       "1793000  LTuZkdKfxRGVceoWkVyg  JNHcPlZPxEMWDnRiyoBf                   NaN   \n",
       "207261   LTuZkdKfxRGVceoWkVyg  JNHcPlZPxEMWDnRiyoBf  puhZPIYqKXeFPaUviSjo   \n",
       "93412    LTuZkdKfxRGVceoWkVyg  JNHcPlZPxEMWDnRiyoBf  puhZPIYqKXeFPaUviSjo   \n",
       "1568591  gecBYcKZCPMcVYdSSzKP  JNHcPlZPxEMWDnRiyoBf                   NaN   \n",
       "1119538  FTjNLDyTrXaWYgZymFkV  dUuXlWzvmhDSyclWRhNP                   NaN   \n",
       "\n",
       "        device_category device_os device_brand device_model  \\\n",
       "1056663          mobile       iOS        Apple          NaN   \n",
       "1409550          mobile       NaN       Xiaomi          NaN   \n",
       "739873           mobile   Android      Samsung          NaN   \n",
       "62902            mobile       NaN        Apple          NaN   \n",
       "1297234          mobile   Android       Xiaomi          NaN   \n",
       "...                 ...       ...          ...          ...   \n",
       "1793000          mobile       NaN      Samsung          NaN   \n",
       "207261           mobile       iOS        Apple          NaN   \n",
       "93412           desktop   Windows          NaN          NaN   \n",
       "1568591          mobile       NaN       Xiaomi          NaN   \n",
       "1119538          mobile       NaN       Xiaomi          NaN   \n",
       "\n",
       "        device_screen_resolution device_browser geo_country          geo_city  \n",
       "1056663                  428x926         Safari      Russia       Novotroitsk  \n",
       "1409550                  393x851         Chrome      Russia        Balashikha  \n",
       "739873                   339x753         Chrome      Russia        Almetyevsk  \n",
       "62902                    414x896         Safari      Russia  Saint Petersburg  \n",
       "1297234                  393x851         Chrome      Russia            Moscow  \n",
       "...                          ...            ...         ...               ...  \n",
       "1793000                  360x800         Chrome      Russia            Moscow  \n",
       "207261                   375x812         Safari      Russia  Saint Petersburg  \n",
       "93412                  1920x1080         Chrome      Russia            Moscow  \n",
       "1568591                  393x851         Chrome      Russia       Novosibirsk  \n",
       "1119538                  393x851         Chrome      Russia  Saint Petersburg  \n",
       "\n",
       "[1460042 rows x 18 columns]"
      ]
     },
     "execution_count": 654,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "id": "420c3d7c-cc04-413e-9658-c2d130d2685b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2vtF_r6P1sen",
    "outputId": "491885c9-2d6a-4a7e-d1a2-1e5325b51986"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train.shape = (1460042, 55)\n"
     ]
    }
   ],
   "source": [
    "X_train_preprocessed = preprocessor.fit_transform(X_train)\n",
    "X_valid_preprocessed = preprocessor.transform(X_valid)\n",
    "\n",
    "print(f'X_train.shape = {X_train_preprocessed.shape}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a260a92f-8924-4c21-be3a-590857de7db5",
   "metadata": {
    "id": "-G6lNagZNfgM",
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "### Выбор метрик  \n",
    "\n",
    "Будем использовать в качестве основной метрики - `roc_auc`. Но также взглянем и на другие метрики: `accuracy`, `precision`, `recall`, `f1`.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "id": "697ca8a7-e512-4fcf-aae6-b50d8fe9cab5",
   "metadata": {
    "id": "klHCY5FJOgDz"
   },
   "outputs": [],
   "source": [
    "def find_best_threshold(\n",
    "    y_true: pd.Series, \n",
    "    y_proba: pd.Series, \n",
    "    metriс_name: str = 'roc_auc', \n",
    "    iterations: int = 250, \n",
    "    learning_rate: float = 0.05\n",
    ") -> float:\n",
    "    \"\"\"Находит лучший порог перевода вероятностей `y_proba` \n",
    "    в принадлежность к классу 1.\n",
    "    \"\"\"\n",
    "    \n",
    "    # Получение функции метрики, которую оптимизируем\n",
    "    metrics = {'roc_auc': roc_auc_score, 'f1': f1_score, \n",
    "               'precision': precision_score, 'recall': recall_score}\n",
    "    metric_function = metrics.get(metriс_name, accuracy_score)\n",
    "\n",
    "    # Получение метрики\n",
    "    def get_metric(threshold: float) -> float:\n",
    "        prediction = (y_proba > threshold).astype(int)\n",
    "        return metric_function(y_true, prediction)\n",
    "\n",
    "    direction = -1\n",
    "    shift = 0.25\n",
    "\n",
    "    best_threshold = 0.5\n",
    "    best_metric = get_metric(best_threshold)\n",
    "\n",
    "    # На каждой итерации\n",
    "    for i in range(iterations):\n",
    "\n",
    "        # Меняем порог\n",
    "        threshold = best_threshold + direction * shift\n",
    "        shift *= (1 - learning_rate)\n",
    "        metric = get_metric(threshold)\n",
    "\n",
    "        # И проверяем, улучшилась ли метрика\n",
    "        if metric > best_metric: \n",
    "            best_threshold = threshold\n",
    "            best_metric = metric\n",
    "        else: \n",
    "            direction *= -1\n",
    "            \n",
    "    return best_threshold"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "id": "7bfd74bc-a35a-4619-9f09-24e459455ba2",
   "metadata": {
    "id": "Gy9AoqAeowRX"
   },
   "outputs": [],
   "source": [
    "def print_metrics(\n",
    "    model: BaseEstimator, \n",
    "    X: pd.DataFrame, \n",
    "    y: pd.Series, \n",
    "    threshold: Union[float, None] = None, \n",
    "    show_roc_curve: bool = False\n",
    ") -> None:\n",
    "    \"\"\"Получает метрики бинарной классификации из модели `model` на данных \n",
    "    `X` и `y`. Если возможно, то через метод `predict_proba` с заданным \n",
    "    порогом перевода вероятностей в классы `threshold`, иначе через `predict`. \n",
    "    Если `threshold` равен None, автоматически найдёт лучший порог.  \n",
    "    ---\n",
    "    Метрики: roc_auc, accuracy, precision, recall, f1, confusion_matrix, \n",
    "    roc_curve.\n",
    "    \"\"\"\n",
    "\n",
    "    # Получим предсказания, если возможно в виде вероятностей\n",
    "    try: \n",
    "        probas = model.predict_proba(X)[:, 1]\n",
    "    except AttributeError:\n",
    "        prediction = model.predict(X)\n",
    "        threshold = None\n",
    "        probas = None\n",
    "    else:\n",
    "        threshold = threshold or find_best_threshold(y, probas, 'roc_auc')\n",
    "        prediction = (probas > threshold).astype(int)\n",
    "\n",
    "    # Распечатаем порог перевода вероятностей в классы\n",
    "    if threshold is None:\n",
    "        print(\"Порог перевода вероятностей в классы: не используется\")\n",
    "    else:\n",
    "        print(f\"Порог перевода вероятностей в классы: {threshold}\")\n",
    "        print(f\"{roc_auc_score(y, probas)} - roc_auc на вероятностях\")\n",
    "\n",
    "    # Распечатаем однострочные метрики\n",
    "    print()\n",
    "    print(f\"{roc_auc_score(y, prediction):0.8f} - roc_auc\")\n",
    "    print(f\"{accuracy_score(y, prediction):0.8f} - accuracy\")\n",
    "    print(f\"{precision_score(y, prediction):0.8f} - precision\")\n",
    "    print(f\"{recall_score(y, prediction):0.8f} - recall\")\n",
    "    print(f\"{f1_score(y, prediction):0.8f} - f1\")\n",
    "\n",
    "    # Распечатаем матрицу ошибок\n",
    "    conf_mat = confusion_matrix(y, prediction)\n",
    "    classes = model.classes_\n",
    "    n_classes = len(classes)\n",
    "    print()\n",
    "    print(\"|\".join(f\"{i:^10}\" for i in [\"prediction\"] + list(classes)))\n",
    "    print(f\"{'true label':^10}\" + (\"|\" + \" \" * 10) * n_classes)\n",
    "    print(\"-\" * ((n_classes * 10) + n_classes + 10))\n",
    "    for i in range(n_classes):\n",
    "        print(\"|\".join(f\"{j:>10}\" for j in [classes[i]] + list(conf_mat[i])))\n",
    "\n",
    "    # Отобразим ROC-кривую\n",
    "    if show_roc_curve:\n",
    "        print()\n",
    "        plt.figure(figsize=(7, 4))\n",
    "        if probas is not None:\n",
    "            plt.plot(*roc_curve(y_test, probas)[:2], \n",
    "                     c='r', label='on probability')\n",
    "        plt.plot(*roc_curve(y_test, prediction)[:2], c='b', label='on class')\n",
    "        plt.plot([0, 1], [0, 1], c='y', label='random', linestyle='dashed')\n",
    "        plt.title('Receiver operating characteristic')\n",
    "        plt.xlabel('False Positive Rate')\n",
    "        plt.ylabel('True Positive Rate')\n",
    "        plt.legend()\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d0e8fa6-2e14-4b19-a47d-dad05e838935",
   "metadata": {
    "id": "Fm7hN89g4g4_",
    "tags": []
   },
   "source": [
    "### Базовая модель  \n",
    "\n",
    "Так как целевая переменная распределена не равномерно, то в качестве бейзлайна можно выбрать стратегию, которая для каждой сессии предсказывает значение 0.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "id": "59101a7e-1a0e-4e69-a993-166d061131be",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MoTDCM7V4FUm",
    "outputId": "9e083a1b-b379-488e-c3f4-f51f4451742a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.5\n",
      "0.5 - roc_auc на вероятностях\n",
      "\n",
      "0.50000000 - roc_auc\n",
      "0.97295000 - accuracy\n",
      "0.00000000 - precision\n",
      "0.00000000 - recall\n",
      "0.00000000 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    194590|         0\n",
      "       1.0|      5410|         0\n"
     ]
    }
   ],
   "source": [
    "# Обучим базовую модель \n",
    "baseline = DummyClassifier(strategy='constant', constant=0)\n",
    "baseline.fit(X_train_preprocessed, y_train)\n",
    "\n",
    "# И получим её метрики\n",
    "print_metrics(baseline, X_valid_preprocessed, y_valid, 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66b4b78d-7419-4ce2-9430-e373a4945d3a",
   "metadata": {
    "id": "PWvFBh-uZo0b",
    "tags": []
   },
   "source": [
    "### Выбор модели  \n",
    "\n",
    "Обучим несколько моделей, оценим на валидационной выборке и выберем лучшую, которую уже будем оптимизировать."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24c0100b-1214-410c-9109-a3d3acbef360",
   "metadata": {
    "id": "mx3teaVL1vJH",
    "tags": []
   },
   "source": [
    "#### Логистическая регрессия"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "id": "8cdadecb-45b6-4a41-9f94-4529d60834b3",
   "metadata": {
    "id": "MQN5cdrGBsAl"
   },
   "outputs": [],
   "source": [
    "logreg = LogisticRegression(random_state=RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "id": "f7ec62f2-66a7-42bd-83bd-553b84db050b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HmSC_wnWt7CO",
    "outputId": "d22897ac-3f5f-47ab-ad03-6b22e0f09a02"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 11.1 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-10 {color: black;background-color: white;}#sk-container-id-10 pre{padding: 0;}#sk-container-id-10 div.sk-toggleable {background-color: white;}#sk-container-id-10 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-10 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-10 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-10 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-10 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-10 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-10 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-10 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-10 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-10 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-10 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-10 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-10 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-10 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-10 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-10 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-10 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-10 div.sk-item {position: relative;z-index: 1;}#sk-container-id-10 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-10 div.sk-item::before, #sk-container-id-10 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-10 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-10 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-10 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-10 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-10 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-10 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-10 div.sk-label-container {text-align: center;}#sk-container-id-10 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-10 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-10\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LogisticRegression(random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-10\" type=\"checkbox\" checked><label for=\"sk-estimator-id-10\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression(random_state=0)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "LogisticRegression(random_state=0)"
      ]
     },
     "execution_count": 219,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "logreg.fit(X_train_preprocessed, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "id": "f11f2105-1b54-42bf-bbf3-5149968a6cc4",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "TwU_sjsouHq4",
    "outputId": "795b7551-fcba-4c81-b120-dacde071370f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.5\n",
      "0.6683645332681568 - roc_auc на вероятностях\n",
      "\n",
      "0.50000000 - roc_auc\n",
      "0.97295000 - accuracy\n",
      "0.00000000 - precision\n",
      "0.00000000 - recall\n",
      "0.00000000 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    194590|         0\n",
      "       1.0|      5410|         0\n"
     ]
    }
   ],
   "source": [
    "print_metrics(logreg, X_valid_preprocessed, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ddf2776-ac48-426d-b585-d189388af7da",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Метод опорных векторов"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "id": "6cedeea3-d504-40d1-9add-14f05218d059",
   "metadata": {
    "id": "FIH3-eifJ3Ik"
   },
   "outputs": [],
   "source": [
    "svc = LinearSVC(class_weight='balanced')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "id": "19e43dd3-45f8-46ea-8f6c-a598ab268140",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rOSy-TIcJ3EE",
    "outputId": "a7249970-0541-43b1-fc57-4195235ab6f1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 10min 29s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-11 {color: black;background-color: white;}#sk-container-id-11 pre{padding: 0;}#sk-container-id-11 div.sk-toggleable {background-color: white;}#sk-container-id-11 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-11 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-11 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-11 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-11 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-11 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-11 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-11 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-11 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-11 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-11 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-11 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-11 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-11 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-11 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-11 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-11 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-11 div.sk-item {position: relative;z-index: 1;}#sk-container-id-11 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-11 div.sk-item::before, #sk-container-id-11 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-11 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-11 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-11 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-11 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-11 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-11 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-11 div.sk-label-container {text-align: center;}#sk-container-id-11 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-11 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-11\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearSVC(class_weight=&#x27;balanced&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-11\" type=\"checkbox\" checked><label for=\"sk-estimator-id-11\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearSVC</label><div class=\"sk-toggleable__content\"><pre>LinearSVC(class_weight=&#x27;balanced&#x27;)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "LinearSVC(class_weight='balanced')"
      ]
     },
     "execution_count": 222,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "svc.fit(X_train_preprocessed, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "id": "bffcdb8e-1111-4ded-8f59-1bd183181c7f",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3MM_qupCKUuU",
    "outputId": "2ded8b17-c265-44fa-f036-03b86b7ec8b1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: не используется\n",
      "\n",
      "0.62056289 - roc_auc\n",
      "0.61588500 - accuracy\n",
      "0.04328417 - precision\n",
      "0.62550832 - recall\n",
      "0.08096565 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    119793|     74797\n",
      "       1.0|      2026|      3384\n"
     ]
    }
   ],
   "source": [
    "print_metrics(svc, X_valid_preprocessed, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a2bbeb2-8fb6-4767-bc70-72c2b8390537",
   "metadata": {
    "id": "Z8jQU-SKCPXh",
    "tags": []
   },
   "source": [
    "#### Нейронная сеть"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 224,
   "id": "57380d52-ad8c-468f-92d5-4cb63f170c69",
   "metadata": {
    "id": "I1CuFrrmCOaX"
   },
   "outputs": [],
   "source": [
    "mlp = MLPClassifier((32,), random_state=RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "id": "961e9bd4-a00d-44a3-a65b-77ebb3929fb9",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tGqKzgskCOa3",
    "outputId": "c2f3a0be-79f6-4d38-df54-7b8a473a3ff7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 1min 9s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-12 {color: black;background-color: white;}#sk-container-id-12 pre{padding: 0;}#sk-container-id-12 div.sk-toggleable {background-color: white;}#sk-container-id-12 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-12 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-12 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-12 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-12 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-12 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-12 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-12 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-12 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-12 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-12 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-12 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-12 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-12 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-12 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-12 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-12 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-12 div.sk-item {position: relative;z-index: 1;}#sk-container-id-12 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-12 div.sk-item::before, #sk-container-id-12 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-12 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-12 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-12 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-12 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-12 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-12 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-12 div.sk-label-container {text-align: center;}#sk-container-id-12 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-12 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-12\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>MLPClassifier(hidden_layer_sizes=(32,), random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-12\" type=\"checkbox\" checked><label for=\"sk-estimator-id-12\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">MLPClassifier</label><div class=\"sk-toggleable__content\"><pre>MLPClassifier(hidden_layer_sizes=(32,), random_state=0)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "MLPClassifier(hidden_layer_sizes=(32,), random_state=0)"
      ]
     },
     "execution_count": 225,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "mlp.fit(X_train_preprocessed, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 226,
   "id": "046583f3-cd6d-41c5-802c-75110c83813f",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "GWrCizlgCOa4",
    "outputId": "1f9249a7-7436-4fb1-b1d8-b9419b75e01c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.5\n",
      "0.6983782119645087 - roc_auc на вероятностях\n",
      "\n",
      "0.50000000 - roc_auc\n",
      "0.97295000 - accuracy\n",
      "0.00000000 - precision\n",
      "0.00000000 - recall\n",
      "0.00000000 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    194590|         0\n",
      "       1.0|      5410|         0\n"
     ]
    }
   ],
   "source": [
    "print_metrics(mlp, X_valid_preprocessed, y_valid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "id": "14c63583-7100-411d-92af-82485c489bf3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|████████████████████████████████████████| 1000/1000 [100%] in 15.6s (64.18/s) \n",
      "|██████████████████████████▋⚠︎            | (!) 1000/1500 [67%] in 15.6s (64.11/s) \n",
      "|████████████████████████████████████████✗︎ (!) 1000/700 [143%] in 15.6s (64.10/s) \n",
      "|████████████████████████████████████████| 1000 in 15.6s (64.16/s) \n"
     ]
    }
   ],
   "source": [
    "from alive_progress import alive_bar\n",
    "import time\n",
    "\n",
    "for x in 1000, 1500, 700, 0:\n",
    "   with alive_bar(x) as bar:\n",
    "       for i in range(1000):\n",
    "           time.sleep(.005)\n",
    "           bar()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ec88c51-8404-42b0-84c2-0f71cbe6b444",
   "metadata": {
    "id": "fnB04lZh17ub",
    "tags": []
   },
   "source": [
    "#### Байесовский классификатор"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 228,
   "id": "7524b120-2f7a-4af6-85e2-5b00297c7804",
   "metadata": {
    "id": "05T2TlUWBxHN"
   },
   "outputs": [],
   "source": [
    "gaussnb = GaussianNB()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 229,
   "id": "4cb0ee68-4e5f-41d9-bac5-3c79d6d24101",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-Xr1YU0v2FZq",
    "outputId": "e6c868b1-c9ea-4efa-fff6-1ad66baccdad"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 1.33 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-13 {color: black;background-color: white;}#sk-container-id-13 pre{padding: 0;}#sk-container-id-13 div.sk-toggleable {background-color: white;}#sk-container-id-13 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-13 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-13 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-13 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-13 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-13 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-13 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-13 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-13 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-13 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-13 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-13 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-13 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-13 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-13 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-13 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-13 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-13 div.sk-item {position: relative;z-index: 1;}#sk-container-id-13 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-13 div.sk-item::before, #sk-container-id-13 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-13 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-13 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-13 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-13 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-13 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-13 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-13 div.sk-label-container {text-align: center;}#sk-container-id-13 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-13 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-13\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GaussianNB()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-13\" type=\"checkbox\" checked><label for=\"sk-estimator-id-13\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">GaussianNB</label><div class=\"sk-toggleable__content\"><pre>GaussianNB()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "GaussianNB()"
      ]
     },
     "execution_count": 229,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "gaussnb.fit(X_train_preprocessed, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 230,
   "id": "ffb10ca1-b31f-4baf-a4b6-485d57ad4616",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mTmSxITR2FTk",
    "outputId": "28dc1f86-e74e-4028-e073-e908cd710fb0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.05419511287276451\n",
      "0.6458839691283222 - roc_auc на вероятностях\n",
      "\n",
      "0.60928251 - roc_auc\n",
      "0.56001500 - accuracy\n",
      "0.03986940 - precision\n",
      "0.66136784 - recall\n",
      "0.07520520 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    108425|     86165\n",
      "       1.0|      1832|      3578\n"
     ]
    }
   ],
   "source": [
    "print_metrics(gaussnb, X_valid_preprocessed, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1179cbb-b70d-4003-ac4a-00ccd52ec4d8",
   "metadata": {
    "id": "BIZBspdkNsSP",
    "tags": []
   },
   "source": [
    "#### Дерево решений"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "id": "251b2b49-172e-4461-9581-0287324a6276",
   "metadata": {
    "id": "UOlsEARaNr98"
   },
   "outputs": [],
   "source": [
    "tree = DecisionTreeClassifier(random_state=RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 232,
   "id": "0df5f1de-98f9-4c99-8cf2-4be90ba31e4d",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2IUaLZBcN2hL",
    "outputId": "e90c5abb-65d5-48f3-f9eb-530da4ecc911"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 26.7 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-14 {color: black;background-color: white;}#sk-container-id-14 pre{padding: 0;}#sk-container-id-14 div.sk-toggleable {background-color: white;}#sk-container-id-14 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-14 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-14 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-14 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-14 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-14 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-14 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-14 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-14 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-14 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-14 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-14 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-14 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-14 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-14 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-14 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-14 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-14 div.sk-item {position: relative;z-index: 1;}#sk-container-id-14 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-14 div.sk-item::before, #sk-container-id-14 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-14 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-14 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-14 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-14 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-14 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-14 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-14 div.sk-label-container {text-align: center;}#sk-container-id-14 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-14 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-14\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>DecisionTreeClassifier(random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-14\" type=\"checkbox\" checked><label for=\"sk-estimator-id-14\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">DecisionTreeClassifier</label><div class=\"sk-toggleable__content\"><pre>DecisionTreeClassifier(random_state=0)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "DecisionTreeClassifier(random_state=0)"
      ]
     },
     "execution_count": 232,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "tree.fit(X_train_preprocessed, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "id": "443b2bb2-09d1-49b9-a08c-6d00583502f0",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "er40gzjIN2dV",
    "outputId": "cfe7c571-0b7c-4122-8b3e-ef2d20fdab55"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.033708156968674276\n",
      "0.5204877305418407 - roc_auc на вероятностях\n",
      "\n",
      "0.52080171 - roc_auc\n",
      "0.93335000 - accuracy\n",
      "0.05183341 - precision\n",
      "0.08465804 - recall\n",
      "0.06429875 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    186212|      8378\n",
      "       1.0|      4952|       458\n"
     ]
    }
   ],
   "source": [
    "print_metrics(tree, X_valid_preprocessed, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19ee19af-db67-4d9d-a245-d5c95c164796",
   "metadata": {
    "id": "ND_qC0SnPczC",
    "tags": []
   },
   "source": [
    "#### Случайный лес"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "id": "965d6c4e-357d-4127-9cf5-715e945dba21",
   "metadata": {
    "id": "Ohu0p_RCPgUd"
   },
   "outputs": [],
   "source": [
    "forest = RandomForestClassifier(random_state=RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "id": "000a8a8e-26dc-465b-8c43-7f3c1b7a94e3",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "X_NwrOzUPgQT",
    "outputId": "79c8b478-f712-4133-de0a-f52b5de26bf2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 7min 3s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-15 {color: black;background-color: white;}#sk-container-id-15 pre{padding: 0;}#sk-container-id-15 div.sk-toggleable {background-color: white;}#sk-container-id-15 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-15 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-15 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-15 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-15 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-15 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-15 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-15 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-15 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-15 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-15 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-15 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-15 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-15 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-15 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-15 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-15 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-15 div.sk-item {position: relative;z-index: 1;}#sk-container-id-15 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-15 div.sk-item::before, #sk-container-id-15 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-15 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-15 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-15 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-15 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-15 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-15 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-15 div.sk-label-container {text-align: center;}#sk-container-id-15 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-15 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-15\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomForestClassifier(random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-15\" type=\"checkbox\" checked><label for=\"sk-estimator-id-15\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(random_state=0)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "RandomForestClassifier(random_state=0)"
      ]
     },
     "execution_count": 235,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "forest.fit(X_train_preprocessed, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "id": "abb3a780-67a3-4a2a-8f9c-98668575ecbc",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MbvfSLq2PdKb",
    "outputId": "089b21ed-6c37-4138-c6a4-fd6583be1973"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.010218123817113232\n",
      "0.6253519433580381 - roc_auc на вероятностях\n",
      "\n",
      "0.59594638 - roc_auc\n",
      "0.59998000 - accuracy\n",
      "0.03952047 - precision\n",
      "0.59168207 - recall\n",
      "0.07409208 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    116795|     77795\n",
      "       1.0|      2209|      3201\n"
     ]
    }
   ],
   "source": [
    "print_metrics(forest, X_valid_preprocessed, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c476bdf2-44e1-4e4b-b8bb-ae0af1a1ae74",
   "metadata": {
    "id": "SJFfsUFc3B5b",
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "#### Градиентный Бустинг"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "id": "7e7eec1d-ae20-4bf7-a617-0eafcc60dccb",
   "metadata": {
    "id": "A12fP4saB4-e"
   },
   "outputs": [],
   "source": [
    "histboost = HistGradientBoostingClassifier(random_state=RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "id": "e1b82446-be9f-4c75-9fc7-89264c8a58b2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "vc9PzIjHDXVm",
    "outputId": "b6c4752e-a560-4ab0-cb1b-b8872be3796e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 15.6 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-16 {color: black;background-color: white;}#sk-container-id-16 pre{padding: 0;}#sk-container-id-16 div.sk-toggleable {background-color: white;}#sk-container-id-16 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-16 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-16 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-16 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-16 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-16 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-16 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-16 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-16 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-16 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-16 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-16 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-16 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-16 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-16 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-16 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-16 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-16 div.sk-item {position: relative;z-index: 1;}#sk-container-id-16 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-16 div.sk-item::before, #sk-container-id-16 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-16 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-16 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-16 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-16 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-16 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-16 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-16 div.sk-label-container {text-align: center;}#sk-container-id-16 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-16 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-16\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>HistGradientBoostingClassifier(random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-16\" type=\"checkbox\" checked><label for=\"sk-estimator-id-16\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">HistGradientBoostingClassifier</label><div class=\"sk-toggleable__content\"><pre>HistGradientBoostingClassifier(random_state=0)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "HistGradientBoostingClassifier(random_state=0)"
      ]
     },
     "execution_count": 238,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "histboost.fit(X_train_preprocessed, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "id": "fd7f38f3-0817-432b-9c42-d8704afdd162",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rs88B_PzDfMF",
    "outputId": "8f825910-95df-4501-a13c-cb648d0d9173"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.02822602984682529\n",
      "0.70696812360298 - roc_auc на вероятностях\n",
      "\n",
      "0.64682393 - roc_auc\n",
      "0.62957000 - accuracy\n",
      "0.04742197 - precision\n",
      "0.66506470 - recall\n",
      "0.08853129 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    122316|     72274\n",
      "       1.0|      1812|      3598\n"
     ]
    }
   ],
   "source": [
    "print_metrics(histboost, X_valid_preprocessed, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd5c64ff-10fc-43c7-ba57-8f7ab2f3ab4a",
   "metadata": {
    "id": "5vNwKUyjToUJ",
    "tags": []
   },
   "source": [
    "#### [CatBoost](https://catboost.ai/) (CatBoost - открытая программная библиотека, разработанная компанией Яндекс и реализующая уникальный патентованный алгоритм построения моделей машинного обучения, использующий одну из оригинальных схем градиентного бустинга.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "id": "bb4bcb1e-c462-49f7-b25d-2ee809780a25",
   "metadata": {
    "id": "SXMJDwsJgFw5"
   },
   "outputs": [],
   "source": [
    "catboost = CatBoostClassifier(\n",
    "    iterations=100, verbose=False, random_state=RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "id": "a6ee8db8-017a-4092-836c-9573109324a9",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "miBsOhczge8L",
    "outputId": "fca7ea04-a64f-490e-c982-7481e4a8392e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 10.8 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x20caf437b80>"
      ]
     },
     "execution_count": 241,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "catboost.fit(X_train_preprocessed, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 242,
   "id": "89f5e9b0-5daa-4eca-bc67-fb4f47a9e162",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YjePcwU2gj2D",
    "outputId": "31fddb47-3904-49bf-f6c5-b748d7951910"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.02533905564192846\n",
      "0.7068194537469606 - roc_auc на вероятностях\n",
      "\n",
      "0.64895457 - roc_auc\n",
      "0.58476000 - accuracy\n",
      "0.04541303 - precision\n",
      "0.71682070 - recall\n",
      "0.08541474 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    113074|     81516\n",
      "       1.0|      1532|      3878\n"
     ]
    }
   ],
   "source": [
    "print_metrics(catboost, X_valid_preprocessed, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92c9206c-03c7-416a-968d-84e98a123d6d",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### [XGBoost](https://github.com/dmlc/xgboost)\n",
    "\n",
    "Основные особенности XGBoost, отличающие его от других алгоритмов градиентного бустинга, включают:\n",
    "\n",
    "* Умная штрафовка деревьев\n",
    "* Пропорциональное уменьшение узлов листьев\n",
    "* Метод Ньютона в оптимизации\n",
    "* Дополнительный параметр рандомизации\n",
    "* Реализация на одиночных, распределенных системах и out-of-core вычислениях\n",
    "* Автоматический отбор признаков\n",
    "\n",
    "Предварительно устанавливается `pip install xgboost` согласно [документации](https://xgboost.readthedocs.io/en/stable/install.html#python)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 243,
   "id": "7dc95607-008e-41f9-b958-39b2954ab1a6",
   "metadata": {
    "id": "XQLRpwg6ZNts"
   },
   "outputs": [],
   "source": [
    "xgboost = XGBClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 244,
   "id": "30541c0a-da64-4fea-9fda-1508714dfb3f",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "vP04Ay7mZNtt",
    "outputId": "09a797df-f4b0-40cc-b722-62ff3c5668bb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 1min 14s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-17 {color: black;background-color: white;}#sk-container-id-17 pre{padding: 0;}#sk-container-id-17 div.sk-toggleable {background-color: white;}#sk-container-id-17 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-17 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-17 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-17 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-17 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-17 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-17 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-17 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-17 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-17 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-17 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-17 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-17 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-17 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-17 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-17 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-17 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-17 div.sk-item {position: relative;z-index: 1;}#sk-container-id-17 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-17 div.sk-item::before, #sk-container-id-17 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-17 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-17 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-17 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-17 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-17 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-17 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-17 div.sk-label-container {text-align: center;}#sk-container-id-17 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-17 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-17\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>XGBClassifier(base_score=None, booster=None, callbacks=None,\n",
       "              colsample_bylevel=None, colsample_bynode=None,\n",
       "              colsample_bytree=None, early_stopping_rounds=None,\n",
       "              enable_categorical=False, eval_metric=None, feature_types=None,\n",
       "              gamma=None, gpu_id=None, grow_policy=None, importance_type=None,\n",
       "              interaction_constraints=None, learning_rate=None, max_bin=None,\n",
       "              max_cat_threshold=None, max_cat_to_onehot=None,\n",
       "              max_delta_step=None, max_depth=None, max_leaves=None,\n",
       "              min_child_weight=None, missing=nan, monotone_constraints=None,\n",
       "              n_estimators=100, n_jobs=None, num_parallel_tree=None,\n",
       "              predictor=None, random_state=None, ...)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-17\" type=\"checkbox\" checked><label for=\"sk-estimator-id-17\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">XGBClassifier</label><div class=\"sk-toggleable__content\"><pre>XGBClassifier(base_score=None, booster=None, callbacks=None,\n",
       "              colsample_bylevel=None, colsample_bynode=None,\n",
       "              colsample_bytree=None, early_stopping_rounds=None,\n",
       "              enable_categorical=False, eval_metric=None, feature_types=None,\n",
       "              gamma=None, gpu_id=None, grow_policy=None, importance_type=None,\n",
       "              interaction_constraints=None, learning_rate=None, max_bin=None,\n",
       "              max_cat_threshold=None, max_cat_to_onehot=None,\n",
       "              max_delta_step=None, max_depth=None, max_leaves=None,\n",
       "              min_child_weight=None, missing=nan, monotone_constraints=None,\n",
       "              n_estimators=100, n_jobs=None, num_parallel_tree=None,\n",
       "              predictor=None, random_state=None, ...)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "XGBClassifier(base_score=None, booster=None, callbacks=None,\n",
       "              colsample_bylevel=None, colsample_bynode=None,\n",
       "              colsample_bytree=None, early_stopping_rounds=None,\n",
       "              enable_categorical=False, eval_metric=None, feature_types=None,\n",
       "              gamma=None, gpu_id=None, grow_policy=None, importance_type=None,\n",
       "              interaction_constraints=None, learning_rate=None, max_bin=None,\n",
       "              max_cat_threshold=None, max_cat_to_onehot=None,\n",
       "              max_delta_step=None, max_depth=None, max_leaves=None,\n",
       "              min_child_weight=None, missing=nan, monotone_constraints=None,\n",
       "              n_estimators=100, n_jobs=None, num_parallel_tree=None,\n",
       "              predictor=None, random_state=None, ...)"
      ]
     },
     "execution_count": 244,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "xgboost.fit(X_train_preprocessed.values, y_train.values);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "id": "4ab0e7a5-ecb1-470a-bbb4-0d3cae12a66d",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ct7RtQdDZNtu",
    "outputId": "745534a6-f726-463c-be58-4dd26cd3a559"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.02784477629170188\n",
      "0.7107721766577036 - roc_auc на вероятностях\n",
      "\n",
      "0.65178082 - roc_auc\n",
      "0.63432000 - accuracy\n",
      "0.04836085 - precision\n",
      "0.67024030 - recall\n",
      "0.09021247 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    123238|     71352\n",
      "       1.0|      1784|      3626\n"
     ]
    }
   ],
   "source": [
    "print_metrics(xgboost, X_valid_preprocessed.values, y_valid.values)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c74d9ee-49cc-4ba9-bfeb-158067325984",
   "metadata": {
    "id": "UB6kxht3n16M",
    "tags": []
   },
   "source": [
    "#### LGBMClassifier — это фреймворк, который предоставляет реализацию деревьев принятия решений с градиентным бустингом."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 246,
   "id": "c42eec45-a444-4eef-83c7-84c4c9cbff08",
   "metadata": {
    "id": "jCi3bHfsn16O"
   },
   "outputs": [],
   "source": [
    "lightgbm = LGBMClassifier(random_state=RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 247,
   "id": "c8affbe3-1da9-435e-be1f-4d3ed0817805",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qwJwaO9Xn16P",
    "outputId": "963405f9-b816-491d-9517-9ae368df1127"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 4.62 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-18 {color: black;background-color: white;}#sk-container-id-18 pre{padding: 0;}#sk-container-id-18 div.sk-toggleable {background-color: white;}#sk-container-id-18 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-18 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-18 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-18 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-18 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-18 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-18 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-18 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-18 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-18 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-18 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-18 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-18 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-18 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-18 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-18 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-18 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-18 div.sk-item {position: relative;z-index: 1;}#sk-container-id-18 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-18 div.sk-item::before, #sk-container-id-18 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-18 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-18 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-18 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-18 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-18 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-18 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-18 div.sk-label-container {text-align: center;}#sk-container-id-18 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-18 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-18\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LGBMClassifier(random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-18\" type=\"checkbox\" checked><label for=\"sk-estimator-id-18\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LGBMClassifier</label><div class=\"sk-toggleable__content\"><pre>LGBMClassifier(random_state=0)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "LGBMClassifier(random_state=0)"
      ]
     },
     "execution_count": 247,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "lightgbm.fit(X_train_preprocessed, y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 248,
   "id": "4b1227e9-9efb-431b-98ba-0baf936d8dec",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "t5bs-yUBn16Q",
    "outputId": "08c92767-db0f-400b-c8f3-e2401e7af308"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Порог перевода вероятностей в классы: 0.028516118176763625\n",
      "0.7069700438449713 - roc_auc на вероятностях\n",
      "\n",
      "0.64701144 - roc_auc\n",
      "0.63535500 - accuracy\n",
      "0.04778107 - precision\n",
      "0.65933457 - recall\n",
      "0.08910483 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    123504|     71086\n",
      "       1.0|      1843|      3567\n"
     ]
    }
   ],
   "source": [
    "print_metrics(lightgbm, X_valid_preprocessed, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56f652ab-54c5-45ba-a72c-e85fdcefbd42",
   "metadata": {
    "id": "UqnvrGRY7wWC",
    "tags": []
   },
   "source": [
    "### Оптимизация модели  \n",
    "\n",
    "Лучшей моделью является LightGBM по следующим причинам:\n",
    "+ Один из лучших показателей `roc_auc`.\n",
    "+ Быстрое обучение. \n",
    "+ Модель интерпретируема, то есть можно получить показатели важности признаков.\n",
    "+ Может предсказывать вероятность класса.\n",
    "+ Нет проблем с процессорами без SSE4 как у CatBoost.\n",
    "\n",
    "Оптимизация модели и конвейера по подготовке данных проводится с помощью байесовской оптимизации. При разных гиперпараметрах модель обучается на тренировочных данных, а оценивается на валидационных."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "id": "d229b860-3fbd-4fbe-8a6d-2d166f2d5080",
   "metadata": {
    "id": "r9DJbmb5m26A"
   },
   "outputs": [],
   "source": [
    "def optimize_lightgbm(\n",
    "    rare_encoder_tol: float, \n",
    "    constant_dropper_tol: float, \n",
    "    correlated_dropper_threshold: float, \n",
    "    n_estimators: float, \n",
    "    learning_rate: float, \n",
    "    reg_lambda: float, \n",
    "    num_leaves: float, \n",
    "    reg_alpha: float, \n",
    "    boosting_type: str = 'goss'\n",
    ") -> float:\n",
    "    \"\"\"Используется Баейсовским оптимизатором для поиска лучших гиперпараметров \n",
    "    конвейера по подготовке данных и модели LGBMClassifier.\n",
    "    \"\"\"\n",
    "\n",
    "    # Создадим конвейер с заданными гиперпараметрами\n",
    "    model = Pipeline(steps=[\n",
    "        # Создание дополнительных признаков\n",
    "        ('indexer', FunctionTransformer(set_index)), \n",
    "        ('imputer', FunctionTransformer(fill_missings)), \n",
    "        ('engineer', FunctionTransformer(create_features)), \n",
    "        ('dropper', DropFeatures(['client_id', 'visit_date', 'visit_time', \n",
    "                                  'device_screen_resolution'])), \n",
    "        # Преобразования численных переменных\n",
    "        ('normalization', YeoJohnsonTransformer()), \n",
    "        ('outlier_remover', Winsorizer()), \n",
    "        ('scaler', SklearnTransformerWrapper(StandardScaler())), \n",
    "        # Преобразования категориальных признаков\n",
    "        ('rare_encoder', RareLabelEncoder(\n",
    "            tol=rare_encoder_tol, replace_with='rare')),\n",
    "        ('onehot_encoder', OneHotEncoder(drop_last_binary=True)), \n",
    "        ('bool_converter', FunctionTransformer(converse_types)), \n",
    "        # Удаление дубликатов и коррелируемых признаков\n",
    "        ('constant_dropper', DropConstantFeatures(tol=constant_dropper_tol)), \n",
    "        ('duplicated_dropper', DropDuplicateFeatures()), \n",
    "        ('correlated_dropper', DropCorrelatedFeatures(\n",
    "            threshold=correlated_dropper_threshold)), \n",
    "        # Лучшая модель с оптимизированными гиперпараметрами\n",
    "        ('model', LGBMClassifier(\n",
    "            n_estimators=int(n_estimators), boosting_type=boosting_type, \n",
    "            learning_rate=learning_rate, num_leaves=int(num_leaves), \n",
    "            reg_lambda=reg_lambda, reg_alpha=reg_alpha, \n",
    "            random_state=RANDOM_SEED))])\n",
    "    \n",
    "    # Обучим и оценим модель\n",
    "    model.fit(X_train, y_train)\n",
    "    prediction = model.predict_proba(X_valid)[:, 1]\n",
    "    return roc_auc_score(y_valid, prediction)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 539,
   "id": "1f97018a-2759-436a-b588-0d673025de35",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>session_id</th>\n",
       "      <th>client_id</th>\n",
       "      <th>visit_date</th>\n",
       "      <th>visit_time</th>\n",
       "      <th>visit_number</th>\n",
       "      <th>utm_source</th>\n",
       "      <th>utm_medium</th>\n",
       "      <th>utm_campaign</th>\n",
       "      <th>utm_adcontent</th>\n",
       "      <th>utm_keyword</th>\n",
       "      <th>device_category</th>\n",
       "      <th>device_os</th>\n",
       "      <th>device_brand</th>\n",
       "      <th>device_model</th>\n",
       "      <th>device_screen_resolution</th>\n",
       "      <th>device_browser</th>\n",
       "      <th>geo_country</th>\n",
       "      <th>geo_city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1056663</th>\n",
       "      <td>547332070992313735.1638727047.1638727047</td>\n",
       "      <td>127435678.163873</td>\n",
       "      <td>2021-12-05</td>\n",
       "      <td>20:57:27</td>\n",
       "      <td>1</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>TmThBvoCcwkCZZUWACYq</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>mobile</td>\n",
       "      <td>iOS</td>\n",
       "      <td>Apple</td>\n",
       "      <td>NaN</td>\n",
       "      <td>428x926</td>\n",
       "      <td>Safari</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Novotroitsk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1409550</th>\n",
       "      <td>7049980726847853060.1629809978.1629809978</td>\n",
       "      <td>1641451550.162934</td>\n",
       "      <td>2021-08-24</td>\n",
       "      <td>15:00:00</td>\n",
       "      <td>5</td>\n",
       "      <td>MvfHsxITijuriZxsqZqt</td>\n",
       "      <td>cpm</td>\n",
       "      <td>FTjNLDyTrXaWYgZymFkV</td>\n",
       "      <td>PkybGvWbaqORmxjNunqZ</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Xiaomi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>393x851</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Balashikha</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>739873</th>\n",
       "      <td>4064566220882386809.1639930788.1639930788</td>\n",
       "      <td>946355569.163992</td>\n",
       "      <td>2021-12-19</td>\n",
       "      <td>19:19:48</td>\n",
       "      <td>2</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>LEoPHuyFvzoNfnzGgfcd</td>\n",
       "      <td>vCIpmpaGBnIQhyYNkXqp</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>mobile</td>\n",
       "      <td>Android</td>\n",
       "      <td>Samsung</td>\n",
       "      <td>NaN</td>\n",
       "      <td>339x753</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Almetyevsk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62902</th>\n",
       "      <td>1033849280328168809.1631647080.1631647080</td>\n",
       "      <td>240711793.163165</td>\n",
       "      <td>2021-09-14</td>\n",
       "      <td>22:00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>gecBYcKZCPMcVYdSSzKP</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Apple</td>\n",
       "      <td>NaN</td>\n",
       "      <td>414x896</td>\n",
       "      <td>Safari</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Saint Petersburg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1297234</th>\n",
       "      <td>6544287746962620171.1638268683.1638268683</td>\n",
       "      <td>1523710728.163827</td>\n",
       "      <td>2021-11-30</td>\n",
       "      <td>13:38:03</td>\n",
       "      <td>1</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>LEoPHuyFvzoNfnzGgfcd</td>\n",
       "      <td>vCIpmpaGBnIQhyYNkXqp</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>mobile</td>\n",
       "      <td>Android</td>\n",
       "      <td>Xiaomi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>393x851</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Moscow</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1793000</th>\n",
       "      <td>8755082352522037809.1633552946.1633552946</td>\n",
       "      <td>2038451459.163355</td>\n",
       "      <td>2021-10-06</td>\n",
       "      <td>23:00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>fDLlAcSmythWSCVMvqvL</td>\n",
       "      <td>(none)</td>\n",
       "      <td>LTuZkdKfxRGVceoWkVyg</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Samsung</td>\n",
       "      <td>NaN</td>\n",
       "      <td>360x800</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Moscow</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>207261</th>\n",
       "      <td>1682284231298311525.1640424805.1640424805</td>\n",
       "      <td>391687320.1640424805</td>\n",
       "      <td>2021-12-25</td>\n",
       "      <td>12:33:25</td>\n",
       "      <td>1</td>\n",
       "      <td>fDLlAcSmythWSCVMvqvL</td>\n",
       "      <td>(none)</td>\n",
       "      <td>LTuZkdKfxRGVceoWkVyg</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>mobile</td>\n",
       "      <td>iOS</td>\n",
       "      <td>Apple</td>\n",
       "      <td>NaN</td>\n",
       "      <td>375x812</td>\n",
       "      <td>Safari</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Saint Petersburg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93412</th>\n",
       "      <td>1170303784354950884.1640345317.1640345317</td>\n",
       "      <td>272482583.164035</td>\n",
       "      <td>2021-12-24</td>\n",
       "      <td>14:28:37</td>\n",
       "      <td>1</td>\n",
       "      <td>fDLlAcSmythWSCVMvqvL</td>\n",
       "      <td>(none)</td>\n",
       "      <td>LTuZkdKfxRGVceoWkVyg</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>puhZPIYqKXeFPaUviSjo</td>\n",
       "      <td>desktop</td>\n",
       "      <td>Windows</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1920x1080</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Moscow</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1568591</th>\n",
       "      <td>7753976912353269271.1634047522.1634047522</td>\n",
       "      <td>1805363435.163405</td>\n",
       "      <td>2021-10-12</td>\n",
       "      <td>17:00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>ZpYIoDJMcFzVoPFsHGJL</td>\n",
       "      <td>banner</td>\n",
       "      <td>gecBYcKZCPMcVYdSSzKP</td>\n",
       "      <td>JNHcPlZPxEMWDnRiyoBf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Xiaomi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>393x851</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Novosibirsk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1119538</th>\n",
       "      <td>5753600408244704247.1626854393.1626854393</td>\n",
       "      <td>1339614486.162685</td>\n",
       "      <td>2021-07-21</td>\n",
       "      <td>10:00:00</td>\n",
       "      <td>1</td>\n",
       "      <td>MvfHsxITijuriZxsqZqt</td>\n",
       "      <td>cpm</td>\n",
       "      <td>FTjNLDyTrXaWYgZymFkV</td>\n",
       "      <td>dUuXlWzvmhDSyclWRhNP</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mobile</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Xiaomi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>393x851</td>\n",
       "      <td>Chrome</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Saint Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1460042 rows × 18 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                        session_id             client_id  \\\n",
       "1056663   547332070992313735.1638727047.1638727047      127435678.163873   \n",
       "1409550  7049980726847853060.1629809978.1629809978     1641451550.162934   \n",
       "739873   4064566220882386809.1639930788.1639930788      946355569.163992   \n",
       "62902    1033849280328168809.1631647080.1631647080      240711793.163165   \n",
       "1297234  6544287746962620171.1638268683.1638268683     1523710728.163827   \n",
       "...                                            ...                   ...   \n",
       "1793000  8755082352522037809.1633552946.1633552946     2038451459.163355   \n",
       "207261   1682284231298311525.1640424805.1640424805  391687320.1640424805   \n",
       "93412    1170303784354950884.1640345317.1640345317      272482583.164035   \n",
       "1568591  7753976912353269271.1634047522.1634047522     1805363435.163405   \n",
       "1119538  5753600408244704247.1626854393.1626854393     1339614486.162685   \n",
       "\n",
       "         visit_date visit_time  visit_number            utm_source utm_medium  \\\n",
       "1056663  2021-12-05   20:57:27             1  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "1409550  2021-08-24   15:00:00             5  MvfHsxITijuriZxsqZqt        cpm   \n",
       "739873   2021-12-19   19:19:48             2  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "62902    2021-09-14   22:00:00             1  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "1297234  2021-11-30   13:38:03             1  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "...             ...        ...           ...                   ...        ...   \n",
       "1793000  2021-10-06   23:00:00             1  fDLlAcSmythWSCVMvqvL     (none)   \n",
       "207261   2021-12-25   12:33:25             1  fDLlAcSmythWSCVMvqvL     (none)   \n",
       "93412    2021-12-24   14:28:37             1  fDLlAcSmythWSCVMvqvL     (none)   \n",
       "1568591  2021-10-12   17:00:00             1  ZpYIoDJMcFzVoPFsHGJL     banner   \n",
       "1119538  2021-07-21   10:00:00             1  MvfHsxITijuriZxsqZqt        cpm   \n",
       "\n",
       "                 utm_campaign         utm_adcontent           utm_keyword  \\\n",
       "1056663  TmThBvoCcwkCZZUWACYq  JNHcPlZPxEMWDnRiyoBf  puhZPIYqKXeFPaUviSjo   \n",
       "1409550  FTjNLDyTrXaWYgZymFkV  PkybGvWbaqORmxjNunqZ                   NaN   \n",
       "739873   LEoPHuyFvzoNfnzGgfcd  vCIpmpaGBnIQhyYNkXqp  puhZPIYqKXeFPaUviSjo   \n",
       "62902    gecBYcKZCPMcVYdSSzKP  JNHcPlZPxEMWDnRiyoBf                   NaN   \n",
       "1297234  LEoPHuyFvzoNfnzGgfcd  vCIpmpaGBnIQhyYNkXqp  puhZPIYqKXeFPaUviSjo   \n",
       "...                       ...                   ...                   ...   \n",
       "1793000  LTuZkdKfxRGVceoWkVyg  JNHcPlZPxEMWDnRiyoBf                   NaN   \n",
       "207261   LTuZkdKfxRGVceoWkVyg  JNHcPlZPxEMWDnRiyoBf  puhZPIYqKXeFPaUviSjo   \n",
       "93412    LTuZkdKfxRGVceoWkVyg  JNHcPlZPxEMWDnRiyoBf  puhZPIYqKXeFPaUviSjo   \n",
       "1568591  gecBYcKZCPMcVYdSSzKP  JNHcPlZPxEMWDnRiyoBf                   NaN   \n",
       "1119538  FTjNLDyTrXaWYgZymFkV  dUuXlWzvmhDSyclWRhNP                   NaN   \n",
       "\n",
       "        device_category device_os device_brand device_model  \\\n",
       "1056663          mobile       iOS        Apple          NaN   \n",
       "1409550          mobile       NaN       Xiaomi          NaN   \n",
       "739873           mobile   Android      Samsung          NaN   \n",
       "62902            mobile       NaN        Apple          NaN   \n",
       "1297234          mobile   Android       Xiaomi          NaN   \n",
       "...                 ...       ...          ...          ...   \n",
       "1793000          mobile       NaN      Samsung          NaN   \n",
       "207261           mobile       iOS        Apple          NaN   \n",
       "93412           desktop   Windows          NaN          NaN   \n",
       "1568591          mobile       NaN       Xiaomi          NaN   \n",
       "1119538          mobile       NaN       Xiaomi          NaN   \n",
       "\n",
       "        device_screen_resolution device_browser geo_country          geo_city  \n",
       "1056663                  428x926         Safari      Russia       Novotroitsk  \n",
       "1409550                  393x851         Chrome      Russia        Balashikha  \n",
       "739873                   339x753         Chrome      Russia        Almetyevsk  \n",
       "62902                    414x896         Safari      Russia  Saint Petersburg  \n",
       "1297234                  393x851         Chrome      Russia            Moscow  \n",
       "...                          ...            ...         ...               ...  \n",
       "1793000                  360x800         Chrome      Russia            Moscow  \n",
       "207261                   375x812         Safari      Russia  Saint Petersburg  \n",
       "93412                  1920x1080         Chrome      Russia            Moscow  \n",
       "1568591                  393x851         Chrome      Russia       Novosibirsk  \n",
       "1119538                  393x851         Chrome      Russia  Saint Petersburg  \n",
       "\n",
       "[1460042 rows x 18 columns]"
      ]
     },
     "execution_count": 539,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 250,
   "id": "6dadbd43-551f-4dc0-b783-173d1e29ae65",
   "metadata": {
    "id": "d0kki5lZWEwh"
   },
   "outputs": [],
   "source": [
    "optimizing_parameters = {\n",
    "    'rare_encoder_tol': (0.01, 0.1), \n",
    "    'constant_dropper_tol': (0.95, 0.999), \n",
    "    'correlated_dropper_threshold': (0.7, 0.99), \n",
    "    'n_estimators': (50, 5000), \n",
    "    'learning_rate': (0.01, 0.25), \n",
    "    'reg_lambda': (0, 50), \n",
    "    'num_leaves': (10, 120), \n",
    "    'reg_alpha': (0, 50)}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 251,
   "id": "be1be778-8908-456c-b1cc-0e0341799f12",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "k4Yax3asoOGT",
    "outputId": "6d280cef-0383-4e6c-da63-2a49a0840d1a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "boosting_type = gbdt\n",
      "|   iter    |  target   | consta... | correl... | learni... | n_esti... | num_le... | rare_e... | reg_alpha | reg_la... |\n",
      "-------------------------------------------------------------------------------------------------------------------------\n",
      "| \u001b[0m1        \u001b[0m | \u001b[0m0.7107   \u001b[0m | \u001b[0m0.9769   \u001b[0m | \u001b[0m0.9074   \u001b[0m | \u001b[0m0.1547   \u001b[0m | \u001b[0m2.747e+03\u001b[0m | \u001b[0m56.6     \u001b[0m | \u001b[0m0.06813  \u001b[0m | \u001b[0m21.88    \u001b[0m | \u001b[0m44.59    \u001b[0m |\n",
      "| \u001b[0m2        \u001b[0m | \u001b[0m0.6836   \u001b[0m | \u001b[0m0.9972   \u001b[0m | \u001b[0m0.8112   \u001b[0m | \u001b[0m0.2      \u001b[0m | \u001b[0m2.668e+03\u001b[0m | \u001b[0m72.48    \u001b[0m | \u001b[0m0.0933   \u001b[0m | \u001b[0m3.552    \u001b[0m | \u001b[0m4.356    \u001b[0m |\n",
      "| \u001b[0m3        \u001b[0m | \u001b[0m0.7094   \u001b[0m | \u001b[0m0.951    \u001b[0m | \u001b[0m0.9415   \u001b[0m | \u001b[0m0.1968   \u001b[0m | \u001b[0m4.357e+03\u001b[0m | \u001b[0m117.6    \u001b[0m | \u001b[0m0.08192  \u001b[0m | \u001b[0m23.07    \u001b[0m | \u001b[0m39.03    \u001b[0m |\n",
      "| \u001b[95m4        \u001b[0m | \u001b[95m0.7113   \u001b[0m | \u001b[95m0.9558   \u001b[0m | \u001b[95m0.8856   \u001b[0m | \u001b[95m0.0444   \u001b[0m | \u001b[95m4.726e+03\u001b[0m | \u001b[95m67.4     \u001b[0m | \u001b[95m0.04732  \u001b[0m | \u001b[95m13.23    \u001b[0m | \u001b[95m38.71    \u001b[0m |\n",
      "| \u001b[0m5        \u001b[0m | \u001b[0m0.7055   \u001b[0m | \u001b[0m0.9724   \u001b[0m | \u001b[0m0.8648   \u001b[0m | \u001b[0m0.01451  \u001b[0m | \u001b[0m3.107e+03\u001b[0m | \u001b[0m77.33    \u001b[0m | \u001b[0m0.06552  \u001b[0m | \u001b[0m47.19    \u001b[0m | \u001b[0m34.09    \u001b[0m |\n",
      "| \u001b[0m6        \u001b[0m | \u001b[0m0.7087   \u001b[0m | \u001b[0m0.9859   \u001b[0m | \u001b[0m0.7406   \u001b[0m | \u001b[0m0.07166  \u001b[0m | \u001b[0m4.72e+03 \u001b[0m | \u001b[0m61.15    \u001b[0m | \u001b[0m0.06792  \u001b[0m | \u001b[0m12.79    \u001b[0m | \u001b[0m47.99    \u001b[0m |\n",
      "| \u001b[0m7        \u001b[0m | \u001b[0m0.709    \u001b[0m | \u001b[0m0.9687   \u001b[0m | \u001b[0m0.7694   \u001b[0m | \u001b[0m0.183    \u001b[0m | \u001b[0m4.745e+03\u001b[0m | \u001b[0m107.8    \u001b[0m | \u001b[0m0.04107  \u001b[0m | \u001b[0m35.41    \u001b[0m | \u001b[0m2.306    \u001b[0m |\n",
      "| \u001b[0m8        \u001b[0m | \u001b[0m0.6953   \u001b[0m | \u001b[0m0.9874   \u001b[0m | \u001b[0m0.9089   \u001b[0m | \u001b[0m0.1418   \u001b[0m | \u001b[0m4.783e+03\u001b[0m | \u001b[0m49.48    \u001b[0m | \u001b[0m0.01103  \u001b[0m | \u001b[0m1.832    \u001b[0m | \u001b[0m11.54    \u001b[0m |\n",
      "| \u001b[0m9        \u001b[0m | \u001b[0m0.7054   \u001b[0m | \u001b[0m0.978    \u001b[0m | \u001b[0m0.8866   \u001b[0m | \u001b[0m0.24     \u001b[0m | \u001b[0m4.735e+03\u001b[0m | \u001b[0m64.69    \u001b[0m | \u001b[0m0.05351  \u001b[0m | \u001b[0m12.02    \u001b[0m | \u001b[0m37.2     \u001b[0m |\n",
      "| \u001b[0m10       \u001b[0m | \u001b[0m0.7092   \u001b[0m | \u001b[0m0.989    \u001b[0m | \u001b[0m0.7662   \u001b[0m | \u001b[0m0.2053   \u001b[0m | \u001b[0m4.746e+03\u001b[0m | \u001b[0m111.7    \u001b[0m | \u001b[0m0.04616  \u001b[0m | \u001b[0m31.5     \u001b[0m | \u001b[0m3.138    \u001b[0m |\n",
      "| \u001b[0m11       \u001b[0m | \u001b[0m0.706    \u001b[0m | \u001b[0m0.9901   \u001b[0m | \u001b[0m0.7797   \u001b[0m | \u001b[0m0.1295   \u001b[0m | \u001b[0m4.706e+03\u001b[0m | \u001b[0m89.92    \u001b[0m | \u001b[0m0.02122  \u001b[0m | \u001b[0m10.07    \u001b[0m | \u001b[0m36.34    \u001b[0m |\n",
      "| \u001b[0m12       \u001b[0m | \u001b[0m0.7072   \u001b[0m | \u001b[0m0.9945   \u001b[0m | \u001b[0m0.7721   \u001b[0m | \u001b[0m0.02904  \u001b[0m | \u001b[0m4.711e+03\u001b[0m | \u001b[0m74.84    \u001b[0m | \u001b[0m0.02862  \u001b[0m | \u001b[0m44.54    \u001b[0m | \u001b[0m32.25    \u001b[0m |\n",
      "| \u001b[0m13       \u001b[0m | \u001b[0m0.7026   \u001b[0m | \u001b[0m0.9845   \u001b[0m | \u001b[0m0.8319   \u001b[0m | \u001b[0m0.224    \u001b[0m | \u001b[0m4.704e+03\u001b[0m | \u001b[0m62.01    \u001b[0m | \u001b[0m0.04982  \u001b[0m | \u001b[0m9.975    \u001b[0m | \u001b[0m21.73    \u001b[0m |\n",
      "| \u001b[0m14       \u001b[0m | \u001b[0m0.7091   \u001b[0m | \u001b[0m0.9731   \u001b[0m | \u001b[0m0.9777   \u001b[0m | \u001b[0m0.2198   \u001b[0m | \u001b[0m2.773e+03\u001b[0m | \u001b[0m59.99    \u001b[0m | \u001b[0m0.0311   \u001b[0m | \u001b[0m34.29    \u001b[0m | \u001b[0m40.44    \u001b[0m |\n",
      "| \u001b[0m15       \u001b[0m | \u001b[0m0.7046   \u001b[0m | \u001b[0m0.9501   \u001b[0m | \u001b[0m0.7732   \u001b[0m | \u001b[0m0.09686  \u001b[0m | \u001b[0m2.74e+03 \u001b[0m | \u001b[0m42.57    \u001b[0m | \u001b[0m0.08392  \u001b[0m | \u001b[0m49.52    \u001b[0m | \u001b[0m40.76    \u001b[0m |\n",
      "=========================================================================================================================\n",
      "boosting_type = goss\n",
      "|   iter    |  target   | consta... | correl... | learni... | n_esti... | num_le... | rare_e... | reg_alpha | reg_la... |\n",
      "-------------------------------------------------------------------------------------------------------------------------\n",
      "| \u001b[0m1        \u001b[0m | \u001b[0m0.7038   \u001b[0m | \u001b[0m0.9769   \u001b[0m | \u001b[0m0.9074   \u001b[0m | \u001b[0m0.1547   \u001b[0m | \u001b[0m2.747e+03\u001b[0m | \u001b[0m56.6     \u001b[0m | \u001b[0m0.06813  \u001b[0m | \u001b[0m21.88    \u001b[0m | \u001b[0m44.59    \u001b[0m |\n",
      "| \u001b[0m2        \u001b[0m | \u001b[0m0.6776   \u001b[0m | \u001b[0m0.9972   \u001b[0m | \u001b[0m0.8112   \u001b[0m | \u001b[0m0.2      \u001b[0m | \u001b[0m2.668e+03\u001b[0m | \u001b[0m72.48    \u001b[0m | \u001b[0m0.0933   \u001b[0m | \u001b[0m3.552    \u001b[0m | \u001b[0m4.356    \u001b[0m |\n",
      "| \u001b[0m3        \u001b[0m | \u001b[0m0.6914   \u001b[0m | \u001b[0m0.951    \u001b[0m | \u001b[0m0.9415   \u001b[0m | \u001b[0m0.1968   \u001b[0m | \u001b[0m4.357e+03\u001b[0m | \u001b[0m117.6    \u001b[0m | \u001b[0m0.08192  \u001b[0m | \u001b[0m23.07    \u001b[0m | \u001b[0m39.03    \u001b[0m |\n",
      "| \u001b[95m4        \u001b[0m | \u001b[95m0.7065   \u001b[0m | \u001b[95m0.9558   \u001b[0m | \u001b[95m0.8856   \u001b[0m | \u001b[95m0.0444   \u001b[0m | \u001b[95m4.726e+03\u001b[0m | \u001b[95m67.4     \u001b[0m | \u001b[95m0.04732  \u001b[0m | \u001b[95m13.23    \u001b[0m | \u001b[95m38.71    \u001b[0m |\n",
      "| \u001b[95m5        \u001b[0m | \u001b[95m0.7098   \u001b[0m | \u001b[95m0.9724   \u001b[0m | \u001b[95m0.8648   \u001b[0m | \u001b[95m0.01451  \u001b[0m | \u001b[95m3.107e+03\u001b[0m | \u001b[95m77.33    \u001b[0m | \u001b[95m0.06552  \u001b[0m | \u001b[95m47.19    \u001b[0m | \u001b[95m34.09    \u001b[0m |\n",
      "| \u001b[0m6        \u001b[0m | \u001b[0m0.7009   \u001b[0m | \u001b[0m0.9859   \u001b[0m | \u001b[0m0.7406   \u001b[0m | \u001b[0m0.07166  \u001b[0m | \u001b[0m4.72e+03 \u001b[0m | \u001b[0m61.15    \u001b[0m | \u001b[0m0.06792  \u001b[0m | \u001b[0m12.79    \u001b[0m | \u001b[0m47.99    \u001b[0m |\n",
      "| \u001b[0m7        \u001b[0m | \u001b[0m0.6941   \u001b[0m | \u001b[0m0.9673   \u001b[0m | \u001b[0m0.7629   \u001b[0m | \u001b[0m0.1495   \u001b[0m | \u001b[0m3.1e+03  \u001b[0m | \u001b[0m71.47    \u001b[0m | \u001b[0m0.09376  \u001b[0m | \u001b[0m14.61    \u001b[0m | \u001b[0m39.34    \u001b[0m |\n",
      "| \u001b[0m8        \u001b[0m | \u001b[0m0.7082   \u001b[0m | \u001b[0m0.9952   \u001b[0m | \u001b[0m0.7474   \u001b[0m | \u001b[0m0.03482  \u001b[0m | \u001b[0m3.122e+03\u001b[0m | \u001b[0m69.63    \u001b[0m | \u001b[0m0.08773  \u001b[0m | \u001b[0m47.67    \u001b[0m | \u001b[0m42.17    \u001b[0m |\n",
      "| \u001b[0m9        \u001b[0m | \u001b[0m0.6855   \u001b[0m | \u001b[0m0.978    \u001b[0m | \u001b[0m0.8866   \u001b[0m | \u001b[0m0.24     \u001b[0m | \u001b[0m4.735e+03\u001b[0m | \u001b[0m64.69    \u001b[0m | \u001b[0m0.05351  \u001b[0m | \u001b[0m12.02    \u001b[0m | \u001b[0m37.2     \u001b[0m |\n",
      "| \u001b[0m10       \u001b[0m | \u001b[0m0.7086   \u001b[0m | \u001b[0m0.9788   \u001b[0m | \u001b[0m0.7382   \u001b[0m | \u001b[0m0.04363  \u001b[0m | \u001b[0m384.9    \u001b[0m | \u001b[0m117.3    \u001b[0m | \u001b[0m0.0828   \u001b[0m | \u001b[0m30.44    \u001b[0m | \u001b[0m4.359    \u001b[0m |\n",
      "| \u001b[0m11       \u001b[0m | \u001b[0m0.702    \u001b[0m | \u001b[0m0.9922   \u001b[0m | \u001b[0m0.8262   \u001b[0m | \u001b[0m0.1057   \u001b[0m | \u001b[0m2.917e+03\u001b[0m | \u001b[0m76.05    \u001b[0m | \u001b[0m0.04535  \u001b[0m | \u001b[0m17.51    \u001b[0m | \u001b[0m43.0     \u001b[0m |\n",
      "| \u001b[0m12       \u001b[0m | \u001b[0m0.7095   \u001b[0m | \u001b[0m0.9787   \u001b[0m | \u001b[0m0.9699   \u001b[0m | \u001b[0m0.1354   \u001b[0m | \u001b[0m1.554e+03\u001b[0m | \u001b[0m107.4    \u001b[0m | \u001b[0m0.02774  \u001b[0m | \u001b[0m34.13    \u001b[0m | \u001b[0m18.62    \u001b[0m |\n",
      "| \u001b[95m13       \u001b[0m | \u001b[95m0.7108   \u001b[0m | \u001b[95m0.9887   \u001b[0m | \u001b[95m0.7021   \u001b[0m | \u001b[95m0.07697  \u001b[0m | \u001b[95m3.026e+03\u001b[0m | \u001b[95m31.61    \u001b[0m | \u001b[95m0.03634  \u001b[0m | \u001b[95m35.23    \u001b[0m | \u001b[95m44.89    \u001b[0m |\n",
      "| \u001b[0m14       \u001b[0m | \u001b[0m0.6929   \u001b[0m | \u001b[0m0.9652   \u001b[0m | \u001b[0m0.9437   \u001b[0m | \u001b[0m0.229    \u001b[0m | \u001b[0m4.437e+03\u001b[0m | \u001b[0m75.66    \u001b[0m | \u001b[0m0.0661   \u001b[0m | \u001b[0m22.42    \u001b[0m | \u001b[0m40.7     \u001b[0m |\n",
      "| \u001b[0m15       \u001b[0m | \u001b[0m0.7001   \u001b[0m | \u001b[0m0.9958   \u001b[0m | \u001b[0m0.7702   \u001b[0m | \u001b[0m0.09209  \u001b[0m | \u001b[0m3.433e+03\u001b[0m | \u001b[0m44.81    \u001b[0m | \u001b[0m0.08533  \u001b[0m | \u001b[0m11.06    \u001b[0m | \u001b[0m9.014    \u001b[0m |\n",
      "=========================================================================================================================\n"
     ]
    }
   ],
   "source": [
    "# байесовский оптимизатор не работает с категориальными значениями, поэтому для \n",
    "# каждого типа бустинга будет проводится своя оптимизация гиперпараметров\n",
    "\n",
    "best_score, best_parameters = 0.0, dict()\n",
    "for boosting_type in ('gbdt', 'goss'):\n",
    "\n",
    "    print(f'boosting_type = {boosting_type}')\n",
    "    optimizer = BayesianOptimization(\n",
    "        partial(optimize_lightgbm, boosting_type=boosting_type), \n",
    "        optimizing_parameters, random_state=RANDOM_SEED)\n",
    "    optimizer.maximize(init_points=5, n_iter=10)\n",
    "    \n",
    "    if best_score < optimizer.max['target']:\n",
    "        best_score = optimizer.max['target']\n",
    "        best_parameters.update(optimizer.max['params'])\n",
    "        best_parameters['boosting_type'] = boosting_type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "id": "38274c1a-629b-4882-9f2d-de35382501a2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "npzZvZ_Um2yH",
    "outputId": "8bff58c0-0014-4216-c895-e2ab6a3032df"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Лучшее значение метрики ROC-AUC=0.711314521294548 при параметрах:\n",
      "\n",
      "constant_dropper_tol = 0.9557954468675777\n",
      "correlated_dropper_threshold = 0.8855770961849819\n",
      "learning_rate = 0.044404788978171135\n",
      "n_estimators = 4726.111139395441\n",
      "num_leaves = 67.40331539250789\n",
      "rare_encoder_tol = 0.04731957459914713\n",
      "reg_alpha = 13.227780605231349\n",
      "reg_lambda = 38.71168447171083\n",
      "boosting_type = gbdt\n"
     ]
    }
   ],
   "source": [
    "print(f'Лучшее значение метрики ROC-AUC={best_score} при параметрах:\\n')\n",
    "for param, value in best_parameters.items():\n",
    "    print(f'{param} = {value}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79d332e9-611f-477f-803e-d048cfec4620",
   "metadata": {
    "id": "n-mXUuF72-6z",
    "tags": []
   },
   "source": [
    "## Оценка модели"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 253,
   "id": "dfda30a4-d290-44fb-b6fe-489f11c4149b",
   "metadata": {
    "id": "JWVR-muRJ77Y"
   },
   "outputs": [],
   "source": [
    "final_pipeline = Pipeline(steps=[\n",
    "\n",
    "    # Создание дополнительных признаков и\n",
    "    # Приведение датафрейма к удобному виду \n",
    "    ('indexer', FunctionTransformer(set_index)), \n",
    "    ('imputer', FunctionTransformer(fill_missings)), \n",
    "    ('engineer', FunctionTransformer(create_features)), \n",
    "    ('dropper', DropFeatures(['client_id', 'visit_date', 'visit_time', \n",
    "                              'device_screen_resolution'])), \n",
    "\n",
    "    # Преобразования численных переменных\n",
    "    ('normalization', YeoJohnsonTransformer()), \n",
    "    ('outlier_remover', Winsorizer()), \n",
    "    ('scaler', SklearnTransformerWrapper(StandardScaler())), \n",
    "\n",
    "    # Преобразования категориальных признаков\n",
    "    ('rare_encoder', RareLabelEncoder(tol=0.047319, replace_with='rare')),\n",
    "    ('onehot_encoder', OneHotEncoder(drop_last_binary=True)), \n",
    "    ('bool_converter', FunctionTransformer(converse_types)), \n",
    "\n",
    "    # Удаление дубликатов и коррелируемых признаков\n",
    "    ('constant_dropper', DropConstantFeatures(tol=0.95579)), \n",
    "    ('duplicated_dropper', DropDuplicateFeatures()), \n",
    "    ('correlated_dropper', DropCorrelatedFeatures(threshold=0.8856)), \n",
    "\n",
    "    # Лучшая модель с оптимизированными гиперпараметрами\n",
    "    ('model', LGBMClassifier(\n",
    "        random_state=RANDOM_SEED, learning_rate=0.04440, boosting_type='gbdt', \n",
    "        n_estimators=4726, reg_lambda=38.7116, reg_alpha=13.22778, num_leaves=67)), \n",
    "    \n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ad9c36e-1fd3-4426-ab0a-f66d0f1914c3",
   "metadata": {
    "id": "oafZ3oICgABQ",
    "tags": []
   },
   "source": [
    "### Метрики модели\n",
    "\n",
    "Для оценки метрик модели обучим её на объектах тренировочной и валидационной выборках и сделаем предсказания на тестовых данных. \n",
    "\n",
    "Целевая метрика `roc-auc=0.6534` (для пресказанных классов) выбранной модели превосходит 0.65, а значит, цель работы выполнена.\n",
    "\n",
    "Хотя метрика `f1=0.0849` довольно низкая, что неудивительно при большом количестве неверно классифицированных объектов нулевого класса.\n",
    "\n",
    "У модели наблюдается совсем небольшое переобучение."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 254,
   "id": "071b3339-d772-4929-baec-37c4261877d2",
   "metadata": {
    "id": "5PrafZ4L3Xnj"
   },
   "outputs": [],
   "source": [
    "final_pipeline.fit(X, y);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 255,
   "id": "35458768-29bb-492c-86d7-f3bae30d83b2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Y3sbdVdEUgw0",
    "outputId": "bc65ace9-343b-4068-de62-f59b1de969c1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Лучший порог перевода вероятностей в класс: 0.025970171560173425\n"
     ]
    }
   ],
   "source": [
    "test_proba = final_pipeline.predict_proba(X_test)[:, 1]\n",
    "best_threshold = find_best_threshold(y_test, test_proba)\n",
    "test_prediction = (test_proba > best_threshold).astype(int)\n",
    "\n",
    "print(f'Лучший порог перевода вероятностей в класс: {best_threshold}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 256,
   "id": "bf65fafe-9126-482c-a441-9eff93e34ff2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "OMPFEzUXUgtP",
    "outputId": "4607bd5c-4aaf-41a3-e2d9-eb5f1f79eaeb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Метрики лучшей модели на обучающей выборке:\n",
      "Порог перевода вероятностей в классы: 0.025970171560173425\n",
      "0.7733082958232637 - roc_auc на вероятностях\n",
      "\n",
      "0.69832994 - roc_auc\n",
      "0.61846869 - accuracy\n",
      "0.05335695 - precision\n",
      "0.78275877 - recall\n",
      "0.09990393 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    991535|    623603\n",
      "       1.0|      9755|     35149\n"
     ]
    }
   ],
   "source": [
    "print(f'Метрики лучшей модели на обучающей выборке:')\n",
    "print_metrics(final_pipeline, X, y, best_threshold)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 257,
   "id": "cb409464-2680-42c1-b7bd-5d785fe66387",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 589
    },
    "id": "QoGbh-kvJ5rI",
    "outputId": "06bda910-e7b9-4eb5-bb2d-426d8125927f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Метрики лучшей модели на тестовой выборке:\n",
      "Порог перевода вероятностей в классы: 0.025970171560173425\n",
      "0.7138627470108961 - roc_auc на вероятностях\n",
      "\n",
      "0.65279266 - roc_auc\n",
      "0.61286000 - accuracy\n",
      "0.04727300 - precision\n",
      "0.69500924 - recall\n",
      "0.08852474 - f1\n",
      "\n",
      "prediction|   0.0    |   1.0    \n",
      "true label|          |          \n",
      "--------------------------------\n",
      "       0.0|    118812|     75778\n",
      "       1.0|      1650|      3760\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbkAAAEWCAYAAAD7HukTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABXK0lEQVR4nO3dd3hU1dbA4d9KL4QACTWU0JsgHRUpgggCiu1aEcGPa2/3WlDsvWFBvRYERGxcUcGKqBcFBanSOwkBAiG0EJKQnv39sScwxCRMyEwmmaz3efJk5tR1TpJZ2fvsIsYYlFJKKV/k5+0AlFJKKU/RJKeUUspnaZJTSinlszTJKaWU8lma5JRSSvksTXJKKaV8liY5VSWIyAYRGeDtOLxNRN4VkUcr+JzTReSZijynp4jIdSLy02nuq7+DVZBoPzlVViKSANQH8oF04EfgDmNMujfj8jUiMgYYZ4w518txTAcSjTGPeDmOJ4BWxphRFXCu6VSCa1blpyU5dbouMsbUALoAXYGHvBtO2YlIQHU8tzfpPVcVTZOcKhdjzD5gHjbZASAiZ4nIYhE5IiJrnKt4RKSOiHwgIntFJEVE5jitGyEiqx37LRaRzk7rEkTkfBFpJCKZIlLHaV1XETkoIoGO9zeKyCbH8eeJSDOnbY2I3C4i24BtxV2TiFzsqJo6IiK/iUj7InE8JCIbHcf/QERCynAN40VkLZAhIgEi8qCIxIlImuOYlzq2bQ+8C5wtIukicsSx/HjVoYgMEJFEEblXRPaLSJKIjHU6X5SIfCsiR0VkuYg8IyJ/lPSzFJFznX5uux0lyUK1ReR7R5xLRaSl036THNsfFZGVItLXad0TIvKFiHwsIkeBMSLSS0T+dJwnSUTeEpEgp306isjPInJYRJJFZIKIDAUmAFc57scax7aRIjLVcZw9jmv0d6wbIyKLROQ1ETkMPOFY9odjvTjW7ReRVBFZKyJniMhNwHXAA45zfev08zvf8drfEVfhz26liDQp6d4qLzLG6Jd+lekLSADOd7xuDKwDJjnexwCHgGHYf6IGO97Xdaz/HvgvUBsIBPo7lncD9gO9AX/gBsd5gos553zgn07xvAy863h9CbAdaA8EAI8Ai522NcDPQB0gtJhrawNkOOIOBB5wHC/IKY71QBPHMRYBz5ThGlY79g11LPsH0Mhxr65ynLuhY90Y4I8i8U13Ot8AIA94yhHrMOAYUNuxfqbjKwzoAOwuejyn4zYF0oBrHMeKAro4nfMw0MtxTz8BZjrtO8qxfQBwL7APCHGsewLIdfxc/IBQoDtwlmP7WGATcI9j+wggyXGcEMf73k7H+rhI3HOA94BwoB6wDLjZ6f7lAXc6zhXqfE+BIcBKoBYg2N+ZhkXvcwm/9/djf+/bOvY9E4jy9t+mfhXzu+3tAPSr6n05/tjTHR+KBvgfUMuxbjzwUZHt52E/8BsCBYUfwkW2eQd4usiyLZxIgs4fMOOA+Y7X4vjw7ud4Pxf4P6dj+GE/+Js53htgYCnX9ijweZH99wADnOK4xWn9MCCuDNdw4ynu7WpgpOP18Q9kp/XHP3yxSS4TCHBavx+bQPyxyaWt07pnih7Pad1DwOwS1k0HphS55s2lXEMKcKbj9RPAwlNc8z2F58Ym2VUlbPcETkkO+1w4G6d/Vhz7/+p0/3YVOcbxewoMBLY67pdfSfe5yO994e/glsKfk35V7i+trlSn6xJjTAT2g7YdEO1Y3gz4h6Mq6oijmu1cbIJrAhw2xqQUc7xmwL1F9muCLeUU9QW2Gq8R0A+buH53Os4kp2McxibCGKf9d5dyXY2AnYVvjDEFju1L2n+nU4yuXMNJ5xaR0U7Vm0eAMzhxL11xyBiT5/T+GFADqIstvTifr7TrbgLElbJ+XzHnAMBRXbrJUeV3BIjk5Gsoes1tROQ7EdnnqMJ8zmn7U8XhrBm21JnkdP/ew5boij23M2PMfOAt4D9AsohMFpGaLp67LHEqL9Ikp8rFGLMA+1/vRMei3diSXC2nr3BjzAuOdXVEpFYxh9oNPFtkvzBjzGfFnPMI8BNwJXAt8Jlx/HvtOM7NRY4TaoxZ7HyIUi5pL/bDE7DPbbAfaHuctnF+9tLUsY+r13D83GKfFb4P3IGt6qqFrQoVF+I8lQPYqrrGJcRd1G6gZSnri+V4/jYe+7Oo7biGVE5cA/z9Ot4BNgOtjTE1sc/aCrcvLY6ix9mNLclFO93vmsaYjqXsc/IBjXnDGNMd6Iitqr7flf1OEaeqRDTJKXd4HRgsIl2Aj4GLRGSI4+F8iKOBRGNjTBK2OvFtEaktIoEi0s9xjPeBW0Skt6NBQLiIDBeRiBLO+SkwGrjc8brQu8BDItIRjjdM+EcZruVzYLiIDBLbkOVe7Aepc5K8XUQai238MgH7jPF0riEc+2F6wBHrWGxJrlAy0Ni5UYarjDH5wFfYxhZhItIOe79K8glwvohcKbZBTJTj53kqEdhkegAIEJHHgFOVhiKAo0C6I65bndZ9BzQQkXtEJFhEIkSkt2NdMhArIn6Oa0zC/rPziojUFBE/EWkpIv1diBsR6en4WQVin4VmYbvFFJ6rRSm7TwGeFpHWjp91ZxGJcuW8qmJpklPlZow5AMwAHjXG7AZGYj/8D2D/472fE79r12OfFW3GPj+6x3GMFcA/sdVHKdjGHmNKOe03QGsg2RizximW2cCLwExHVdh64MIyXMsWbEOKN4GDwEXY7hI5Tpt9iv1wjXd8PXM612CM2Qi8AvyJ/VDthG3IUmg+sAHYJyIHXb0GJ3dgqw73AR8Bn2ETdnGx7MI+a7sXW8W7GtuY4lTmYf9x2Yqtus2i9GpRgPuwJfA07D8Ghf8kYIxJwzb6ucgR9zbgPMfqWY7vh0TkL8fr0UAQsBF7z7/AVo27oqbj/CmO2A9xokZiKtDBUQ06p5h9X8X+Q/QTNmFPxTZsUZWMdgZXqgzEdoQfZ4z5xduxlJWIvAg0MMbc4O1YlKooWpJTykeJSDtHNZqISC/g/4DZ3o5LqYqkIwAo5bsisFWUjbBVw68AX3s1IqUqmFZXKqWU8llaXamUUspnVbnqyujoaBMbG+vtMJRSSlUiK1euPGiMqVt0eZVLcrGxsaxYscLbYSillKpERGRnccu1ulIppZTP0iSnlFLKZ2mSU0op5bOq3DO54uTm5pKYmEhWVpa3Q1HFCAkJoXHjxgQGBno7FKVUNeMTSS4xMZGIiAhiY2Oxg8arysIYw6FDh0hMTKR58+beDkcpVc14rLpSRKaJnVZ+fQnrRUTeEJHtYqed73a658rKyiIqKkoTXCUkIkRFRWkpWynlFZ58JjcdGFrK+guxo8i3Bm7CzjF12jTBVV76s1FKeYvHqiuNMQtFJLaUTUYCMxyTXS4RkVoi0tAxR5RSSqnKIDsbMjIgMxMOHYK0NMjLg9xc+z0vD/bvBz8/uywnx35t3Qp1/9Y3+7jdRyP4Lc2PjMye3PJpvxK3Ky9vPpOL4eR5pxIdy/6W5ETkJmxpj6ZNm1ZIcFXJE088QY0aNbjvvvtc2j4hIYERI0awfv3fa5LHjRvHv//9bzp06HC84310dDTnnHMOixcvJiEhgcWLF3Pttde6+zKUUp5mjE1UhUnr4EFYscImp3XrYN8+m6wyMux2mzaV/5yOmpxME8Lv9OVHhjKPC9gW2Irp09sz97mXuaX8ZymRN5NccXVYxY4WbYyZDEwG6NGjR7UcUTovL4+AAM//uKZMmVLs8sWL7cTYCQkJfPrpp5rklKpsCgpsiWrfPkhNhfh4mD8fjh2Dv/6Co0fh8OFTH6dGDejRA1q2hPPOs6WyM86A0FAIC4PoaAgOhoAACAy03wtfR0ZCUBAEB2OCgtm0I4R582DePFiwACIjd3LNNa/QZFNLxp4fTMeOPzF7baxHb4s3k1wi0MTpfWNgr5diKbdXX32VadOmAbY0dM8995CQkMCFF17Iueeey+LFi4mJieHrr78mNPTkCYTHjBlDSEgIGzZsIDk5mVdffZURI0Ywffp0vv/+e7KyssjIyOCLL77gxhtvJD4+nrCwMCZPnkznzp0BWLNmDQMHDmT37t088MAD/POf/yQ9PZ2RI0eSkpJCbm4uzzzzDCNHjgRs0rzhhhtYtWoVbdq0YcaMGYSFhTFgwAAmTpxIjx49ToqxRo0apKen8+CDD7Jp0ya6dOnCDTfcwFdffcWbb75Jly5dAOjTpw/vvPPO8biUUm507Bhs3w5r1thS1tq1trpw/XrYs6f4ffz8oGdPiIqCc86Btm2hQ4cTSatFC2jeHMLDj5e6TldKCvzvfxxPbLsddXVduqQyadLztG79On5+wuOPX0GtWv2AVuU6nyu8meS+Ae4QkZlAbyDVLc/j7rkHVq8u92FO0qULvP56iatXrlzJBx98wNKlSzHG0Lt3b/r370/t2rXZtm0bn332Ge+//z5XXnklX375JaNGjfrbMRISEliwYAFxcXGcd955bN++HYA///yTtWvXUqdOHe688066du3KnDlzmD9/PqNHj2a141rXrl3LkiVLyMjIoGvXrgwfPpx69eoxe/ZsatasycGDBznrrLO4+OKLAdiyZQtTp06lT58+3Hjjjbz99tsuVXe+8MILTJw4ke+++w6AOnXqMH36dF5//XW2bt1Kdna2JjilyisjA374ASZPhp077XOxXbuK37Z2bejWDS69FEJCbAmsZUtb2urVyy7zkPx8W9v54482qS1daguUNWvC+efDI4/kc/bZ73L06BPk5h6kfv3rad78WUJCmpzy2O7isSQnIp8BA4BoEUkEHgcCAYwx7wI/AMOA7cAxYKynYvG0P/74g0svvZTw8HAALrvsMn7//XcuvvhimjdvfryU0717dxISEoo9xpVXXomfnx+tW7emRYsWbN68GYDBgwdTp06d4+f58ssvARg4cCCHDh0iNTUVgJEjRxIaGkpoaCjnnXcey5YtY/jw4UyYMIGFCxfi5+fHnj17SE5OBqBJkyb06dMHgFGjRvHGG2+4/EzP2T/+8Q+efvppXn75ZaZNm8aYMWPKfAylqr3Dh2HRIpg9GxYuhLi4E+sCAuw/2iNH2gzSrRu0aweNG4MX2ijs2XOipPbLLzZ0EVvDOWECDB0KvXvbsI3xY9WqjwkPP4OWLV8hIuK0e4qdNk+2rrzmFOsNcLvbT1xKictTSpt4Njg4+Phrf39/MjMzi92uaDP7wveFibOk8xRuV9z+n3zyCQcOHGDlypUEBgYSGxt7vL9aSecrq7CwMAYPHszXX3/N559/rjNEKFWc/HxYvNgWdTZssJkhORkSEux3Z/XqwSWXwD//CQMG2CpFL8rKgt9/P5HYCturNWwIF18MQ4bYUlt0tF2elraSjRsfp127qQQF1adTp7kEBER6rSuRjl3pBv369WPOnDkcO3aMjIwMZs+eTd++fct0jFmzZlFQUEBcXBzx8fG0bdu22PN88sknAPz2229ER0dTs2ZNAL7++muysrI4dOgQv/32Gz179iQ1NZV69eoRGBjIr7/+ys6dJ2ai2LVrF3/++ScAn332Geeee65LcUZERJCWlnbSsnHjxnHXXXfRs2fP46VOpaq1nTth0iS4+mpbdRgQAP36wf33w/Tp8M03sHevfTY2YADcfrvdvjD5zZ4Nw4Z5JcEZA5s32/LChRdCnTpwwQXw5ptQvz689JJ9JLhnD3zwgb3E6GjIytrNpk3Xs3JlD9LSlpKRYVtmBgbW8mpfWZ8Y1svbunXrxpgxY+jVqxdgP/S7du1aYtVkcdq2bUv//v1JTk7m3XffJaSYevQnnniCsWPH0rlzZ8LCwvjwww+Pr+vVqxfDhw9n165dPProozRq1IjrrruOiy66iB49etClSxfatWt3fPv27dvz4YcfcvPNN9O6dWtuvfVWl+Ls3LkzAQEBnHnmmYwZM4Z//etfdO/enZo1azJ2bJWtcVbq9GVm2sYgn39uWzM6WiIfV6eObfgxbJgt+nTpYhuDVCJHjpzcYKTw8V+bNjBunK2C7N/ftk0pyhjDjh2PkJj4KsYYmjQZT7NmDxEQEFmh11ASKa2qrTLq0aOHKVoltmnTJtq3b++liMpvzJgxjBgxgiuuuMLboZyWvXv3MmDAADZv3oxfCX+8Vf1npNRxx47ZKscFC2zjkF9/PXn9pZdCRATccQd07WpLcZVMfj6sXHkiqS1ZYpdFRNiqxyFD7FdsbMnHMMYcL6Ft3HgdILRo8SwhIc0q5BqKEpGVxpgeRZdXvruvqpQZM2bw8MMP8+qrr5aY4JSq0lJS4OefYc4cWw3pXFJr2hRuvhnat7df559f6UpphfbuPZHUfv75RIOR7t3hwQdtUjvrLNvdrTTGGA4f/oH4+Adp3/5TatToRPv2MxDxr5gLKSNNcpXA9OnTvR3CaRs9ejSjR4/2dhhKuU92tu2G9J//wJ9/2qpIZ3ffbevxhg61fcwqqezskxuMrFtnlzdoACNG2PDPP7/Ukbf+Ji1tNXFx93LkyHxCQ1uTl2dbd1fWBAea5JRS1d2+fTBjBnz99d+fp9WrZ0tqvXvbok6jRt6J0QXG2OEiC5Pab7/ZmtXAQOjbF1580V5C586n1+d769Zb2bv3PQIC6tCq1Rs0anQLfn6Vf45ITXJKqepnyxZ49VXb2dpZ8+a2xeNVV9mWFpV8rNzU1JMbjBQ2oG7dGm680Sa1AQPsSF2nIz//GH5+oYgIQUENaNLkPpo2nUBgYC13XYLHaZJTSvm+nBz47juYOBGWLbOtLMB29urdG667DqpAw6+CgpMbjPz554kGIwMHwvjxNrGVtxa1oCCPffumsWPHY7RrN42oqGHExj7unouoYJrklFK+5/Bh+xBq0SI75tSWLXbwYrBjOF5wgR2e44wzvBunC5KS4KefTjQYOXjQLu/e/URSO/vsUzcYcYVtVPIjcXH3c+zYBmrW7ENQUIPyH9iLNMlVYoWDIiulXLB1K8yaZTtVHzhw8rr27eHZZ+Gaa4rv7FWJZGfDH3+cKK2tXWuX169vO2cPGQKDB9vHhe62efMNJCd/RGhoKzp2/JLo6Eur/KTHmuSUUlWTMXaMqfnzbYlt1iy7vH59W7QZNcq2uOjQAfwrb+s/Y2DbthNJ7ddfTzQY6dMHnn/etoTs3NkzvROys/cSGFgXP79A6tQZQo0a3YiJuQ0/vyD3n8wbjDFV6qt79+6mqI0bN/5tWUV75ZVXTMeOHU3Hjh3Na6+9ZowxZseOHaZdu3Zm3LhxpkOHDmbw4MHm2LFjf9t337595pJLLjGdO3c2nTt3NosWLTLGGBMeHm6MMSYtLc0MHDjQdO3a1Zxxxhlmzpw5xhhj0tPTzbBhw0znzp1Nx44dzcyZM40xxowfP960b9/edOrUydx7770VcPWnVhl+RspHrFplzLPPGtOrlzE2R9iviy4yZsECYwoKvB3hKaWmGjN7tjG33GJM8+YnLqFlS2Nuv92Yb74x5uhRz8aQm5tm4uMfNwsWhJnExLc8e7IKAKwwxeQMnyvJeWGmnXJPtXPXXXfRv39/Zs+eTX5+/t+qKENCQoqdMufHH3+kUaNGfP/99wCkpqZy+PBhZs+ezebNmxERjhw54t6boVRFy8+3VZHffWefr82ff2Ldww/b52vdu1fqasiCAli1ypbUfvzRNhjJy7OtHgcOhPvus9WQLVt6PhZj8tm3bzo7djxKTk4SdeteSZ06F3r+xF7ic0nOG8o71c78+fOZMWMGYGcqiIw8ecw3Y0yxU+Z06tSJ++67j/HjxzNixAj69u1LXl4eISEhjBs3juHDhzNixAiPXrtSHpGcbMeC/O03+Oqrk9cNHAivvWarISvhkFmF9u07ucFI4WPCrl1PJLVzzrETaVekTZuuZ//+z6hZ82w6dvySyMizKzaAClZ5f0NOkxdm2nHLVDulKWnKnDZt2rBy5Up++OEHHnroIS644AIee+wxli1bxv/+9z9mzpzJW2+9xXzn/3yVqqwWLbIj9C9ZcmI+F7APpHr2tMN09OxZ7tmrPSUnx15C4bO1whqlunVPjAU5eLB9ZFjR0tPXExwcQ2BgbRo1upXo6EupW/eKKt+oxBU+l+S8oV+/fowZM4YHH3wQYwyzZ8/mo48+cnn/QYMG8c4773DPPfeQn59PRkbG8Sl0gBKnzNm7dy916tRh1KhR1KhRg+nTp5Oens6xY8cYNmwYZ511Fq1aeX56eaVOmzHw2Wfwxht2rjWATp3giSdOjDJSiT+It28/UQX56692Qu+AANtg5LnnbPjenHQgOzuJhITHSUqaStOmD9CixfPUqlW2acCqOk1yblDeqXYmTZrETTfdxNSpU/H39+edd97h7LNPVCGUNGXOunXruP/++/Hz8yMwMJB33nmHtLQ0Ro4cSVZWFsYYXnvtNbdfr1LlkpsL774LX35pR/IHqFkTnnnGDqFVOPtmJZSWZh8JFpbW4uPt8hYtYPRom9TOO89ejjfl52ewe/cr7Nr1Esbk0Ljx3TRpcr93g/ISnWpHVQj9GSmSkuDJJ+G9904s69sXhg+3LcacqvYri4ICW+1YmNQWLbINRsLD7aPBwmrIylZhsnnzWPbtm07dulfQvPnzhIVVsgA9QKfaUUpVvMxMO5r/f/8Lhf+cDhwIt95q512rhP3X9u+3DUZ+/NE2GCkcKKVLF7j3XpvU+vSp+AYjp5KS8j9CQmIJDW1J06YTaNhwHJGRfbwdltdpklNKud/GjXbw40mT7PsWLWyGuPJKcFTrVxY5OXbygcLS2qpVdnl0tO2dMGSI/d6gko5ulZGxgbi4Bzh8+AcaNbqFNm3eISysNdDa26FVCprklFLukZ5u+7K99podBBmgWzdbFXn99V4Nrai4uBNJbf58G3pAgB0o5dlnbWLr2rXSzn8KQE5OMjt2PE5S0vv4+0fQosXLxMTc4e2wKh1Nckqp02eMLQa99x4UtiiuWdM2+3/hBTjzTO/G55Cebls//vijTWxxcXZ58+Z29K8hQ2wtqrcbjJTFrl0vsW/fVGJi7qBZs0cJCqq8DXa8SZOcUqrsDh2ynVKfeebEspgYWz150UVef2BVUABr1pzcYCQ3F8LCbOvHu++2ebhVq0rdQ+EkxhSQnPwxoaGtiIw8h2bNJtCo0c2EhbXxdmiVmiY5pZRr0tPhm29sq4xPP7VZo29f6NHDVkl6eYLR/fttQ5F582yIycl2eefO8K9/nWgwUgkbcZ5SSsqvxMXdS3r6Kho0uJHIyHMIDIwiMDDK26FVeprkKqnY2FhWrFhBdCXuM6SqgZQU2zpywwb4+mvbWhLg3HNth+1Bg7wWWm6uHQOysAryr7/s8sLp4gobjDRs6LUQyy0jYzPx8Q9w6NC3BAc3pX37T6hX72pvh1WlaJLzgMLRr/0q81NrpUqzdy889hhMnXpi2ejRMHIk9O9vM4kXxMef3GAkLc32Qjj7bHj6aVsF2a1b5W4wUhaHD8/lyJEFtGjxAjExd+PvH+LtkKocTXJukpCQwIUXXsh5553Hn3/+SZcuXVi3bh2ZmZlcccUVPPnkk4Atod1www18++235ObmMmvWLNq1a8ehQ4e45pprOHDgAL169TppPMxXX32VadOmAXY0lXvuuYeEhASGDh3Kueeey5IlSzjzzDMZO3Ysjz/+OPv37+eTTz45PgKLUi47cgTuuutEI5LevW1zw/79vTIYcnq6HaO5MLFt22aXN2sG1157osFIkTHNq6z8/EwSEycREhJL/fpXExNzO/XrjyIoqK63Q6uyfDLJrVo14G/L6tW7kpiY28jPP8batcP+tr5BgzE0bDiGnJyDbNhwxUnrunb9zaXzbtmyhQ8++IC3336bw4cPU6dOHfLz8xk0aBBr166lc+fOAERHR/PXX3/x9ttvM3HiRKZMmcKTTz7Jueeey2OPPcb333/P5MmTgdKn8dm+fTuzZs1i8uTJ9OzZk08//ZQ//viDb775hueee445c+aU6b6paio7G6ZMOdH7OTvbZo1PPrGjkVQgY+xM2IXjQf7xx4kGIwMGwB132MTWpk3VaTDiCmMK2L//M+LjJ5CdvYuGDW+mfv2r8fML0gRXTj6Z5LylWbNmnHXWWQB8/vnnTJ48mby8PJKSkti4cePxJHfZZZcBduqdrxzTiCxcuPD46+HDh1O7dm3g1NP4dOrUCYCOHTsyaNAgRIROnTq5PG6mqsY2bLDT2fznP7a1JMAZZ9j3/fpVWBgHDpzcYGTfPru8UyfbCnLIEPsIMMRHa+pSU5ewffudpKWtoEaNbrRr9yG1aw/wdlg+wyeTXGklL3//sFLXBwVFu1xyK6owEe3YsYOJEyeyfPlyateuzZgxY8jKyjq+XeH0O/7+/uTl5R1fXty0F65O4+Pn53f8vZ+f30nHVeokq1fbyUZ/+MG+r1/fvn/8cQgM9Pjpc3PtbDqFVZArV9oSXJ06diqaoUNtg5FGjTweSqWQk7OPnJx9tGs3g/r1r0PERx4oVhJ6Nz3g6NGjhIeHExkZSXJyMnPnzj3lPv369eOTTz4BYO7cuaSkpBxfPmfOHI4dO0ZGRgazZ8+mb9/qNVWGcpN33rEJrWtXm+D69IFNm2zR6ZlnPJrgEhJsf/FLL7XDZfXrZ/uKBwfbMZuXLrVdAGbOhDFjfDvB5eQcYNu2O0lIsH0Mo6NH0qvXNho0uF4TnAd4tCQnIkOBSYA/MMUY80KR9ZHAx0BTRywTjTEfeDKminDmmWfStWtXOnbsSIsWLejT59SDpD7++ONcc801dOvWjf79+9PU0eeovNP4qGruyBF4+WXbcfvYMdt45Pbb7ZcHZ4XIyDi5wcjWrXZ506Zw1VW2CnLQIKhVy2MhVDr5+Vns2fMGO3c+S35+Bo0b3w3YGhxtNek5HptqR0T8ga3AYCARWA5cY4zZ6LTNBCDSGDNeROoCW4AGxpicko6rU+1UTfozqkDG2LEj33oLPv7YLouNtcWn99/3yGgkxsC6dSeS2u+/24GPQ0Ntw8yhQ21ia9vWtxqMuColZT6bN99IdvZOoqJG0KLFi4SHd/B2WD7FG1Pt9AK2G2PiHQHMBEYCG522MUCE2IdRNYDDgD5MUup05ObaRiMvvnii9cbll8O4cTbLuNmhQ7bByI8/2gYjSUl2+RlnwJ132qTWt6/vNhhxRUFBHn5+AQQE1CYwMIp27aZSu7b3OtBXR55McjHAbqf3iUDvItu8BXwD7AUigKuMMQVFDyQiNwE3Acer8ZRSDtu2wbRpNsGlpdkqyeuvh6eesiU4N8nLO7nByIoVtgRXu7ZtMFI4wkjjxm47ZZV17Ng24uPH4+8fQfv2HxIR0ZXu3VcU27hMeZYnk1xxP82idaNDgNXAQKAl8LOI/G6MOXrSTsZMBiaDra4s7mTGGP0FqqSq2uzzVcaSJbb1xtdf2/d9+9pWG2PHuq1OcOfOE0ntl1/g6FE7mkjv3nZUryFD7NCVlXDuU6/IzT1EQsJT7N37NiLBNGv20PHPJv188g5PJrlEoInT+8bYEpuzscALxn4KbheRHUA7YFlZThQSEsKhQ4eIiorSX6RKxhjDoUOHCKnOdVbutHOnHY3kww9h+3ZbarvtNvvVsWO5D3/sGCxYcGI8yC1b7PImTex8p4UNRhzdOJWTw4fnsWHDVeTnp9Gw4ThiY58kOLiSzrRajXgyyS0HWotIc2APcDVwbZFtdgGDgN9FpD7QFogv64kaN25MYmIiBw4cKGfIyhNCQkJorHVY5bNsme3L9ssv9n2DBnD//Xb0/3K0tzcG1q8/ucFIdrZ9jta/P9x8s01s7dtXzwYjp2KMIS8vhcDAOoSHd6J27fOIjX2aGjXO8HZoysFjSc4YkycidwDzsF0IphljNojILY717wJPA9NFZB22enO8MeZgWc8VGBhI8+bN3Ri9UpWAMfD99/Dgg3Z0EoAOHeygyY6RdU7HoUM2VxYmtr17Txz69ttPNBgJDXXDNfiw1NTFxMXdi0gQXbr8RnBwI844Y7a3w1JFeLSfnDHmB+CHIsvedXq9F7jAkzEoVeX873+2ueKXX56Ywvrcc2HOnNMa/T8vzxYEC6sgly+3+bNWrZMbjDRpcspDKSAzM474+Ac5cOALgoIa0rz5M9jmBlrUrYx8clgvpaqk9HS47z47NAjYkUkmTbKNSWrWLNOhdu06ucFIaqptMNKrl51BZ8gQ6NnTKxMLVGmHD//MunXDEQkkNvYJmjS5D3//cG+HpUqhv+JKeVtcHLzyii257d9v+7VNnFim+WMyM22DkcLEtmmTXR4TY7vKDR1qG4zUqeOha/BhBQXZZGXtJCysDZGRfYiJuYMmTe4jONiHxx7zIZrklPKWX3+FRx6BxYvt+5Yt7QglgwefcldjYOPGE1WQCxfaBiPBwbbByLhxtrTWoYM2GDldxhgOHPiS+PjxgNCr1yb8/cNo1epVb4emykCTnFIVbcECePRR25QRbDHrmWege/dSdzt8+OQGI3v22OXt28Ott9qk1q+fnXtNlU9q6hLi4u7l6NHFhIV1pGXLifj5eX6GBuV+muSUqig7dsD48TBrln3/2GN2Fu4SGpPk5dlGIoVJbdkyKCiwtZjnn39iShodBMi9jhz5ndWr+xEU1IA2bd6nQYMx+PnpR2VVpT85pTwpLw8++8yOSvLtt/b9RRfZxiUNG/5t8927T24wcuSIrW7s1cvWbA4ZYl9rgxH3ys1NIT19DbVrDyAysg+tW79F/fo3EBBQw9uhqXLSPxWlPOHIETsyyeuvQ3w8hIfbIUMeecQOxe+QmWmfpxUmto2O4csbNbJzrw0ZYkttp9FzQLmgoCCHvXvfJSHhScBw9tmJ+PuHERNzu7dDU26iSU4pd0pNtVWShd0AYmNhyhS48UYQwRjYtPFEUluwALKybIORvn3tZkOG2BG6tMGI5xhjOHhwDvHxD5CZuZ1atQbRsuVE/P31gaav0SSnVHkVjo115502a4GdGXTUKBgxgpQU+OWLE4ktMdFu0q7diWGz+vfXBiMVKT19DRs2XEZYWHs6dfqeOnUu1HFvfZQmOaXKY/FiuOUWO2MoQK9e5L86ieX+Z9mk9hwsXWobjNSsaaseH33UJrZmzbwbenWTlbWTI0d+o0GDG4iI6EKnTnOpXft8bVTi41z+6YpIuDEmw5PBKFVlrFhhi2F//QV167LngUnMCxjGvO2t+PkiSEmx1Y09esCECbYlZO/e2mDEG/LyUtm58zkSEyfh5xdEVNTFBAbWJirK/RPJqsrnlH9yInIOMAU7c3dTETkTuNkYc5ung1Oq0tm1Cx56iKxPv+L3wIH82OZb5vkNZcNL9k+pYUMYOfJEg5HoaC/HW40VFOSyd+977Nz5JLm5B6lffzTNmz9LYKDOE1SduPJ/5WvYyU2/ATDGrBGRfh6NSqlKxvyxiM23TmLe+kbMYxQL/KaSmRtCUIJtMHKDo8FIp07aYKSyyM7eQ1zcvURGnkPLlq8QEdHN2yEpL3Cp8sQYs7vIQ9l8z4SjVOWRnZHHd7f9wLwv05mX0YddfA5Am8bHGHdpCEOH2gYj4To+b6WRlraSAwdm06LFM4SGxtKjx2rCwtppo5JqzJUkt9tRZWlEJAi4C9jk2bCU8q706V8w/NamLMy6mAiOcn7TbUy4vRZDrowkNlabQVY2WVm72bFjAsnJHxMYWJeYmNsJDm5IeHh7b4emvMyVJHcLMAmIARKBnwB9Hqd8086dpN36AMPm3sGfdGPaP+Yy6qMhBAaXPq6k8o68vHR27XqOxMTXAGja9CGaNn2QgICyTU2kfJcrSa6tMeY65wUi0gdY5JmQlPKCzEx4/HHSJr7HheZ7lsjZfDo9lytHX+jtyFSp8klKmkrdulfQvPmzhIToQJ7qZH4ubPOmi8uUqnqMgcmToXFjjr78LkNrL2GJfx8++68/V44O8XZ0qgg7Usl3rF9/BQUFeQQERNKr1xbat/9IE5wqVoklORE5GzgHqCsi/3ZaVRPw93RgSnnc8uV2FoAlSzjasC1DW//B8h3R/Pe/dqJRVbmkpa0iLu4+jhyZT2hoG7KzEwkNjSUwsJa3Q1OVWGnVlUHYvnEBQITT8qPAFZ4MSimP2rkT/v1v+OorAFIHX8HQ1Jms+Muf//4XLrvMy/Gpk+TlHWXbtrtITp5BQEAdWrV6k0aNbtb53ZRLSkxyxpgFwAIRmW6M2VmBMSnlGdnZ8OST8Pzz9v1tt5F692MMGV2flX/B55/bkf9V5WCMQUTw8wsjI2M9TZrcT7NmEwgIiPR2aKoKcaXhyTEReRnoCBx/SGGMGeixqJRytxkz7BiTmZnQtStMmsSRTn0ZMgRWrYIvvrAjlSjvKyjIY9++aezZ8xZdu/5BQEBNundfiog+JVFl50rDk0+AzUBz4EkgAVjuwZiUcp+0NBgzBm64wfba/uorWLmSlDP6MniwJrjKxBjDoUNzWbGiC1u33kxAQCS5uYcANMGp0+ZKkosyxkwFco0xC4wxNwJneTgupcrHGPj0U2jeHD78EG691c5xc+mlpBwRBg+GNWvgyy/h4ou9HazKz89g7doLWLduGMZk07HjV3TpspDQ0ObeDk1Vca5UV+Y6vieJyHBgL9DYcyEpVU6//AJXXGEnMG3fHmbNgvPOA+DwYRg82E7/9tVXMGKEl2Ot5vLzj+HvH4a/fziBgfVo1WoSjRrdgp9fkLdDUz7ClZLcMyISCdwL3IedkeAeTwal1GlZtMhmrcGDISICnn0WVq8+KcGdf75NcLNna4Lzpry8dHbseII//2xCZuYOADp0+ITGje/SBKfc6pQlOWPMd46XqcB5cHzEE6Uqh6NH4brr4DvHr+r998PDD0PkiVZ4hw7ZBLdpE8yZAxfqQCZeYUw+SUkfkJDwKDk5+6hb9ypEdJI95TmldQb3B67Ejln5ozFmvYiMACYAoUDXiglRqVIsWgQ33ghbt0JQkM1iLVqctMnBgzbBbd5sE9xQnSvTKwoKcvjrr7NIT19FzZrn0LHjbCIj9fG+8qzS/oWaCjQBlgFviMhO4GzgQWPMnAqITamSZWTASy/BU0/Z97Nm2edwRRw8CIMG2Rz4zTdwwQUVHKciK2s3ISFN8PMLIjr6Mpo2nUDdupfr9DeqQpSW5HoAnY0xBSISAhwEWhlj9lVMaEqV4Kef4I47YNs2GDYM3nsPGv+9LdSBAzbBbdtmE9zgwV6ItRrLzk4iIeFxkpKm0bXrQiIjzyE29hFvh6WqmdIanuQYYwoAjDFZwNayJjgRGSoiW0Rku4g8WMI2A0RktYhsEJEFZTm+qmbmzoVzz7VTcG/bZkcu+f77YhPc/v0wcKDd7NtvNcFVpPz8DBISnmLp0tbs2zedxo3vIiysnbfDUtVUaSW5diKy1vFagJaO9wIYY0zn0g7seKb3H2Awdh665SLyjTFmo9M2tYC3gaHGmF0iUu/0L0X5rL174V//suNuhYbC00/bgZVrFj9nWGGCi4+3bVEGDargeKsxYwpYubIXx45tdEx/8zxhYa28HZaqxkpLcuWdUrcXsN0YEw8gIjOBkcBGp22uBb4yxuwCMMbsL+c5lS8pKIBXXrHjTR47ZhPbc8/ZkUtKkJxsE9yOHTbBDdTB5ypEauoiatY8GxE/mjV7hJCQpkRGaiNs5X2lDdBc3kGZY4DdTu8Tgd5FtmkDBIrIb9iZDiYZY2YUPZCI3ATcBNC0qc4ZVS0UFMCoUfDZZ7af2xtvwBlnlLrLvn02qe3cCT/8AAMGVEyo1VlGxgbi4u7n8OG5dOjwOfXq/YP69a/xdlhKHefJDirFNZ0yxZy/OzAI2y3hTxFZYozZetJOxkwGJgP06NGj6DGUr0lJgSuvtCOX3HgjTJkCp2iJl5RkE9yuXTbB9e9fQbFWUzk5yezY8RhJSVPw94+gRYuXiY7W8dFU5ePJJJeI7YJQqDF2SLCi2xw0xmQAGSKyEDgT2IqqnpYtsz21Dx+GBx+01ZMuJLjzzrNDU86dC/36VVCs1ZQxhjVrhnDs2AZiYu6gWbNHCQqK9nZYShXLpSQnIqFAU2PMljIceznQWkSaA3uAq7HP4Jx9DbwldsiDIGx15mtlOIfyFfv326kAliyx799803YTOIW9e22C27PHJri+fT0cZzVlTAH7988kOvoS/P3DaN36LYKC6hEW1sbboSlVqlOOXSkiFwGrgR8d77uIyDen2s8YkwfcAcwDNgGfG2M2iMgtInKLY5tNjuOuxXY6n2KMWX+a16Kqqp9/ttlp2TK4/nrYsMGlBLdnj33utncv/PijJjhPSUmZz8qV3dm06TqSkz8GoFatczXBqSpBjCn9EZeIrAQGAr8ZY7o6lq09VRcCT+nRo4dZsWKFN06t3C0nBx591I5cEh0N778Pl1zi0q579tgSXFKSTXB9tCGf22VkbCI+/gEOHfqO4OCmtGjxPPXqXY2IK+O6K1WxRGSlMaZH0eWuVFfmGWNSdQge5TbGwMyZ8MILsHYtXHYZTJ9uZw5wQWKiTXDJyTBvHpxzjmfDra62bbudtLSVtGjxAjExd+PvH+LtkJQqM1eS3HoRuRbwF5HWwF3AYs+GpXzaddfZrgH16tlkd9VVLu+6e7dNcAcO2NG9ztLxfd0mPz+TPXveoH790QQHN6RNm8kEBEQSFFTX26EpddpcSXJ3Ag8D2cCn2Gdsz3gyKOWjPv0U/vlP27G7UydYvhyCg13efdcum+AOHrQJrnfRXpfqtBhTQHLyp+zYMYHs7N34+YXTuPEdOlKJ8gmuJLm2xpiHsYlOqbI7ehSuvto2fwwOhtGj4YMPwM/1Zzs7d9oEd/iwbafSq5cH461GjhxZwPbt95KevpIaNbrRrt0Matce4O2wlHIbV5LcqyLSEJgFzDTGbPBwTMqXzJ0Ll18OmZm2WnLaNAgLK9MhEhJsgktJsQmuZ0/PhFodJSVNJTd3P+3afUT9+tdqoxLlc075G22MOQ8YABwAJovIOhHR+TJU6dLSbEvJYcPsoMqffWafv51GghswAI4csQOgaIIrn5ycA2zbdidpaasBaNXqNXr12kKDBqM0wSmf5FJncMcUO2+IyK/AA8Bj6HM5VZJ162yLye3boUkTWLMGatcu82F27LAluKNHbYLr3t0DsVYT+flZ7NkziZ07nyM/P4OwsHZERHQhMDDK26Ep5VGnTHIi0h64CrgCOATMBO71cFyqKjLGltjGjrUDLM+e7XK/t6Li422CS0uzCa5bN/eGWp3s3z+LuLj7yM7eRVTUCFq0eInw8PJOMqJU1eBKSe4D4DPgAmNM0bEnlbK2b4cbboDFiyEmxk5meuaZp3WouDib4DIy4H//g65d3RxrNZORsY7AwDq0a/cBtWvr3EOqenHlmdxZxphJmuBUib77zia0pUttB+9du8qV4AYM0ARXHseObWP9+ss4eNCOvtes2cN0775CE5yqlkosyYnI58aYK0VkHSdPkePSzOCqGli3znYHWL0aoqLs99atT/tw27fbBJeVBfPnn3aerLZycw+RkPAUe/e+jUgwdepcCICfn+t9EZXyNaVVV97t+D6iIgJRVcy339o537KyYNAg+O9/baI7Tdu22QSXk2MTXGf9F6pMkpKmsn37veTnp9Gw4ThiY58kOLiBt8NSyutKmxk8yfHyNmPMeOd1IvIiMP7veymfZ4zt63bvveDvb8fZaty4XIfcutU+gytMcJ06uSlWH2cHVy9AxB/wIzLyHFq2fJnw8I7eDk2pSsOVjjGDi1l2obsDUVVAUhJccAGMGwd5ebZndjkT3JYttgSXmwu//qoJzlWpqYv466+z2bPnLQAaNBhD584/aIJTqogSk5yI3Op4HtdWRNY6fe3Azv+mqpM334RGjWx7/iuvhEOH4Oyzy3XIzZttgsvPtwnujDPcE6ovy8yMY/36K1i16lyys3cRGFgPAJ0lRKnilfZM7lNgLvA88KDT8jRjzGGPRqUql/ffh7vuss/cfvrJLZ3WNm2CgQNt7eevv0KHDm6I08clJr5FXNy/EQkkNvYJmjS5D3//cG+HpVSlVlqSM8aYBBG5vegKEamjia6aeO45O7FpUJB9eFanTrkPuXGjTXBgE1x77ZdcooKCHAoKcggIqEF4+BnUrz+a5s2fJji4obdDU6pKOFVJbgSwEtuFwLk+xAAtPBiX8jZj4JFHbJLr0sV2WnNDgtuwwSY4Pz+b4Nq1K3+ovsgYw4EDXxIfP56oqIto3fp1atceoDMEKFVGpbWuHOH43rziwlGVwo4dcPPNJ+a0+f13W5Irp/XrbYILCLAJrm1bN8Tqg1JTlxAXdy9Hjy4mPPwMoqKGeTskpaqsU7auFJE+IhLueD1KRF4VkaaeD015xcKFcM45NsHddZcdpssNCW7dOpvgAgPht980wZVk9+7XWbXqbLKy4mnbdgo9eqymTp0LvB2WUlWWK2NXvgOcKSJnYmcgmAp8BPT3ZGCqgm3aBI8/DrNm2fd//glnneWWQ69da/uLBwfbElw5BkXxSbm5KRQUHCM4OIaoqOHk5aXQpMn9BATU8HZoSlV5rvSTyzO21+lIYJIxZhIQ4dmwVIXJyYGHHrLNG2fNstlo1y63Jbg1a2wJLjjYluA0wZ1QUJBDYuIbLF3aiq1bbfuusLDWNG/+pCY4pdzElZJcmog8BFwP9BU7vEKgZ8NSFSIuzs77tnatbVzyn//Yqko3Wb0azj/fzpn666/QqpXbDl2lGWM4eHAO8fEPkJm5ndq1z6d58ye9HZZSPsmVJHcVcC1wozFmn+N53MueDUt5XHy8bVSSkWGH6Ro71q2HX7XKJrjwcJvgWrZ06+GrtD173mT79rsJC+tAp04/UKfOUO3MrZSHnDLJORLbJ0BPERkBLDPGzPB8aMpjli6Fq6+2E5uuWuX2jmp//WUTXESETXAttLMJWVk7yc1NISKiC/XrX4+fXygNGozFz8+V/zOVUqfLldaVVwLLgH8AVwJLReQKTwemPOSjj2DIEEhIgJkz3Z7gVq60Ca5mTfsMrronuLy8VOLixrN0aVu2bbsNgMDA2jRq9E9NcEpVAFf+yh4Gehpj9gOISF3gF+ALTwamPOCZZ+zoJc2awYIFbp+wbcUKGDwYIiNtgouNdevhq5SCglz27n2PnTufJDf3kGOkkme8HZZS1Y4rSc6vMME5HMK1VpmqMnn9dZvgLrgAvvnGNnd0o+XL7aFr1bIJrlkztx6+yklO/ojt2++kVq2BtGw5kYgIneJcKW9wJcn9KCLzgM8c768CfvBcSMrtHnwQXnwROna03QTcnOCWLbMJrk4d+wyuuia4o0dXkJu7n6ioYdSvfz3BwTHUrn2BNipRyotcaXhyv4hcBpyLHb9ysjFmtscjU+7x7LM2wQ0bBl98Ydvzu9HSpTbBRUfbBNe0Go6Fk5W1ix07HiY5+WPCw8+kTp0L8fMLpE6dId4OTalqr8QkJyKtgYlAS2AdcJ8xZk9FBabKKScH/vlPmDED+vSBOXPsmFputGSJbcNSt65NcE2auPXwlV5e3lF27XqBxMTXAGja9CGaNn1QS25KVSKlPVubBnwHXI6dieDNsh5cRIaKyBYR2S4iD5ayXU8RyddWm26ya5ed823GDLjoIpg/3+0J7s8/bQmubl37DK66JTiA1NTf2bXreerWvYJevbbQosVzBATU9HZYSiknpVVXRhhj3ne83iIif5XlwI6RUf4DDAYSgeUi8o0xZmMx270IzCvL8VUJFi2yo5ikpsLnn8M//uH2UyxebEtwDRvaElxMjNtPUSkZYzh06Huys3cRE3MbdeoMo2fPjYSH64R4SlVWpSW5EBHpyol55EKd3xtjTpX0egHbjTHxACIyEzv+5cYi290JfAn0LGPsqqiVK232iYqCKVNsKc7NFi2CoUOrX4JLS1tFXNx9HDkynxo1utCo0c2I+GuCU6qSKy3JJQGvOr3f5/TeAANPcewYYLfT+0Sgt/MGIhIDXOo4VolJTkRuAm4CaFodWza44vvv4ZJLbCe1776DTp3cforff4cLL7SJ7ddfoVEjt5+i0snO3kt8/ASSk2cQGBhF69Zv0bDhTdgKCKVUZVfapKnnlfPYxT19N0Xevw6MN8bkl/aw3hgzGZgM0KNHj6LHUM89Z2fxLhwJ2QMJbuFC20CzcWN7ioYN3X6KSik39yAHDsyiSZMHaNbsIQICIr0dklKqDDw5rlAi4NwcoTGwt8g2PYCZjgQXDQwTkTxjzBwPxuVb3noLHn7Ydk5bvdr2xnazBQtg+HDbuOTXX6FBA7efotIoKMhj376pZGRsonXr16lRozNnn51IYGBtb4emlDoNnkxyy4HWItIc2ANcjZ3N4DhjTPPC1yIyHfhOE1wZTJoE99xjS24rVrhlBu+ifvvNJrhmzWwjTV9NcMYYDh+eS1zc/Rw7tpHIyL4UFGTj5xesCU6pKsxjSc4Ykycid2BbTfoD04wxG0TkFsf6dz117mph4UKb4AIDbXt+DyS4X3+1Ca55c5vg6td3+ykqhczMeLZuvZmUlF8IDW1Fx45fER19ifZ3U8oHnDLJif1Lvw5oYYx5yjGfXANjzLJT7WuM+YEiQ4CVlNyMMWNciljZgSJHjLAPyP74w07a5mbz59tTtGhhX9er5/ZTeJ0xBhHBzy+MY8e20arVJBo1ugU/P/f/w6CU8g5XSnJvAwXYFpBPAWlok3/vOXzYdg3w84NffvHIQJG//GJP0aoV/O9/vpfg8vLS2b37ZY4e/ZPOnecRHNyA3r2369Q3SvkgV2YT6G2MuR3IAjDGpAD6r643JCRAhw6QnGxbU7Zt6/ZT/PyzTXCtW/teCc6YfPbufZ9ly1qzc+dTBATUIT8/A0ATnFI+ypW/7FzHqCQGjs8nV+DRqNTfJSTY2UiPHoUvv7SjmrjZTz/ByJHQpo0twUVHu/0UXnPs2HY2bLiUjIz11Kx5Dh07ziYy8ixvh6WU8jBXktwbwGygnog8C1wBPOLRqNTJli+3w4wcPQo//GBnJnWzefNsgmvXzlZX+kqCy88/hr9/GMHBMQQG1qVDh1nUrXu5NipRqppwZaqdT0RkJTAI28H7EmPMJo9Hpqxly2wJLi0NNm6E9u4fRurHH+1gKe3b2wQXFeX2U1S47Oy97NjxGEeO/ErPnhvw9w+lS5f53g5LKVXBXGld2RQ4BnzrvMwYs8uTgSlgzx645hrIzYW//vJIgvvhB7j0Ujuf6i+/2IlPq7L8/Ax2757Irl0vYUwuMTF3YEwuEOLt0JRSXuBKdeX32Odxgv2kaA5sATp6MC6Vnm6rJZOSbGuQrl3dforvv7eP9s44w56iqie4zMwEVq06h5ycJOrWvYLmzZ8nLKyVt8NSSnmRK9WVJw2EKCLdgJs9FpGCggIYNQo2bYKZM+Hcc91+iu++swmuc2eb4GpX4UE9srJ2ExLShJCQZkRHj6R+/VFERvbxdlhKqUrAlS4EJ3FMsaN95Dzpgw/g669h/Hi46iq3H/7bb22C69LFVlFW1QSXkbGBtWuHsXx5R3Jy9iMitGnzjiY4pdRxrjyT+7fTWz+gG3DAYxFVd8uXw6232iLWE0+4/fBff23nUe3a1bao9MB4zh6Xnb2PhITHSUqagr9/BM2aPaazAyiliuXKM7kIp9d52Gd0X3omnGpu3Tq4+GLw97eluRD3NpaYM8cmuO7dbYKLrIJ5IScnmWXL2lBQkElMzJ3Exj5KYKAPNAdVSnlEqUnO0Qm8hjHm/gqKp/pKSbF94Q4csBmoWze3Hn72bLjySujRw3YZqEoJzpgCjh5dQmTkOQQF1Sc29imiooYTFtba26EppSq5Ep/JiUiAMSYfWz2pPCk5GQYMgH377EOyQYPcevgvv7QJrmfPqleCS0mZz8qV3Vm1qi/Hjm0BoEmTezTBKaVcUlpJbhk2wa0WkW+AWUBG4UpjzFcejq162LrVJrikJJg61b52oy++gKuvht69Ye5cqFnTrYf3mIyMTcTF3c/hw98THNyM9u0/JjRUE5tSqmxceSZXBziEnYWgsL+cATTJldexY/YZ3P79dvI2Nye4WbNsX/KzzrIJLiLi1PtUBnl5qaxc2RMRf1q0eJGYmLvw99fO3EqpsistydVztKxcz4nkVsh4NKrq4o47YMsWeO89tye4//4XrrsOzj7bjmpS2RNcfn4mBw9+Rb161xIQEEmHDp9Ss+bZBAXV9XZoSqkqrLQk5w/U4OTkVkiTXHnNmGFbUI4aBTfd5NZDz5xpE1yfPjbB1ajh1sO7lTEFJCd/wo4dD5OdvZvQ0LbUrNmD6OiLvR2aUsoHlJbkkowxT1VYJNXJ22/D7bfb8bTeeceth/70U7j+eujb145qUpkTXErKb8TF3Ut6+l/UqNGddu1mULNmD2+HpZTyIaUlOZ2LxBOefdZOeDpwoG3X78Ys9MknMHo09OtnE1x4uNsO7Xb5+Zls3Hg1fn5BtGv3EfXrX4tImQfgUUqpUpWW5Nzbjl2dSHA9etixtcLC3Hbojz+GG26A/v3toStjgsvJOcDevW/TtOkE/P1D6dx5LmFh7fD3D/V2aEopH1VikjPGHK7IQHze1q3w9NPQrBn8+ScEuNKw1TUzZsCYMXDeeW7PnW6Rn5/Fnj2T2LnzOfLzM6hV6zxq1epHRIT7Z1ZQSiln7vukVSXLz4cbb4TAQPjjD7cmuA8/hLFjbf/xr7+uXAnOmAL2759JfPxDZGfvIirqIlq0eJHwcPfPi6eUUsXRJFcRHnkEFi2ynb0bN3bbYT/4AP7v/+zE4V9/DaGVrtbPsGvXCwQGRtGu3QfUrj3Q2wEppaoZTXKe9u238MILtqg1dqzbDjttGowbZ+dVnTOn8iS4Y8e2snPns7RqNYnAwFp07vwjQUENtFGJUsor9JPHk9auhUsvhQYNbFFL3NNgdcoUW4K74ILKU4LLzT3Etm13s3x5Rw4e/Ir09JUABAc30gSnlPIaLcl5Smambc+fn2+H/XdTc8f337d9x4cOtT0Q3DwbT5kZY9i9+xV27nyG/Pw0Gjb8J82bP0lQUH3vBqaUUmiS8wxjbHPHNWtst4Ezz3TLYSdPhptvhmHD7MwC3k5wACJCauoCIiP70LLly4SHd/B2SEopdZwmOU/48EP4/HMYMQImTHDLId99104YPny4TXDBwW457GlJTV1EfPwE2rWbRmhoSzp0+Fz7uimlKiV9WOJuBw7AXXdBx462PtEN3n7bJrgRI7yb4DIz41i//gpWrTqXzMztZGXtBNAEp5SqtLQk505HjtjmjmlptjTnhv5w//mPnazg4ott4dBbCS4ubjyJia8hEkRs7JM0aXIv/v6VcFgVpZRy4tGSnIgMFZEtIrJdRB4sZv11IrLW8bVYRNzz8MobMjJgyBD7HO6zz6B793If8s03bYIbOdLODVfRCa6gIM/pdTYNGoyhd+9txMY+pglOKVUleCzJiYg/8B/gQqADcI2IFG2VsAPob4zpDDwNTPZUPB6VlQWXXQbLlsHdd9upuMvpjTdsrecll9gSXFBQ+cN0lTGG/fu/YNmydhw5shCAVq1eo23byQQHN6y4QJRSqpw8WZLrBWw3xsQbY3KAmcBI5w2MMYuNMSmOt0sA9w0HUpEeegh++smObPL66+U+3Ouv21x52WUVn+BSU5ewatW5bNz4D/z9wxCxVa7ipj5+SilVkTz5TC4G2O30PhHoXcr2/wfMLW6FiNwE3ATQtGlTd8XnHr//brPSpZfaAZjL6bXX4N//hssvt7WegYHlD9FVW7bcQlLSewQFNaBt2yk0aDAGWyBXSqmqyZNJzuUZxUXkPGySO7e49caYyTiqMnv06FF5ZiVPSoKLLoKYGNtLu5xeeQXuuw+uuMJOfloRCS439wgBATUR8aNGjc40a/Y4TZrcR0BAJZ5tVSmlXOTJJJcINHF63xjYW3QjEekMTAEuNMYc8mA87nfvvZCaausUo6LKdaiJE+H+++HKK+3ccJ5OcAUFOezd+w4JCU/RqtXrNGhwPTExt3n2pEopVcE8meSWA61FpDmwB7gauNZ5AxFpCnwFXG+M2erBWNzvp59sfeKVV9pBJMvhpZdg/HjbXuWjj9w6E8/fGGM4eHA28fHjyczcTu3a51OjRtVt1KqUUqXx2MepMSZPRO4A5gH+wDRjzAYRucWx/l3gMSAKeNvRsCHPGNPDUzG5zdKltuNahw7w3nvlOtQLL9h2K9dcYyc/9WSCA9i8eSzJyR8SFtaBTp1+oE6dodqoRCnls8SYyvOIyxU9evQwK1as8F4AS5dCv34QGWm7DMTGnvahnn/ejvp17bVu6zterMzMBIKC6uLvH86hQ3PJzt5NgwY34uenYwEopXyDiKwsrpCkw3qVxc6dNsEFBsL8+eVKcM8+axPcqFGeK8Hl5h4hLu4Bli1ry+7drwAQFXUhjRrdpAlOKVUt6CddWdxyC+TlwYIFcMYZp32Yp5+Gxx6D66+3s3v7u7mVfkFBLnv3vktCwpPk5R2mfv3RNGhwo3tPopRSVYAmOVf99pudF+5f/4Kzzjrtwzz1FDz+uJ1qbto09yc4gK1bb2LfvunUqjWQli0nEhHR1f0nUUqpKkCTnCsKCmzzx8hIePLJ0z7ME0/Y3ceMsbN7uzPBHT26guDghgQHx9C48b+Ijr6cqKjh2qhEKVWt6TM5V8yYYRuZPP88RESUeXdjbOntySdh7FiYOtV9CS4raycbN47ir796snPn8wDUqNGZ6OgRmuCUUtWeluROZcsW+yyuSxe46aYy716Y4J5+Gv7v/+zs3n5u+NciLy+VXbteYPfu1xARmjadQNOm48t/YKWU8iGa5EpjjE1s/v7wxRdlLn4ZA48+altSjhtnu9S5I8EB7NjxCHv2vEX9+tfTvPkzhIRUsjE9lVKqEtAkV5q5c2HhQnjuOWjZsky7GgMPP2xrOG+6Cd55p3wJzhjDoUPfERzchIiILjRt+hANGowhIqL889YppZSv0mdypXniCahb145RWQbG2D5wzz8PN99c/gSXlvYXa9YMYv36i0lMfB2A4OBGmuCUUuoUtCRXklmzYPly+yCtDBO6GQMPPmjHo7z1VnjrrdNPcFlZu9mx42GSkz8iMDCa1q3fomHDsj8XVEqp6kqTXHEyMuy03LVr2+kBXGQMPPCA3eW222yCK08Dx337prN//+c0aTKeZs0eIiAg8vQPppRS1ZAmueK8+CLs2wdz5kCtWi7tYoydKueVV+COO+CNN8qe4AoK8ti3byrBwY2JihpOkyb/pn796wkNjS3rFSillEKfyf2dMbZfXL9+MHKky7vce69NcHfdVfYEZxuV/MCKFWeydest7N//OQD+/uGa4JRSqhw0yRX1zTd2IOZRo1za3Bg70tdrr8Hdd8Prr5ctwaWnr2XNmsGsWzccY3Lo2PEr2rWbflqhK6WUOplWVzpLSbGtRZo1cynJGQP33GNLbvfcA6++WvYqyvT0NaSnr6JVq0k0anQLfn6uN3JRSilVOk1yzt56C5KS4LvvIDS01E2NsVWTb70F//63bWziSoLLy0tn9+6XCAqqT0zM7dSvfx1RURcRGFjLPdeglFLqOE1yhY4csQ1Ozj4bhg8vdVNj4M474T//gfvus90FTpXgjMknKWkaO3Y8Sm5uMg0b3gyAiJ8mOKWU8hBNcoVefdV2HXj55VI3KyiwrSffece2pnzxxVMnuNTURWzdegsZGeupWfMczjhjDpGRpz9dj1JKKddokgM4fNgmt379oE+fEjcrKIDbb4d337Uz7zz/fOkJzhiDiGBMAfn5mXToMIu6dS/X2QGUUqqCaJIDePttyMqypbkSFBTYNimTJ8NDD9lBl0vKVdnZe9mx4zH8/cNp3XoStWr1pVevzfj56e1WSqmKpF0IjLEzmPbvD92LHwuyoMDOtjN5sh2TsqQEl5+fQULCkyxd2prk5Bn4+QVjjAHQBKeUUl6gn7zz59t+cY8+WuzqggI7i8DUqfDII/DUU8UnuJSUX9m06TpycpKoW/cKWrR4gdDQss1coJRSyr00yb34IjRuDFdf/bdVBQXwz3/CtGnw2GN2UoKiCS4/PxN//1BCQpoRGtqajh2/IDLynIqJXSmlVKmqd5LbsQN+/tmW4sLDT1qVn28nOp0+3c7s/cQTJ++anr6e+Pj7AT86d/6e0NAWdO26oKIiV0op5YLq/UxuyhRbNLvxxpMW5+fbGXamT7fJzTnBZWfvY8uWm1ix4kyOHl1C7doDMaagIqNWSinloupbkjMG/vtf220gNvb44vx8GDsWPvrIPn9zflSXkvI/1q0biTHZxMTcSWzsowQGRlV87EoppVxSfZPc6tUQF2f7Azjk58OYMfDxx/D007ahiTEF5OQkERwcQ40a3alb93KaNXuEsLDWXgtdKaWUa6pvkps3z34fPBiAvDy44Qb49FPbRWDCBEhJmU9c3L0Yk0ePHqsJDKxF+/YfejFopZRSZVF9n8ktWAA1a0LTpuTlwejRNsE9/zzcffcm1q4dwZo1g8jNTaFp04cBHaVEKaWqmupZksvIgB9/hLvvJi8Prr8eZs6EF16Am29eyPLlA/H3r0GLFi8RE3Mn/v4h3o5YKaXUaaieSW7lSgDyuvbkuutgzpxM3nprM7ff3pWCgnOIjX2MRo1uIygo2suBKqWUKg+PVleKyFAR2SIi20XkwWLWi4i84Vi/VkS6eTKe45YuJZcArv3yUlJSPuLbb9vQpctQ8vMz8fMLIDb2MU1wSinlAzxWkhMRf+A/wGAgEVguIt8YYzY6bXYh0Nrx1Rt4x/Hdo3KXreK+Xk9x/oi+tGnzFxERPWjZciL+/qVPlKqUUqpq8WR1ZS9guzEmHkBEZgIjAeckNxKYYewoxktEpJaINDTGJHkqqPx8uGtnf6566RZycprQvv3H1Kt3DSLVtw2OUkr5Kk9+sscAu53eJzqWlXUbROQmEVkhIisOHDhQrqD8/KC2fxt2fv8QgwZtoX796zTBKaWUj/JkSa64NvfmNLbBGDMZmAzQo0ePv60vU1ACz/15HnBeeQ6jlFKqCvBkESYRaOL0vjGw9zS2UUoppU6LJ5PccqC1iDQXkSDgauCbItt8A4x2tLI8C0j15PM4pZRS1YvHqiuNMXkicgcwD/AHphljNojILY717wI/AMOA7cAxYKyn4lFKKVX9eLQzuDHmB2wic172rtNrA9zuyRiUUkpVX9qsUCmllM/SJKeUUspnaZJTSinlszTJKaWU8lli235UHSJyANjphkNFAwfdcBxfpPemZHpvSqb3pmR6b0rmrnvTzBhTt+jCKpfk3EVEVhhjeng7jspI703J9N6UTO9NyfTelMzT90arK5VSSvksTXJKKaV8VnVOcpO9HUAlpvemZHpvSqb3pmR6b0rm0XtTbZ/JKaWU8n3VuSSnlFLKx2mSU0op5bN8PsmJyFAR2SIi20XkwWLWi4i84Vi/VkS6eSNOb3Dh3lznuCdrRWSxiJzpjTi94VT3xmm7niKSLyJXVGR83uTKvRGRASKyWkQ2iMiCio7RW1z4m4oUkW9FZI3j3lSLmVdEZJqI7BeR9SWs99znsDHGZ7+wU/zEAS2AIGAN0KHINsOAudhZys8Clno77kp0b84BajteX6j3ptjt5mNn2rjC23FXlnsD1AI2Ak0d7+t5O+5KdG8mAC86XtcFDgNB3o69Au5NP6AbsL6E9R77HPb1klwvYLsxJt4YkwPMBEYW2WYkMMNYS4BaItKwogP1glPeG2PMYmNMiuPtEuzM7dWBK783AHcCXwL7KzI4L3Pl3lwLfGWM2QVgjKku98eVe2OACBERoAY2yeVVbJgVzxizEHutJfHY57CvJ7kYYLfT+0THsrJu44vKet3/h/1Pqzo45b0RkRjgUuBdqhdXfm/aALVF5DcRWSkioyssOu9y5d68BbQH9gLrgLuNMQUVE16l5rHPYY9OmloJSDHLivaZcGUbX+TydYvIedgkd65HI6o8XLk3rwPjjTH59p/yasOVexMAdAcGAaHAnyKyxBiz1dPBeZkr92YIsBoYCLQEfhaR340xRz0cW2Xnsc9hX09yiUATp/eNsf9BlXUbX+TSdYtIZ2AKcKEx5lAFxeZtrtybHsBMR4KLBoaJSJ4xZk6FROg9rv5NHTTGZAAZIrIQOBPw9STnyr0ZC7xg7IOo7SKyA2gHLKuYECstj30O+3p15XKgtYg0F5Eg4GrgmyLbfAOMdrTuOQtINcYkVXSgXnDKeyMiTYGvgOurwX/hzk55b4wxzY0xscaYWOAL4LZqkODAtb+pr4G+IhIgImFAb2BTBcfpDa7cm13YEi4iUh9oC8RXaJSVk8c+h326JGeMyRORO4B52JZP04wxG0TkFsf6d7Et44YB24Fj2P+0fJ6L9+YxIAp421FiyTPVYCR1F+9NteTKvTHGbBKRH4G1QAEwxRhTbNNxX+Li783TwHQRWYetohtvjPH5KXhE5DNgABAtIonA40AgeP5zWIf1Ukop5bN8vbpSKaVUNaZJTimllM/SJKeUUspnaZJTSinlszTJKaWU8lma5JQCHDMJrHb6ii1l23Q3nG+6iOxwnOsvETn7NI4xRUQ6OF5PKLJucXljdByn8L6sd4yeX+sU23cRkWHuOLdS7qBdCJTCJi5jTA13b1vKMaYD3xljvhCRC4CJxpjO5TheuWM61XFF5ENgqzHm2VK2HwP0MMbc4e5YlDodWpJTqhgiUkNE/ucoZa0Tkb/NQiAiDUVkoVNJp69j+QUi8qdj31kicqrksxBo5dj3345jrReRexzLwkXke8ccZOtF5CrH8t9EpIeIvACEOuL4xLEu3fH9v84lK0cJ8nIR8ReRl0Vkudj5u2524bb8iWPQXBHpJXaOwVWO720do3w8BVzliOUqR+zTHOdZVdx9VMqjvD3PkH7pV2X4AvKxA+euBmZjRwOq6VgXjR2JobDmI93x/V7gYcdrfyDCse1CINyxfDzwWDHnm45jDjrgH8BS7KDG64Bw7DQsG4CuwOXA+077Rjq+/4YtNR2PyWmbwhgvBT50vA7CjvQeCtwEPOJYHgysAJoXE2e60/XNAoY63tcEAhyvzwe+dLweA7zltP9zwCjH61rY8SvDvf3z1q/q8+XTw3opVQaZxpguhW9EJBB4TkT6YYemigHqA/uc9lkOTHNsO8cYs1pE+gMdgEWOodCCsCWg4rwsIo8AB7CzPAwCZhs7sDEi8hXQF/gRmCgiL2KrOH8vw3XNBd4QkWBgKLDQGJPpqCLtLCdmNI8EWgM7iuwfKiKrgVhgJfCz0/Yfikhr7GjxgSWc/wLgYhG5z/E+BGhK9RjLUlUCmuSUKt512JmbuxtjckUkAfsBfZwxZqEjCQ4HPhKRl4EU4GdjzDUunON+Y8wXhW9E5PziNjLGbBWR7tix/Z4XkZ+MMU+5chHGmCwR+Q07xctVwGeFpwPuNMbMO8UhMo0xXUQkEvgOuB14AzsG46/GmEsdjXR+K2F/AS43xmxxJV6l3E2fySlVvEhgvyPBnQc0K7qBiDRzbPM+MBXohp1BvY+IFD5jCxORNi6ecyFwiWOfcGxV4+8i0gg4Zoz5GJjoOE9RuY4SZXFmYge87YsdPBjH91sL9xGRNo5zFssYkwrcBdzn2CcS2ONYPcZp0zRstW2hecCd4ijWikjXks6hlCdoklOqeJ8APURkBbZUt7mYbQYAq0VkFfa52SRjzAHsh/5nIrIWm/TauXJCY8xf2Gd1y7DP6KYYY1YBnYBljmrDh4Fnitl9MrC2sOFJET8B/YBfjDE5jmVTgI3AXyKyHniPU9TsOGJZg51C5iVsqXIR9nldoV+BDoUNT7AlvkBHbOsd75WqMNqFQCmllM/SkpxSSimfpUlOKaWUz9Ikp5RSymdpklNKKeWzNMkppZTyWZrklFJK+SxNckoppXzW/wNWfOZtYsgzvwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(f'Метрики лучшей модели на тестовой выборке:')\n",
    "print_metrics(\n",
    "    final_pipeline, X_test, y_test, best_threshold, show_roc_curve=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aee5076d-756e-42fa-a4e5-e42f604131a2",
   "metadata": {
    "id": "rmqEm1P2nT94",
    "tags": []
   },
   "source": [
    "### Обучение на всех данных\n",
    "\n",
    "Для анализа обработки данных и важности признаков разобъём финальный конвейер на препроцессор и модель и обучим их на всех данных. А перед сохранением модели объединим обратно."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 258,
   "id": "80eb6add-d351-492f-b820-722a1b6138e2",
   "metadata": {
    "id": "40ri9TC6MbLJ"
   },
   "outputs": [],
   "source": [
    "final_model = final_pipeline.named_steps['model']\n",
    "final_preprocessor = final_pipeline.set_params(model=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 259,
   "id": "3661db53-0665-4131-9e9b-091601da5446",
   "metadata": {
    "id": "Zb-xUv8GzPtR"
   },
   "outputs": [],
   "source": [
    "sessions_preprocessed = final_preprocessor.fit_transform(sessions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 260,
   "id": "c88b35d4-9cfd-46b8-a5e6-bcff6f745140",
   "metadata": {
    "id": "-kdJDViDVV6B"
   },
   "outputs": [],
   "source": [
    "final_model.fit(sessions_preprocessed, target);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2453418-a984-4876-b08b-d51e4a21e630",
   "metadata": {
    "id": "nAkkyr8O4wjG",
    "tags": []
   },
   "source": [
    "### Анализ обработки данных\n",
    "\n",
    "В итоге всех преобразований получается 62 признака, при том, что ещё 16 признаков были удалены из-за корреляций и т.п..\n",
    "\n",
    "Датасет имеет 175 034 дубликатов, но как показали эксперименты, удаление дубликатов из тренировочной выборки ведёт к небольшому ухудшению метрик. \n",
    "\n",
    "Признаков, коррелируемых с целевой переменной, нет."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 540,
   "id": "b5b13f86-110d-40ae-a553-d67b78b97922",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 319
    },
    "id": "O08IthBdYwd2",
    "outputId": "28a29ed1-06c0-441b-e5d0-c254e372a2f6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1860042, 58)"
      ]
     },
     "execution_count": 540,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sessions_preprocessed.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 262,
   "id": "6d544cd4-40f7-429f-a84f-97c427d1a7bc",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5SWe0lAcbtwE",
    "outputId": "766f231b-d80e-4239-ccd6-5dbb824f3cc7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Index: 1860042 entries, 9055434745589932991.1637753792.1637753792 to 9055430416266113553.1640968742.1640968742\n",
      "Data columns (total 58 columns):\n",
      " #   Column                                                     Dtype  \n",
      "---  ------                                                     -----  \n",
      " 0   visit_number                                               float64\n",
      " 1   visit_date_weekday                                         float64\n",
      " 2   visit_date_weekend                                         float64\n",
      " 3   visit_date_day                                             float64\n",
      " 4   visit_time_hour                                            float64\n",
      " 5   visit_time_minute                                          float64\n",
      " 6   visit_time_night                                           float64\n",
      " 7   utm_medium_added_is_organic                                float64\n",
      " 8   utm_source_added_is_social                                 float64\n",
      " 9   device_screen_resolution_width                             float64\n",
      " 10  device_screen_resolution_height                            float64\n",
      " 11  device_screen_resolution_ratio                             float64\n",
      " 12  geo_city_added_is_moscow_region                            float64\n",
      " 13  geo_city_added_is_big                                      float64\n",
      " 14  geo_city_is_big_or_in_moscow_region                        float64\n",
      " 15  geo_city_added_distance_from_moscow                        float64\n",
      " 16  utm_source_ZpYIoDJMcFzVoPFsHGJL                            float64\n",
      " 17  utm_source_MvfHsxITijuriZxsqZqt                            float64\n",
      " 18  utm_source_kjsLglQLzykiRbcDiGcD                            float64\n",
      " 19  utm_source_rare                                            float64\n",
      " 20  utm_source_fDLlAcSmythWSCVMvqvL                            float64\n",
      " 21  utm_source_BHcvLfOaCWvWTykYqHVe                            float64\n",
      " 22  utm_source_bByPQxmDaMXgpHeypKSM                            float64\n",
      " 23  utm_medium_cpm                                             float64\n",
      " 24  utm_medium_cpc                                             float64\n",
      " 25  utm_medium_rare                                            float64\n",
      " 26  utm_medium_referral                                        float64\n",
      " 27  utm_campaign_LEoPHuyFvzoNfnzGgfcd                          float64\n",
      " 28  utm_campaign_(nan)                                         float64\n",
      " 29  utm_campaign_gecBYcKZCPMcVYdSSzKP                          float64\n",
      " 30  utm_campaign_rare                                          float64\n",
      " 31  utm_adcontent_vCIpmpaGBnIQhyYNkXqp                         float64\n",
      " 32  utm_adcontent_xhoenQgDQsgfEPYNPwKO                         float64\n",
      " 33  utm_adcontent_rare                                         float64\n",
      " 34  utm_adcontent_(nan)                                        float64\n",
      " 35  utm_adcontent_JNHcPlZPxEMWDnRiyoBf                         float64\n",
      " 36  utm_keyword_puhZPIYqKXeFPaUviSjo                           float64\n",
      " 37  utm_keyword_rare                                           float64\n",
      " 38  device_category_mobile                                     float64\n",
      " 39  device_os_Android                                          float64\n",
      " 40  device_os_iOS                                              float64\n",
      " 41  device_os_Windows                                          float64\n",
      " 42  device_brand_Huawei                                        float64\n",
      " 43  device_brand_Samsung                                       float64\n",
      " 44  device_brand_Xiaomi                                        float64\n",
      " 45  device_brand_Apple                                         float64\n",
      " 46  device_brand_rare                                          float64\n",
      " 47  device_browser_Chrome                                      float64\n",
      " 48  device_browser_rare                                        float64\n",
      " 49  device_browser_Safari                                      float64\n",
      " 50  device_browser_YaBrowser                                   float64\n",
      " 51  geo_city_rare                                              float64\n",
      " 52  geo_city_Moscow                                            float64\n",
      " 53  geo_city_Saint Petersburg                                  float64\n",
      " 54  geo_city_added_distance_from_moscow_category_1000-3000 km  float64\n",
      " 55  geo_city_added_distance_from_moscow_category_500-1000 km   float64\n",
      " 56  geo_city_added_distance_from_moscow_category_100-500 km    float64\n",
      " 57  geo_city_added_distance_from_moscow_category_no distance   float64\n",
      "dtypes: float64(58)\n",
      "memory usage: 837.3+ MB\n"
     ]
    }
   ],
   "source": [
    "sessions_preprocessed.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 263,
   "id": "cb1c9191-a0b4-4aa6-b8b1-9066c8a01932",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ENj_Uf0YzbOl",
    "outputId": "f1eb2712-0b39-4b53-be2a-e71e4890b823"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Количество дубликатов: 176667\n"
     ]
    }
   ],
   "source": [
    "print('Количество дубликатов:', sessions_preprocessed.duplicated().sum())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 264,
   "id": "1f90dfb3-dfb9-4833-9128-f109f96c0e79",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "f2yUUnezly5y",
    "outputId": "3d868753-ff6c-4bff-a229-6903cb26ce07"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Корреляция с целевой переменной:\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "target                                1.000000\n",
       "utm_medium_added_is_organic           0.050813\n",
       "utm_medium_referral                   0.049207\n",
       "utm_adcontent_JNHcPlZPxEMWDnRiyoBf    0.044788\n",
       "utm_source_bByPQxmDaMXgpHeypKSM       0.040561\n",
       "Name: target, dtype: float64"
      ]
     },
     "execution_count": 264,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('Корреляция с целевой переменной:')\n",
    "correlation = pd.concat([sessions_preprocessed, target], axis=1).corr()\n",
    "correlation['target'].sort_values(ascending=False, key=abs).head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "607c5685-2046-492e-a12e-c33202598cb7",
   "metadata": {
    "id": "pCDZ_HVmgDaJ",
    "tags": []
   },
   "source": [
    "### Важность признаков\n",
    "\n",
    "Самыми важными признаками после преобразования оказались численные переменные: день (месяца и недели), час, минута (скорее всего именно нулевая минута часа) и номер посещения, размеры экрана и расстояние до Москвы.\n",
    "\n",
    "После этих признаков важность остальных сильно падает. Возможно такова особенность работы с категориальными признаками (особенно с большим количеством неравномерно распределенных уникальных значений).\n",
    "\n",
    "Самыми важными исходными признаками окзаались: размеры экрана, дата, время и номер посещения, город пользователя, а также признаки с дополнительными данными (как индикатор органического трафика). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 265,
   "id": "b5a531b7-3590-4c58-8550-a5b41b81fcc5",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YL5n0vtek5-h",
    "outputId": "df021af6-99c0-4b38-87a9-76d4b78be43a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Признаки, удалённые во время feature selection\n",
      "\n",
      "constant_dropper:\n",
      "\tvisit_date_added_holiday\n",
      "\tdevice_screen_resolution_area\n",
      "\tdevice_category_tablet\n",
      "\tdevice_os_rare\n",
      "\tdevice_model_(nan)\n",
      "\tgeo_country_Russia\n",
      "\tgeo_city_added_distance_from_moscow_category_>= 3000 km\n",
      "duplicated_dropper:\n",
      "\tgeo_city_added_distance_from_moscow_category_moscow\n",
      "\tutm_medium_(none)\n",
      "correlated_dropper:\n",
      "\tutm_keyword_(nan)\n",
      "\tutm_campaign_FTjNLDyTrXaWYgZymFkV\n",
      "\tdevice_category_desktop\n",
      "\tgeo_city_added_distance_from_moscow_category_< 100 km\n",
      "\tdevice_os_(nan)\n",
      "\tdevice_brand_(nan)\n",
      "\tutm_campaign_LTuZkdKfxRGVceoWkVyg\n",
      "\tutm_medium_banner\n",
      "\tdevice_screen_resolution_ratio_greater_1\n",
      "Всего удалено: 18\n"
     ]
    }
   ],
   "source": [
    "print('Признаки, удалённые во время feature selection\\n')\n",
    "all = 0\n",
    "\n",
    "for step in ('constant_dropper', 'duplicated_dropper', 'correlated_dropper'):\n",
    "    print(step + ':')\n",
    "    for column in final_preprocessor.named_steps[step].features_to_drop_:\n",
    "        print(f'\\t{column}')\n",
    "        all += 1\n",
    "\n",
    "print(f'Всего удалено: {all}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 266,
   "id": "beb07ea2-ff9e-4855-b1f6-5aba3e252efe",
   "metadata": {
    "id": "zUGLmCB4gvWu"
   },
   "outputs": [],
   "source": [
    "feature_importance = pd.Series(\n",
    "    data=final_model.feature_importances_, \n",
    "    index=sessions_preprocessed.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 267,
   "id": "c200abc8-f2ed-4e3e-91bb-f96089ecc10b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 608
    },
    "id": "virCtTBOgvSl",
    "outputId": "ce6e6a81-84b8-49d3-ed51-534ce311ad13"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABF4AAAJPCAYAAABFHhD7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAACi5UlEQVR4nOzdefzm9bz/8cezReu0UDqyNEQS1VSTdoVsx1KRQicnopMlRAghncORjhOhOolGhLQ5FSrSRuvMNM1USrTI0S+hfV9evz8+729dXX23meb7/dbM4367ze26rvfnvbzen+v6urlevd/vK1WFJEmSJEmS5r9FJjoASZIkSZKkBZWJF0mSJEmSpDFi4kWSJEmSJGmMmHiRJEmSJEkaIyZeJEmSJEmSxoiJF0mSJOlJIp3FJjoOSdLomXiRJEmSnsCSvDzJKUn+BNwGbD/RMUmSRs/EiyRJC6kk70gyPckdSW5I8oskm090XJIe0f4mjwa+CTy3qiZV1Y8nOCxJ0lww8SJJ0kIoyUeBrwFfAlYBngMcDGwzgWFJeqz/APaoqpOr6sGJDkaSNPdMvEiStJBJsjywH/CBqjq+qu6sqvur6qSq+nir89Ik5yW5pa2G+WaSp/T0UUku6Xm9aJK/JPlzT9m1SbYeZPwfJNm35/UbksxqY52bZJ1hYq8kd7ZVOn9M8taea3u3stuTXJ5ku55rb07y+yQrt9fTkvxHe/78JH9Kskl7vW+SH/S0XayNO7m/bV9s70lyZs/rNZP8Msk/klyZZIdh5nVmknvavO5IcneSa/vu5afavG5OckSSJdu1rfru+w4t3ve0129N8ufW71VJtu/rd+ue1/1z+HqS65PclmRGki16rj18n5IsmeSsJPu315NbDIu11y9trx9z39r1XZI82DP/O5I8lGSrdn2JJF9rn7G/tOdL9LTfpn2Gbmufgdf2XJuW5L6e+9p7r8blPWrXh/ycJ3lR6/+WJJcleVPP0C8FXp3k//XPfeC9T/LpJH9rMezU0+/rk1zc7sv1efTf3eptzrcnubH3vUmyfJIjk9yU5Lok+yRZZJD36rYkv07yzHZtxSQnt3Y3t+fP6ruH7+l5vfUg93Dr9nzZFtdveq5v2xPzHen5u5SkJzITL5IkLXw2AZYEThimzoPAnsBKrf4rgff31XlKkg3b89cDt8xtIEnWB74L/BvwNOB/gBN7v1QPYt2qWpYueXRIT/kfgS2A5YEvAD9I8gyAqjoe+BZwUpKlesZfCTgZ2LOqzpvb+IeZ1zLAL4EfAk8H3g4cnOTFwzT7YFUt2+b2xkGu7wS8BlgdWAPYZ5BxFwf+Hbihp/g8YL3W74fo7vFoXQRMAZ7a5nJMbzKhjbkY8BPg91X1ySH6+QrwfyOMdd7A/Fusf+m59hlg4xbLunTJiH3a+C8FjgQ+DqwAvAy4tqftIsD+rc/X9cQ9bu/RcJ/z9p6dBJzW4tgDOCrJC1ufSwEvBtbpn3vzT3R/p88E/hU4rKftncA72315PfC+JNu2a38F/hlYju7evifJ2u3aN+j+jp4HbNn6eFfPmOe1e/B04F66/62A7l4fAaxGt4rubrotUvPi48D9fWWHAv9ZVZPanCTpScHEiyRJC5+nAX+rqgeGqlBVM6rq/Kp6oKqupfuiuGVfte8AA//1+j3t9dx6L/A/VXVBVT1YVd+j+yK38SjaLgb8vSfmY6rqL1X1UFUdDVxF9yV14PrXgSvpvmgvQpd8+l/g2Ko6bh5iH84bgGur6oh2D2cCx/H4DkX9ZlVdX1X/AL5Ilyjo92/ABcDvBwqq6s9VdVN7GWDmaAesqh9U1d/bHL4KLAG8sKdK6N73ZYHdB+sjyRvo7vevRjvuIHYC9quqv7a5fAHYuV3bFfhuVf2yvff/V1VX9LR9CnDfIH2O53s03Od8Y7r79+Wquq+qfk2XDOx9f4ea+4DPVtW9VXUW8DNgB4CqOrOq5rT7Mhv4Ee3vuKpur6o/VlXRvY83An9JsiiwI/CpVuda4KuDjAnd+7oI7e+wfVaOq6q7qur2dg/6/3djRElWoXtf/3uQy4slydz2KUkTycSLJEkLn78DK2WYn6RNskbbJvD/ktxGdxbMSn3VTga2SvJ84BnAjEG6+mnbPnF9kq8M8oVpNeBjrc4tSW4Bng2sOkz8M5PcQbeCZb+emN/Zs5XjFuAlvTG3lRqb0a0MeCvwAbrkzSsHiWuHnn7+NkgMe7XrNyY5PsnTBpnXRn3z2oludcK8ur7n+XX03aMkk4BPAJ/tb5juIOU7gWPav14/7YnxoL52H0vyuyS3tuvL8+jPwXbAi+hWZKw8SMyLAP/Z4no8VqWb84De+T+bbrXTUJ4K3DxI+Xi+R8N9zlcFrq+qh/raPrM9v4+h5w5wc1XdOdj1JBslOaNt/bmVLjnW+zfxnFb+B+A3wO3t+lMGGfOZPa83bnO4BXguMK31t3SS/2nbk24DzgZWaMmcAQf13IOfMrh96Vbd/KOvfBdgb7qVNIP9XUrSE5KJF0mSFj7nAfcA2w5T5xDgCuAFVbUc8Gm6/yre6wG67UrH0r54DWLbqloB2BT4F7ptGL2uB75YVSv0/Fu6qn40TGzrt20O69FtDXlOktWAbwMfBJ7Wxry0L+bP0s19S7qVL+e0uO6hWynS6ycD8fDYhBPAf7VrzwOWptsW0T+vs/rmtWxVvW+YeY3k2T3Pn8Ojt+LQYvhJVV3XV05V/bCqlgG2Ar6eZK2ey9v2zPVDA4XpznP5JN3qiRXb9Vt59D29GngF3aqXgweJeRfgyqo6fxTzG85f6JIXA3rnfz3d1p6hrEHPCqAe4/keDfc5/wvw7IEzVHraDmzN+hNDzx1gxbZtarDrPwROBJ5dVcvTbdV5+P2rqj+18mfS/V3sSpfQuH+QMXu3ip3fPg9LAj/gkb//j9GtiNqo/e/Gy1p572fmQz2ft215rDXo/nfioEGu/ZLuM7gzg/9dStITkokXSZIWMlV1K/A54FvtsMqlkyye5HVJvtKqTQJuA+5IsiYw1JfRw4DfAUeNMOztdIma/v/v8W1g9/Zf5pNkmXQHgk4axVQepPsv8ysAywAF3ASQ5F10K15or9cC3g18tKruBmYBF1b3KzG7A/smmZeVDvcAdw0yr5OBNZLs3O7t4kk2TPKieRhjwAeSPCvJU+kSYUf3XJtEdwbHF/sbJXlhz7ksS9B9Cb57FONNonvPbqLb3vE5uvNAes2qqjvotr+smWTHvuufAT41irFG8iNgnyQrt3N5Pkf3hR+6pM+7krwyySJJnpnu0NzFkuxOt43nN4P0OZ7v0XCf8wvozmL5RIthK7rzYwZ+Mnq4uQ/4QpKntGTZG3hkVdMk4B9VdU+6s3DeMdCgJ07o/o4WBe5ufxM/Ab6YZFJLan50kDGh+5t7kEdWO02i+2zd0vr+/Kjv3CP2odtaNdhn9GPAX6qqf9WWJD2hmXiRJGkhVFX/Tfdlah+6L9bX060W+Wmrshfdl7Tb6b40Hv3YXqCqrq6qt1fVLUMM9aN0vyJzaev7lL720+nOv/gm3XaQP9CtkhjOJW2r0ZnAl6pqdlVdTncOxXl0Z1WsDfwWoG0j+h9gn56zTnpjuJJuJcDXRhi314favP5E91/9/6uvz9uBVwNvo1t98P+A/ekSH/Pqh3QHsF7d/vX+QtBywEFVNdiWmrcC1ye5HTgceH9VXTOK8U4FfkG3WuQ6uiTT9YNVrKp76RI/X2vJgQEnV9VVoxhrJP8BTAdmA3Pozqn5jzb2hW3sA+lWQ5xFt1pj11a+TVXdNUjM4/YeDfc5r6r7gDfRHfz7N7qVQ+/sOafmS22+c/rn3vy/1udf6BKgu/e0fT+wX3vvP0eXUBmwNnBxu3Yu8HPg++3aHnTJoKvpklY/pDsceMAm7W/wVuDNdP/bAd3f0FJtHufT9/c+Sn+nOyz5UZKsTpd46T/kW5Ke8NKdpyVJkqQnqnQ/ufueqno8B9RqDE3Ee9RWx/ygqp41QlVJ0gRyxYskSZIkSdIYMfEiSZIkSZI0RtxqJEmSJEmSNEZc8SJJkiRJkjRGTLxIkiRJkiSNkcUmOgBpYbHSSivV5MmTJzoMSZIkSdIYmDFjxt+qauX+chMv0jiZPHky06dPn+gwJEmSJEljIMl1g5WbeJHGyQM3/YObDvnBRIchSZIkSU8aK7/vXyY6hMfNM14kSZIkSZLGiIkXSZIkSZKkMWLiRZIkSZIkaYyYeJEkSZIkSRojJl4kSZIkSZLGyJMm8ZLk3PY4Ock7xnisM5NMHaR8lyTfnMu+rk2y0ijr7ptkr/Z8vyRbD1N32yRrzU0s81uSlZNckOTiJFtMZCySJEmSJD0RPWkSL1W1aXs6GRjTxMsTQVV9rqp+NUyVbYEJTbwArwSuqKr1quqc3gtJFp2gmCRJkiRJesKY58RLks8muSLJL5P8KMleSVZPckqSGUnOSbJmq7taktOTzG6Pzxmm31WSnJDkkvZv01Z+R6vyZWCLJLOS7NnGmdLT/rdJ1hmi75cmObet0Dg3yQtb+VJJftziOxpYqqfNu5L8PslZwGY95SsnOS7JRe3fZq38aUlOa2P8D5AR7uNnklyZ5FfAC3vKpyXZvj3/cpLLW3z/1e7Jm4AD2n1YPcl7WxyXtLiW7unnoDbfqwf6bNc+kWROa/PlVjboezhI3FOArwD/3GJYKskdbaXOBcAmST6a5NL27yOt3eT2uTm8lR+VZOv2vl2V5KXD3Kt9k3yv3d9rk7w5yVfaHE5Jsnir98p2/+ck+W6SJQa7j61sqM/bYLF/IsmH2vMDk/y6Z7wfDPc+S5IkSZIWTvOUeEm3DectwHrAm4GBbTmHAXtU1QbAXsDBrfybwJFVtQ5wFHDQMN0fBJxVVesC6wOX9V3fGzinqqZU1YHA4cAuLa41gCWqavYQfV8BvKyq1gM+B3yplb8PuKvF90Vgg9bfM4Av0CVcXsWjV5h8HTiwqjZs9+LwVv554DdtjBOB4ZJMGwBv45H7uOEgdZ4KbAe8uMX3H1V1buv74+0+/BE4vqo2bPftd8CuPd08A9gceANd4ookr6NbNbNRa/OVVneo9/BRqmoW3T08usVwN7AMcGlVbQTcDbwL2AjYGHhvkvVa8+e3+7cOsCbdCqbN23ifHup+NasDrwe2AX4AnFFVa7fxXp9kSWAasGMrXwx432D3sfX3mM9be18Gi/1sYGBL1VRg2Zbs2Rx41IqfAUl2SzI9yfS/33HbCFOTJEmSJC1oFpvHdpsD/9u+bJPkJGBJYFPgmOThRR5LtMdN6BILAN/nkS/5g3kF8E6AqnoQuHWEWI4BPpvk48C76b50D2V54HtJXgAUsHgrfxktGVRVs5MMJG42As6sqpvaPI8G1mjXtgbW6pnrckkmtb7e3Pr6WZKbh4lnC+CEqrqr9X/iIHVuA+4BDk/yM+DkIfp6SZL/AFYAlgVO7bn206p6CLg8ySo98R8xMHZV/SPJsgz9Ho7Gg8Bx7fnmbW53trkd3+Z7InBNVc1p5ZcBp1dVJZlDt5VsOL+oqvtb3UWBU1r5QNsXtv5/38q/B3yALvk32H18zOctyVCxHwJs0N7ne4GZdAmYLYAPDRZsVR1Gl8xiymrPqxHmJkmSJElawMxr4mWw7TOLALdU1ZRRtJ9vX0Cr6q4kv6RbAbEDj6y+Gcy/062Q2C7JZODMUcQ0VPkiwCYDyacBLWExN/Mbtm5VPdC237ySbnXMB+mSBf2mAdtW1SVJdgG26rl2b2+IPY/9Y8/NeziYe1ryonecwfTG81DP64cY+TN5L0BVPZTk/qoamMNA20HHnYv7OGTsLeFzLd1qmHOB2cDL6Vbh/G6EuCVJkiRJC6F5PePlN8AbkyzZVkm8HrgLuCbJWwHSWbfVP5fuyy7ATq39UE6n2/pDkkWTLNd3/XZgUl/Z4XQrVi6qqn8M0/fywP+157v0lJ/d4iLJS+i2wABcAGyV7tyWxYG39rQ5je7LO63dlEH6eh2w4jDxnA1s185HmQS8sb9Cu7/LV9XPgY8AA+P034dJwA0tzp2GGbM3/nfnkbNgnlpVtzH0ezi3zga2TbJ0kmXotvkMuh1nPrsCmJzk+e31zsBZw9zHwT5vw8V+Nt2WqLNb2e7ArJ4EkCRJkiRJD5unxEtVXUS3ZeQS4HhgOt2WoJ2AXZNcQnc2yzatyYeAd7UtPDsDHx6m+w8DL29bSWYAL+67Pht4oB2EumeLZwbdlpwjRgj9K8B/Jvkt3TaVAYfQndcxG/gEcGHr9wZgX+A84Fd0W0sGfAiY2g5qvZzuCzh0Z8K8LMlM4NXAn4YKpqpmAkcDs+i26AyWmJgEnNxiOwvYs5X/GPh4O0R2deCzdImiX9IlH4ZVVafQvYfTk8yiSybA0O/hXGlzm0Z3Ly8ADq+qi+elr7kc9x66FSnHtM/QQ8ChDH0fH/N5GyH2c+jOzDmvqm6k2740HgklSZIkSdKTUOb1P9QnWbaq7mgrJs4GdmtfWMddklXptg2t2c4ykZ5wpqz2vPrl3vtNdBiSJEmS9KSx8vv+ZaJDGLUkM6rqMcefzPPPSQOHtZUSM4HjJjDp8k66VQmfMekiSZIkSZKeSOb1cF2q6h2PZ+Akn+HRZ6YAHFNVX5zLOI4Ejuzr+108djvTb6vqA3Md6HyQ5Gl0Z4n0e2VV/X2845lb8+u9movxnlDvnyRJkiRJ82qetxpJmjtuNZIkSZKkubMgbDWa5xUvkubOYis/9Un1PxqSJEmSpMfv8ZzxIkmSJEmSpGGYeJEkSZIkSRojJl4kSZIkSZLGiGe8SOPk/ptu5MZDvjrRYUiSJEkLvVXe97GJDkELEVe8SJIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIYMfEiSZIkSZI0Rky8SJIkSZIkjRETL5ovkqya5NgR6pzbHicnecdc9n9mkqkj1PlIkqXnpt8h+tkqycmPtx9JkiRJkky8aL6oqr9U1fYj1Nm0PZ0MzFXiZZQ+AjzuxIskSZIkSfOLiRfNtST7J3l/z+t9k3wsyaXt9YuTXJhkVpLZSV7Qyu9oTb4MbNGu7znEGEsl+XFrfzSwVM+1Q5JMT3JZki+0sg8BqwJnJDmjlb06yXlJZiY5Jsmyw8zptUmuSPIb4M095S9Ncm6Si9vjC1v5OUmm9NT7bZJ15upGSpIkSZIWeCZeNC9+DOzY83oH4KKe17sDX6+qKcBU4M997fcGzqmqKVV14BBjvA+4q6rWAb4IbNBz7TNVNRVYB9gyyTpVdRDwF+DlVfXyJCsB+wBbV9X6wHTgo4MNlGRJ4NvAG4EtgH/quXwF8LKqWg/4HPClVn44sEtrvwawRFXNHqTv3VqSaPo/7rhziKlKkiRJkhZUJl4016rqYuDp7VyXdYGbgT/1VDkP+HSSTwKrVdXd8zDMy4AftPFmA71JjR2SzAQuBl4MrDVI+41b+W+TzAL+FVhtiLHWBK6pqquqqgbGbZYHjmmreQ5s4wEcA7whyeLAu4Fpg3VcVYdV1dSqmvrUZZcZfsaSJEmSpAXOYhMdgJ60jgW2p1sd8uPeC1X1wyQXAK8HTk3ynqr69TyMUf0FSZ4L7AVsWFU3J5kGLDlI2wC/rKq3z+tYzb8DZ1TVdkkmA2cCVNVdSX4JbEO34mfYg38lSZIkSQsnV7xoXv0YeBtd8uVRv2aU5HnA1W37z4l0W4J63Q5MGqH/s4GdWn8v6eljOeBO4NYkqwCvG6Lf84HNkjy/9bF02xI0mCuA5yZZvb3uTdYsD/xfe75LX7vDgYOAi6rqHyPMR5IkSZK0EDLxonlSVZfRJTn+r6pu6Lu8I3Bp2+KzJnBk3/XZwANJLhnqcF3gEGDZJLOBTwAXtnEvodtidBnwXeC3PW0OA36R5IyquokuUfKj1sf5LZbB5nIPsBvws3a47nU9l78C/GeS3wKL9rWbAdwGHDHEHCRJkiRJC7l0R1pImltJVqXberRmVT00Uv11V3t2nbb3R8Y6LEmSJEkjWOV9H5voELQASjKj/RDMo7jiRZoHSd4JXED3C0sjJl0kSZIkSQsnD9fVhEryGmD/vuJrqmq7MRrvBOC5fcWfrKpT56afqjqSx26hkiRJkiTpUUy8aEK1hMdcJT0e53hjktCRJEmSJGkwJl6kcbL4yqu4l1SSJEmSFjKe8SJJkiRJkjRGTLxIkiRJkiSNERMvkiRJkiRJY8QzXqRxct9fr+H6b+w00WFIkiRJY+rZexw10SFITyiueJEkSZIkSRojJl4kSZIkSZLGiIkXSZIkSZKkMWLiRZIkSZIkaYyYeJEkSZIkSRojJl4015KsmuTYEeqc2x4nJ3nHCHWnJPnnntdvSrL3/Il20PGuTbLSWPUvSZIkSdIAEy+aa1X1l6rafoQ6m7ank4FhEy/AFODhxEtVnVhVX348MY63dPx7kiRJkiQ9il8UNawk+yd5f8/rfZN8LMml7fWLk1yYZFaS2Ule0MrvaE2+DGzRru85SP9PAfYDdmx1dkyyS5JvtuvTkhyS5IwkVyfZMsl3k/wuybSefl6d5LwkM5Mck2TZEaa2R6s7J8marY+nJvlpm8f5SdbpmfNePWNd2lbyTG5xHAzMBJ49t/dXkiRJkrRgM/GikfwY2LHn9Q7ART2vdwe+XlVTgKnAn/va7w2cU1VTqurA/s6r6j7gc8DRrc7Rg8SwIvAKYE/gJOBA4MXA2m2b0krAPsDWVbU+MB346Ajz+lurewgwkFT5AnBxVa0DfBo4coQ+AF4IHFlV61XVdf0Xk+yWZHqS6f+4455RdCdJkiRJWpAsNtEB6Imtqi5O8vQkqwIrAzcDf+qpch7wmSTPAo6vqqvGIIyTqqqSzAFurKo5AEkuo9vK9CxgLeC3SQCe0uIazvHtcQbw5vZ8c+AtAFX16yRPS7L8CP1cV1XnD3Wxqg4DDgNY5zlPqxH6kiRJkiQtYEy8aDSOBbYH/oluBczDquqHSS4AXg+cmuQ9VfXr+Tz+ve3xoZ7nA68XAx4EfllVb5+HPh/kkb+DDFKvgAd49OqwJXue3zkXY0qSJEmSFjJuNdJo/Bh4G13y5VG/ZpTkecDVVXUQcCKwTl/b24FJI/Q/mjrDOR/YLMnzW0xLJ1ljHvo5G9ip9bEV3Xak24BrgfVb+frAcx9HrJIkSZKkhYiJF42oqi6jS4z8X1Xd0Hd5R+DSJLOANXnsuSizgQeSXDLY4brNGcBaA4frzkN8NwG7AD9KMpsuEbPm3PYD7AtMbX18GfjXVn4c8NQ2x/cBv5+HviVJkiRJC6FUeeyENB7Wec7T6mcff+1EhyFJkiSNqWfvcdREhyBNiCQzqmpqf7krXiRJkiRJksaIh+tq3CR5DbB/X/E1VbXdGI13Ao89j+WTVXXqWIwnSZIkSVI/txpJ42Tq1Kk1ffr0iQ5DkiRJkjQG3GokSZIkSZI0zky8SJIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIY8VeNpHFy101/YOahb5zoMCRJkjSfrL/7SRMdgqQnAVe8SJIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIYMfEiSZIkSZI0RhbqxEuSfZPsNQ/t9kuy9VjEtKBLMi3J9iPU2SrJpj2vd0/yzrGP7jFxrJDk/T2vV01y7HjHIUmSJEl68vJXjeZBVX1uomNIslhVPTCO4y1aVQ+O03BbAXcA5wJU1aFjNdAI93EF4P3AwS2OvwDDJo0kSZIkSeq10K14SfKZJFcm+RXwwla2epJTksxIck6SNZMsn+TaJIu0OksnuT7J4r2rNpJsmOTcJJckuTDJpCSLJjkgyUVJZif5t2HieUaSs5PMSnJpki1a+WuTzGz9nt7K9k1yWJLTgCOTrJzkuDbORUk2a/WWSfLdVnZxkm1a+S5Jjm9zvSrJV0a4V3e01T0XAJsk+Zc2x1lJ/qfNc9F2Py5NMifJnq3tlCTnt/mfkGTFQfq/NslK7fnUJGcmmQzsDuzZxtmid2XSUP22tvu3+H4/cB+HmNcuSY5JchJwWpJlk5ze7vecgfsFfBlYvcVxQJLJSS5tfSyZ5IhW/+IkLx/uXkqSJEmSFk4L1YqXJBsAbwPWo5v7TGAGcBiwe1VdlWQj4OCqekWSS4AtgTOANwKnVtX9SQb6ewpwNLBjVV2UZDngbmBX4Naq2jDJEsBvk5xWVdcMEtY7Wr9fTLIosHSSlYFvAy+rqmuSPLWn/gbA5lV1d5IfAgdW1W+SPAc4FXgR8Bng11X17iQrABe2RBPAlDb/e4Erk3yjqq4f4pYtA1xaVZ9L8iLgk8Bm7R4cDOwEXAY8s6pe0u7JCq3tkcAeVXVWkv2AzwMfGWKch1XVtUkOBe6oqv9qfb6yp8pw/S5WVS9N8s+tfLjtYJsA61TVP5IsBmxXVbe1RND5SU4E9gZeUlVTWhyTe9p/oMW7dpI16RI4a1TVPb2DJNkN2A3gn5661EjTlyRJkiQtYBaqxAuwBXBCVd0F0L5cLwlsChwzkFABlmiPRwM70iVe3kbbctLjhcANVXURQFXd1vp9NbBOHjnLZHngBcBgiZeLgO8mWRz4aVXNSrIVcPZAoqaq/tFT/8Squrs93xpYqyfu5ZJMAl4NvCmPnF+zJPCc9vz0qrq1xXk5sBowVOLlQeC49vyVdEmfi9p4SwF/BU4CnpfkG8DP6BIQywMrVNVZre33gGOGGGPURtHv8e1xBjB5hO5+2XNfA3wpycuAh4BnAquM0H5z4BsAVXVFkuuANYDZvZWq6jC6xB5rrbZCjdCnJEmSJGkBs7AlXgD6v/wuAtwysKqhz4nAf7YVJxsAv+67nkH6Gyjfo6pOHTGYqrPbF/7XA99PcgBwyxD9AtzZF/smPYmYbvAuM/KWqrqyr3wjupUuAx5k+M/APT3nugT4XlV9qr9SknWB19CtAtkB2HOYPns9wCPb3ZYcZZvhDMxtpHnBo+/jTsDKwAZtNc+1o4gnI1yXJEmSJGmhO+PlbGC7JEu1lSFvBO4CrknyVuiSFi2RQFXdAVwIfB04eZDDZa8AVk2yYWs7qW1bORV4X1vFQpI1kiwzWEBJVgP+WlXfBr4DrA+cB2yZ5LmtzlMHawucBnywp68p7empwB4tAUOS9UZ1d4Z3OrB9kqcPxJRktbY1Z5GqOg74LLB+W1Fzc885KzsDZw3S57V0CS2At/SU3w5M6q88F/3OreXp3oP721ktqw0XR3M2XcKGJGvQrSi6coi6kiRJkqSF1EK14qWqZiY5GpgFXAec0y7tBBySZB9gceDHwCXt2tF021m2GqS/+5LsCHwjyVJ057tsDRxOt9VlZkt+3ARsO0RYWwEfT3I/3S/5vLOqbmpngxyf7nDfvwKvGqTth4BvJZlN916eTXcw7b8DXwNmt/GvBd4w7M0ZQVVd3u7PaS2m++lWuNwNHNHKAAZWxPwrcGiSpYGrgXcN0u0XgO8k+TRwQU/5ScCx7ZDbPfrajKbfuXUUcFKS6XSfjSsAqurvSX7bDtT9BfCtnjYHtzjm0K3c2aWq7kWSJEmSpB6p8tgJaTystdoK9YNPDfljS5IkSXqSWX/3kyY6BElPIElmVNXU/vKFbauRJEmSJEnSuFmothpNpCRrA9/vK763qjaaiHh6JbmAR37JacDOVTVnIuKZX5K8Bti/r/iaqtpuIuKRJEmSJC18TLyMk5bEmDLRcQzmiZD8GQvtV6VG/GUpSZIkSZLGiokXaZwsvfLz3QcsSZIkSQsZz3iRJEmSJEkaIyZeJEmSJEmSxoiJF0mSJEmSpDHiGS/SOLntb1fxq8P/eaLDkCRJWqBs/Z6fT3QIkjQsV7xIkiRJkiSNERMvkiRJkiRJY8TEiyRJkiRJ0hgx8SJJkiRJkjRGTLxIkiRJkiSNkQU28ZJk3yR7zUO7/ZJsPRYxLeiSTEuy/Qh1tkqyac/r3ZO8cz7GsEuSb85lm8OTrDVCnUHnlmRyknfMbZySJEmSpIWDPyfdp6o+N9ExJFmsqh4Yx/EWraoHx2m4rYA7gHMBqurQcRp3SFX1nsfRfDLwDuCH8ycaSZIkSdKCZIFa8ZLkM0muTPIr4IWtbPUkpySZkeScJGsmWT7JtUkWaXWWTnJ9ksV7VzYk2TDJuUkuSXJhkklJFk1yQJKLksxO8m/DxPOMJGcnmZXk0iRbtPLXJpnZ+j29le2b5LAkpwFHJlk5yXFtnIuSbNbqLZPku63s4iTbtPJdkhzf5npVkq+McK/uaKt7LgA2SfIvbY6zkvxPm+ei7X5cmmROkj1b2ylJzm/zPyHJioP0f22SldrzqUnOTDIZ2B3Ys42zRe/KpKH6bW33b/H9fuA+DmPVwe5DklcnOa/d+2OSLNvT/9T2fNc2xplJvt23euZl7fNwdc/qly8DW7T57DlCXJIkSZKkhcwCk3hJsgHwNmA94M3Ahu3SYcAeVbUBsBdwcFXdClwCbNnqvBE4taru7+nvKcDRwIeral1ga+BuYFfg1qrasI3x3iTPHSKsd7R+pwDrArOSrAx8G3hL6/etPfU3ALapqncAXwcObOO8BTi81fkM8OtW/nLggCTLtGtTgB2BtYEdkzx7mFu2DHBpVW0E/L2126zF+iCwU+vvmVX1kqpaGziitT0S+GRVrQPMAT4/zDgPq6prgUPbvKZU1Tl9VYbrd7GqeinwkVGMN4W++9CSQPsAW1fV+sB04KO9jZKsCnwW2Bh4FbBmX7/PADYH3kCXcAHYGzinzefA/kCS7JZkepLpt95+3whhS5IkSZIWNAvSVqMtgBOq6i6AJCcCSwKbAsckGai3RHs8mu7L+Rl0CZuD+/p7IXBDVV0EUFW3tX5fDazTs+JheeAFwDWDxHQR8N0kiwM/rapZSbYCzq6qa1q//+ipf2JV3d2ebw2s1RP3ckkmAa8G3pRHzq9ZEnhOe356SyqR5HJgNeD6Qe9Wl1w5rj1/JV3S56I23lLAX4GTgOcl+QbwM+C0JMsDK1TVWa3t94Bjhhhj1EbR7/HtcQbd9p7hDHYfVgDWAn7b5vgU4Ly+di8Fzhp4T5IcA6zRc/2nVfUQcHmSVUYzr6o6jC75xxqTl6/RtJEkSZIkLTgWpMQLQP8X20WAW9oqjn4nAv+Z5Kl0SYdf913PIP0NlO9RVaeOGEzV2UleBrwe+H6SA4BbhugX4M6+2DfpScR0g3dZg7dU1ZV95RsB9/YUPcjw7+89Pee6BPheVX2qv1KSdYHXAB8AdgBGu53mAR5ZUbXkKNsMZ2BuI82rt25v/QC/rKq3D9Muw1zr73ekupIkSZIkLThbjYCzge2SLNVWhrwRuAu4JslboUtatEQCVXUHcCHdlp6TBzlc9gq6s0I2bG0nJVkMOBV4X1vFQpI1erb6PEqS1YC/VtW3ge8A69OtsthyYHtSS/wM5jTggz19TWlPTwX2aAkYkqw3qrszvNOB7ZM8fSCmJKu17TmLVNVxdFtw1m8rSW7uOWdlZ+CsQfq8li6hBd1WqQG3A5P6K89Fv/PqfGCzJM+Hh8/1WaOvzoV0782K7b1+S38ngxh0PpIkSZIkwQK04qWqZiY5GpgFXAcMnB+yE3BIkn2AxYEf053vAt12o2Pofmmnv7/7kuwIfCPJUnTnu2xNd9bKZGBmS37cBGw7RFhbAR9Pcj/dL/m8s6puSrIbcHy6w33/SneeSL8PAd9KMpvufTqb7mDafwe+Bsxu419Ld+bIPKuqy9v9Oa3FdD/dCpe7gSNaGcDAiph/BQ5NsjRwNfCuQbr9AvCdJJ8GLugpPwk4Nt2hwHv0tRlNv/Ok3fddgB8lGdhutg/w+546/5fkSy3evwCXA7eO0PVs4IEklwDTBjvnRZIkSZK08EqVx05IA5IsW1V3tBUvJwDfraoT5kffa0xevg7eZ7P50ZUkSZKard/z84kOQZIASDKjqqb2ly9IW42k+WHfJLOAS+kOTP7phEYjSZIkSXpSW2C2Gk2kJGsD3+8rvrf9VPOESnIBj/yS04Cdq2rORMQzvyR5DbB/X/E1VbXd4+m3qvYauZYkSZIkSaNj4mU+aEmMKRMdx2CeCMmfsdB+VWrEX5aSJEmSJGkimXiRxslyK73APciSJEmStJDxjBdJkiRJkqQxYuJFkiRJkiRpjJh4kSRJkiRJGiOe8SKNk5v/dhXHHvHaiQ5DkiQ9gWz/rlMmOgRJ0hhzxYskSZIkSdIYMfEiSZIkSZI0Rky8SJIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIYMfEikqya5NgR6pzbHicneccIdack+eee129Ksvf8iXbejGaOI7TfNsla8zMmSZIkSdKCz8SLqKq/VNX2I9TZtD2dDAybeAGmAA8nXqrqxKr68uOJ8fEazRxHsC1g4kWSJEmSNFdMvCxkkuyf5P09r/dN8rEkl7bXL05yYZJZSWYneUErv6M1+TKwRbu+5yD9PwXYD9ix1dkxyS5JvtmuT0tySJIzklydZMsk303yuyTTevp5dZLzksxMckySZYeZ07VJvtTqT0+yfpJTk/wxye6tzuSeOe6S5PgkpyS5KslXevq6o+f59i3eTYE3AQe0Oa3e/p2SZEaSc5KsOddvhiRJkiRpgWfiZeHzY2DHntc7ABf1vN4d+HpVTQGmAn/ua783cE5VTamqA/s7r6r7gM8BR7c6Rw8Sw4rAK4A9gZOAA4EXA2u3bUorAfsAW1fV+sB04KMjzOv6qtoEOAeYBmwPbEyXBBrMFLr7sDZdkujZQ3VcVecCJwIfb3P6I3AYsEdVbQDsBRw8WNsku7Vk0PTb7rhvhClIkiRJkhY0i010ABpfVXVxkqcnWRVYGbgZ+FNPlfOAzyR5FnB8VV01BmGcVFWVZA5wY1XNAUhyGd1WpmfRbev5bRKAp7S4hnNie5wDLFtVtwO3J7knyQqD1D+9qm5t414OrAZcP5rg2+qbTYFjWnwASwxWt6oOo0vSsPrk5Ws0/UuSJEmSFhwmXhZOx9KtCPknuhUwD6uqHya5AHg9cGqS91TVr+fz+Pe2x4d6ng+8Xgx4EPhlVb19PvY5VH3aeAN1epMjSw4x1iLALW1VkCRJkiRJQ3Kr0cLpx8Db6JIvj/qlnyTPA66uqoPoVpGs09f2dmDSCP2Pps5wzgc2S/L8FtPSSdZ4HP3NjRuTvCjJIsB2PeUPz6mqbgOuSfLWFl+SrDtO8UmSJEmSnkRMvCyEquoyuiTC/1XVDX2XdwQuTTILWBM4su/6bOCBJJcMdrhucwaw1sDhuvMQ303ALsCPksymS8SM1+G1ewMnA78Geu/Nj4GPJ7k4yerATsCuSS4BLgO2Gaf4JEmSJElPIqny2AlpPKw+efna//ObTHQYkiTpCWT7d50y0SFIkuaTJDOqamp/uSteJEmSJEmSxoiH62qeJXkNsH9f8TVVtd1g9efDeCcAz+0r/mRVnToW40mSJEmS9Hi51UgaJ1OnTq3p06dPdBiSJEmSpDHgViNJkiRJkqRxZuJFkiRJkiRpjJh4kSRJkiRJGiMmXiRJkiRJksaIv2okjZO//f0qvnPkayY6DEnSE9Cu7/QH+iRJWlC54kWSJEmSJGmMmHiRJEmSJEkaIyZeJEmSJEmSxoiJF0mSJEmSpDFi4kWSJEmSJGmMmHjRmEuyapJjR6hzbnucnOQd4xPZoHHcMVFjS5IkSZIWPCZeNOaq6i9Vtf0IdTZtTycDE5Z4eTyS+PPskiRJkqRHMfGi+SrJ/kne3/N63yQfS3Jpe/3iJBcmmZVkdpIXtPKBlSZfBrZo1/ccYoxdkhyf5JQkVyX5Ss+1O3qeb59kWns+LckhSc5IcnWSLZN8N8nvBur0tPtqkplJTk+ycitbvY03I8k5Sdbs6fe/k5wB7P+4b6AkSZIkaYFi4kXz24+BHXte7wBc1PN6d+DrVTUFmAr8ua/93sA5VTWlqg4cZpwpbZy1gR2TPHsUsa0IvALYEzgJOBB4MbB2kimtzjLAzKpaHzgL+HwrPwzYo6o2APYCDu7pdw1g66r6WP+ASXZLMj3J9Ntvv28UIUqSJEmSFiRujdB8VVUXJ3l6klWBlYGbgT/1VDkP+EySZwHHV9VV8zjU6VV1K0CSy4HVgOtHaHNSVVWSOcCNVTWntb+MbovTLOAh4OhW/wfA8UmWBTYFjkky0NcSPf0eU1UPDjZgVR1Gl7Rh8nOXr7mZoCRJkiTpyc/Ei8bCscD2wD/RrYB5WFX9MMkFwOuBU5O8p6p+PQ9j3Nvz/EEe+Sz3JjeWHKLNQ33tH2Lov4WiWxl2S1ulM5g7RwpWkiRJkrRwcquRxsKPgbfRJV8e9WtGSZ4HXF1VBwEnAuv0tb0dmPQ4xr4xyYuSLAJsNw/tF6GLG7pDfn9TVbcB1yR5K0A66z6OGCVJkiRJCwkTL5rvquoyuuTJ/1XVDX2XdwQuTTILWBM4su/6bOCBJJcMdbjuCPYGTgZ+DfSPPRp3Ai9OMoPuPJj9WvlOwK5JLgEuA7aZh74lSZIkSQuZVHnshDQeJj93+frsFzae6DAkSU9Au77z1IkOQZIkPU5JZlTV1P5yV7xIkiRJkiSNEQ/X1RNWktcA+/cVX1NV83J2iyRJkiRJ486tRtI4mTp1ak2fPn2iw5AkSZIkjQG3GkmSJEmSJI0zEy+SJEmSJEljxMSLJEmSJEnSGDHxIkmSJEmSNEb8VSNpnNz4j6v4rx+9ZqLDkCTNR3u9/dSJDkGSJD3BueJFkiRJkiRpjJh4kSRJkiRJGiMmXiRJkiRJksaIiRdJkiRJkqQxstAkXpLsm2SveWi3X5KtxyKmBV2SaUm2H6HOVkk27Xm9e5J3jn10D4+3apJjh7h2ZpKp7fmne8onJ7l0vGKUJEmSJD15+atGI6iqz010DEkWq6oHxnG8RavqwXEabivgDuBcgKo6dJzGpY33F2DY5FDzaeBLYxyOJEmSJGkBs0CveEnymSRXJvkV8MJWtnqSU5LMSHJOkjWTLJ/k2iSLtDpLJ7k+yeK9qzaSbJjk3CSXJLkwyaQkiyY5IMlFSWYn+bdh4nlGkrOTzEpyaZItWvlrk8xs/Z7eyvZNcliS04Ajk6yc5Lg2zkVJNmv1lkny3VZ2cZJtWvkuSY5vc70qyVdGuFd3tNU9FwCbJPmXNsdZSf6nzXPRdj8uTTInyZ6t7ZQk57f5n5BkxUH6vzbJSu351LaaZDKwO7BnG2eL3pVJQ/Xb2u7f4vv9wH0cYl4/T7JOe35xks+15/+e5D29q1eSLJXkx228o4GlWvmXgaVajEe1rhdN8u0klyU5LclSw91fSZIkSdLCaYFNvCTZAHgbsB7wZmDDdukwYI+q2gDYCzi4qm4FLgG2bHXeCJxaVff39PcU4Gjgw1W1LrA1cDewK3BrVW3YxnhvkucOEdY7Wr9TgHWBWUlWBr4NvKX1+9ae+hsA21TVO4CvAwe2cd4CHN7qfAb4dSt/OXBAkmXatSnAjsDawI5Jnj3MLVsGuLSqNgL+3tpt1mJ9ENip9ffMqnpJVa0NHNHaHgl8sqrWAeYAnx9mnIdV1bXAoW1eU6rqnL4qw/W7WFW9FPjICOOdDWyRZDngAWCzVr450D/e+4C72nhfpLv/VNXewN0txp1a3RcA36qqFwO30L0nj5FktyTTk0y/4/b7hglTkiRJkrQgWpC3Gm0BnFBVdwEkORFYEtgUOCbJQL0l2uPRdMmGM+gSNgf39fdC4Iaqugigqm5r/b4aWCePnGWyPN2X8msGieki4LtJFgd+WlWzkmwFnF1V17R+/9FT/8Squrs93xpYqyfu5ZJMAl4NvCmPnF+zJPCc9vz0llQiyeXAasD1g96tLrlyXHv+Srqkw0VtvKWAvwInAc9L8g3gZ8BpSZYHVqiqs1rb7wHHDDHGqI2i3+Pb4wxg8jBdnQN8iO79+BnwqiRLA5Or6sq26mbAy4CDAKpqdpLZw/R7TVXNGimGqjqMLtnHs5+3fA3TnyRJkiRpAbQgJ14A+r/oLgLc0lZx9DsR+M8kT6VLOvy673oG6W+gfI+qOnXEYKrOTvIy4PXA95McQLdaYqgv5Hf2xb5JTyKmG7zLjLylqq7sK98IuLen6EGGf7/v6TnXJcD3qupT/ZWSrAu8BvgAsAOw5zB99nqAR1ZYLTnKNsMZmNtI87oImApcDfwSWAl4L12yZDCjTY7031u3GkmSJEmSHmOB3WpEt8Vku3ZuxyS67UN3AdckeSt0SYuWSKCq7gAupNvSc/Igh8teAayaZMPWdlKSxYBTgfe1VSwkWaNnq8+jJFkN+GtVfRv4DrA+cB6w5cD2pJb4GcxpwAd7+prSnp4K7NESMCRZb1R3Z3inA9snefpATElWa2e0LFJVxwGfBdZvK2pu7jlnZWfgrEH6vJa2dYdHb8u5HZjUX3ku+h1WVd1Ht8pnB+B8uhUwe/HYbUbQfWZ2AkjyEmCdnmv3D7zHkiRJkiSN1gK74qWqZrYDUmcB1/HIF+2dgEOS7AMsDvyY7nwX6LYbHUP3Szv9/d2XZEfgG+0g1bvptv8cTrfNZGZLftwEbDtEWFsBH09yP90v+byzqm5KshtwfLrDff8KvGqQth8CvtW2vyxGlyTYHfh34GvA7Db+tcAbhr05I6iqy9v9Oa3FdD/dCpe7gSNaGcDAiph/BQ5tW3iuBt41SLdfAL6T7meZL+gpPwk4Nt2hwHv0tRlNv6NxDvDKqroryTnAsxg88XII3fxm031uLuy5dhjdPZ5Jd66OJEmSJEkjSpXHTkjj4dnPW74+/MWNJzoMSdJ8tNfbR9xpLEmSFhJJZlTV1P7yBXmrkSRJkiRJ0oRaYLcaTaQkawPf7yu+t/1U84RKcgGP/JLTgJ2ras5ExDO/JHkNsH9f8TVVtd1ExCNJkiRJEph4GRMtiTFlouMYzBMh+TMW2q9Kud5bkiRJkvSEYuJFGierPPUFngUgSZIkSQsZz3iRJEmSJEkaIyZeJEmSJEmSxoiJF0mSJEmSpDHiGS/SOLn+5qv4yHGvnegwJD2JfO0tp0x0CJIkSXqcXPEiSZIkSZI0Rky8SJIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIYMfEiSZIkSZI0Rky8LMCSrJrk2BHqnNseJyd5x1z2f2aSqSPU+UiSpeem37GSZN8ke41QZ3KSS8crJkmSJEnSgs3EywKsqv5SVduPUGfT9nQyMFeJl1H6CPCESLxIkiRJkjTeTLwsIJLsn+T9Pa/3TfKxgdUbSV6c5MIks5LMTvKCVn5Ha/JlYIt2fc8hxlgqyY9b+6OBpXquHZJkepLLknyhlX0IWBU4I8kZrezVSc5LMjPJMUmWHWKslyY5vj3fJsndSZ6SZMkkV7fy1ZOckmRGknOSrNnKV05yXJKL2r/NBun/vUl+0ea0QZJLkpwHfKCnzuTW78z2b9NW/v0k2/TUOyrJm4Z/hyRJkiRJCyMTLwuOHwM79rzeAbio5/XuwNeragowFfhzX/u9gXOqakpVHTjEGO8D7qqqdYAvAhv0XPtMVU0F1gG2TLJOVR0E/AV4eVW9PMlKwD7A1lW1PjAd+OgQY80E1mvPtwAuBTYENgIuaOWHAXtU1QbAXsDBrfzrwIFVtSHwFuDw3o6TfBB4I7BtVd0NHAF8qKo26Yvhr8CrWqw7Age18sOBd7W+lgc2BX4+2CSS7NYSUtPvvu2+IaYqSZIkSVpQLTbRAWj+qKqLkzw9yarAysDNwJ96qpwHfCbJs4Djq+qqeRjmZbTkQ1XNTjK759oOSXaj+0w9A1gLmN3XfuNW/tskAE9pcQ02nweS/CHJi4CXAv/dxl8UOKetlNkUOKb1BbBEe9waWKunfLkkk9rznemSTttW1f0tcbJCVZ3Vrn8feF17vjjwzSRTgAeBNVpsZyX5VpKnA28GjquqB4aYx2F0CSJWWX35GqyOJEmSJGnBZeJlwXIssD3wT3QrYB5WVT9McgHweuDUJO+pql/PwxiPSR4keS7dipMNq+rmJNOAJQdpG+CXVfX2UY51Dl0S5H7gV8A0usTLXnSrtW5pK3j6LQJs0laz9MYJ3cqZKcCzgGtaTEMlRPYEbgTWbX3e03Pt+8BOwNuAd49yPpIkSZKkhYxbjRYsP6ZLBGxPl4R5WJLnAVe37T8n0m0J6nU7MInhnU2XbCDJS3r6WA64E7g1ySo8smKkv9/zgc2SPL/1sXSSNUYY7yPAeVV1E/A0YE3gsqq6DbgmyVtbX0mybmt3GvDBnrlP6enzYuDfgBOTrFpVt7S4N2/Xd+qpuzxwQ1U9RLdSZtGea9NabFTVZcPMQZIkSZK0EDPxsgBpCYBJwP9V1Q19l3cELk0yiy55cWTf9dnAA+2Q2UEP1wUOAZZtW4w+AVzYxr2ELqFxGfBd4Lc9bQ4DfpHkjJY82QX4Uevj/BbLUC4AVqFLwAzEOLuqBlao7ATsmuSSNvbAgbcfAqa2Q4Avpzvf5mFV9Ru6VTM/a+fOvAv4Vjtct3eVzMHAvyY5n26b0Z09fdwI/I7ufBhJkiRJkgaVR77DShqtJEsDc4D1q+rW0bRZZfXl6+1f6T+/V5KG9rW3nDLRIUiSJGmUksxoPzrzKK54keZSkq2BK4BvjDbpIkmSJElaOHm4rh4jyWuA/fuKr6mq7cZovBOA5/YVf7KqTh2L8R6vqvoV8JyJjkOSJEmS9MRn4kWP0RIe45b0GKuEjiRJkiRJE83EizROnr3iCzyvQZIkSZIWMp7xIkmSJEmSNEZMvEiSJEmSJI0REy+SJEmSJEljxDNepHFy1S3X8rr/3XWiwxjWL7b5zkSHIEmSJEkLFFe8SJIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIYMfEiSZIkSZI0Rky8SJIkSZIkjZExSbwk2SXJqmPR90RIsnuSd87nPq9NslJf2S5Jbkoyq+ffWsP0MTnJ3a3e5UkOTbJIK7+0r+6+Sfaax1j74zpyXvoZou9pSf4vyRLt9UpJru25fkCSy5IcMA99r5Lkh0muTjIjyXlJthtFu7cm+V2SM0Y5zlZJTp7b+CRJkiRJC76x+jnpXYBLgb+MUf/jqqoOHcfhjq6qD85F/T9W1ZQkiwG/BrYFZj4B4pobDwLvBg4Z5Nq/AStX1b1z02GSAD8FvldV72hlqwFvGkXzXYH3V9WoEi+SJEmSJA3lca146V9ZkWSv9noqcFRbHbFUW93xpbbiYHqS9ZOcmuSPSXYfYYxPJJmT5JIkX25l701yUSs7LsnSrXxakkOSnNFWOWyZ5Ltt9cK0nj7vSPLVJDOTnJ5k5RH6fXi1SJINk8xuczlgYP5tVcjxSU5JclWSrzyee9t3DzIwVrsXO/bXqaoHgHOB54+ivzOTTG3PH15hkuScJFN66v02yTpD9PGiJBf2vJ7c7svUnpUxc5JUuz4lyfmtzglJVuzp7mvAni151DvGicAywAVJdmzv70FJzm3v7/at3n49Y/5fkiOAVwD39SbNquq6qvpGa7N0kp+0eI5OckGL/XPA5sCh7Z4vmuS/2lxmJ9mjtX9tkiuS/AZ480j3XJIkSZK0cBqLrUbHAtOBnapqSlXd3cqvr6pNgHOAacD2wMbAfkN1lOR1dCs4NqqqdYGBZMbxVbVhK/sd3QqFASvSfeneEzgJOBB4MbB2T1JhGWBmVa0PnAV8fhT9DjgC2L3N5cG+a1OAHYG1gR2TPHuouQ1jxzx6q9FSdF/spwDrAlsDByR5Rm+jliR6JTCnFa3e2w8wbIKrOZxutRJJ1gCWqKrZg8T1rqr6HfCUJM8buA78pKqmt/d9CnAK8F/t+pHAJ6tqnRbjwD0H+BPwG2Dn3mCq6k3A3a2/o1vxM+gSI28Avtzqfa6NtyXwd+CbdO/5cCt/3g/c3OL5d2CD1td+PPL5/TiwG/BcYL1W96gkSwLfBt4IbAH801CDJNmtJRun33fbPcOEI0mSJElaEI3n4bontsc5wAVVdXtV3QTck2SFIdpsDRxRVXcBVNU/WvlL2uqMOcBOdF+yB5xUVdXGubGq5lTVQ8BlwORW5yFg4Iv8D+i+yI/ULy3OSVV1biv6YV+8p1fVrVV1D3A5sNrQt2NIRw8kLnoSV5sDP6qqB6vqRrpk0Yat/uotsfJb4GdV9YtW/sfefoDRbJc6BnhDksXptv5MGyKuI1rZT4Ad2vMdeeSekmQHYH1g7yTLAytU1Vnt8veAl/WN/SXg44z8mfxpVT1UVZcDq/SMF+Ao4MCqmtHfKMm32kqmi1rR5sCPAarqUmB2f5tma+DQtqJo4DO4JnBNVV3VPms/GCrYqjqsqqZW1dSnLLfkCFOTJEmSJC1oHu8ZLw/w6C/Kw32zHDij46Ge5wOvh4ojQA1SPg3YtqouSbILsNXjHGdgjOH6HYhnOL3jPTjMeHNruHH/2BIrc6P3fXv4Pauqu5L8EtiGLqEydYR+jgaOSXJ817yuAkjyYuALwMuq6sEuJzK8qvpDSyDtMELV3nvc2/G+wJ97kkKXAW/p6f8D6Q4znj5I2+EM9RkcrEySJEmSpEd5vCtebgSenuRp6X6V5g2t/HZg0uPsG+A04N09Z608tZVPAm5oKzN2mod+F6Hb6gTwDrptLiP2W1U3A7cn2bgVvW0exp4XZ9Nt9Vk03Xk0LwMuHKHNcK6lba3hkfsw4HDgIOCinhVGg6qqP9IlmD5LW+3SVrf8GHhnW9FEVd0K3Jxki9Z0Z7pVO/2+CMz1Ly8leQPwKuBDPcW/BpZM8r6esqV7nv+GluRJ98tRaw/R/WnA7gPnz7TP4BXAc5Os3uq8fW5jliRJkiQtHB7Xioyquj/JfsAFwDV0X0ihWzlyaJK7gU0eR/+ntHNZpie5D/g58Gm6L/oXANfRbSma2yTPncCLk8wAbqXbJsMo+90V+HaSO4EzW/t5NTvJQ+35T+i2u+yYZPOeOu8HTqC7j5fQrbT4RFX9vyST53Hc/wJ+kmRnugTFw6pqRpLb6M6yGY2jgQPozkGB7kye1eju0UCfU4B/pftMLA1cDbyrv6OquizJTLotSnPjY8CqwIVtzBOr6nNJtgUOTPIJ4Ca69/2Trc3BwPeSzAYuprv3g72XhwNr0L1X9wPfrqpvJtkN+FmSv9ElcV4ylzFLkiRJkhYC6Y6oWLgkuaOqlp3HtstW1R3t+d7AM6rqw/M1wAmUZFW6hNKa7WycBVKSRYHFq+qetnLldGCNqrpvrMZc/vkr1aZf3Wasup8vfrHNdyY6BEmSJEl6Ukoyo6oec2TH/DqDZGHy+iSfort319F+BWhBkOSddNt9ProgJ12apYEz2rayAO8by6SLJEmSJGnh9IRIvCRZG/h+X/G9VbXRWIw3r6tdWtuj6fn1npEkuQBYoq9456qaM1j9iVRVR9L97PMCr6puZ+TDgyVJkiRJelwWyq1G0kSYOnVqTZ8+feSKkiRJkqQnnaG2Gj3eXzWSJEmSJEnSEEy8SJIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIYeUL8qpG0MLjqlhv45xP+Y773+/Pt9pnvfUqSJEmS5g9XvEiSJEmSJI0REy+SJEmSJEljxMSLJEmSJEnSGDHxIkmSJEmSNEZMvEiSJEmSJI0REy8LsCS7JFl1ouOYX5LsnuSdEx2HJEmSJEmj5c9JL9h2AS4F/jLBccwXVXXoWPafZNGqenAU9RarqgfGMhZJkiRJ0oLBFS8LgCSTk1za83qv9noqcFSSWUmWSnJtki8lOS/J9CTrJzk1yR+T7D7CGJ9IMifJJUm+3Mrem+SiVnZckqVb+bQkhyQ5I8nVSbZM8t0kv0syrafPO5J8NcnMJKcnWXmEfvdNsld7vmGS2W0uBwzMv63yOT7JKUmuSvKVEeZ1R5L9klwAbJLkc23sS5McliSt3pnt3p0FfDjJBknOSjKj3cNnzOXbJkmSJElaCJh4WXAdC0wHdqqqKVV1dyu/vqo2Ac4BpgHbAxsD+w3VUZLXAdsCG1XVusBAMuP4qtqwlf0O2LWn2YrAK4A9gZOAA4EXA2snmdLqLAPMrKr1gbOAz4+i3wFHALu3ufSvUpkC7AisDeyY5NlDza3FcGlVbVRVvwG+2cZ+CbAU8IaeuitU1ZbAQcA3gO2ragPgu8AXB+s8yW4tyTX9vtvuHCYMSZIkSdKCyMTLwufE9jgHuKCqbq+qm4B7kqwwRJutgSOq6i6AqvpHK39JknOSzAF2okusDDipqqqNc2NVzamqh4DLgMmtzkPA0e35D4DNR9EvLc5JVXVuK/phX7ynV9WtVXUPcDmw2tC3gweB43pevzzJBW3sV/SNPRDrC4GXAL9MMgvYB3jWYJ1X1WFVNbWqpj5luWWGCUOSJEmStCDyjJcFwwM8Oom25DB1722PD/U8H3g91OchQA1SPg3YtqouSbILsNXjHGdgjOH6HYhnOL3jPTjMeAD3DJzrkmRJ4GBgalVdn2RfHn0vB5asBLisrbaRJEmSJGlIrnhZMNwIPD3J05IswSPbY24HJs2H/k8D3t1z1spTW/kk4IYki9OtTJlbi9BtdQJ4B/Cb0fRbVTcDtyfZuBW9bR7GHsxAkuVvSZbtia3flcDKSTYBSLJ4khcPUVeSJEmStBBzxcsCoKruT7IfcAFwDXBFuzQNODTJ3cA8r86oqlPauSzTk9wH/Bz4NPDZNuZ1dFuK5jbJcyfw4iQzgFvpzmVhlP3uCnw7yZ3Ama3941JVtyT5dhvzWuCiIerdl2R74KAky9P9HX2NbhuVJEmSJEkPS3cMhzT+ktxRVcvOY9tlq+qO9nxv4BlV9eH5GuB8tvzzn1mbHfC++d7vz7fbZ773KUmSJEmaO0lmVNXU/nJXvOjJ6vVJPkX3Gb4O2GViw5EkSZIk6bFMvOhhSdYGvt9XfG9VbTQW483rapfW9mge+ZWhESW5AFiir3jnqpozrzFIkiRJkjQStxpJ42Tq1Kk1ffr0iQ5DkiRJkjQGhtpq5K8aSZIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIYMfEiSZIkSZI0RvxVI2mcXHXLX3n98d+cb/397M0fnG99SZIkSZLGhiteJEmSJEmSxoiJF0mSJEmSpDFi4kWSJEmSJGmMmHiRJEmSJEkaIwtl4iXJLklWneg4+iW5NslK7fm5T4B49k2y1yDlk5NcOpd9TUuy/TDXD0+y1rzEOZ6eLHFKkiRJkp4YFtZfNdoFuBT4ywTHMaSq2nSiYxhPVfWeseg3yWJV9cD86m+s4pQkSZIkLZgW6BUv/SszkuzVXk8FjkoyK8lSbaXJl5Kcl2R6kvWTnJrkj0l2H6b/rZKcleQnSX6f5MtJdkpyYZI5SVZv9VZOclySi9q/zVr505KcluTiJP8DpKfvO3rGOLmn/JtJdmnP5zXuZZOcnmRmi3ObnmufSXJlkl8BL+wp3yDJJUnOAz7QU75okgPavGYn+bdWnhbr5Ul+Bjx9hPfqzCRTW3/TklzaYttzmDZTkpzfxj0hyYo9fX0pyVnAh5Ns2Oqc12K9tNWbnOScdh9mJtm0556fmeTYJFckOSpJeuNsz1/b2l2S5PTh5idJkiRJWjgt0ImXIRwLTAd2qqopVXV3K7++qjYBzgGmAdsDGwP7jdDfusCHgbWBnYE1quqlwOHAHq3O14EDq2pD4C3tGsDngd9U1XrAicBz5mE+8xL3PcB2VbU+8HLgqy1RsgHwNmA94M3Ahj1tjgA+1MbqtStwa5vbhsB7kzwX2I4ucbM28F5gtCt4pgDPrKqXVNXabdyhHAl8sqrWAebQ3c8BK1TVllX11dbH7i32B3vq/BV4VbsPOwIH9VxbD/gIsBbwPGCz3oGTrAx8G3hLVa0LvHWwAJPs1pJi0++79Y7hZy5JkiRJWuAsrFuNBnNie5wDLFtVtwO3J7knyQpVdcsQ7S6qqhsAkvwROK2nn5e351sDa7VFEwDLJZkEvIwuwUFV/SzJzeMUd4AvJXkZ8BDwTGAVYAvghKq6q83nxPa4PF0i46zW/vvA69rzVwPr5JHzW5YHXtDm9qOqehD4S5Jfj3I+VwPPS/IN4Gc8cj8fPYHHxvQ94JieKke3eisAk6pq4MycHwJvaM8XB76ZZApdQmaNnvYXVtWfWx+zgMnAb3qubwycXVXXAFTVPwaLs6oOAw4DWP75z6mhpy1JkiRJWhAt6ImXB3j0qp4lh6l7b3t8qOf5wOvh7lN/3d5+BtotAmzSs7oGgJaIGenL+EhzmJe4dwJWBjaoqvuTXNvT72DxZJg4A+xRVac+qjD552HaDKmqbk6yLvAaui1NOwDvntt+gDt74hvKnsCNdKuWFqFbCTSg914+yGPv5XD3RJIkSZIkYMHfanQj8PR2lsoSPLLS4XZg0jjGcRrwwYEXbYUFwNl0SRCSvA5YcZC219GtllmirfJ45XyIZ3ngry3p8nJgtZ54tkt37s0k4I0AbdXMrUk2b/V26unrVOB9SRZv81gjyTKtr7e1M1uewSOrf4aV7ledFqmq44DPAusPVq+qbgVuTrJFK9oZOGuQejfTrQDauBW9re8+3FBVD7X2i44mxuY8YMu2rYokT52LtpIkSZKkhcQCveKlJRb2Ay4ArgGuaJemAYcmuRvoP7NkLHwI+FaS2XT3/Gxgd+ALwI+SzKRLGvypv2FVXZ/kJ8Bs4Crg4vkQz1HASUmmA7No96WqZiY5upVdR3duzIB3Ad9NchddsmXA4XTbcGa2A2hvArYFTgBeQbcF6vcMkhQZwjOBI5IMJAU/NUzdf6V7H5em26L0riHq7Qp8O8mdwJnAra38YOC4JG8FzuCRVTIjqqqbkuwGHN9i/SvwqtG2lyRJkiQtHFLlbgkt2JIsW1UDvxK1N/CMqvrweMex/POfU5t/5RPzrb+fvfmDI1eSJEmSJI2LJDOqamp/+QK94kVqXp/kU3Sf9+uAXSY2HEmSJEnSwsLEyygkWZvul3x63VtVG01EPKP1RIw7yQnAc/uKP9l/OG9fm2/R93POwNerarifmn5YVR1N+5UjSZIkSZLGk4mXUaiqOcCUiY5jbj0R466q7eahzQfGIhZJkiRJksaaiRdpnLxghad7LoskSZIkLWQW9J+TliRJkiRJmjAmXiRJkiRJksaIiRdJkiRJkqQx4hkv0jj5w81/5w3HTXvc/Zz8ll0edx+SJEmSpPHhihdJkiRJkqQxYuJFkiRJkiRpjJh4kSRJkiRJGiMmXiRJkiRJksaIiRdJkiRJkqQxYuLlSS7JLklWHcfx9k2y13zu8yNJln4c7bdKsukox3nnvI4zTL8/TvKC+d2vJEmSJOnJz8TLk98uwLglXsbIR4B5TrwAWwHDJl6SLAa8G/jh4xhnKIcAnxiDfiVJkiRJT3ImXp4kkkxOcmnP673a66nAUUlmJVkqybVJvpTkvCTTk6yf5NQkf0yy+zD9L5vk9CQzk8xJsk3Ptc8kuTLJr4AX9pQ/P8mvklzS2q2ezgFJLm397NjqbpXkzCTHJrkiyVGt7ofoEkdnJDmj1X11i39mkmOSLNvKr03yhZ4Y10wyGdgd2LPdgy2GmOIrgJlV9UDr68wk+ye5MMnvB9q1+3xOG2PmwEqaoeJvfZ8DbN2SO5IkSZIkPczEy5PbscB0YKeqmlJVd7fy66tqE7qEwDRge2BjYL9h+roH2K6q1gdeDny1JUY2AN4GrAe8Gdiwp81RwLeqal26FSc3tDpTgHWBrYEDkjyj1V+PbnXLWsDzgM2q6iDgL8DLq+rlSVYC9gG2brFMBz7aM+bfWvkhwF5VdS1wKHBguwfnDDG/zYAZfWWLVdVLW0yfb2V/BV7VxtgROKin/mPiB6iqh4A/tDk/SpLdWgJs+n233T5EaJIkSZKkBZWJlwXTie1xDnBBVd1eVTcB9yRZYYg2Ab6UZDbwK+CZwCrAFsAJVXVXVd020HeSScAzq+oEgKq6p6ruAjYHflRVD1bVjcBZPJKsubCq/twSFbOAyYPEsTFdYuO3SWYB/wqs1nP9+PY4Y4j2Q3kGcFNf2WB9LQ58O8kc4JgWy4Dh4v8rg2z5qqrDqmpqVU19ynKT5iJcSZIkSdKCwK0RTx4P8OhE2ZLD1L23PT7U83zg9VDv+U7AysAGVXV/kmt7xqhB6meQsuHKe+MCeHCIWAL8sqrePkIfQ7Ufyt089p4N1teewI10q1cWoVsJ1F9/sPGXbGNIkiRJkvQwV7w8edwIPD3J05IsAbyhld8OzI+lFMsDf21Jl5fzyCqTs4Ht2vkxk4A3ArTVL39Osi1AkiXaLxOdDeyYZNEkKwMvAy4cYezeOZwPbJbk+a3fpZOsMRfth/I74Pkj1IHuPtzQVrXsDCw6ijYAawCXjbKuJEmSJGkhYeLlSaKq7qc7o+UC4GTginZpGnDowOG6j2OIo4CpSabTrX65oo07EziabmvNcXTnxgzYGfhQ2550LvBPwAnAbOAS4NfAJ6rq/40w9mHAL5Kc0bZE7QL8qPV7PrDmCO1PoksODXe47i/okkAjORj41yTn0yVT7hypQZJVgLur6oZR9C9JkiRJWoikarBdJNKCJ8kJdImgq+Zzv3sCt1XVd4art8Lqz63Nv/L54aqMyslv2eVx9yFJkiRJmr+SzKiqqf3lrnjRwmRvukN257dbgO+NQb+SJEmSpCc5D9ddyCRZG/h+X/G9VbXRRMQzvyX5DPDWvuJjquqLVXUlcOX8HrOqjpjffUqSJEmSFgwmXhYyVTUHmDLRcYyVqvoi8MWJjkOSJEmSJDDxIo2b56/4NM9nkSRJkqSFjGe8SJIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIY8YwXaZz84eabecOxP5mntidvv8N8jkaSJEmSNB5c8SJJkiRJkjRGTLxIkiRJkiSNERMvkiRJkiRJY8TEiyRJkiRJ0hgx8SJJkiRJkjRGTLxMoCT7JtlrHtrtl2Tr+RTDVklOnh99jackr0syPcnvklyR5L9a+bQk2090fJIkSZIkgT8n/aRUVZ8b7zGTLFpVD47zmItV1QODlL8E+Cbw+qq6IsliwG5z0e+4z0WSJEmStHByxcs4S/KZJFcm+RXwwla2epJTksxIck6SNZMsn+TaJIu0OksnuT7J4r2rOpJsmOTcJJckuTDJpCSLJjkgyUVJZif5txHCWi7JCUkuT3Joz5h3tNU1FwCbJPlokkvbv4+0Op9I8qH2/MAkv27PX5nkBy2Waa3NnCR7DjXnVj4tyX8nOQPYf4h4PwF8saquAKiqB6rq4J7rL2v35Oqe+7RVkjOS/BCYk2TJJEe0mC5O8vJWb5ckP01yUpJrknywzfviJOcneepw8UuSJEmS1MsVL+MoyQbA24D16O79TGAGcBiwe1VdlWQj4OCqekWSS4AtgTOANwKnVtX9SQb6ewpwNLBjVV2UZDngbmBX4Naq2jDJEsBvk5xWVdcMEdpLgbWA64BTgDcDxwLLAJdW1eda7O8CNgICXJDkLOBs4GPAQcBUYIkkiwObA+cAU4BnVtVLWswrtDEfM2fgFe3aGsDWw6xKeQnw1WFu9TPa+GsCJ7a5DMzzJVV1TZKPAVTV2i1pclqSNXr6Xw9YEvgD8MmqWi/JgcA7ga+NEP/DkuxGW42z1EorDROyJEmSJGlBZOJlfG0BnFBVdwEkOZHuy/2mwDEDCRVgifZ4NLAjXeLlbXRf7nu9ELihqi4CqKrbWr+vBtbpOetkeeAFwFCJlwur6urW9kd0SYtjgQeB41qdzVvsd7Z6x7f5HAJskGQScC9dMmlqu/Yh4AbgeUm+AfyMLsGx7DBzBjjmcW4F+mlVPQRcnmSVvnkO3IPNgW8AtO1K19ElfADOqKrbgduT3Aqc1Mrn0N3XkeJ/WFUdRpekYYXVV6/HMSdJkiRJ0pOQiZfx1//lexHglqqaMkjdE4H/bNtbNgB+3Xc9g/Q3UL5HVZ06jzENvL6nJwESBtFW4FxLtxrmXGA28HJgdeB3VVVJ1gVeA3wA2AH4CEPPGeDOEeK9jO5+XDLE9Xt7nvfGfecQ5cO1f6jn9UN0fzPDvWeSJEmSJD3MM17G19nAdkmWaitE3gjcBVyT5K0A6awLUFV3ABcCXwdOHmQVyBXAqkk2bG0npTto9lTgfW3LD0nWSLLMMHG9NMlz29kuOwK/GSL2bdtZM8sA29FtJRq4tld7PAfYHZjVki4rAYtU1XHAZ4H128qcQec8SgcAnx7YGpRkkSQfnYv2AzHv1NqvATwHuHI0DedD/JIkSZKkhYSJl3FUVTPptg/NotvCM5C42AnYtZ3pchmwTU+zo4F/aY/9/d1Hlyj5Rmv7S7qtS4cDlwMzk1wK/A/Dr246D/gycCnddqQThoh9Gl0i6ALg8Kq6uF0+h+5clfOq6kbgnp65PRM4M8ms1v5To5jzsKpqNt2qmR8l+V2L+xmjbd8cDCyaZA7dvd2lqu4doU2veY5fkiRJkrTwSJXHTkjjYYXVV6/N9//PeWp78vY7zOdoJEmSJEnzU5IZVTW1v9wVL5IkSZIkSWPEw3UXEknWBr7fV3xvVW00EfGMRpJ3AR/uK/5tVX1gIuKRJEmSJGluudVIGidTp06t6dOnT3QYkiRJkqQx4FYjSZIkSZKkcWbiRZIkSZIkaYyYeJEkSZIkSRojJl4kSZIkSZLGiL9qJI2TP9x8K9sc+/N5avu/2//zfI5GkiRJkjQeXPEiSZIkSZI0Rky8SJIkSZIkjRETL5IkSZIkSWPExIskSZIkSdIYmefES5Jdkqw6P4OZSEl2T/LOiY5jMEmuTTInyaz2uE2SVyc5L0lanUXb9U2H6eeOnuf/nOSqJM9JcmprO/DvL0ku6Km7V5Irklya5JKB+5TkzCRXtrLfJnlhT/mfBmJrZT/tHX+Q2HZJ8qO+spWS3JTkP5Ps31O+WpKrk6zQV3+rJCe356skObnFdnmSn7fyRZIc1OYyJ8lFSZ47TFxnJpnank9u9+w1baxbk1yc5HdJPj9UH5IkSZKkhdfjWfGyC7DAJF6q6tCqOnKi4xjGy6tqCrA9cFBVnQZcB+zaru8BXFRV547UUZJXAt8AXltVf6qq11TVlNb/ZsBtwD6t7u7Aq4CXVtVLgJcB6elup6paF/gecEBP+S2tL1qC5BkjhHU88KokS/eUbQ+cCOwHbJPkRa3868Bnq+qWYfrbD/hlVa1bVWsBe7fyHek+t+tU1drAdi3WYSV5FnAq8LGqOrUVn1NV6wFTgX9JssFI/UiSJEmSFi4jJl7af+W/tOf1Xu31VOCotkJiqbYq40ttFcb0JOu3lRR/bF/ehxvjE231wSVJvtzK3ttWI1yS5LiBL+RJpiU5JMkZbdXDlkm+21YdTOvp844kX00yM8npSVYeod99k+zVnm+YZHabywED82+rMo5Pckpb+fCVEea1a5Lft1UT307yzVa+chv7ovZvIEGxbJIj2r2YneQtg3S7HHBze74n8KkkLwY+CHyy9bNKkhPaHC9JzyqYJFsA3wZeX1V/HKT/rwM/r6pfttefBt5fVbcBVNWtVfW9QdqdDTy/5/WPgbe152+mS6z03ptHveet/7OBN/ZUexvwo6q6G/gocHCS1wGTquqo1s9r063G+U0bZ8AzgD8PvKiq2T3lN1TVQ638z1V1c5I35ZEVP1cmuaanr38CTgP2qaoT+ydeVXcCM4DVB7kvkiRJkqSF2LyueDkWmE632mFK+2IMcH1VbQKcA0yjW7GwMd3qg0G1L9LbAhu1lRMDyYzjq2rDVvY7HlnZAbAi8Aq6xMNJwIHAi4G1k0xpdZYBZlbV+sBZwOdH0e+AI4Dd21we7Ls2hW7VxNrAjkmePcS8VgU+2+b/KmDNnstfBw6sqg2BtwCHt/LPArdW1dpVtQ7w6542Z7QE0Fm01ShVdQPwNeA84D+q6h+t7kHAWW2O6wOXtfIlgP8Ftq2qKwaJeTu6hNqn2utJdEmOwRI0/d4IzOl5fTrwsiSL0iVQju4ZZ6j3/Eet7sD9WwM4o83158A/gCOB97c6S9Ilkd4IbEGXIBnwLeA7LUH3mTyyLe4nwBtbguWrSdZr/Z/Ys+rnEuC/evo6EvhmVR0z2MSTPI3ufb5skGu7tUTk9Ptuu3XQGydJkiRJWnDN78N1B1YDzAEuqKrbq+om4J70ncfRY2vgiKq6C6AnefCSJOckmQPsRJdYGXBSVVUb58aqmtNWMFwGTG51HuKRL/s/ADYfRb8D22Im9WzZ+WFfvKe3VR/3AJcDqw0xr5fSJT/+UVX3A71f2rcGvplkFt09W64lObamSxjQ7sXNPW1e3rb6rN3aLtvKvwUsWlXTeuq+Ajik9fFgVQ18478fOJdBkk1JnkmXsHlHVd07UAzUEPMbcFSbx2bAXj3lDwK/oUtSLVVV1/bNf7D3/GRg8yTLATsAx1ZVb+LrW3Tbqa5sr9cErqmqq9rn4QcDFdt2oOfRJWbWBC5OsnJV/Rl4IV1y6SHg9HRbrwbuwyeAu6vq4fcB+BWwcx69DQpgiyQX062G+XJVPSbxUlWHVdXUqpr6lOWWf8zNkyRJkiQt2EaTeHmgr96Sw9Qd+ML+UM/zgdeLDdFmqC/304APtnM4vtA37ryMMzDGcP0OxDOc3vEeHGa84fpZBNhkYIVFVT2zqm5nFImOtvrkRmCt9vqhkdr0eIguobFhkk8/HGgSujNavlxVl/eMdRtwZ5LnDdPnwKqnbavq+r5rP6Y7S+YnfeWDzrOtnDqF7tyVt9GtgOmP/6H+ZkMF1pJeP6yqnYGL6M6noarurapfVNXHgS/Rrb4ZOPvmrUD/1rivABcAxyTpfb/Pqar1qmqDqjp0qDgkSZIkSQuv0SRebgSenuRpSZYA3tDKbwcmzYcYTgPe3XPWylNb+STghiSL061MmVuL0G11AngH3eqLEfttq0xuT7JxK3pbf51RuhDYMsmK7ct673ktp9GdyQJAz/ao/vIV+ztN8nTguXQH6w7ldOB9rf6ibQUJAG2VyRuAnZIMrHzZC7inb5XHgP8EvjXQR5Llkuw2zNi9zmnt+xMoQ73ntLofBVYBzh+h/yuA5yYZOFvl7QMXkryip/9JdOev/Cnd2UOrtvJFgHWA65KsBhwM7NCzda7XnnSHDn+nJaokSZIkSRrRiImXtk1mP7r/4n8y3Zdd6FaOHNrOylhqXgOoqlPotttMb1tWBrarfLaN+cueMefGncCLk8yg23ozcM7MaPrdFTgsyXl0qzPm+nCOqvo/utUUF9BtVbm8p58PAVPbAbqX88gKi/8AVkz72Wbg5T1dntHuzxnA3lV14zDDfxh4edtONYO+7VRta89rgX2SbNPGfVEe/ZPSZ7Tqh7QxL+o5Y+auUd6Dqqr/qqq/9ZUP9Z5Dl5RZFTi6bR8arv97gN2An7XDdXuTURu0/mfTnYFzeFVdBDwdOKnNZTbdiq5v0v1K19OAE9r8f94/F+Bf6Q7nHfZQZUmSJEmSBmSE77ZPWknuqKplR645aNtlq+qO9nxv4BlV9eF57aeteDkB+G5VnTAvMenJb4XVX1Bb7v/1eWr7v9v/83yORpIkSZI0PyWZUVVT+8uHOp9kYff6JJ+iuz/X0a2GmBf7Jtma7hyZ04CfzpfoJEmSJEnSk8K4JV6SrA18v6/43qraaCzGm9fVLq3t0fT8/PFIklxA91PNvXauqr0Gqy9JkiRJkhYO45Z4qao5wJTxGm88jVXySJIkSZIkPbm51UgaJ89fcXnPapEkSZKkhcxofk5akiRJkiRJ88DEiyRJkiRJ0hgx8SJJkiRJkjRGTLxIkiRJkiSNEQ/XlcbJH2++g+2O+81ctzvhLZuPQTSSJEmSpPHgihdJkiRJkqQxYuJFkiRJkiRpjJh4kSRJkiRJGiMmXjSsJLskWXWi45AkSZIk6cnIxItGsgvwhEy8JJmvh0OnM6q/iSSLzs+xJUmSJEkLJhMvAiDJ5CSX9rzeq72eChyVZFaSpZJcm+RLSc5LMj3J+klOTfLHJLsP0/8zkpzd+rk0yRat/O1J5rSy/Xvq39HzfPsk09rzaUn+O8kZwP5Jnp/kV0kuSTIzyeqt3seTXJRkdpIvjDDv3yU5GJgJPDvJIW1ul/W2bXP/XJLfAG9N8up2H2YmOSbJsnN94yVJkiRJCzQTLxrOscB0YKeqmlJVd7fy66tqE+AcYBqwPbAxsN8wfb0DOLWqpgDrArPaFqb9gVcAU4ANk2w7irjWALauqo8BRwHfqqp1gU2BG5K8GngB8NLW7wZJXjZMfy8Ejqyq9arqOuAzVTUVWAfYMsk6PXXvqarNgV8B+7Q41qe7Tx8dReySJEmSpIXIfN2qoYXGie1xDrBsVd0O3J7kniQrVNUtg7S5CPhuksWBn1bVrCSvAM6sqpsAkhwFvAz46QjjH1NVDyaZBDyzqk4AqKp7Wj+vBl4NXNzqL0uXiDl7iP6uq6rze17vkGQ3ur+PZwBrAbPbtaPb48at/LdJAJ4CnNffcetnN4ClVlplhGlJkiRJkhY0Jl404AEevQJqyWHq3tseH+p5PvB60M9UVZ3dVp28Hvh+kgOA24YZo4aJ5c72mCHaBvjPqvqfYfofrD+SPBfYC9iwqm5uW5yWHKRugF9W1duH67iqDgMOA1hx9TVruLqSJEmSpAWPW4004Ebg6UmelmQJ4A2t/HZg0uPtPMlqwF+r6tvAd4D1gQvotvKs1A6rfTtw1kA8SV7UDrvdbrA+q+o24M8D25OSLJFkaeBU4N0DZ64keWaSp48y1OXokiu3JlkFeN0Q9c4HNkvy/DbG0knWGOUYkiRJkqSFhCteBEBV3Z9kP7pkyDXAFe3SNODQJHcDmzyOIbYCPp7kfuAO4J1VdUOSTwFn0K0g+XlV/W+rvzdwMnA9cCnddqHB7Az8T4v9fuCtVXVakhcB57VtQHcA/wL8daQgq+qSJBcDlwFXA78dot5NSXYBftQSVdCd+fL7kcaQJEmSJC08UuXuB2k8rLj6mrXVVw6f63YnvGXzMYhGkiRJkjQ/JZnRfqjlUdxqJEmSJEmSNEbcaqT5KsnawPf7iu+tqo0mIp4BSZ4GnD7IpVdW1d/HOx5JkiRJ0sLBxIvmq6qaA0yZ6Dj6teTKlImOQ5IkSZK0cHGrkSRJkiRJ0hhxxYs0TlZfcVkPypUkSZKkhYwrXiRJkiRJksaIiRdJkiRJkqQxYuJFkiRJkiRpjHjGizROrr75HnY47vK5bveTt6w1BtFIkiRJksaDK14kSZIkSZLGiIkXSZIkSZKkMWLiRZIkSZIkaYyYeJEkSZIkSRojJl4kSZIkSZLGyIQnXpLskmTV+dDPVklOnh8xzW9JJie5dD73eWaSqYOUfzHJrJ5/v0/yYJJl272+qZVfnuS9rc0uSb6Z5NVJzkuSVr5oq7vpaOeW5L1JZiZZMcmJSXbuufbtJB8foa+7++J/ylBxD9PPtUlW6nk94mcjye5J3tmeb5zkgjbe75Ls28rflGTv4fqRJEmSJKnXE+HnpHcBLgX+MsFxzDdJFquqByZi7Kr6DPCZnliOAn5SVXe0fMrRVfXBJE8HLktyYk/b05K8G9gVOBzYA7ioqs4dzdgtybIH8IqqujnJh4AzkpwErAVsBLx/hG7+WFVT+vodNO6qunE0cY1GVR3a8/J7wA5VdUmSRYEXtjonAicO1l6SJEmSpMGM24qXQVZG7NVeTwWOaqsLlmqrFb7UVl5MT7J+klOT/DHJ7qMca8MkFyd5XpINkpyVZEbr5xlJVk8ys6f+C9r1lyY5vpVt01ZfPCXJkkmubuVTkpyfZHaSE5Ks2MrPbHGfBXy4jXtJkvOAD4wQ7y5J/jfJKUmuTPL5Ye7Zvj1N35rkwraqZYtB+v0X4PnAvv3XquqvwB+B1fou7Ql8KsmLgQ8Cn2x9Pbe9Jxcl+fckd/SNtQOwN/DqqvpbG+Na4DDgK8DBwAer6v62kuaA1tfsJP823P0ZKu4kh7TPyGVJvjBS2ySLtM/XCj1lf0iySpJ9k+zVip8O3NDGe7CqLm91d0nyzfZ8tSSnt/hPT/Kc0c5BkiRJkrTwmOitRscC04GdqmpKVd3dyq+vqk2Ac4BpwPbAxsB+I3WYblvMocA2wPXAN4Dtq2oD4LvAF6vqj8CtSaa0Zu9q48wE1mtlW9CtxNmQbqXGBa38SOCTVbUOMAf4fM/wK1TVllX1VeAI4ENtHqPxUmAnYApdQuUx24gGsVhVvRT4SF8cJJkMfJnu3j5m9U2S5wHPA/7QW15VNwBfA84D/qOq/tEufR04pKo2BP5fX3erAd+kS7r0X/sv4LXAZVV1divbFbi19bUh8N4kz23XVs8j24y+NULcn6mqqcA6wJZJ1umv3ze3h4D/BbZrfW0EXDvIypkDgStbYu3fkiw5SHffBI5sn4OjgIMGGzPJbi05NP3e2/4xWBVJkiRJ0gJsohMvQxnYzjEHuKCqbq+qm4B7elcrDOJFdCss3lhVf6LbIvIS4JdJZgH7AM9qdQ8H3tW2kuwI/LAlKP6Q5EV0iZD/Bl5Gl4Q5J8nydMmVs1of32vXBxwNMEi9749izr+sqr+35NPxwOajaHN8e5wBTB4obHP6AfDZqvpDX5sd2734EfBvPYmVXt8CFq2qaT1lm7U28Nj53AT8CdhhkL7WAQKsmWTg8/Zq4J0tjguApwEvaNf+2JJwU6qqd6XQYHHv0FYuXQy8mG47E0ANEsdA2dF07zfA29rrR1es2o9uJdZpwDuAUwbpbxPgh+359xni/aqqw6pqalVNXWK5pw5WRZIkSZK0ABvPM14e4NGJnsFWEQy4tz0+1PN84PVwMd/Q+l2P7syY0K20GGzVyXF0q0R+Dcyoqr+38nOA1wH3A7+iWwmzKLDXIH30u7M9hsG//A+nv34x8j0buDcP8uj7sg9wQ1UdMcg4R1fVB4cNpOqhJMMlL/rdRXfPfpPkr1V1FHRbe+i2GO0M7A68jy6pE2CPqjq1t5O2Smcoj4q7rZDZC9iwnSczjUfuz9+BFYG/tddP7Xl+HvD8JCsD2wL/MdhgbVXUIUm+DdyU5GnDxAZz/35LkiRJkhYC47ni5Ubg6UmelmQJ4A2t/HZg0nwa4xbg9cCXkmwFXAmsnGQTgCSLt7NLqKp7gFOBQ+i2BQ04m27rznltlc3TgDXpEji3Ajf3nKeyM3AWfarqFrqtTAOrIHYaReyvSvLUJEvRJQR+y9D3bEhJNqY7sHi3UYw5N35Lt0IEBplPu1evpbv3r2nF/wZcVVVnAh8FPtESHqcC70uyeIt5jSTLzGU8y9Elum5Nsgpd4mfAmXTvzcDqn38BzmhxFnAC3Wqm3/Uk3B6W5PVJd6Iv3UqcB+k+W73O5dH34zdzGb8kSZL+f3t3Hm5JVd57/PtjkKkBZRTQiCCDjA00ICiISIwaB1AMKA6gCcHrgAPm6jUqYoISvXIxoggoHQkoAqJIFFAEQcCmBxq6mRwAA0pAZWwmGd77R60jm8OZuul9Tg/fz/OcZ1etWrXWW7WL7dOva62SpKXAuI14aYuqHkE3teQm4Pp2aCpwXJIH6aZvPN1+bk/yWuBHwDvp1of5Upv+sxzd+iXXtOqnAG+gm1IyYBqwLl0CBuBq4I72D3aAd7R4VwZupFsfZigHAd9I8gBdomE0P6ebsvICumlPMwCGuWcj+TSwMt3bhHrL3ziGc0dyKHBqkkPpRgs9RVXdlOR1wA+TvIFuYd4XtWO/T3IM3UK776KbGjWrJTj+QJdsGrP2xqEr6b7LG+kSQwM+Qzda5Sq60TXn0k29GnAaMJ0uQTWUtwFHt+/uUbp1ch5r93PgOXg/3ff7kRb/cM+BJEmSJGkplifyCUuf9hab1avqExMcx4HAlNGmAC1KksyrqkkTHcd4SvJhYLWq+tSolYewxsZb1V7/9p35Pu87b9xi9EqSJEmSpAmVZGZ7AcyTjOcaL4uUJGcBGwN7TnQsWvSle5X5gXQjpCRJkiRJGpPFLvGSZGue+ladh6tq5/lpp6r2WXhRjU1b++SoQcU3tVimjnc8T8fSNtqlqo6je025JEmSJEljttglXqpqDjB5ouNYEO0tPmNZ70WSJEmSJC0BFrvEi7S42uhZK7peiyRJkiQtZcbzddKSJEmSJElLFRMvkiRJkiRJfWLiRZIkSZIkqU9c40UaJ7+/+xEOP+v3833e4fus34doJEmSJEnjwREvkiRJkiRJfWLiRZIkSZIkqU9MvEiSJEmSJPWJiRdJkiRJkqQ+MfEiSZIkSZLUJyZeFhFJDk9y2AKcd0SSvRZSDHskOWdhtDWGvi5KMmWE4+9MMifJ1UnmJnn9eMQlSZIkSdLC5OukF3NV9cnx7jPJslX1WB/bfw7wcWD7qronySRg7X71J0mSJElSvzjiZQIl+XiSG5L8BNislW2c5NwkM5NckmTzJKsnuTnJMq3OykluSbJ8kqlJ9m3lOya5LMlVSa5IsmqSZZN8Psn0NnrkH0cJa7UkZyW5NslxPX3Oa6NrpgG7JPlka3NukuOTpNW7KMlRrf9fJtmtla+U5NsthtOAlUaIYR3gPmAeQFXNq6qbWjv/0Pq9KsmZSVZu5VOTfDXJhUluTPLSJN9Icl2Sqa3Osq3e3Daa5oM9MU9p22slubltH5jku+37+FWSf+v57t7Vru+iJCck+fKYvnRJkiRJ0lLFxMsESbIDsD+wHfAGYMd26HjgfVW1A3AY8JWquge4Cnhpq/Na4LyqeqSnvWcApwGHVtW2wF7Ag8C7gHuqasfWxz8kef4Ioe0EfBjYGti4xQawCjC3qnauqp8DX66qHatqK7okymt62liuqnYCPgB8qpW9G3igqrYB/hXYYYQYrgJuB25KclKS1/Yc+27rd1vgunZ9A54F7Al8EPgBcDSwJbB1ksnAZGCDqtqqqrYGThohhgGTgf3a/dgvyXOTrA98AngR8NfA5sOdnOTgJDOSzHjg3j+NoTtJkiRJ0pLExMvE2Q04q6oeqKp7gbOBFYFdgdOTzAa+BqzX6p9GlwCALmFz2qD2NgNuq6rpAFV1b1U9CrwCeHtrbxqwJrDJCHFdUVU3tqlE3wJe0sofA87sqfeyJNOSzKFLdmzZc+y77XMmsGHb3h34zxbb1cDVwwXQ+n4lsC/wS+DoJIe3w1u1kUBzgAMG9fuDqipgDnB7Vc2pqseBa1ocNwIbJfn3JK8E7h3hPgy4oKruqaqHgGuB59Elp35WVXe25NfpI1zL8VU1paqmrLzammPoTpIkSZK0JHGNl4lVg/aXAe6uqslD1D0b+GySNehGi/x00PEM0d5A+fuq6rwFjGlg/6GBdV2SrAh8BZhSVbe0pMiKPec83D4f48nP2FDxDR1El0C5ArgiyY/pRqccDkwF9q6qq5IcCOwxRL+P92wP7C9XVXcl2Rb4G+A9wN8B7wQe5YkkZO919LbZez0Z63VIkiRJkpZujniZOBcD+7S1T1almz70AN30mjcBpLMtdOuc0CUijgHOGWJx2+uB9ZPs2M5dNclywHnAu5Ms38o3TbLKCHHtlOT5bW2X/YCfD1FnIDnxx3QL3+47xus9oMWwFbDNcBWTrJ9k+56iycBv2/aqwG3teg4YQ7+97a4FLFNVZ9JNFRro42aemPo0lmu5Anhpkme1e/zG+YlDkiRJkrT0cMTLBKmqWW2R2dl0SYVL2qEDgK8m+WdgeeDbdGueQDe96HSePMpjoL0/J9kP+PckK9Gt77IXcCLdNJtZbQHcPwB7jxDa5cDn6NY0uRg4a4i+7k5yAt2UnpuB6WO45K8CJyW5ul3zFSPUXR74QltL5aEW8yHt2Cfopkz9tvW/6hj6HrBBi2Eg4fix9vkF4DtJ3sZTRxI9RVX9LsmRLY7f001Bumc+4pAkSZIkLSXSzeiQND+STKqqeW3Ey1nAN6rqKUmqXuu/YNs6+PM/mu++Dt9n/QWMUpIkSZI0XpLMrKopg8udaiQtmMPbgsVzgZuA701oNJIkSZKkRZJTjZZCSbYGTh5U/HBV7TwBsUwDVhhU/LaqmjPescyPqjpsomOQJEmSJC36nGokjZMpU6bUjBkzJjoMSZIkSVIfONVIkiRJkiRpnJl4kSRJkiRJ6hMTL5IkSZIkSX1i4kWSJEmSJKlPfKuRNE7uvOtRTjnzD/N1zgFvXLtP0UiSJEmSxoMjXiRJkiRJkvrExIskSZIkSVKfmHiRJEmSJEnqExMvkiRJkiRJfWLiZQmQ5MAk6090HAtLkkOSvH0ht5kkP02y2kJud+0k5y7MNiVJkiRJSw4TL0uGA4ElJvFSVcdV1TcXcrOvBq6qqnsXZqNV9QfgtiQvXpjtSpIkSZKWDCZeFiNJNkwyt2f/sLY/BTglyewkKyW5OcmRSS5PMiPJ9knOS/KbJIeM0sc/JZmT5Kokn2tl/5Bkeis7M8nKrXxqkq8muTDJjUlemuQbSa5LMrWnzXlJ/m+SWUkuSLL2KO0enuSwtr1jkqvbtXx+4PrbKJ/vJjk3ya+S/Nsot+8A4Ps99/G6JCckuSbJ+UlWGsO1finJZe1a9+1p+3utfUmSJEmSnsTEy+LvDGAGcEBVTa6qB1v5LVW1C3AJMBXYF3gRcMRwDSV5FbA3sHNVbQsMJDO+W1U7trLrgHf1nPYsYE/gg8APgKOBLYGtk0xudVYBZlXV9sDPgE+Nod0BJwGHtGt5bNCxycB+wNbAfkmeO9y1AS8GZvbsbwIcW1VbAncDbxxDTOsBLwFeA3yup3wGsNtQnSY5uCW/Ztx7759GCE+SJEmStCQy8bLkOrt9zgGmVdV9bVrMQ0meOcw5ewEnVdUDAFV1ZyvfKsklSebQjezYsuecH1RVtX5ur6o5VfU4cA2wYavzOHBa2/5PuuTFaO3S4ly1qi5rRacOiveCqrqnqh4CrgWeN/ztYI2quq9n/6aqmt22Z/bEOlJM36uqx6vqWmDdnvI7GGaqV1UdX1VTqmrKaqutOUJ4kiRJkqQlkYmXxcujPPk7W3GEug+3z8d7tgf2lxvmnAA1RPlU4L1VtTXw6UH9Lkg/A32M1O5APCPp7e+xEfoDeDRJ770b7tyRYuo9pze2FYEHkSRJkiRpEBMvi5fbgXWSrJlkBbopLwD3AasuhPbPB97Zs67JGq18VboFZJdnwdYyWYZuqhPAW4Cfj6XdqroLuC/Ji1rR/gvQ94AbgI3GUG9BrnVTYO6otSRJkiRJS52RRghoEVNVjyQ5ApgG3ARc3w5NBY5L8iCwy9No/9y2LsuMJH8Gfgj8H+ATrc/f0k0pmt8kz/3AlklmAvfQrcvCGNt9F3BCkvuBi9r5C+K/gD2AX49Sb0Gu9WWtfUmSJEmSniTd8hxS/ySZV1WTFvDcSVU1r21/FFivqg5dgHbWA75ZVX+9IHGM0vbFwOvbCJ1hbbTx5PrMv/14vto+4I1rP53QJEmSJEnjJMnMqpoyuNwRL1rU/W2Sj9E9q78FDlyQRqrqtvb66NWq6t6FFVx7NfYXR0u6SJIkSZKWTiZelkJJtgZOHlT8cFXt3I/+FnS0Szv3NJ54I9KokkwDVhhU/Lb2tqXvLGgcw2lvivrewm5XkiRJkrRkMPGyFKqqOcDkiY6jH/qVPJIkSZIkaUGYeJHGyRrPWs41WyRJkiRpKePrpCVJkiRJkvrExIskSZIkSVKfmHiRJEmSJEnqExMvkiRJkiRJfeLiutI4uffOR/nJqX+Yr3P2eouL8UqSJEnS4swRL5IkSZIkSX1i4kWSJEmSJKlPTLxIkiRJkiT1iYkXSZIkSZKkPjHxsoCSHJhk/YmOY6K1+/DlYY7Nm8+2Dk9y2AjHj0iy1/zG+HQlOTHJFqPUuSjJlPGKSZIkSZK0ePCtRgvuQGAu8PsJjuMpkixXVY9OdBwLW1V9coL6/fuJ6FeSJEmStPhzxMsokmyYZG7P/mFtfwpwSpLZSVZKcnOSI5NcnmRGku2TnJfkN0kOGaH99ZJc3NqZm2S3Vv7mJHNa2VE99ef1bO+bZGrbnprki0kuBI5K8oIkP0lyVZJZSTZu9T6SZHqSq5N8epRr/16SmUmuSXJwT/lBSX6Z5GfAi3vKn9+uf3qSzwxqa8h+k3w8yQ1JfgJsNko8U5Ps27Y/l+Ta1t4XRjjnTe0eXpXk4la2YpKT2v29MsnLWvmySb7Qyq9O8r5W/pfRLEm+2r7fa0a7f5IkSZIkOeJlwZwB7AEcVlUzAJIA3FJVuyQ5GphKl5RYEbgGOG6Ytt4CnFdV/5pkWWDlNoXpKGAH4C7g/CR7V9X3RolrU2CvqnosyTTgc1V1VpIVgWWSvALYBNgJCHB2kt2r6uJh2ntnVd2ZZCVgepIzgWcAn26x3QNcCFzZ6h8DfLWqvpnkPQONDNcvcD+wP7Ad3bM4C5g5yjWSZA1gH2Dzqqokzxyh+ieBv6mq3/XUew9AVW2dZHO6+7spcBDwfGC7qnq09TPYx9s9WRa4IMk2VXX1CLEeDBwMsM5azxnt0iRJkiRJSxhHvCxcZ7fPOcC0qrqvqv4APDRCcmA6cFCSw4Gtq+o+YEfgoqr6Q5sydAqw+xj6P70lXVYFNqiqswCq6qGqegB4Rfu7ki7JsTldQmQ4709yFfAL4Lmt7s49sf0ZOK2n/ouBb7Xtk3vKh+t3N+Csqnqgqu7lifs3mnuBh4ATk7wBeGCEupcCU5P8A7BsK3vJQHxVdT3wW1rSCjhuYJpWVd05RHt/l2RWu5YtgRHXfqmq46tqSlVNWX3VNcd4eZIkSZKkJYUjXkb3KE9OUK04Qt2H2+fjPdsD+0Pe66q6uI3++Fvg5CSfp0ssDKdGiOX+9plhzg3w2ar62gjtdxWTPegSEbtU1QNJLurpr4Y5bbhjQ/ab5AOjtDV0B91olJ2Al9ONmHkvsOcwdQ9JsjPd/Z2dZDIj359h40nyfOAwYMequqtN8xrpeZAkSZIkLeUc8TK624F1kqyZZAXgNa38PmDVp9t4kucBd1TVCcDXge2BacBLk6zVprS8GfjZQDxJXphkGbrpNk/RRo/cmmTv1scKSVYGzgPemWRSK98gyTrDhLY6cFdLumwOvKiVTwP2aPdjeeBNPedcSpcIATigp3y4fi8G9km3Rs6qwGtHuV208ycBq1fVD4EPAJNHqLtxVU1rC/P+kW7kzsUD8bUpRn8F3ACcDxySZLl2bPBUo9Xoklv3JFkXeNVY4pUkSZIkLb0c8TKKqnokyRF0CYebgOvboanAcUkeBHZ5Gl3sAXwkySPAPODtVXVbko/RrZ8S4IdV9f1W/6PAOcAtdG9VmjRMu28DvtZifwR4U1Wdn+SFwOVtTZp5wFuBO4Y4/1y6JMTVdEmJXwC02A4HLgduo5s6NDCF51Dg1CSHAmcONDRcv1U1K8lpwGy66T6XjOmOdQmv77e1awJ8cIS6n0+ySat3AXAV3Xd4XJI5dCOaDqyqh5OcSDfl6Or2fZwA/OVV2VV1VZIr6dbsuZEu0SRJkiRJ0rBSNd8zPSQtgE03mlxf+Zcfz9c5e71l7T5FI0mSJElamJLMrKopg8udaiRJkiRJktQnTjUaJ0m25slv+gF4uKp2noh4BiRZk24KzmAvr6o/jXc8AEmOpXtDUq9jquqkEc75OE9ebwa6tzz968KOT5IkSZKksTLxMk6qag4jLAI7UVpyZfJEx9Grqt6zAOf8K2CSRZIkSZK0SDHxIo2T1dZYzjVbJEmSJGkp4xovkiRJkiRJfWLiRZIkSZIkqU9MvEiSJEmSJPWJa7xI4+SBPz7KlSfeMeb62/39On2MRpIkSZI0HhzxIkmSJEmS1CcmXiRJkiRJkvrExIskSZIkSVKfmHiRJEmSJEnqExMvkiRJkiRJfWLiZTGQ5PAkhy3AeUck2WshxbBHknMWRlvjJcm6Sc5JclWSa5P8cAznvD/JdUlOmY9+1k9yxtOLVpIkSZK0JPJ10kuwqvrkePeZZNmqemyc+1yuqh4d4tARwI+r6phWb5sxNPe/gFdV1U3z0ffvgX3HHLAkSZIkaanhiJdFVJKPJ7khyU+AzVrZxknOTTIzySVJNk+yepKbkyzT6qyc5JYkyyeZmmTfVr5jksva6I8rkqyaZNkkn08yPcnVSf5xlLBWS3JWGz1yXE+f89rommnALkk+lGRu+/tAq/NPSd7fto9O8tO2/fIk/9limdrOmZPkg8NdcyufmuSLSS4Ejhom3vWAWwd2qurqdu6kJBckmdX6en0rPw7YCDg7yQeT7NTu2ZXtc+B7ODDJ6Ul+AJyfZMMkc8f+7UqSJEmSlhaOeFkEJdkB2B/Yju47mgXMBI4HDqmqXyXZGfhKVe2Z5CrgpcCFwGuB86rqkSQD7T0DOA3Yr6qmJ1kNeBB4F3BPVe2YZAXg0iTnjzDaYydgC+C3wLnAG4AzgFWAuVX1yRb7QcDOQIBpSX4GXAx8GPgSMAVYIcnywEuAS4DJwAZVtVWL+Zmtz6dcM7BnO7YpsNcII2yOBU5L8l7gJ8BJbXTKQ8A+VXVvkrWAXyQ5u6oOSfJK4GVV9cd2n3avqkfblK0jgTe2tncBtqmqO5NsOEz/JDkYOBjg2Ws8Z7hqkiRJkqQllImXRdNuwFlV9QBAkrOBFYFdgdMHEirACu3zNGA/usTL/nTJiV6bAbdV1XSAqrq3tfsKYJuBUTHA6sAmwHCJlyuq6sZ27rfokiZnAI8BZ7Y6L2mx39/qfbddz1eBHZKsCjxMl0ya0o69H7gN2CjJvwP/RTeSZNII1wxw+kjTmqrqvCQbAa8EXgVcmWQr4G7gyCS7A48DGwDrAv8zqInVgf9IsglQwPI9x35cVXcO13dPDMfTJY/YYsPJNVp9SZIkSdKSxcTLomvwP9KXAe6uqslD1D0b+GySNYAdgJ8OOp4h2hsof19VnbeAMQ3sP9STAAlDaCNwbqYbDXMZcDXwMmBj4LqqqiTbAn8DvAf4O+ADDH/NAPePGnCXHDkVODXd4sC7A6sCawM79MS14hCnfwa4sKr2aaNaLpqfviVJkiRJco2XRdPFwD5JVmojRF4LPADclORNAOlsC1BV84ArgGOAc4YYBXI9sH6SHdu5qyZZDjgPeHeb8kOSTZOsMkJcOyV5flvbZT/g58PEvndba2YVYB+6qUQDxw5rn5cAhwCzW9JlLWCZqjoT+ASwfRuZM+Q1j0WSPZOsPHDNdEme/6YbyXJHS7q8DHjeME2sDvyubR841n4lSZIkSRrgiJdFUFXNSnIaMJtuPZWBxMUBwFeT/DPdtJdvA1e1Y6cBpwN7DNHen5PsB/x7kpXo1nfZCzgR2BCYlW4uzx+AvUcI7XLgc8DWdMmTs4aJfSpdIgjgxKq6sm1fAnwcuLyq7k/yUM+1bQCcNLBgL/CxMVzzaHYAvpzkUbok44ltjZubgB8kmUF3j68f5vx/o5tq9CGeOopIkiRJkqRRpcplJ6TxsMWGk+uUfz5/zPW3+/t1+hiNJEmSJGlhSjKzqqYMLneqkSRJkiRJUp841UhPkmRr4ORBxQ9X1c4TEc9YJDkIOHRQ8aVV9Z6JiEeSJEmSpAEmXvQkVTUHmDzRccyPqjoJOGmi45AkSZIkaTATL9I4WXmt5Vy3RZIkSZKWMq7xIkmSJEmS1CcmXiRJkiRJkvrExIskSZIkSVKfuMaLNE7+fPsj3PJ//2fM9Z/74Wf3MRpJkiRJ0nhwxIskSZIkSVKfmHiRJEmSJEnqExMvkiRJkiRJfWLiRZIkSZIkqU9MvEiSJEmSJPWJiZelXJIDk6w/0XEMluTmJGu17csmOh5JkiRJkhaEiRcdCCxyiZdeVbXrePSTZNl+1JUkSZIkLb1MvCwlkmyYZG7P/mFtfwpwSpLZSVZqI02OTHJ5khlJtk9yXpLfJDlkhPb3SPKzJN9J8sskn0tyQJIrksxJsnGrt3aSM5NMb38vbuVrJjk/yZVJvgakp+15PX2c01P+5SQHtu2nE/eFSU4F5rSy7yWZmeSaJAf3xpHkiCTTgF2SvLVd3+wkXzMZI0mSJEkazMTL0u0MYAZwQFVNrqoHW/ktVbULcAkwFdgXeBFwxCjtbQscCmwNvA3YtKp2Ak4E3tfqHAMcXVU7Am9sxwA+Bfy8qrYDzgb+agGuZ0Hj3gn4eFVt0fbfWVU70CWl3p9kzVa+CjC3qnYG/gTsB7y4qiYDjwEHDG44ycEtETTjzvv/tACXJEmSJElanC030QFokXR2+5wDTKqq+4D7kjyU5JlVdfcw502vqtsAkvwGOL+nnZe17b2ALZK/DGhZLcmqwO7AGwCq6r+S3DWOcV9RVTf17L8/yT5t+7nAJnSJlseAM1v5y4EdgOntWlYC7hjccFUdDxwPsM1zt60FuCZJkiRJ0mLMxMvS41GePMJpxRHqPtw+H+/ZHtgf6ZkZXLe3nYHzlgF26RldA0BLXoyWmBjtGhY07vt74tiDLjm0S1U9kOSinn4eqqrHBqoC/1FVHxslZkmSJEnSUsypRkuP24F12loqKwCvaeX3AauOYxznA+8d2EkyuW1eTJuqk+RVwLOGOPe3dKNlVkiyOt2ok4VtdeCulnTZnG6q0lAuAPZNsk6LeY0kz+tDPJIkSZKkxZiJl6VEVT1Ct9bJNOAc4Pp2aCpw3MDiuuMQyvuBKUmuTnItMLDw7aeB3ZPMAl4B/PfgE6vqFuA7wNXAKcCVfYjvXGC5JFcDnwF+MVSlqroW+Gfg/Fb3x8B6fYhHkiRJkrQYS5XLTkjjYZvnblv/9YHzxlz/uR9+dh+jkSRJkiQtTElmVtWUweWOeJEkSZIkSeoTF9fVfEmyNXDyoOKH2yuWF1mLa9ySJEmSpMWbiRfNl6qaA0ye6Djm16IQ9zPWXd7pQ5IkSZK0lHGqkSRJkiRJUp+YeJEkSZIkSeoTEy+SJEmSJEl9YuJFkiRJkiSpT1xcVxonj9z+EP/zxWvHVPfZH9qiz9FIkiRJksaDI14kSZIkSZL6xMSLJEmSJElSn5h4kSRJkiRJ6hMTL5IkSZIkSX0ybomXJAcmWX+8+psISR5LMjvJVUlmJdm1lW+YZO6guocnOWwB+tgjyTlDlO+W5JrW/0pJtkzy0yS/TPKrJJ9IkjG0v1OSi5PckOT6JCcmWSXJH5M8q9VZL0kleUnPeX9IsuYwbW6bZHbP/puTPJBk+ba/dZKrk5zV4v91knva9uyB+ziozafcvyQHJzmtZ3+1JL9J8vwRrvcp9zPJ1CT7tj4+O+jY5CTXDdeeJEmSJEm9xnPEy4HAIpl4SbKw3u70YFVNrqptgY8Bnx3thIXoAOALVTW57Z8NfK6qNgW2BXYF/tdIDSRZFzgd+N9VtRnwQuBcYBIwDdilVd0VuLJ9kmQz4I9V9adhmp4DPC/Jqj3nXw9s17N/aVXt0+L/e+CSdi8nV9VlY7sFnAA8J8lebf8I4BtVddMYzx/sW8B+g8r2B05dwPYkSZIkSUuZhZ54GTy6I8lhbX8KcErPiIybkxyZ5PIkM5Jsn+S8NkLhkBHaX6+NyJidZG6S3Vr5m5PMaWVH9dSf17O9b5KpbXtqki8muRA4KskLkvykZ7TKxq3eR5JMbyMyPj0ft2I14K4x3rOn9J3ktCSv7qkzNckbhzn/74G/Az6Z5BTgLXSJjPMBquoB4L3AR1v9nZJcluTK9rlZa+o9wH9U1eXtvKqqM6rqduBSWqKlfX6RJydiLkuyevtel2n9rJzkFmBZYDqwc6u/A3DsoPaGTK4kuSTJ5J79S5NsM6jOPyT5EbAi8G7g/yWZArwc+Hyrs0O7v5cn+XwGjUAaSlXdANydZOee4r8Dvt2+o3OTzGwxbj5ae5IkSZKkpc94jXg5A5gBHNBGMDzYym+pql2AS4CpwL7Ai+hGKgznLcB5bWTEtsDsdFOYjgL2BCYDOybZewxxbQrsVVUfBk4Bjm2jVXYFbkvyCmATYKfW7g5Jdh+hvZVaQuh64ETgMz3HNu6ZOjMb6E0uPaVv4Nu00RZJnkGXRPjhUJ1W1Yl0I1w+UlUHAFsCMwfV+Q0wKclqdKNNdq+q7YBPAke2alsNPq/HZTyRKNkJ+B7w3LY/MGLlHuAq4KWt/LV039UjA+cnWQV4HLiIJydeLh2m3xPpRkuRZFNghaq6euBgkve2fvauqgfbsfOAC4D3V9WfW9WT2v4uPNVug76b1/Uc+xbdKBeSvAj4U1X9CjgeeF9V7QAcBnxlqODb9KcZSWb86f47h7lESZIkSdKSaqIX1z27fc4BplXVfVX1B+ChJM8c5pzpwEFJDge2rqr7gB2Bi6rqD1X1KF0iY6QEyYDTq+qxNgVmg6o6C6CqHmqjRF7R/q4EZgGb0yVihjMw1Whz4JXAN5O/rKvym56pM5OB4wBG6PtHwJ5JVgBeBVzck7AaTYAa5lgBqwOnt1EfR9MlakZzBbBdS5wsX1XzgBuTvIAnj1g5jSem5+zf9uGJETM7AdNbIugFSdYGJlXVjcP0ezrwmnTrwbyTLkE34G109+aNVfVwT/mxwO+q6kKAJKsDz6yqn7XjJw/q45JB383ZPce+DezbRvHsD3wryaR2Lae3RM3XgPWGCr6qjq+qKVU1Zc1V1hjmEiVJkiRJS6qFtbZJr0d5ckJnxRHqDvxj+fGe7YH9IWOrqovbqJO/BU5O8nng3hH66E1ADI7l/vY53KKzAT5bVV8bof2hO626PMlawNqjVB2y76p6KMlFwN/QJTK+NR/dX8OgxFOSjYB5VXVfkn8HLqyqfZJsSDf6ZOC8HYDvDxHPA0l+TZf8mNWKfwG8GlgHuKGVnQ18Nskara2f9tTdEXgJcHkru5UumTHsGi6t3x8Dr6eb5jOl5/BcupFIzwF613F5vP395fIZPhE1oqq6JcnNdKN43kg3vWoZ4O6e9XQkSZIkSRpSP0a83A6sk2TNNlrjNa38PmDV4U8bmyTPA+6oqhOArwPb0y38+tIkayVZFngzMDC64fYkL2wjFvYZqs2quhe4dWB6UpIVkqxMN2XlnW2EA0k2SLLOGOPcnG5tk+EWnB2tb+hGWxwE7NZiGatTgJekLTKbZCXgS8C/teOrA79r2wf2nPdl4B29a5okeWuSZ7fdS4EP8ETi5HLgUOAXVVXteubRjY45Bjinqh5r5fcBt7T+es//ACMkXpoTW/zTq6p3vs6VwD8CZ2eEN2ZV1d3APXniLUwHjNLfYN+iGxn0m6q6tX1nNyV5E0A6285nm5IkSZKkpcBCT7y09TyOoEuGnEO3ngh0U0SOa+torPQ0utiDbl2XK+lGIBxTVbfRvUXoQro1RmZV1cCojY+2OH5Kt3bKcN4GvD/J1XSJgGe3xWlPBS5PModurZqRkkcDa7zMppti846BxMMontJ3Kz+fbuTKT3rWKgF4eZJbe/6etG5Jm5L0euCfk9xAN5VrOl1iBboEzGeTXEqXHBo473a6EShfSPc66evokj4DI4ouBTbiicTJLLrRJoMTJ6cBb+WJaUYDLqVbo+WWtn95a2/ExEtVzWwxnDTEsZ/TrbHyX22E0XAOAo5Ncjkw1ilbA06nm4717Z6yA4B3JbmKbqTQ6+ezTUmSJEnSUiBtoIK0yGqjWS4CNq+qx0epPpb2NqQbjbPV021rfmz73K3qvA9+Z0x1n/2hLfocjSRJkiRpYUoys6qmDC6f6MV1pREleTvd6KmPL4ykiyRJkiRJ46kfi+suFEm25qlvn3m4qnYeqv54SbIm3auKB3t5VY24novmX1V9E/jmQm7zZrpXZ0uSJEmS1FeLbOKlqubQvbFmkdKSK5MnOg5JkiRJkrToW2QTL9KSZvl1V3TtFkmSJElayrjGiyRJkiRJUp+YeJEkSZIkSeoTEy+SJEmSJEl94hov0jh55I553H7MpaPWW/fQF49DNJIkSZKk8eCIF0mSJEmSpD4x8SJJkiRJktQnJl4kSZIkSZL6xMSLJEmSJElSn5h4kSRJkiRJ6pPFLvGS5MAk6090HP2SZMMkc4con5LkS/N73gj190hyTs/+3kmuTnJ9krlJ9u05NrV3f34N7qun/HVJPtq2D0/yuySzk1yb5M099S5KMmU++rs5yZz2d22Sf0myQju2fpIzeuru1Nr/VZJZSf4rydajtP9Yi/OaJFcl+VCSxe6/JUmSJElS/y2Or5M+EJgL/H6C43iKJMtV1aP9aLuqZgAz+tF2km2BLwB/XVU3JXk+8JMkN1XVzH70CVBVZwNn9xQdXVVfSLIJMDPJGVX1yAI2/7Kq+mOSScDx7e8dVfV7YF+AJOsC3wHeUlWXtbKXABsDc0Zo+8GqmtzqrwOcCqwOfGoBY5UkSZIkLaEW2f+XfvAIjiSHtf0pwCltxMFKbXTDkUkuTzIjyfZJzkvymySHjND+ekkubu3MTbJbK39zGykxN8lRPfXn9Wzvm2Rq256a5ItJLgSOSvKCJD9pIyFmJdm41ftIkultVMmnx3gPNkpyZZIde0eNJHlpi3t2O77qCG3s2Pq8PMnnhxkVcxhwZFXdBNA+jwQ+PEK7R/TE8LskJyX5TJJDe+r8a5L3DxHPle3aDkzy5cFtV9WvgAeAZ/UUvzXJZe172am1Nan1O6dd4xuHaGsecAiwd5I1Bj1X7wX+YyDp0ur/vKq+19pfN8lZ7bu8KsmuQ7R/B3Aw8N4kGe5+SZIkSZKWTots4mUYZ9CN+jigqiZX1YOt/Jaq2gW4BJhKN6LhRcARI7T1FuC8NnJhW2B2uilMRwF7ApOBHZPsPYa4NgX2qqoPA6cAx1bVtsCuwG1JXgFsAuzU2t0hye4jNZhkM+BM4KCqmj7o8GHAe1rsuwEPMryTgEPa/XlsmDpbAoNHtswAthiu0ar6ZOv/pcCfgC8DXwfe0eJfBtif7n4MXNOuwHHA66vqxuHaTrI98KuW1BiwSlXtCvwv4But7BPAPVW1dVVtA/x0mFjvBW6i+w56bQnMGi4O4EvAz9p3uT1wzTDt30j339I6Q1zLwS0hOOPOeXeP0JUkSZIkaUm0uCVehjMwXWUOMK2q7quqPwAPJXnmMOdMBw5KcjiwdVXdB+wIXFRVf2hThk4BRkyQNKdX1WNt5MkGVXUWQFU9VFUPAK9of1fS/UN/c56aBOi1NvB94K1VNXuI45cCX2yjSZ453PSmdu2r9ozoOHWY/gLUEGUjaiM8TqGbIjSzqm4G/pRkO9r1VtWfWvUX0k33eW1V/fcwTX4wyQ3ANODwQce+BVBVFwOrtWvbCzh2oEJV3TVSuGO4nmlJrktyTCvaE/hqa/uxqrpnftuvquOrakpVTVlj0jNHC0GSJEmStIRZlBMvj/Lk+FYcoe7D7fPxnu2B/SHXsWn/gN8d+B1wcpK3M/I/znsTE4Njub99Dnd+gM+2UTqTq+oFVfX1Efq6B7gFePEwsX8O+HtgJeAXSTYfod+xuIZuClev7Rl9TZnDgVur6qSeshPp1uE5iCdGpgDcBjwEbDdCe0dX1WbAfsA3k/Te58GJoWLohNFTtITYhsAvBx26hu46uwardqYbRbP6aG0Oan8jutFEd4xWV5IkSZK0dFmUEy+3A+skWTPdG2le08rvA4Zd02SskjwPuKOqTqCbIrM93UiLlyZZK8mywJuBnw3Ek+SFbQrNPkO12aa03DowPSnJCklWBs4D3pluoVeSbJBuUdbh/BnYG3h7krcMEfvGVTWnqo6iS44MmXhpI0DuS/KiVrT/MP19AfhYkg1b+xsCHwA+P1yASV4D/DXw/kGHzgJeSTd66Lye8ruBvwWOTLLHcO22uL9Ld13v6Cner/X7ErrpRfcA59Ot0zIQU++aMANlk4CvAN8bYkTMscCBg9ZuWbln+wLg3a2dZZOsNkT7a9NNn/pyVY2aBJIkSZIkLV0W2bcaVdUjSY6gS4bcBFzfDk0FjkvyILDL0+hiD+AjSR4B5gFvr6rbknwMuJBuNMUPq+r7rf5HgXPoRqLMBSYN0+7bgK+12B8B3lRV5yd5IXB5W391HvBWRhghUVX3t+TGj5PcTzcKZuAf9h9I8jK6URbXAj8C1gM2S3JrTzMfBN4FnNDauKi1M7iv2Un+N/CDluTakO6tQDf0VPtakv/Xtm+hG72yPnBFu6az27ovf0630PDdVfWkNWWq6vYkrwV+lOSdw117cwRwapIT2v5dSS4DVgMGzv0X4Ni2WO5jwKeB77ZjF7apUMvQJYM+M8R1/0+S/egWRd6A7vv4I0+sDXQocHySd7X23w1cDqyUZDawPN3IrJOBL45yPZIkSZKkpVD8P+kXD+2NPa+rqneMWvnJ501qb/YhyUeB9arq0FHO+RywM/A3VfXn+exvGbp1bN7U3k6kZtu/2rzO//BIM8w66x465AwzSZIkSdIiLMnMqhq8jMeiO+JFT0jyOuBfeWKkx/z42zaKZzngt3Trr4yoqj66AP2QZAu6UUFnmXSRJEmSJGkpSLwk2ZpuKkivh9tCqhMmyZp0a4gM9vKeNwEBUFVn88Sbm+ZLVZ0GnLYg5y5AX9cCG41HX5IkSZIkLQ6W+MRLVc0BJk90HIO15MrkiY5DkiRJkiT1zxKfeJEWFcuvM8n1WyRJkiRpKbMov05akiRJkiRpsWbiRZIkSZIkqU9MvEiSJEmSJPWJa7xI4+TRO+7hji//cNR667z31eMQjSRJkiRpPDjiRZIkSZIkqU9MvEiSJEmSJPWJiRdJkiRJkqQ+MfEiSZIkSZLUJyZeJEmSJEmS+sTEy2IqyeFJDluA845IstdCimGPJOcsjLbG0NdFSaYMc2zVJL9JsknbXz7JnCQ7t/3L+hzb65J8tJ99SJIkSZIWT75OeilTVZ8c7z6TLFtVj/Wr/aq6L8nHgGOBVwCHAZdV1bR2fNd+9d3aPxs4u599SJIkSZIWT454WYwk+XiSG5L8BNislW2c5NwkM5NckmTzJKsnuTnJMq3OykluaSNBpibZt5XvmOSyJFcluaKNHFk2yeeTTE9ydZJ/HCWs1ZKcleTaJMf19Dmvja6ZBuyS5JOtzblJjk+SVu+iJEe1/n+ZZLdWvlKSb7cYTgNWGimIqvoO8HiSfwIOAT7Wc9/mtc9JSS5IMquNiHl9T50PtdjmJvlAK9swyfVJTmzlpyTZK8mlSX6VZKdW78AkXx7btyhJkiRJWpo44mUxkWQHYH9gO7rvbRYwEzgeOKSqftWm1nylqvZMchXwUuBC4LXAeVX1SMt3kOQZwGnAflU1PclqwIPAu4B7qmrHJCsAlyY5v6puGia0nYAtgN8C5wJvAM4AVgHmDoywSXJtVR3Rtk8GXgP8oLWxXFXtlOTVwKeAvYB3Aw9U1TZJtmnXO5oPANcBB1fVnUMcfwjYp6ruTbIW8IskZwPbAwcBOwMBpiX5GXAX8ALgTcDBwHTgLcBLgNcB/wfYe6SAkhzczuU5z1p7DJcgSZIkSVqSOOJl8bEbcFZVPVBV99JNbVkR2BU4Pcls4GvAeq3+acB+bXv/tt9rM+C2qpoOUFX3VtWjdFN13t7amwasCWwyQlxXVNWNbSrRt+iSEgCPAWf21HtZkmlJ5gB7Alv2HPtu+5wJbNi2dwf+s8V2NXD1CDEMeCVwG7DVMMcDHJnkauAnwAbAui3ms6rq/qqa1+LZrZ1zU1XNqarHgWuAC6qqgDk9sQ6rqo6vqilVNWXNSauP4RIkSZIkSUsSR7wsXmrQ/jLA3VU1eYi6ZwOfTbIGsAPw00HHM0R7A+Xvq6rzFjCmgf2HBtZ1SbIi8BVgSlXdkuRwuqTRgIfb52M8+ZkcKr4hJVkfeD/dCJwLk3y9JWx6HQCsDezQRv/c3OLICE0/3LP9eM/+4/jfjyRJkiRpFI54WXxcDOzT1j5ZlW760APATUneBJDOtgBt5MYVwDHAOUMsbns9sH6SHdu5qyZZDjgPeHeS5Vv5pklWGSGunZI8v63tsh/w8yHqDCRZ/phkErDvGK/3gBbDVsA2o9Q/Gjiyqm4FPgQcO7COTI/VgTta0uVlwPN6+tq7rYWzCrAPcMkYYpQkSZIkaUQmXhYTVTWLbrrQbLopPAOJgQOAd7U1Xa4BXt9z2mnAW3nqNCOq6s90iZJ/b+f+mC5BciJwLTAryVy66Usjjey4HPgcMBe4CThriL7uBk6gm57zPbq1UkbzVWBSmxb0T3RJpCEl+Wvgr4Cvt/5+QLc+y9sHVT0FmJJkBt19u77VnwVMbX1MA06sqivHEKMkSZIkSSNKt1yFpH6b/Feb1Pn/dMyo9dZ576vHIRpJkiRJ0sKUZGZVTRlc7ogXSZIkSZKkPnFxUI0qydbAyYOKH66qnScglmnACoOK31ZVc8Y7FkmSJEmSRmPiRaNqSY3JEx0HwEQkeyRJkiRJWlAmXqRxstw6q7t+iyRJkiQtZVzjRZIkSZIkqU98q5E0TpLcB9ww0XFIg6wF/HGig5AG8bnUosZnUosin0stipb25/J5VbX24EKnGknj54ahXi0mTaQkM3wutajxudSixmdSiyKfSy2KfC6H5lQjSZIkSZKkPjHxIkmSJEmS1CcmXqTxc/xEByANwedSiyKfSy1qfCa1KPK51KLI53IILq4rSZIkSZLUJ454kSRJkiRJ6hMTL1KfJXllkhuS/DrJRyc6Hi35ktycZE6S2UlmtLI1kvw4ya/a57N66n+sPZ83JPmbnvIdWju/TvKlJJmI69HiKck3ktyRZG5P2UJ7DpOskOS0Vj4tyYbjeoFaLA3zXB6e5HftN3N2klf3HPO5VF8leW6SC5Ncl+SaJIe2cn8vNWFGeC79vVxAJl6kPkqyLHAs8CpgC+DNSbaY2Ki0lHhZVU3ueZ3fR4ELqmoT4IK2T3se9we2BF4JfKU9twBfBQ4GNml/rxzH+LX4m8pTn5mF+Ry+C7irql4AHA0c1bcr0ZJkKkP/lh3dfjMnV9UPwedS4+ZR4MNV9ULgRcB72rPn76Um0nDPJfh7uUBMvEj9tRPw66q6sar+DHwbeP0Ex6Sl0+uB/2jb/wHs3VP+7ap6uKpuAn4N7JRkPWC1qrq8usXAvtlzjjSqqroYuHNQ8cJ8DnvbOgN4uaOyNJphnsvh+Fyq76rqtqqa1bbvA64DNsDfS02gEZ7L4fhcjsLEi9RfGwC39Ozfysg/WtLCUMD5SWYmObiVrVtVt0H3P6bAOq18uGd0g7Y9uFx6Ohbmc/iXc6rqUeAeYM2+Ra4l3XuTXN2mIg1M6fC51LhqUy22A6bh76UWEYOeS/D3coGYeJH6a6isra8SU7+9uKq2p5vi9p4ku49Qd7hn1GdX42lBnkOfUS0sXwU2BiYDtwH/t5X7XGrcJJkEnAl8oKruHanqEGU+l+qLIZ5Lfy8XkIkXqb9uBZ7bs/8c4PcTFIuWElX1+/Z5B3AW3ZS329twT9rnHa36cM/orW17cLn0dCzM5/Av5yRZDlidsU8hkf6iqm6vqseq6nHgBLrfTPC51DhJsjzdP25PqarvtmJ/LzWhhnou/b1ccCZepP6aDmyS5PlJnkG36NTZExyTlmBJVkmy6sA28ApgLt1z945W7R3A99v22cD+bWX559MtenZFG9Z8X5IXtfm2b+85R1pQC/M57G1rX+Cnbf64NF8G/nHb7EP3mwk+lxoH7Rn6OnBdVX2x55C/l5owwz2X/l4uuOUmOgBpSVZVjyZ5L3AesCzwjaq6ZoLD0pJtXeCstjbZcsCpVXVukunAd5K8C/hv4E0AVXVNku8A19KtYP+eqnqstfVuujeArAT8qP1JY5LkW8AewFpJbgU+BXyOhfccfh04Ocmv6f4fsv3H4bK0mBvmudwjyWS6Ie43A/8IPpcaNy8G3gbMSTK7lf0f/L3UxBruuXyzv5cLJktwUkmSJEmSJGlCOdVIkiRJkiSpT0y8SJIkSZIk9YmJF0mSJEmSpD4x8SJJkiRJktQnJl4kSZIkSZL6xMSLJEmSJElSn5h4kSRJkiRJ6hMTL5IkSZIkSX3y/wHe6jdMQxSRxwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(16, 10))\n",
    "most_important = feature_importance.sort_values(ascending=False).head(25)\n",
    "sns.barplot(x=most_important, y=most_important.index, orient='h')\n",
    "plt.title('Самые важные признаки после преобразования');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 268,
   "id": "48727f10-5b52-4f0a-ba42-f7a28fdc8d56",
   "metadata": {
    "id": "H0QGo6sROydQ"
   },
   "outputs": [],
   "source": [
    "# Из важности производных признаков получим важность оригинальных\n",
    "# Также получим важность признаков с добавленными данными '_added_'\n",
    "\n",
    "original_columns = list(sessions.columns) + ['_added_']\n",
    "column_importances = list()\n",
    "for col in original_columns:\n",
    "    imp = feature_importance[feature_importance.index.str.contains(col)].sum()\n",
    "    column_importances.append(imp)\n",
    "\n",
    "column_importances = pd.Series(column_importances, index=original_columns)\n",
    "column_importances.sort_values(ascending=False, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "id": "9b40f5b3-3ce7-41d8-8084-e6290d79ac85",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 607
    },
    "id": "tglso3v3VOFB",
    "outputId": "c0fb8d2c-2f6e-4de5-ac17-7900970cd957"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABH8AAAJOCAYAAAAu3auwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABVMklEQVR4nO3debhdVX3/8fcHwkwYFFBwijKIIhDMBQRFUSkd1AoKhopa1JZiLaIVrf1hFWmtUttah4qmVik4gCBYxNagiICCkMGQgIKoYLFQRGUKIOP398dZV0+u997cm+nc7Lxfz5PnnLP23mt99zk5j+TjWuukqpAkSZIkSVI3rTfoAiRJkiRJkrT6GP5IkiRJkiR1mOGPJEmSJElShxn+SJIkSZIkdZjhjyRJkiRJUocZ/kiSJEmSJHWY4Y8kSZIkSVKHGf5IkqQVkuTGJPclWZrk9iRfSfKEQdclSZKkZRn+SJKklfGSqtoc2B64FfjIgOuRJEnSCIY/kiRppVXVr4CzgacPtyV5UZLvJrkryU1JTuw7tlNre3Z7fVSSb7XnmyS5LMlx7fWBSX7aP16SbyU5qj1fL8k7k/wkyc+SnJZky75zn9P6u6ONeVSS2W3G0tIkDyf51fDrds2JST4zkXtPslGSf0lyc/vzL0k26q89yf9L8vM2W+rIvmtPTfJ3fa//K0klmdZef7O/tuH3of/9aq/f3q47aLT6k0xrx2cs77Ppu6aS3NPGfXC4zpFjj7hmeZ/VqNe2vw/Vnj+qvWcvaa83T/LDJK8ZY8yR79F9SW7sO35jkr9O8r02Q+3TSTYerd4kr2j3/Sft9eGtlqVJrk9y2Ih+D+p7/SdJvtn3+kPtvb0ryYIkB/Qd+/Xnk2TjJBcnObnv+B8muab9nf1mkqeNGHd4xt3/JvmL0d4XSZL6Gf5IkqSVlmRTYDbwnb7me4DXAFsBLwLekOQQgKr6IXA4cEaSnfv6WQ84Hbiyqj40weGPan+eDzwF2Bz4aOvvicB/05uRtC0wE1hUVWdW1eZt1tKlwF/0vZ6sE4Bntb73BPYB3tl3/LHANsDjgD8G5iR56shOkhwI7DFK/7+uraoeP8p1WwNvAu6YRM1jfjatz+H/RtyjvSefnUTfK6Wqfgm8Dvi3JNsBH6T3mZ02zmX9n99LRjl+JPC7wI7ALiz7+QCQZAPgb4Fb+povB/Zq/b4J+MQkbmUevb8TjwI+B5w1HDr1jTkN+ALwg6r6q9a2C/B54M30/s7+F/DlJBv2XTo84+6VwIeTbDGJuiRJ6yDDH0mStDK+lOQO4C7gd4APDB+oqm9W1ZKqeqSqFtP7B+3z+o5/B3gXvXBm29b8j8B2wF9OooYjgX+uqh9X1VLgr4Ej2j+sjwS+XlWfr6oHq+oXVbVoBe91vPFPqqqfVdVtwHuAV48452+q6v6quhj4CvCK/oNJAvwDvfdjsk4APgXcOdELlvfZAMNBwwMrUM9Kq6oLgLOAC+mFU3+2kl1+tKpuasHSe4E/GuWcPwOuAH7QV8dP22cKEGDhRAesqs+0v28PVdU/ARsB/aFfgH+nF1Ye09c+G/hKVX2tqh6k953YBNh/lGGm0fvuDeRzkiStPQx/JEnSyjikqrai9w/bvwAuTvJYgCT7JrkoyW1J7qT3D9xtRlz/O8AvgZOAWcAL6f0DebsR5+3QlsDc0cKmZ/UfA37S9/on9P5R/BjgCcCPVvDeXtHG+3mSryV5yhjnjTb+Dn2vb6+qe8Y5Dr0w6BfANyZTYJvZ9Ar6QrdR6r8D+PmI65b32TxquPYxhn5W6/uX6S2pG+o7Nt5ntbxr+80BngF8uqp+McY5E3VT3/Pfev+TTAfeDvzNyAuTvDLJPfTCqLNGHP5S331+eMR1b03y/SR3tuNbsux7fCjwNGA3fhN+woi/T1X1SKv/cSPGvQu4APj7tuxSkqQxGf5IkqSVVlUPV9U5wMPAc1rz54DzgCdU1ZbAx+nNdgAgye8AQ8Bz6S31uoNeGHQK8C8jhri5qrYa/sOyy8tuBp7U9/qJwEP0NqC+id5SnxXxhTbWDsD/AH8/xnmjjX9z3+utk2w2zvHh5UZ/tQI1/h3wD1V19yjHvtD3fo0M3cb9bOgtjbqlzaQazXdav9sCX6Mts2vG+6yWdy0ASdant8TqNHpL0nYao46J6v8VupHvP8Db6L1fPxnRTlV9rqo2Aw4EPpTk6X2HD+m7zzf11X8Avc/zFcDW7fidLPse/xh4Ab3ZPx/ra1/m71ObFfYE4H9HjLtFu5fjkuw35p1LkoThjyRJWgXS81Jga+D7rXk68Muq+lWSfejtTzJ8/sb0/sH7523WwmXAj6rqZ8D7gJlJfm+Cw38eeEuSJyfZnF5Ic2ZVPURvr5qD2ka+05I8OsnMydxbVT0ALGXs/276PPDOJNsm2Ybe0q2Rm0W/J8mGLRR4McvOIHk1cFlbfjUZOwH7Mrl9aIaN99lsA7wD+NLyOqmqh+mFGpP+b8rlXPv/2uPr6C17Oq0FQivqjUken+RRre8z+45NB15LbznYMpI8tW+fno3ohTf3TWC86fQCyNuAaUneBYzcl2dRC9feA+yaZHZr/wLwoiQvbPsQvRW4n953ZKSH2+O2oxyTJOnXpg26AEmStFb7cpKHgaK3VOWPq+qaduzPgX9K8lHgYnr/qN2qHXsnvRkgXx/ZYVXdn+QY4N+TPGMCNXyK3uycS4CNgbnAsa2v/0nyB/QChE/SCxveCSyaQL+HpvdLUKE3S+P1Y5z3d/T+YT8c3pzV2ob9H73lUzcD9wLHVNW1fce3ZpTlRhPwGODYti/MZI332ZxBb5bJO8a5fu/85leybqW35G+ixr02ySx6ez7tXVUPt1/BelGr57cCmgn6HL0lUjsA/8myn88WwN9V1WhL3A6nN7NmY3rvyZ9X1Q0TGG8uvb2sfkBvc+0PsuzSs19rf99fS28p14VVdV2SV9HbpPxx9P6uvqSFkMOGv3f30guyvjKBmiRJ67BU1aBrkCRJ6qT2C16fGe1XurRmpPez738yWtAoSdK6wmVfkiRJkiRJHWb4I0mSJEmS1GEu+5IkSZIkSeowZ/5IkiRJkiR1mL/2pUnZZpttasaMGYMuQ5IkSZIk9VmwYMHPq2rb0Y4Z/mhSZsyYwfz58wddhiRJkiRJ6pPkJ2Mdc9mXJEmSJElShznzR5Py0G2/5LZTPjPoMiRJkiRJWuW2fcOrBl3CauHMH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeqwSYc/SU5McvwKXHdSkoMme50gyalJDlvOOQcm2b/v9TFJXrP6q5MkSZIkSVPZtDU1UFW9a02NNZYk06rqoTU43vpV9fAaGu5AYClwGUBVfXwNjStJkiRJkqawCc38SXJCkuuSfB14amvbMclXkyxIcmmSXZNsmeTGJOu1czZNclOSDfpnryTZO8llSa5KcmWS6UnWT/KBJPOSLE7yZ+PUs32SS5IsSnJ1kgNa++8lWdj6vbC1nZhkTpILgNOSbJvki22ceUme3c7bLMmnWtt3k7y0tR+V5Jx2r9cn+YflvFdL2yynK4D9kryq3eOiJJ9o97l+ez+uTrIkyVvatTOTfKfd/7lJth6l/xuTbNOeDyX5ZpIZwDHAW9o4B/TP0Bqr33btya2+Hwy/j5IkSZIkqTuWG/4kmQUcAewFvAzYux2aAxxbVbOA44GPVdWdwFXA89o5LwHmVtWDff1tCJwJHFdVewIHAfcBrwfurKq92xh/muTJY5T1ytbvTGBPYFGSbYF/A17e+j287/xZwEur6pXAh4APtnFeDnyynXMC8I3W/nzgA0k2a8dmArOB3YHZSZ4wzlu2GXB1Ve0L/KJd9+xW68PAka2/x1XVM6pqd+DT7drTgL+qqj2AJcC7xxnn16rqRuDj7b5mVtWlI04Zr99pVbUP8OaxxktydJL5Seb/YuldEylJkiRJkiRNERNZ9nUAcG5V3QuQ5DxgY2B/4Kwkw+dt1B7PpBd4XEQvNPrYiP6eCtxSVfMAququ1u/BwB59e9tsCewM3DBKTfOATyXZAPhSVS1KciBwSVXd0Pr9Zd/551XVfe35QcDT++reIsl04GDgD/v2M9oYeGJ7fmELtkjyPeBJwE2jvlu9gOeL7fkL6QVP89p4mwA/A74MPCXJR4CvABck2RLYqqoubtf+B3DWGGNM2AT6Pac9LgBmjNZHVc2hF/Yx80lPqZWtSZIkSZIkrTkT3fNn5D/41wPuaLNZRjoPeF+SR9ELPr4x4nhG6W+4/diqmrvcYqouSfJc4EXA6Uk+ANwxRr8A94yofb++MKg3eC+deXlVXTeifV/g/r6mhxn/fftV3z4/Af6jqv565ElJ9gR+F3gj8ArgLeP02e8hfjNja+MJXjOe4Xtb3n1JkiRJkqS10ET2/LkEODTJJm2GzEuAe4EbkhwOveCkhRlU1VLgSnrLq84fZcPja4Edkuzdrp2eZBowF3hDm81Dkl36ll0tI8mTgJ9V1b8B/w48E7gceN7wUrEWPo3mAuAv+vqa2Z7OBY5tIRBJ9prAe7M8FwKHJdluuKYkT2p79qxXVV8E/gZ4ZptZdHvfvjuvBi4epc8b6YVq0Fu2NuxuYPrIkyfRryRJkiRJ6qDlzvSoqoVJzgQWAT8BhveTORI4Jck7gQ2AM+jt9wO9pV9n0fsFqpH9PZBkNvCRJJvQ2+/nIHp778wAFrYA5jbgkDHKOhB4W5IH6f3C1Wuq6rYkRwPnpLfh9M+A3xnl2jcB/5pkcbv/S+htlvy3wL8Ai9v4NwIvHvfNWY6q+l57fy5oNT1Ib6bPfcCnWxvA8MygPwY+nmRT4MfAa0fp9j3Avyf5f8AVfe1fBs5uG1UfO+KaifQrSZIkSZI6KFVu4aKJm/mkp9TX3nHSoMuQJEmSJGmV2/YNrxp0CSssyYKqGhrt2IR+6l2SJEmSJElrpym9wW+S3YHTRzTf335GfaCSXMFvfuFs2Kuraskg6pEkSZIkSRrNlA5/WpAyc9B1jGYqBFCSJEmSJEnL47IvSZIkSZKkDjP8kSRJkiRJ6rApvexLU8+0bR+1Vu9+LkmSJEnSusaZP5IkSZIkSR1m+CNJkiRJktRhhj+SJEmSJEkdZvgjSZIkSZLUYYY/kiRJkiRJHeavfWlSHrztZv7vlPcMugxJkiRprffYN7x70CVIWkc480eSJEmSJKnDDH8kSZIkSZI6zPBHkiRJkiSpwwx/JEmSJEmSOszwR5IkSZIkqcMMfyRJkiRJkjrM8EeSJEmSJKnDDH8kSZIkSZI6zPBnDUmyQ5Kzl3POZe1xRpJXLufcmUn+oO/1HyZ5x6qpVpIkSZIkdYXhzxpSVTdX1WHLOWf/9nQGMG74A8wEfh3+VNV5VfX+lalRkiRJkiR1j+HPapDk5CR/3vf6xCRvTXJ1e71bkiuTLEqyOMnOrX1pu+T9wAHt+FtG6X9D4CRgdjtndpKjkny0HT81ySlJLkry4yTPS/KpJN9PcmpfPwcnuTzJwiRnJdl8jPs5Osn8JPN/sfTeVfQuSZIkSZKkNcHwZ/U4A5jd9/oVwLy+18cAH6qqmcAQ8NMR178DuLSqZlbVB0d2XlUPAO8CzmznnDlKDVsDLwDeAnwZ+CCwG7B7WzK2DfBO4KCqeiYwH/jL0W6mquZU1VBVDT16802Xc+uSJEmSJGkqmTboArqoqr6bZLskOwDbArcD/9N3yuXACUkeD5xTVdevhjK+XFWVZAlwa1UtAUhyDb1lZY8Hng58OwnAhq0uSZIkSZLUIYY/q8/ZwGHAY+nNBPq1qvpckiuAFwFzk/xJVX1jFY9/f3t8pO/58OtpwMPA16rqj1bxuJIkSZIkaQpx2dfqcwZwBL0AaJlf+UryFODHVfVh4DxgjxHX3g1MX07/EzlnPN8Bnp1kp1bTpkl2WYn+JEmSJEnSFGT4s5pU1TX0wpn/rapbRhyeDVydZBGwK3DaiOOLgYeSXDXahs/NRcDThzd8XoH6bgOOAj6fZDG9MGjXyfYjSZIkSZKmtlTVoGvQWmTPJ+1Qc9/xZ4MuQ5IkSVrrPfYN7x50CZI6JMmCqhoa7ZgzfyRJkiRJkjrMDZ+nuCS/C5w8ovmGqjp0EPVIkiRJkqS1i+HPFFdVc4G5g65DkiRJkiStnVz2JUmSJEmS1GGGP5IkSZIkSR3msi9Nygbb7uCvEkiSJEmStBZx5o8kSZIkSVKHGf5IkiRJkiR1mOGPJEmSJElShxn+SJIkSZIkdZgbPmtS7rvth1zzsT8cdBmSJEmds9ufnzfoEiRJHeXMH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/xZQ5LskOTs5ZxzWXuckeSVk+z/m0mGlnPOm5NsOpl+JUmSJEnS2s3wZw2pqpur6rDlnLN/ezoDmFT4M0FvBgx/JEmSJElahxj+rAZJTk7y532vT0zy1iRXt9e7JbkyyaIki5Ps3NqXtkveDxzQjr9ljDE2SXJGu/5MYJO+Y6ckmZ/kmiTvaW1vAnYALkpyUWs7OMnlSRYmOSvJ5mOMdXTrb/7tSx9Y6fdHkiRJkiStOYY/q8cZwOy+168A5vW9Pgb4UFXNBIaAn464/h3ApVU1s6o+OMYYbwDurao9gPcCs/qOnVBVQ8AewPOS7FFVHwZuBp5fVc9Psg3wTuCgqnomMB/4y9EGqqo5VTVUVUNbb77hcm9ekiRJkiRNHdMGXUAXVdV3k2yXZAdgW+B24H/6TrkcOCHJ44Fzqur6FRjmucCH23iLkyzuO/aKJEfT+3y3B54OLB5x/bNa+7eTAGzY6pIkSZIkSR1i+LP6nA0cBjyW3kygX6uqzyW5AngRMDfJn1TVN1ZgjBrZkOTJwPHA3lV1e5JTgY1HuTbA16rqj1ZgXEmSJEmStJZw2dfqcwZwBL0AaJlf+UryFODHbSnWefSWZ/W7G5i+nP4vAY5s/T2jr48tgHuAO5M8Bvj9Mfr9DvDsJDu1PjZNssuE706SJEmSJK0VDH9Wk6q6hl7Q8r9VdcuIw7OBq5MsAnYFThtxfDHwUJKrxtrwGTgF2Lwt93o7cGUb9yrgu8A1wKeAb/ddMwf47yQXVdVtwFHA51sf32m1SJIkSZKkDknVb60cksa025O2qi/81XMHXYYkSVLn7Pbn5w26BEnSWizJgvbjT7/FmT+SJEmSJEkd5obPU1yS3wVOHtF8Q1UdOoh6JEmSJEnS2sXwZ4qrqrnA3EHXIUmSJEmS1k4u+5IkSZIkSeowZ/5oUjbZdic3I5QkSZIkaS3izB9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA7z1740KXf//Hou/OSLBl2GJEnqsBf+yVcGXYIkSZ3izB9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMOftUSSo5J8dIxjSyfZ14lJjl81lUmSJEmSpKnM8EeSJEmSJKnDDH+mgCRfSrIgyTVJju5rf22SHyS5GHh2X/uTk1yeZF6Svx3R19ta++Ik7+lrPyHJdUm+Djx1kvUdnWR+kvl33P3Ait+oJEmSJEla4wx/pobXVdUsYAh4U5JHJ9keeA+90Od3gKf3nf8h4JSq2hv4v+HGJAcDOwP7ADOBWUmem2QWcASwF/AyYO/JFFdVc6pqqKqGtpq+4YreoyRJkiRJGoBpgy5AQC/wObQ9fwK9AOexwDer6jaAJGcCu7Rzng28vD0/HTi5PT+4/flue71562s6cG5V3dv6Om/13YokSZIkSZpKDH8GLMmBwEHAflV1b5JvAhu3wzXOpaMdC/C+qvrEiDHevJy+JEmSJElSR7nsa/C2BG5vwc+uwLNa+xXAgW0J2AbA4X3XfJveMi6AI/va5wKvS7I5QJLHJdkOuAQ4NMkmSaYDL1mN9yNJkiRJkqYQZ/4M3leBY5IsBq4DvgNQVbckORG4HLgFWAis3645DvhckuOALw53VFUXJHkacHkSgKXAq6pqYVs2tgj4CXDpGrgvSZIkSZI0BaTK1UCauKfO2LI+9s7nDLoMSZLUYS/8k68MugRJktY6SRZU1dBox1z2JUmSJEmS1GEu+1qDkjwauHCUQy+sql8MoJ5/pffLYf0+VFWfXtO1SJIkSZKk1cPwZw1qAc/MQdcxrKreOOgaJEmSJEnS6uWyL0mSJEmSpA4z/JEkSZIkSeowl31pUqZvs7O/wCFJkiRJ0lrEmT+SJEmSJEkdZvgjSZIkSZLUYYY/kiRJkiRJHWb4I0mSJEmS1GGGP5IkSZIkSR3mr31pUn758+s549O/O+gyJElSRxzx2rmDLkGSpM5z5o8kSZIkSVKHGf5IkiRJkiR1mOGPJEmSJElShxn+SJIkSZIkdZjhjyRJkiRJUocZ/kiSJEmSJHWY4Y8kSZIkSVKHGf5IkiRJkiR1mOHPOiDJZe1xRpJXDroeSZIkSZK05hj+rAOqav/2dAZg+CNJkiRJ0jrE8GcAkvxNkmuTfC3J55Mcn2THJF9NsiDJpUl2bec+KcmFSRa3xyeO0+9jkpyb5Kr2Z//WvrSd8n7ggCSLkryljTOz7/pvJ9ljlH6PTjI/yfy7lz6wSt8LSZIkSZK0ehn+rGFJhoCXA3sBLwOG2qE5wLFVNQs4HvhYa/8ocFpV7QF8FvjwON1/GLi4qvYEnglcM+L4O4BLq2pmVX0Q+CRwVKtrF2Cjqlo8stOqmlNVQ1U1NH3zDSd7y5IkSZIkaYAMf9a85wD/WVX3VdXdwJeBjYH9gbOSLAI+AWzfzt8P+Fx7fnq7fiwvAE4BqKqHq+rO5dRyFvDiJBsArwNOnfTdSJIkSZKkKW3aoAtYB2WUtvWAO6pq5gSur1VVSFXdm+RrwEuBV/CbWUiSJEmSJKkjnPmz5n0LeEmSjZNsDrwIuBe4IcnhAOnZs51/GXBEe35ku34sFwJvaH2sn2SLEcfvBqaPaPskveVi86rqlyt4T5IkSZIkaYoy/FnDqmoecB5wFXAOMB+4k16w8/okV9Hbq+el7ZI3Aa9Nshh4NXDcON0fBzw/yRJgAbDbiOOLgYfaZtBvafUsAO4CPr0Kbk+SJEmSJE0xLvsajH+sqhOTbApcAvxTVd0A/N7IE6vqRnp7+SxXVd3Kb0Kj/vbN2+ODwAv7jyXZgV4IeMEk70GSJEmSJK0FnPkzGHPaxs4LgS9W1cJBFJHkNcAVwAlV9cggapAkSZIkSauXM38GoKpeuTLXJzkBOHxE81lV9d5J1nEacNrK1CJJkiRJkqY2w5+1UAt5JhX0SJIkSZKkdZPLviRJkiRJkjrMmT+alEdtszNHvHbuoMuQJEmSJEkT5MwfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQO89e+NCk/++X1fOSzvzvoMiSN49gj/UU+SZIkSb/hzB9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMOfCUqyQ5Kzl3POZe1xRpJXrpnKRq1j6aDGliRJkiRJU4vhzwRV1c1Vddhyztm/PZ0BDCz8WRlJpg26BkmSJEmStOoY/owiyclJ/rzv9YlJ3prk6vZ6tyRXJlmUZHGSnVv78Iyb9wMHtONvGWOMo5Kck+SrSa5P8g99x5b2PT8syant+alJTklyUZIfJ3lekk8l+f7wOX3X/VOShUkuTLJta9uxjbcgyaVJdu3r95+TXAScvNJvoCRJkiRJmjIMf0Z3BjC77/UrgHl9r48BPlRVM4Eh4Kcjrn8HcGlVzayqD44zzsw2zu7A7CRPmEBtWwMvAN4CfBn4ILAbsHuSme2czYCFVfVM4GLg3a19DnBsVc0Cjgc+1tfvLsBBVfXWkQMmOTrJ/CTzl971wARKlCRJkiRJU4VLfEZRVd9Nsl2SHYBtgduB/+k75XLghCSPB86pqutXcKgLq+pOgCTfA54E3LSca75cVZVkCXBrVS1p119Db7nZIuAR4Mx2/meAc5JsDuwPnJVkuK+N+vo9q6oeHm3AqppDLzjiiU/ZsiZzg5IkSZIkabAMf8Z2NnAY8Fh6M4F+rao+l+QK4EXA3CR/UlXfWIEx7u97/jC/+Tz6A5aNx7jmkRHXP8LYn2fRm+V1R5utNJp7llesJEmSJEla+7jsa2xnAEfQC4CW+ZWvJE8BflxVHwbOA/YYce3dwPSVGPvWJE9Lsh5w6Apcvx69uqG38fS3quou4IYkhwOkZ8+VqFGSJEmSJK0FDH/GUFXX0Atw/reqbhlxeDZwdZJFwK7AaSOOLwYeSnLVWBs+L8c7gPOBbwAjx56Ie4Ddkiygtz/QSa39SOD1Sa4CrgFeugJ9S5IkSZKktUiq3MJFE/fEp2xZb/vbZw26DEnjOPbIuYMuQZIkSdIalmRBVQ2NdsyZP5IkSZIkSR3mhs+rWZLfBU4e0XxDVa3IXj6SJEmSJEmTYvizmlXVXMA1GJIkSZIkaSBc9iVJkiRJktRhhj+SJEmSJEkd5rIvTcp2j9rZXxKSJEmSJGkt4swfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQO89e+NCk/vf16jj/79wZdxmrxj4d9ddAlSJIkSZK0yjnzR5IkSZIkqcMMfyRJkiRJkjrM8EeSJEmSJKnDDH8kSZIkSZI6zPBHkiRJkiSpwwx/JEmSJEmSOszwR5IkSZIkqcMMfyRJkiRJkjrM8KdJclSSHQZdx6qS5Jgkrxl0HZIkSZIkabCmDbqAKeQo4Grg5gHXsUpU1ccHXYMkSZIkSRq8dW7mT5IZSa7ue318ez0EfDbJoiSbJLkxyd8nuTzJ/CTPTDI3yY+SHLOcMd6eZEmSq5K8v7X9aZJ5re2LSTZt7acmOSXJRUl+nOR5ST6V5PtJTu3rc2mSf0qyMMmFSbZdTr8nJjm+Pd87yeJ2Lx8Yvv822+mcJF9Ncn2Sfxjjfo5u78H8e+96YCXefUmSJEmStKatc+HPGM4G5gNHVtXMqrqvtd9UVfsBlwKnAocBzwJOGqujJL8PHALsW1V7AsOByjlVtXdr+z7w+r7LtgZeALwF+DLwQWA3YPckM9s5mwELq+qZwMXAuyfQ77BPA8e0e3l4xLGZwGxgd2B2kieMvLiq5lTVUFUNbbrFhmPduiRJkiRJmoIMf8Z3XntcAlxRVXdX1W3Ar5JsNcY1BwGfrqp7Aarql639GUkuTbIEOJJeuDPsy1VVbZxbq2pJVT0CXAPMaOc8ApzZnn8GeM4E+qXVOb2qLmtNnxtR74VVdWdV/Qr4HvCksd8OSZIkSZK0tlkXw5+HWPa+Nx7n3Pvb4yN9z4dfj7VfUoAapf1U4C+qanfgPSPGXZFxhscYr9/hesbTP97D44wnSZIkSZLWQuti+HMrsF2SRyfZCHhxa78bmL4K+r8AeF3f3juPau3TgVuSbEBvhs5krUdv2RnAK4FvTaTfqroduDvJs1rTESswtiRJkiRJWkutc7M8qurBJCcBVwA3ANe2Q6cCH09yH7DfSvT/1bZPz/wkDwD/Bfw/4G/amD+ht7xrskHTPcBuSRYAd9Lbp4cJ9vt64N+S3AN8s10vSZIkSZLWAeltNaOpLsnSqtp8Ba/dvKqWtufvALavquNWpK/H7rhlverkFc7GprR/POyrgy5BkiRJkqQVkmRBVQ2Ndmydm/mzjnpRkr+m93n/BDhqsOVIkiRJkqQ1xfBnBSXZHTh9RPP9VbXv6hhvRWf9tGvP5De/FCZJkiRJktYhhj8rqKqWADMHXYckSZIkSdJ41sVf+5IkSZIkSVpnGP5IkiRJkiR1mMu+NCmP33pnfxVLkiRJkqS1iDN/JEmSJEmSOszwR5IkSZIkqcMMfyRJkiRJkjrM8EeSJEmSJKnD3PBZk3L9Hf/D73/p2EGXMWn/fchHBl2CJEmSJEkD4cwfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8WUWSHJVkh0HXIUmSJEmS1M/wZ9U5CpiS4U+SaYOuQZIkSZIkDYbhzyQlmZHk6r7Xx7fXQ8BnkyxKskmSG5P8fZLLk8xP8swkc5P8KMkx4/S/fZJLWj9XJzmgtf9RkiWt7eS+85f2PT8syant+alJ/jnJRcDJSXZK8vUkVyVZmGTHdt7bksxLsjjJe1b1+yVJkiRJkgbLGSGrxtnAgcDxVTUfIAnATVW1X5IPAqcCzwY2Bq4BPj5GX68E5lbVe5OsD2zalpOdDMwCbgcuSHJIVX1pOXXtAhxUVQ8nuQJ4f1Wdm2RjYL0kBwM7A/sAAc5L8tyqumSF3gVJkiRJkjTlGP6sXue1xyXA5lV1N3B3kl8l2aqq7hjlmnnAp5JsAHypqhYleQHwzaq6DSDJZ4HnAl9azvhnteBnOvC4qjoXoKp+1fo5GDgY+G47f3N6YdAy4U+So4GjATbedvMJ37wkSZIkSRo8w5/Je4hll8ttPM6597fHR/qeD78e9b2vqkuSPBd4EXB6kg8Ad40zRo1Tyz3tMWNcG+B9VfWJcfqnquYAcwC23Gm7Gu9cSZIkSZI0tbjnz+TdCmyX5NFJNgJe3NrvBqavbOdJngT8rKr+Dfh34JnAFcDzkmzTloL9EXDxcD1JnpZkPeDQ0fqsqruAnyY5pI2xUZJNgbnA65Js3tofl2S7lb0HSZIkSZI0dTjzZ5Kq6sEkJ9ELZG4Arm2HTgU+nuQ+YL+VGOJA4G1JHgSWAq+pqluS/DVwEb3ZOv9VVf/Zzn8HcD5wE3A1vaVbo3k18IlW+4PA4VV1QZKnAZe3PYqWAq8CfrYS9UuSJEmSpCkkVa7i0cRtudN2tf8/zh50GZP234d8ZNAlSJIkSZK02iRZUFVDox1z2ZckSZIkSVKHuexrQJLsDpw+ovn+qtp3EPVIkiRJkqRuMvwZkKpaAswcdB2SJEmSJKnbXPYlSZIkSZLUYYY/kiRJkiRJHWb4I0mSJEmS1GHu+aNJ2XmrJ/qz6ZIkSZIkrUWc+SNJkiRJktRhhj+SJEmSJEkdZvgjSZIkSZLUYYY/kiRJkiRJHWb4I0mSJEmS1GH+2pcm5fo7buVF5/zTwMb/ysveOrCxJUmSJElaGznzR5IkSZIkqcMMfyRJkiRJkjrM8EeSJEmSJKnDDH8kSZIkSZI6zPBHkiRJkiSpwwx/JEmSJEmSOszwR5IkSZIkqcMMfyRJkiRJkjrM8GeCkhyVZIdB1zFSkhuTbNOeXzboeiRJkiRJ0tRi+DNxRwFTLvzpV1X7D7oGSZIkSZI0tRj+jJBkRpKr+14f314PAZ9NsijJJm3Gzd8nuTzJ/CTPTDI3yY+SHDNO/wcmuTjJF5L8IMn7kxyZ5MokS5Ls2M7bNskXk8xrf57d2h+d5IIk303yCSB9fS/tG+P8vvaPJjmqPZ903UmObufOf+DOe1bm7ZUkSZIkSWuY4c/EnA3MB46sqplVdV9rv6mq9gMuBU4FDgOeBZy0nP72BI4DdgdeDexSVfsAnwSObed8CPhgVe0NvLwdA3g38K2q2gs4D3jiCtzPpOquqjlVNVRVQxtuudkKDCdJkiRJkgZl2qALWMud1x6XAJtX1d3A3Ul+lWSrqrpjjOvmVdUtAEl+BFzQ18/z2/ODgKcnv57Ys0WS6cBzgZcBVNVXkty+BuuWJEmSJElrGcOf3/YQy86I2nicc+9vj4/0PR9+Pd57O/Lc/n6Gr1sP2K9vlhEALQyqcfqG5d/DitYtSZIkSZLWMi77+m23Atu1vXU2Al7c2u8Gpq/BOi4A/mL4RZKZ7eklwJGt7feBrUe59if0Zg1tlGRL4IWrt1RJkiRJkjRVGf6MUFUP0tv75grgfODaduhU4OPDGz6vgVLeBAwlWZzke8DwZszvAZ6bZCFwMPA/Iy+sqpuALwCLgc8C310D9UqSJEmSpCkoVctbQST9xpY7PaGe8w9vHtj4X3nZWwc2tiRJkiRJU1WSBVU1NNoxZ/5IkiRJkiR1mJv7riZJdgdOH9F8f1XtO4h6JEmSJEnSusnwZzWpqiXAzEHXIUmSJEmS1m0u+5IkSZIkSeowwx9JkiRJkqQOc9mXJmXnrR7jL25JkiRJkrQWceaPJEmSJElShxn+SJIkSZIkdZjhjyRJkiRJUocZ/kiSJEmSJHWY4Y8kSZIkSVKH+WtfmpQf3v5zXvzFT62x8c5/+evW2FiSJEmSJHWRM38kSZIkSZI6zPBHkiRJkiSpwwx/JEmSJEmSOszwR5IkSZIkqcMMfyRJkiRJkjrM8EeSJEmSJKnDDH8kSZIkSZI6zPBHkiRJkiSpw9b58CfJUUl2WIPjnZjk+FXc55uTbLoS1x+YZP9VWZMkSZIkSZoa1vnwBzgKWGPhz2ryZmCFwx/gQMDwR5IkSZKkDlpnwp8kM5Jc3ff6+PZ6CPhskkVJNklyY5K/T3J5kvlJnplkbpIfJTlmnP43T3JhkoVJliR5ad+xE5Jcl+TrwFP72ndK8vUkV7XrdkzPB5Jc3fqZ3c49MMk3k5yd5Nokn23nvoleeHVRkovauQe3+hcmOSvJ5q39xiTv6atx1yQzgGOAt7T34IBR7u3o9l7Mf+CupSv3QUiSJEmSpDVqnQl/xnA2MB84sqpmVtV9rf2mqtoPuBQ4FTgMeBZw0jh9/Qo4tKqeCTwf+KcWzswCjgD2Al4G7N13zWeBf62qPenNvLmlnTMT2BM4CPhAku3b+XvRm+XzdOApwLOr6sPAzcDzq+r5SbYB3gkc1GqZD/xl35g/b+2nAMdX1Y3Ax4EPtvfg0pE3VlVzqmqoqoY23GLzcd4CSZIkSZI01UwbdAFT1HntcQmweVXdDdyd5FdJtqqqO0a5JsDfJ3ku8AjwOOAxwAHAuVV1L0CS89rjdOBxVXUuQFX9qrU/B/h8VT0M3JrkYnqB0V3AlVX103beImAG8K0RdTyLXjj07SQAGwKX9x0/pz0uoBc0SZIkSZKkDluXwp+HWHam08bjnHt/e3yk7/nw67HesyOBbYFZVfVgkhv7xqhRzs8Y/YzV3l8XwMNj1BLga1X1R8vpY6zrJUmSJElSh6xLy75uBbZL8ugkGwEvbu13A9NXQf9bAj9rwc/zgSe19kuAQ9t+QtOBlwBU1V3AT5McApBko/aLXZcAs5Osn2Rb4LnAlcsZu/8evgM8O8lOrd9Nk+wyieslSZIkSVKHrDPhT1U9SG/PniuA84Fr26FTgY8Pb/i8EkN8FhhKMp/eLKBr27gLgTOBRcAX6e0jNOzVwJuSLAYuAx4LnAssBq4CvgG8var+bzljzwH+O8lFVXUbvV8w+3zr9zvArsu5/sv0AqpRN3yWJEmSJElrr1SNtiJJGt1WO86o5/zDu9bYeOe//HVrbCxJkiRJktZWSRZU1dBox9aZmT+SJEmSJEnrIjf8naQkuwOnj2i+v6r2HUQ9kiRJkiRJ4zH8maSqWgLMHHQdkiRJkiRJE+GyL0mSJEmSpA5z5o8mZaett3ETZkmSJEmS1iLO/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jB/7UuT8sPbb+fFZ5292sc5//DDVvsYkiRJkiStC5z5I0mSJEmS1GGGP5IkSZIkSR1m+CNJkiRJktRhhj+SJEmSJEkdZvgjSZIkSZLUYYY/kiRJkiRJHWb4I0mSJEmS1GGGP5IkSZIkSR1m+CNJkiRJktRh61z4k+TEJMevwHUnJTloFdVwYJLzV0VfkiRJkiRJ45k26ALWFlX1rjU9ZpL1q+rhNTzmtKp6aE2OKUmSJEmSVp91YuZPkhOSXJfk68BTW9uOSb6aZEGSS5PsmmTLJDcmWa+ds2mSm5JskOTUJIe19r2TXJbkqiRXJpmeZP0kH0gyL8niJH+2nLK2SHJuku8l+XjfmEvbLKMrgP2S/GWSq9ufN7dz3p7kTe35B5N8oz1/YZLPtFpObdcsSfKWse65tZ+a5J+TXAScPMr7d3SS+UnmP3DXXSv9eUiSJEmSpDWn8zN/kswCjgD2one/C4EFwBzgmKq6Psm+wMeq6gVJrgKeB1wEvASYW1UPJhnub0PgTGB2Vc1LsgVwH/B64M6q2jvJRsC3k1xQVTeMUdo+wNOBnwBfBV4GnA1sBlxdVe9qtb8W2BcIcEWSi4FLgLcCHwaGgI2SbAA8B7gUmAk8rqqe0Wreqo35W/cMvKAd2wU4aLSZRlU1p13LVjvuWMt90yVJkiRJ0pTR+fAHOAA4t6ruBUhyHrAxsD9w1nCoA2zUHs8EZtMLf46gF5D0eypwS1XNA6iqu1q/BwN7DM8OArYEdgbGCn+urKoft2s/Ty+4ORt4GPhiO+c5rfZ72nnntPs5BZiVZDpwP71Aa6gdexNwC/CUJB8BvgJckGTzce4Z4Kw1vcRMkiRJkiStfutC+AMwcrbKesAdVTVzlHPPA96X5FHALOAbI45nlP6G24+tqrkrWNPw61/1hTBhFG0m0o30ZgVdBiwGng/sCHy/qirJnsDvAm8EXgG8mbHvGeCeCdYtSZIkSZLWIuvCnj+XAIcm2aTNlHkJcC9wQ5LDAdKzJ0BVLQWuBD4EnD/KbJhrgR2S7N2unZ5kGjAXeENbfkWSXZJsNk5d+yR5ctvrZzbwrTFqP6TtPbQZcCi9ZV3Dx45vj5cCxwCLWvCzDbBeVX0R+BvgmW2G0qj3LEmSJEmSuqvzM3+qamGSM4FF9PbXGQ5PjgROSfJOYAPgDOCqduxM4CzgwFH6eyDJbOAjSTaht9/PQcAngRnAwvTWVd0GHDJOaZcD7wd2pxfgnDtG7afSC6MAPllV323PLwVOAC6vqnuS/Krv3h4HfHp4E2ngrydwz5IkSZIkqYNS5f69mritdtyxnvP+3/pBsFXu/MMPW/5JkiRJkiQJgCQLqmpotGPrwrIvSZIkSZKkdVbnl30NUpLdgdNHNN9fVfsOoh5JkiRJkrTuMfxZjapqCTBz0HVIkiRJkqR1l8u+JEmSJEmSOsyZP5qUnbbe2s2YJUmSJElaizjzR5IkSZIkqcMMfyRJkiRJkjrM8EeSJEmSJKnDDH8kSZIkSZI6zPBHkiRJkiSpw/y1L03KD2+/i0PO/vpqHeNLhx20WvuXJEmSJGld4swfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jDDnzEkOTHJ8Stw3UlJDlpFNRyY5PxV0dcExvpmkqE1MZYkSZIkSVpzpg26gK6pqnet6TGTrF9VD6/pcSVJkiRJ0tTnzJ8+SU5Icl2SrwNPbW07JvlqkgVJLk2ya5Itk9yYZL12zqZJbkqyQZJTkxzW2vdOclmSq5JcmWR6kvWTfCDJvCSLk/zZcsraIsm5Sb6X5ON9Yy5ts4yuAPZL8q7W59VJ5iRJO++bSU5u4/8gyQGtfZMkZ7QazgQ2Ged9OTrJ/CTzH7jrzpV9myVJkiRJ0hpk+NMkmQUcAewFvAzYux2aAxxbVbOA44GPVdWdwFXA89o5LwHmVtWDff1tCJwJHFdVewIHAfcBrwfurKq92xh/muTJ45S2D/BWYHdgx1YbwGbA1VW1b1V9C/hoVe1dVc+gF+S8uK+PaVW1D/Bm4N2t7Q3AvVW1B/BeYNZYBVTVnKoaqqqhDbfYcpxSJUmSJEnSVGP48xsHAOdW1b1VdRdwHrAxsD9wVpJFwCeA7dv5ZwKz2/Mj2ut+TwVuqap5AFV1V1U9BBwMvKb1dwXwaGDnceq6sqp+3JZ1fR54Tmt/GPhi33nPT3JFkiXAC4Dd+o6d0x4XADPa8+cCn2m1LQYWj1ODJEmSJElaS7nnz7JqxOv1gDuqauYo554HvC/Jo+jNmvnGiOMZpb/h9mOrau4K1jT8+lfD+/wk2Rj4GDBUVTclOZFecDXs/vb4MMt+5qPVJ0mSJEmSOsSZP79xCXBo2wtnOr2lXPcCNyQ5HCA9ewJU1VLgSuBDwPmjbLh8LbBDkr3btdOTTAPmAm9IskFr3yXJZuPUtU+SJ7e9fmYD3xrlnOGg5+dJNgcOm+D9HtlqeAawxwSukSRJkiRJaxln/jRVtbBtfLwI+AlwaTt0JHBKkncCGwBn0NvvB3pLvc4CDhylvweSzAY+kmQTevv9HAR8kt7Sq4VtU+bbgEPGKe1y4P309vy5BDh3lLHuSPJvwBLgRmDeBG75FODTSRa3e75yAtdIkiRJkqS1TKpc+aOJ22rHXerAkz+2Wsf40mEHrdb+JUmSJEnqmiQLqmpotGMu+5IkSZIkSeowl31NAUl2B04f0Xx/Ve07iHokSZIkSVJ3GP5MAVW1BJg56DokSZIkSVL3uOxLkiRJkiSpwwx/JEmSJEmSOsxlX5qUnbbewl/jkiRJkiRpLeLMH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMDd81qT8+Pb7OPyLi1eqj7NevscqqkaSJEmSJC2PM38kSZIkSZI6zPBHkiRJkiSpwwx/JEmSJEmSOszwR5IkSZIkqcMMfyRJkiRJkjrM8EeSJEmSJKnDDH8kSZIkSZI6zPBHkiRJkiSpwwx/JEmSJEmSOszwZzVLcmKS41fgupOSHLQ6apIkSZIkSeuOaYMuQKOrqncNugZJkiRJkrT2c+bPapDkhCTXJfk68NTWtmOSryZZkOTSJLsm2TLJjUnWa+dsmuSmJBskOTXJYa197ySXJbkqyZVJpidZP8kHksxLsjjJn41TT9q5VydZkmR2a98+ySVJFrVjB4xx/dFJ5ieZf/9dt6/y90uSJEmSJK0+zvxZxZLMAo4A9qL3/i4EFgBzgGOq6vok+wIfq6oXJLkKeB5wEfASYG5VPZhkuL8NgTOB2VU1L8kWwH3A64E7q2rvJBsB305yQVXdMEpZLwNmAnsC2wDzklwCvLKN994k6wObjnZPVTWn1c+jdtytVvItkiRJkiRJa5Dhz6p3AHBuVd0LkOQ8YGNgf+Cs4VAH2Kg9ngnMphf+HAF8bER/TwVuqap5AFV1V+v3YGCP4dlBwJbAzsBo4c9zgM9X1cPArUkuBvYG5gGfSrIB8KWqWrQS9y1JkiRJkqYgw5/VY+TsmPWAO6pq5ijnnge8L8mjgFnAN0Yczyj9DbcfW1VzJ1BPRmusqkuSPBd4EXB6kg9U1WkT6E+SJEmSJK0l3PNn1bsEODTJJkmm01vKdS9wQ5LD4dd78OwJUFVLgSuBDwHnt9k5/a4Fdkiyd7t2epJpwFzgDW3WDkl2SbLZODXNbvsEbQs8F7gyyZOAn1XVvwH/DjxzVb0JkiRJkiRpanDmzypWVQuTnAksAn4CXNoOHQmckuSdwAbAGcBV7diZwFnAgaP090DboPkjSTaht9/PQcAngRnAwvTWkt0GHDJGWecC+7XxCnh7Vf1fkj8G3pbkQWAp8JoVvnFJkiRJkjQlpcr9ezVxj9pxt3rhP3x+pfo46+V7rKJqJEmSJEkSQJIFVTU02jGXfUmSJEmSJHWYy746JMnuwOkjmu+vqn0HUY8kSZIkSRo8w58OqaolwMxB1yFJkiRJkqYOl31JkiRJkiR1mDN/NClP2XoTN2yWJEmSJGkt4swfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQO89e+NCm33PEg7z33lnHPOeHQ7ddQNZIkSZIkaXmc+SNJkiRJktRhhj+SJEmSJEkdZvgjSZIkSZLUYYY/kiRJkiRJHWb4I0mSJEmS1GGGP5IkSZIkSR1m+CNJkiRJktRhhj+SJEmSJEkdZvgzAUmOSrLDKujnwCTnr4qaVrUkM5JcPeg6JEmSJEnSqmX4MzFHASsd/kwlSaYNugZJkiRJkrT6Gf70GTn7Jcnx7fUQ8Nkki5JskuTGJH+f5PIk85M8M8ncJD9KcswEx9o7yXeTPCXJrCQXJ1nQ+tk+yY5JFvadv3M7vk+Sc1rbS5Pcl2TDJBsn+XFrn5nkO0kWJzk3ydat/Zut7ouB49q4VyW5HHjjOLUe3e5z/j13/WJF3lpJkiRJkjQghj/LdzYwHziyqmZW1X2t/aaq2g+4FDgVOAx4FnDS8jpMsj/wceClwE3AR4DDqmoW8CngvVX1I+DOJDPbZa9t4ywE9mptBwBXA3sD+wJXtPbTgL+qqj2AJcC7+4bfqqqeV1X/BHwaeFO7jzFV1ZyqGqqqoc22ePTybk+SJEmSJE0hLv1Zcee1xyXA5lV1N3B3kl8l2aqq7hjjuqcBc4CDq+rmJM8AngF8LQnA+sAt7dxPAq9N8pfAbGCfqnooyQ+TPA3YB/hn4LntukuTbEkv4Lm49fEfwFl9458JMMp5pwO/v6JvhiRJkiRJmpoMf5b1EMvOhtp4nHPvb4+P9D0ffj3e+3pL63cv4GYgwDVjzL75Ir1ZO98AFlTV8JqrS+kFNQ8CX6c3I2h94Phxxh12T3sMUBM4X5IkSZIkrcVc9rWsW4Htkjw6yUbAi1v73cD0VTTGHcCLgL9PciBwHbBtkv0AkmyQZDeAqvoVMBc4hd4SrWGXAG8GLq+q24BHA7vSC5HuBG5PckA799XAxYzQZibdmeQ5renIVXR/kiRJkiRpCnHmT5+qejDJSfT2zrkBuLYdOhX4eJL7gHH3x5ngOLcmeQnw38Dr6O0X9OG2FGsa8C/ANe30zwIvAy7o6+IK4DH0QiCAxcDPqmp4Js8ft3o3BX5Mb7+g0bwW+FSSe+mFTJIkSZIkqWPym7xAU1GS44Etq+pvBl0LwON22rP+/ANfHfecEw7dfg1VI0mSJEmSAJIsqKqh0Y4582cKS3IusCPwgkHXIkmSJEmS1k6GP6tBkt3p/XpWv/urat/J9FNVh666qiRJkiRJ0rrI8Gc1qKolwMxB1yFJkiRJkuSvfUmSJEmSJHWY4Y8kSZIkSVKHuexLk7L9Vhv4a16SJEmSJK1FnPkjSZIkSZLUYYY/kiRJkiRJHWb4I0mSJEmS1GGGP5IkSZIkSR1m+CNJkiRJktRhhj+alDtuf4gvnfVzvnTWzwddiiRJkiRJmgDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeqwzoc/SU5McvwKXHdSkoNWR00TGPuQJE8fxNiSJEmSJKlbOh/+rKiqeldVfX1Awx8CrNbwJ8n6q7N/SZIkSZI0NXQy/ElyQpLrknwdeGpr2zHJV5MsSHJpkl2TbJnkxiTrtXM2TXJTkg2SnJrksNa+d5LLklyV5Mok05Osn+QDSeYlWZzkz5ZT09uTLGl9vL+1/Wm7/qokX2zj7w/8IfCBJIta3b9Ve989faf1cVKSpa09rbar25izW/uBSS5K8jlgSZK/TXJcX43vTfKmUWo/Osn8JPPvuusXK/8BSZIkSZKkNWbaoAtY1ZLMAo4A9qJ3fwuBBcAc4Jiquj7JvsDHquoFSa4CngdcBLwEmFtVDyYZ7m9D4ExgdlXNS7IFcB/weuDOqto7yUbAt5NcUFU3jFLT79ObzbNvVd2b5FHt0DlV9W/tnL8DXl9VH0lyHnB+VZ3djl04snbgBcCHgA9V1eeTHNM35MuAmcCewDbAvCSXtGP7AM+oqhuSzADOAT7UArAj2vFlVNWc9v6x044za/mfgiRJkiRJmio6F/4ABwDnVtW9AC1I2RjYHzhrONQBNmqPZwKz6YU/R9ALVvo9FbilquYBVNVdrd+DgT2GZwcBWwI7A78V/gAHAZ8erqmqftnan9FCn62AzYG5Iy9Msvk4te9HL1QC+Bzwj+35c4DPV9XDwK1JLgb2Bu4CrhwOqKrqxiS/SLIX8Bjgu1Xl1B5JkiRJkjqki+EPwMjZKesBd1TVzFHOPQ94X5uNMwv4xojjGaW/4fZjq+q3Apsxzh2tj1OBQ6rqqiRHAQeOcs54tY833ljuGfH6k8BRwGOBT01iDEmSJEmStBbo4p4/lwCHJtkkyXR6S7nuBW5Icjj8ek+cPQGqailwJb0lVOe32TL9rgV2SLJ3u3Z6kmn0Zum8IckGrX2XJJuNUdMFwOuSbNrOHV72NR24pfVxZN/5d7djwzONRq0d+A7w8vb8iBHvwey2L9G2wHPbPY7mXOD36M0MmkiQJUmSJEmS1iKdC3+qaiG9pVyLgC8Cl7ZDRwKvb3v8XAO8tO+yM4FXtceR/T1Ab1nYR9q1X6O3jOyTwPeAhUmuBj7BGDOpquqr9GYYzU+yCBj+6fm/Aa5ofV7bd8kZwNuSfDfJjuPU/mbgL5NcCWwP3NnazwUWA1fRm8n09qr6vzFqe4DekrcvjBJ8SZIkSZKktVyq3L93bdVmEt1XVZXkCOCPquqly7tuRB/r0dsU+/Cqun555++048z6x/d/HYBDDt9mBaqWJEmSJEmrWpIFVTU02rGu7vmzrpgFfDS9naDvAF43mYuTPB04n94G2csNfiRJkiRJ0trH8GcVSrI7cPqI5vurat/VMV5VXUrv59xX9PrvAU9ZdRVJkiRJkqSpxvBnFaqqJcDMQdchSZIkSZI0rHMbPkuSJEmSJOk3nPmjSdlq62lu9CxJkiRJ0lrEmT+SJEmSJEkdZvgjSZIkSZLUYYY/kiRJkiRJHWb4I0mSJEmS1GGGP5IkSZIkSR1m+CNJkiRJktRhhj+SJEmSJEkdZvgjSZIkSZLUYYY/kiRJkiRJHWb4I0mSJEmS1GGGP5IkSZIkSR1m+CNJkiRJktRhhj+SJEmSJEkdZvgjSZIkSZLUYYY/kiRJkiRJHWb4M4okJyY5fgWuOynJQaujppWVZOmqOEeSJEmSJK1dpg26gC6pqncNugZJkiRJkqR+zvxpkpyQ5LokXwee2tp2TPLVJAuSXJpk1yRbJrkxyXrtnE2T3JRkgySnJjmste+d5LIkVyW5Msn0JOsn+UCSeUkWJ/mzceo5MMnFSb6Q5AdJ3p/kyNbXkiQ7tvOelOTC1t+FSZ7Y2p+c5PI21t+O6PttfTW8ZwLvzdFJ5ieZf9ttt63weyxJkiRJktY8wx8gySzgCGAv4GXA3u3QHODYqpoFHA98rKruBK4CntfOeQkwt6oe7OtvQ+BM4Liq2hM4CLgPeD1wZ1Xt3cb40yRPHqe0PYHjgN2BVwO7VNU+wCeBY9s5HwVOq6o9gM8CH27tHwJOaWP9X19tBwM7A/sAM4FZSZ473vtTVXOqaqiqhrbddtvxTpUkSZIkSVOM4U/PAcC5VXVvVd0FnAdsDOwPnJVkEfAJYPt2/pnA7Pb8iPa631OBW6pqHkBV3VVVDwEHA69p/V0BPJpeEDOWeVV1S1XdD/wIuKC1LwFmtOf7AZ9rz08HntOePxv4fF/7sIPbn+8CC4Fdl1ODJEmSJElai7nnz2/UiNfrAXdU1cxRzj0PeF+SRwGzgG+MOJ5R+htuP7aq5k6wpvv7nj/S9/oRxv7saozn/TW8r6o+McEaJEmSJEnSWsyZPz2XAIcm2STJdHpLue4FbkhyOEB69gSoqqXAlfSWVp1fVQ+P6O9aYIcke7drpyeZBswF3pBkg9a+S5LNVrL2y+jNPgI4EvhWe/7tEe3D5gKvS7J5q+FxSbZbyRokSZIkSdIU5cwfoKoWJjkTWAT8BLi0HToSOCXJO4ENgDPo7fcDvaVeZwEHjtLfA0lmAx9Jsgm9/X4OordXzwxgYZIAtwGHrGT5bwI+leRtrb/XtvbjgM8lOQ74Yl9tFyR5GnB5rwSWAq8CfraSdUiSJEmSpCkoVaOtDJJGNzQ0VPPnzx90GZIkSZIkqU+SBVU1NNoxl31JkiRJkiR1mMu+BizJ7iz7a1wA91fVvoOoR5IkSZIkdYvhz4BV1RJg5qDrkCRJkiRJ3eSyL0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/yRJEmSJEnqMMMfSZIkSZKkDjP8kSRJkiRJ6jDDH0mSJEmSpA4z/JEkSZIkSeowwx9JkiRJkqQOM/zpsCQzk/zBoOuQJEmSJEmDY/jTbTOBUcOfJNPWbCmSJEmSJGkQDH8mIMnfJLk2ydeSfD7J8Ul2TPLVJAuSXJpk13buk5JcmGRxe3ziOP0+Jsm5Sa5qf/Zv7X+Z5Or2582tbUaSq/uuPT7Jie35N5OcnOTKJD9IckCSDYGTgNlJFiWZneTEJHOSXACc1uqe2dfnt5PsserfQUmSJEmSNCiGP8uRZAh4ObAX8DJgqB2aAxxbVbOA44GPtfaPAqdV1R7AZ4EPj9P9h4GLq2pP4JnANUlmAa8F9gWeBfxpkr0mUOq0qtoHeDPw7qp6AHgXcGZVzayqM9t5s4CXVtUrgU8CR7X73AXYqKoWj/IeHJ1kfpL5t9122wRKkSRJkiRJU4Xhz/I9B/jPqrqvqu4GvgxsDOwPnJVkEfAJYPt2/n7A59rz09v1Y3kBcApAVT1cVXe288+tqnuqailwDnDABOo8pz0uAGaMc955VXVfe34W8OIkGwCvA04d7YKqmlNVQ1U1tO22206gFEmSJEmSNFW478vyZZS29YA7qmrmBK6vVTAewEMsG9ZtPOL4/e3xYcb/XO/5dWFV9yb5GvBS4BX8ZlaTJEmSJEnqCGf+LN+3gJck2TjJ5sCLgHuBG5IcDpCePdv5lwFHtOdHtuvHciHwhtbH+km2AC4BDkmyaZLNgEOBS4Fbge2SPDrJRsCLJ1D73cD05ZzzSXrLz+ZV1S8n0KckSZIkSVqLGP4sR1XNA84DrqK3tGo+cCe9YOf1Sa4CrqE3ewbgTcBrkywGXg0cN073xwHPT7KE3nKt3apqIb3lV1cCVwCfrKrvVtWD9DZwvgI4H7h2AuVfBDx9eMPnMe5vAXAX8OkJ9CdJkiRJktYyqZrsqqR1T5LNq2ppkk3pzcw5uoU0a70kOwDfBHatqkeWd/7Q0FDNnz9/tdclSZIkSZImLsmCqhp1Oxdn/kzMnLax80Lgix0Kfl5DbybRCRMJfiRJkiRJ0trHDZ8noP0s+gpLcgJw+Ijms6rqvSvT78qqqtOA0wZZgyRJkiRJWr0Mf9aAFvIMNOiRJEmSJEnrJpd9SZIkSZIkdZjhjyRJkiRJUocZ/kiSJEmSJHWY4Y8kSZIkSVKHGf5IkiRJkiR1mOGPJEmSJElShxn+SJIkSZIkdZjhjyRJkiRJUocZ/kiSJEmSJHWY4Y8kSZIkSVKHGf5IkiRJkiR1mOGPJEmSJElShxn+SJIkSZIkdZjhjyRJkiRJUocZ/kiSJEmSJHWY4Y8kSZIkSVKHGf5IkiRJkiR1mOGPJEmSJElShxn+TAFJTkxyfHt+UpKDVrCfmUn+YDnn/GGSd4xxbOmKjCtJkiRJkqauaYMuQMuqqnetxOUzgSHgv8bp/zzgvJUYQ5IkSZIkrUWc+TMASV6TZHGSq5KcPuLYqUkOa89nJbk4yYIkc5Ns39q/meTkJFcm+UGSA5JsCJwEzE6yKMnsMcY+KslH2/MnJ7k8ybwkfztOvUcnmZ9k/m233baq3gZJkiRJkrQGGP6sYUl2A04AXlBVewLHjXHeBsBHgMOqahbwKeC9fadMq6p9gDcD766qB4B3AWdW1cyqOnMC5XwIOKWq9gb+b6yTqmpOVQ1V1dC22247gW4lSZIkSdJU4bKvNe8FwNlV9XOAqvplktHOeyrwDOBr7fj6wC19x89pjwuAGStYy7OBl7fnpwMnr2A/kiRJkiRpijL8WfMC1ATPu6aq9hvj+P3t8WFW7nOcSC2SJEmSJGkt5bKvNe9C4BVJHg2Q5FFjnHcdsG2S/dp5G7QlY+O5G5g+iVq+DRzRnh85ieskSZIkSdJawvBnDauqa+jt3XNxkquAfx7jvAeAw4CT23mLgP2X0/1FwNPH2/B5hOOANyaZB2w5wVuQJEmSJElrkVS56kcTNzQ0VPPnzx90GZIkSZIkqU+SBVU1NNoxZ/5IkiRJkiR1mBs+d1SS1/LbPyP/7ap64yDqkSRJkiRJg2H401FV9Wng04OuQ5IkSZIkDZbLviRJkiRJkjrM8EeSJEmSJKnDDH8kSZIkSZI6zPBHkiRJkiSpwwx/JEmSJEmSOszwR5IkSZIkqcMMfyRJkiRJkjrM8EeSJEmSJKnDDH8kSZIkSZI6zPBHkiRJkiSpwwx/JEmSJEmSOszwR5IkSZIkqcMMfyRJkiRJkjrM8EeSJEmSJKnDDH8kSZIkSZI6zPBHkiRJkiSpwwx/prAkOyQ5exX3+V9Jthql/cQkx6/KsSRJkiRJ0uBNG3QBGltV3Qwctor7/INV2Z8kSZIkSZranPmziiTZLMlXklyV5Ooks5PMSnJxkgVJ5ibZvp37piTfS7I4yRmt7XlJFrU/300yPcmMJFe34xsn+XSSJe3481v7UUnOSfLVJNcn+Yfl1Hljkm3a8xOSXJfk68BTx7nm6CTzk8y/7bbbVtE7JkmSJEmS1gRn/qw6vwfcXFUvAkiyJfDfwEur6rYks4H3Aq8D3gE8uaru71uCdTzwxqr6dpLNgV+N6P+NAFW1e5JdgQuS7NKOzQT2Au4Hrkvykaq6abxik8wCjmjXTQMWAgtGO7eq5gBzAIaGhmoib4YkSZIkSZoanPmz6iwBDkpycpIDgCcAzwC+lmQR8E7g8e3cxcBnk7wKeKi1fRv45yRvAraqqodY1nOA0wGq6lrgJ8Bw+HNhVd1ZVb8Cvgc8aQL1HgCcW1X3VtVdwHmTvmNJkiRJkjTlGf6sIlX1A2AWvRDofcDLgWuqamb7s3tVHdxOfxHwr+38BUmmVdX7gT8BNgG+02b39Ms4w9/f9/xhJj6jy1k8kiRJkiR1nOHPKpJkB+DeqvoM8I/AvsC2SfZrxzdIsluS9YAnVNVFwNuBrYDNk+xYVUuq6mRgPjAy/LkEOLL1tQvwROC6lSj5EuDQJJskmQ68ZCX6kiRJkiRJU5R7/qw6uwMfSPII8CDwBnpLuj7c9v+ZBvwL8APgM60twAer6o4kf9s2cX6Y3tKt/wa27+v/Y8DHkyxp/R7V9gxaoWKramGSM4FF9JaQXbpCHUmSJEmSpCktVa780cQNDQ3V/PnzB12GJEmSJEnqk2RBVQ2NdsxlX5IkSZIkSR3msq+OSnIFsNGI5ldX1ZJB1CNJkiRJkgbD8KejqmrfQdcgSZIkSZIGz2VfkiRJkiRJHWb4I0mSJEmS1GGGP5IkSZIkSR1m+CNJkiRJktRhhj+SJEmSJEkdlqoadA1aiyS5G7hu0HVIU8g2wM8HXYQ0xfi9kJbld0Jalt8JaVmr6jvxpKradrQD/tS7Juu6qhoadBHSVJFkvt8JaVl+L6Rl+Z2QluV3QlrWmvhOuOxLkiRJkiSpwwx/JEmSJEmSOszwR5M1Z9AFSFOM3wnpt/m9kJbld0Jalt8JaVmr/Tvhhs+SJEmSJEkd5swfSZIkSZKkDjP8kSRJkiRJ6jDDH01Ikt9Lcl2SHyZ5x6DrkValJJ9K8rMkV/e1PSrJ15Jc3x637jv21+27cF2S3+1rn5VkSTv24SRp7RslObO1X5Fkxhq9QWmSkjwhyUVJvp/kmiTHtXa/F1pnJdk4yZVJrmrfi/e0dr8XWqclWT/Jd5Oc3177ndA6K8mN7e/yoiTzW9uU+E4Y/mi5kqwP/Cvw+8DTgT9K8vTBViWtUqcCvzei7R3AhVW1M3Bhe037u38EsFu75mPtOwJwCnA0sHP7M9zn64Hbq2on4IPAyavtTqRV4yHgrVX1NOBZwBvb332/F1qX3Q+8oKr2BGYCv5fkWfi9kI4Dvt/32u+E1nXPr6qZVTXUXk+J74ThjyZiH+CHVfXjqnoAOAN46YBrklaZqroE+OWI5pcC/9Ge/wdwSF/7GVV1f1XdAPwQ2CfJ9sAWVXV59XbSP23ENcN9nQ28cDi9l6aiqrqlqha253fT+4/6x+H3Quuw6lnaXm7Q/hR+L7QOS/J44EXAJ/ua/U5Iy5oS3wnDH03E44Cb+l7/tLVJXfaYqroFev8QBrZr7WN9Hx7Xno9sX+aaqnoIuBN49GqrXFqF2nTivYAr8HuhdVxb3rII+Bnwtarye6F13b8Abwce6WvzO6F1WQEXJFmQ5OjWNiW+E9MmfStaF42WJNYar0KaGsb6Poz3PfE7pLVSks2BLwJvrqq7xvk/lvxeaJ1QVQ8DM5NsBZyb5BnjnO73Qp2W5MXAz6pqQZIDJ3LJKG1+J9Q1z66qm5NsB3wtybXjnLtGvxPO/NFE/BR4Qt/rxwM3D6gWaU25tU25pD3+rLWP9X34aXs+sn2Za5JMA7bkt5eZSVNKkg3oBT+frapzWrPfCwmoqjuAb9Lbg8HvhdZVzwb+MMmN9LaFeEGSz+B3Quuwqrq5Pf4MOJfeFipT4jth+KOJmAfsnOTJSTaktynVeQOuSVrdzgP+uD3/Y+A/+9qPaDvtP5neBmxXtimcdyd5Vlt3+5oR1wz3dRjwjbZ+V5qS2t/hfwe+X1X/3HfI74XWWUm2bTN+SLIJcBBwLX4vtI6qqr+uqsdX1Qx6/z74RlW9Cr8TWkcl2SzJ9OHnwMHA1UyR74TLvrRcVfVQkr8A5gLrA5+qqmsGXJa0yiT5PHAgsE2SnwLvBt4PfCHJ64H/AQ4HqKprknwB+B69X0R6Y1sGAPAGer8ctgnw3+0P9P4RfXqSH9JL5o9YA7clrYxnA68GlrT9TQD+H34vtG7bHviP9kss6wFfqKrzk1yO3wupn/9boXXVY+gtCYZe1vK5qvpqknlMge9EDE4lSZIkSZK6y2VfkiRJkiRJHWb4I0mSJEmS1GGGP5IkSZIkSR1m+CNJkiRJktRhhj+SJEmSJEkdZvgjSZIkSZLUYYY/kiRJkiRJHfb/AS7E3YU6TelQAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1296x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(18, 10))\n",
    "sns.barplot(x=column_importances, y=column_importances.index, orient='h')\n",
    "plt.title('Важность оригинальных признаков');"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "caf6e11f-32dd-402c-a10d-346bfd54bba0",
   "metadata": {
    "id": "SD8WsuyWgHfD",
    "tags": []
   },
   "source": [
    "### Сохранение модели"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 589,
   "id": "fe92d155-cb94-4eb6-b8c6-0f977e742f6f",
   "metadata": {
    "id": "XiidXrOEY8bU"
   },
   "outputs": [],
   "source": [
    "# Объединим препроцессор и модель обратно\n",
    "\n",
    "final_pipeline = final_preprocessor.set_params(model=final_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 591,
   "id": "f32d4805-27d6-4127-9735-600928899e0c",
   "metadata": {
    "id": "OAGb1h73XpXq"
   },
   "outputs": [],
   "source": [
    "# Добавим метаданные для модели\n",
    "\n",
    "metadata = {\n",
    "    'name': 'SberAutopodpiska: target event prediction', \n",
    "    'descripton': ('Модель по предсказанию совершения пользователем одного из '\n",
    "                   'целевых действий \"Заказать звонок\" или \"Оставить заявку\" '\n",
    "                   'на сайте сервиса СберАвтоподписка.'), \n",
    "    'model_type': final_model.__class__.__name__, \n",
    "    'version': 1.1, \n",
    "    'training_datetime': datetime.now(), \n",
    "    'author': 'Demir Uzun', \n",
    "    'threshold': best_threshold, \n",
    "    'metrics': {\n",
    "        'roc_auc': roc_auc_score(y_test, test_proba), \n",
    "        'roc_auc_by_class': roc_auc_score(y_test, test_prediction),\n",
    "        'accuracy': accuracy_score(y_test, test_prediction), \n",
    "        'precision': precision_score(y_test, test_prediction), \n",
    "        'recall': recall_score(y_test, test_prediction), \n",
    "        'f1': f1_score(y_test, test_prediction),\n",
    "    }\n",
    "}\n",
    "\n",
    "final_pipeline.metadata = metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 592,
   "id": "4df262ee-b770-42e6-b038-ba2a5c7b38c3",
   "metadata": {
    "id": "y_UVM_E4XeSH"
   },
   "outputs": [],
   "source": [
    "# Сохраним модель\n",
    "\n",
    "models_folder.mkdir(exist_ok=True)\n",
    "filename = f'model_{datetime.now():%Y%m%d%H%M%S}.pkl'\n",
    "\n",
    "with open(models_folder / filename, 'wb') as file:\n",
    "    dill.dump(final_pipeline, file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0bb39fad-d0da-41be-9105-f20ee93dc2b1",
   "metadata": {
    "id": "YuL-rxUw4JmK"
   },
   "source": [
    "## Выводы"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "768b39bc-e7fe-4b36-b22d-c4da4e389eb2",
   "metadata": {
    "id": "Cq2Ps1_Ymsw4"
   },
   "source": [
    "Для преобразования входных данных, со структурой как в файле `ga_sessions.csv`, в удобный для предсказания вид понадобилось четыре этапа:\n",
    "1. Заполнение пропусков и генерация признаков. В том числе добавление новых данных, как-то органический трафик или расстояние до Москвы.\n",
    "2. Преобразование численных переменных: нормализация и удаление выбросов.\n",
    "3. Преобразование категориальных признаков. Основная сложность с ними была в многообразии редких уникальных значений. В итоге только самые популярные значения были закодированы методом one-hot.\n",
    "4. Удаление дублирующих и коррелируемых признаков. Признаки могут коррелировать до 0.95, но именно с таким порогом финальная модель даёт лучший результат.\n",
    "\n",
    "Было проверено 10 моделей с гиперпараметрами по умолчанию. В тройке лучших оказались алгоритмы бустинга от sklearn, lightgbm и catboost.\n",
    "\n",
    "В итоге в качестве лучшей модели был выбран `LightGBM` по следующим причинам: \n",
    "+ Один из лучших показателей `roc_auc`.\n",
    "+ Быстрое обучение. \n",
    "+ Модель интерпретируема, то есть можно получить показатели важности признаков.\n",
    "+ Может предсказывать вероятность класса.\n",
    "+ Нет проблем с процессорами без SSE4 как у CatBoost.\n",
    "\n",
    "Шаг оптимизации модели помог выбрать лучшие гиперпараметры для модели: n_estimators=800, learning_rate=0.07, reg_lambda=10, num_leaves=26, reg_alpha=10, boosting_type='goss'.\n",
    "\n",
    "Качество модели по метрике `roc-auc` составляет **0.7148** (0.6535 при предсказании классов). Но метрика `f1`=0.0871 оставляет желать лучшего. Тем не менее, переобучения нет и цель проекта выполнена - `roc-auc` > 0.65.\n",
    "\n",
    "Для улучшения качества предсказания можно было бы: \n",
    "1. Увеличить количество данных. Тестовая выборка пойдёт на дообучение модели, так что может модель будет лучше в конечном итоге.\n",
    "2. Провести ребалансировку классов. Но если уменьшить выборку с отрицательной целевой переменной, то модель теряет в качестве, а если увеличивать выборку с положительным классом, то модели обучаются слишком долго. \n",
    "3. Провести более тщательный поиск наилучших гиперпараметров - это займёт много времени, а прирост качества будет небольшим. \n",
    "4. Провести дополнительную генерацию признаков. Но хорошие идеи придумывать сложно.\n",
    "5. Попробовать более сложные модели, например, нейронные сети глубокого обучения. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 593,
   "id": "19217d2f-faac-4cee-a187-f9d1330df04a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 622,
   "id": "c6f079ff-e3aa-4c11-ab90-50890471814d",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('data/exemples/examples.json', 'rb') as file:\n",
    "    examples = json.load(file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 623,
   "id": "8734404d-5c3f-43d5-8a64-b08cea31ce3c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Запрос предсказания класса для одного объекта.\n"
     ]
    }
   ],
   "source": [
    "print('Запрос предсказания класса для одного объекта.')\n",
    "data = json.dumps(examples[0])#.encode(\"utf-8\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 655,
   "id": "84c3699c-7a5e-4e98-b018-ff7d31886dc3",
   "metadata": {},
   "outputs": [],
   "source": [
    "#df = pd.DataFrame.from_dict([examples[0]])\n",
    "df = pd.DataFrame.from_dict(examples)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 657,
   "id": "04576a8c-c1b3-419c-8a0a-abd6f559f5af",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5, 18)"
      ]
     },
     "execution_count": 657,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 625,
   "id": "b511a669-f69b-4d35-9d79-5715d1196353",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Expected 2D array, got scalar array instead:\narray={\"session_id\": \"905544597018549464.1636867290.1636867290\", \"client_id\": \"210838531.1636867225\", \"visit_date\": \"2021-11-14\", \"visit_time\": \"08:21:30\", \"visit_number\": 1, \"utm_source\": \"MvfHsxITijuriZxsqZqt\", \"utm_medium\": \"cpm\", \"utm_campaign\": \"FTjNLDyTrXaWYgZymFkV\", \"utm_adcontent\": \"xhoenQgDQsgfEPYNPwKO\", \"utm_keyword\": \"IGUCNvHlhfHpROGclCit\", \"device_category\": \"mobile\", \"device_os\": \"Android\", \"device_brand\": \"Samsung\", \"device_model\": null, \"device_screen_resolution\": \"385x854\", \"device_browser\": \"Samsung Internet\", \"geo_country\": \"Russia\", \"geo_city\": \"Moscow\"}.\nReshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32mC:\\Users\\HPZ2~1\\AppData\\Local\\Temp/ipykernel_9240/1972417851.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmodel\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'model'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpredict_proba\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\lightgbm\\sklearn.py\u001b[0m in \u001b[0;36mpredict_proba\u001b[1;34m(self, X, raw_score, start_iteration, num_iteration, pred_leaf, pred_contrib, **kwargs)\u001b[0m\n\u001b[0;32m    995\u001b[0m                       pred_leaf=False, pred_contrib=False, **kwargs):\n\u001b[0;32m    996\u001b[0m         \u001b[1;34m\"\"\"Docstring is set after definition, using a template.\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 997\u001b[1;33m         \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0msuper\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mraw_score\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstart_iteration\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnum_iteration\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpred_leaf\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpred_contrib\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    998\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mcallable\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_objective\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mand\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mraw_score\u001b[0m \u001b[1;32mor\u001b[0m \u001b[0mpred_leaf\u001b[0m \u001b[1;32mor\u001b[0m \u001b[0mpred_contrib\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    999\u001b[0m             _log_warning(\"Cannot compute class probabilities or labels \"\n",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\lightgbm\\sklearn.py\u001b[0m in \u001b[0;36mpredict\u001b[1;34m(self, X, raw_score, start_iteration, num_iteration, pred_leaf, pred_contrib, **kwargs)\u001b[0m\n\u001b[0;32m    795\u001b[0m             \u001b[1;32mraise\u001b[0m \u001b[0mLGBMNotFittedError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Estimator not fitted, call fit before exploiting the model.\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    796\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mpd_DataFrame\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdt_DataTable\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 797\u001b[1;33m             \u001b[0mX\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_LGBMCheckArray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maccept_sparse\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mTrue\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mforce_all_finite\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mFalse\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    798\u001b[0m         \u001b[0mn_features\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    799\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_n_features\u001b[0m \u001b[1;33m!=\u001b[0m \u001b[0mn_features\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py\u001b[0m in \u001b[0;36mcheck_array\u001b[1;34m(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator, input_name)\u001b[0m\n\u001b[0;32m    892\u001b[0m             \u001b[1;31m# If input is scalar raise error\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    893\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0marray\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mndim\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 894\u001b[1;33m                 raise ValueError(\n\u001b[0m\u001b[0;32m    895\u001b[0m                     \u001b[1;34m\"Expected 2D array, got scalar array instead:\\narray={}.\\n\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    896\u001b[0m                     \u001b[1;34m\"Reshape your data either using array.reshape(-1, 1) if \"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: Expected 2D array, got scalar array instead:\narray={\"session_id\": \"905544597018549464.1636867290.1636867290\", \"client_id\": \"210838531.1636867225\", \"visit_date\": \"2021-11-14\", \"visit_time\": \"08:21:30\", \"visit_number\": 1, \"utm_source\": \"MvfHsxITijuriZxsqZqt\", \"utm_medium\": \"cpm\", \"utm_campaign\": \"FTjNLDyTrXaWYgZymFkV\", \"utm_adcontent\": \"xhoenQgDQsgfEPYNPwKO\", \"utm_keyword\": \"IGUCNvHlhfHpROGclCit\", \"device_category\": \"mobile\", \"device_os\": \"Android\", \"device_brand\": \"Samsung\", \"device_model\": null, \"device_screen_resolution\": \"385x854\", \"device_browser\": \"Samsung Internet\", \"geo_country\": \"Russia\", \"geo_city\": \"Moscow\"}.\nReshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample."
     ]
    }
   ],
   "source": [
    "y = model['model'].predict_proba(data)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 627,
   "id": "21bda15e-6e92-45d7-816d-b83e0232f925",
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'str' object has no attribute 'reshape'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[1;32mC:\\Users\\HPZ2~1\\AppData\\Local\\Temp/ipykernel_9240/1725176514.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mAttributeError\u001b[0m: 'str' object has no attribute 'reshape'"
     ]
    }
   ],
   "source": [
    "data.reshape(-1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 629,
   "id": "c20ee7e8-f839-4747-9050-bbb78298cbf8",
   "metadata": {
    "id": "JWVR-muRJ77Y"
   },
   "outputs": [],
   "source": [
    "my_pipi = Pipeline(steps=[\n",
    "\n",
    "    # Создание дополнительных признаков и\n",
    "    # Приведение датафрейма к удобному виду \n",
    "    ('indexer', FunctionTransformer(set_index)), \n",
    "    ('imputer', FunctionTransformer(fill_missings)), \n",
    "    ('engineer', FunctionTransformer(create_features)), \n",
    "    ('dropper', DropFeatures(['client_id', 'visit_date', 'visit_time', \n",
    "                              'device_screen_resolution'])), \n",
    "\n",
    "    # Преобразования численных переменных\n",
    "    ('normalization', YeoJohnsonTransformer()), \n",
    "    ('outlier_remover', Winsorizer()), \n",
    "    ('scaler', SklearnTransformerWrapper(StandardScaler())), \n",
    "\n",
    "    # Преобразования категориальных признаков\n",
    "    ('rare_encoder', RareLabelEncoder(tol=0.047319, replace_with='rare')),\n",
    "    ('onehot_encoder', OneHotEncoder(drop_last_binary=True)), \n",
    "    ('bool_converter', FunctionTransformer(converse_types)), \n",
    "\n",
    "    # Удаление дубликатов и коррелируемых признаков\n",
    "    ('constant_dropper', DropConstantFeatures(tol=0.95579)), \n",
    "    ('duplicated_dropper', DropDuplicateFeatures()), \n",
    "    ('correlated_dropper', DropCorrelatedFeatures(threshold=0.8856)), \n",
    "\n",
    "    # # Лучшая модель с оптимизированными гиперпараметрами\n",
    "    # ('model', LGBMClassifier(\n",
    "    #     random_state=RANDOM_SEED, learning_rate=0.04440, boosting_type='gbdt', \n",
    "    #     n_estimators=4726, reg_lambda=38.7116, reg_alpha=13.22778, num_leaves=67)), \n",
    "    \n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 630,
   "id": "27176265-1633-46de-b6bb-d33bdd876f66",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-27 {color: black;background-color: white;}#sk-container-id-27 pre{padding: 0;}#sk-container-id-27 div.sk-toggleable {background-color: white;}#sk-container-id-27 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-27 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-27 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-27 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-27 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-27 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-27 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-27 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-27 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-27 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-27 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-27 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-27 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-27 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-27 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-27 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-27 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-27 div.sk-item {position: relative;z-index: 1;}#sk-container-id-27 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-27 div.sk-item::before, #sk-container-id-27 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-27 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-27 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-27 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-27 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-27 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-27 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-27 div.sk-label-container {text-align: center;}#sk-container-id-27 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-27 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-27\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;indexer&#x27;,\n",
       "                 FunctionTransformer(func=&lt;function set_index at 0x0000020CF5CC4940&gt;)),\n",
       "                (&#x27;imputer&#x27;,\n",
       "                 FunctionTransformer(func=&lt;function fill_missings at 0x0000020CF5CC4040&gt;)),\n",
       "                (&#x27;engineer&#x27;,\n",
       "                 FunctionTransformer(func=&lt;function create_features at 0x0000020CCAA9D4C0&gt;)),\n",
       "                (&#x27;dropper&#x27;,\n",
       "                 DropFeatures(features_to_drop=[&#x27;client_id&#x27;, &#x27;visit_date&#x27;,\n",
       "                                                &#x27;visit_time&#x27;,\n",
       "                                                &#x27;device_s...\n",
       "                 RareLabelEncoder(replace_with=&#x27;rare&#x27;, tol=0.047319)),\n",
       "                (&#x27;onehot_encoder&#x27;, OneHotEncoder(drop_last_binary=True)),\n",
       "                (&#x27;bool_converter&#x27;,\n",
       "                 FunctionTransformer(func=&lt;function converse_types at 0x0000020CF5CC4D30&gt;)),\n",
       "                (&#x27;constant_dropper&#x27;, DropConstantFeatures(tol=0.95579)),\n",
       "                (&#x27;duplicated_dropper&#x27;, DropDuplicateFeatures()),\n",
       "                (&#x27;correlated_dropper&#x27;,\n",
       "                 DropCorrelatedFeatures(threshold=0.8856))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-121\" type=\"checkbox\" ><label for=\"sk-estimator-id-121\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Pipeline</label><div class=\"sk-toggleable__content\"><pre>Pipeline(steps=[(&#x27;indexer&#x27;,\n",
       "                 FunctionTransformer(func=&lt;function set_index at 0x0000020CF5CC4940&gt;)),\n",
       "                (&#x27;imputer&#x27;,\n",
       "                 FunctionTransformer(func=&lt;function fill_missings at 0x0000020CF5CC4040&gt;)),\n",
       "                (&#x27;engineer&#x27;,\n",
       "                 FunctionTransformer(func=&lt;function create_features at 0x0000020CCAA9D4C0&gt;)),\n",
       "                (&#x27;dropper&#x27;,\n",
       "                 DropFeatures(features_to_drop=[&#x27;client_id&#x27;, &#x27;visit_date&#x27;,\n",
       "                                                &#x27;visit_time&#x27;,\n",
       "                                                &#x27;device_s...\n",
       "                 RareLabelEncoder(replace_with=&#x27;rare&#x27;, tol=0.047319)),\n",
       "                (&#x27;onehot_encoder&#x27;, OneHotEncoder(drop_last_binary=True)),\n",
       "                (&#x27;bool_converter&#x27;,\n",
       "                 FunctionTransformer(func=&lt;function converse_types at 0x0000020CF5CC4D30&gt;)),\n",
       "                (&#x27;constant_dropper&#x27;, DropConstantFeatures(tol=0.95579)),\n",
       "                (&#x27;duplicated_dropper&#x27;, DropDuplicateFeatures()),\n",
       "                (&#x27;correlated_dropper&#x27;,\n",
       "                 DropCorrelatedFeatures(threshold=0.8856))])</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-122\" type=\"checkbox\" ><label for=\"sk-estimator-id-122\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">FunctionTransformer</label><div class=\"sk-toggleable__content\"><pre>FunctionTransformer(func=&lt;function set_index at 0x0000020CF5CC4940&gt;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-123\" type=\"checkbox\" ><label for=\"sk-estimator-id-123\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">FunctionTransformer</label><div class=\"sk-toggleable__content\"><pre>FunctionTransformer(func=&lt;function fill_missings at 0x0000020CF5CC4040&gt;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-124\" type=\"checkbox\" ><label for=\"sk-estimator-id-124\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">FunctionTransformer</label><div class=\"sk-toggleable__content\"><pre>FunctionTransformer(func=&lt;function create_features at 0x0000020CCAA9D4C0&gt;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-125\" type=\"checkbox\" ><label for=\"sk-estimator-id-125\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">DropFeatures</label><div class=\"sk-toggleable__content\"><pre>DropFeatures(features_to_drop=[&#x27;client_id&#x27;, &#x27;visit_date&#x27;, &#x27;visit_time&#x27;,\n",
       "                               &#x27;device_screen_resolution&#x27;])</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-126\" type=\"checkbox\" ><label for=\"sk-estimator-id-126\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">YeoJohnsonTransformer</label><div class=\"sk-toggleable__content\"><pre>YeoJohnsonTransformer()</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-127\" type=\"checkbox\" ><label for=\"sk-estimator-id-127\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Winsorizer</label><div class=\"sk-toggleable__content\"><pre>Winsorizer()</pre></div></div></div><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-128\" type=\"checkbox\" ><label for=\"sk-estimator-id-128\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">scaler: SklearnTransformerWrapper</label><div class=\"sk-toggleable__content\"><pre>SklearnTransformerWrapper(transformer=StandardScaler())</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-129\" type=\"checkbox\" ><label for=\"sk-estimator-id-129\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">transformer: StandardScaler</label><div class=\"sk-toggleable__content\"><pre>StandardScaler()</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-130\" type=\"checkbox\" ><label for=\"sk-estimator-id-130\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">StandardScaler</label><div class=\"sk-toggleable__content\"><pre>StandardScaler()</pre></div></div></div></div></div></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-131\" type=\"checkbox\" ><label for=\"sk-estimator-id-131\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RareLabelEncoder</label><div class=\"sk-toggleable__content\"><pre>RareLabelEncoder(replace_with=&#x27;rare&#x27;, tol=0.047319)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-132\" type=\"checkbox\" ><label for=\"sk-estimator-id-132\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OneHotEncoder</label><div class=\"sk-toggleable__content\"><pre>OneHotEncoder(drop_last_binary=True)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-133\" type=\"checkbox\" ><label for=\"sk-estimator-id-133\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">FunctionTransformer</label><div class=\"sk-toggleable__content\"><pre>FunctionTransformer(func=&lt;function converse_types at 0x0000020CF5CC4D30&gt;)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-134\" type=\"checkbox\" ><label for=\"sk-estimator-id-134\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">DropConstantFeatures</label><div class=\"sk-toggleable__content\"><pre>DropConstantFeatures(tol=0.95579)</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-135\" type=\"checkbox\" ><label for=\"sk-estimator-id-135\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">DropDuplicateFeatures</label><div class=\"sk-toggleable__content\"><pre>DropDuplicateFeatures()</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-136\" type=\"checkbox\" ><label for=\"sk-estimator-id-136\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">DropCorrelatedFeatures</label><div class=\"sk-toggleable__content\"><pre>DropCorrelatedFeatures(threshold=0.8856)</pre></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "Pipeline(steps=[('indexer',\n",
       "                 FunctionTransformer(func=<function set_index at 0x0000020CF5CC4940>)),\n",
       "                ('imputer',\n",
       "                 FunctionTransformer(func=<function fill_missings at 0x0000020CF5CC4040>)),\n",
       "                ('engineer',\n",
       "                 FunctionTransformer(func=<function create_features at 0x0000020CCAA9D4C0>)),\n",
       "                ('dropper',\n",
       "                 DropFeatures(features_to_drop=['client_id', 'visit_date',\n",
       "                                                'visit_time',\n",
       "                                                'device_s...\n",
       "                 RareLabelEncoder(replace_with='rare', tol=0.047319)),\n",
       "                ('onehot_encoder', OneHotEncoder(drop_last_binary=True)),\n",
       "                ('bool_converter',\n",
       "                 FunctionTransformer(func=<function converse_types at 0x0000020CF5CC4D30>)),\n",
       "                ('constant_dropper', DropConstantFeatures(tol=0.95579)),\n",
       "                ('duplicated_dropper', DropDuplicateFeatures()),\n",
       "                ('correlated_dropper',\n",
       "                 DropCorrelatedFeatures(threshold=0.8856))])"
      ]
     },
     "execution_count": 630,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_pipi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 645,
   "id": "b728a2c8-055f-472d-90d5-f8ea0069fc7e",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "The resulting dataframe will have no columns after dropping all constant or quasi-constant features. Try changing the tol value.",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32mC:\\Users\\HPZ2~1\\AppData\\Local\\Temp/ipykernel_9240/3470311679.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0md\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmy_pipi\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdf\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\sklearn\\pipeline.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, **fit_params)\u001b[0m\n\u001b[0;32m    399\u001b[0m         \"\"\"\n\u001b[0;32m    400\u001b[0m         \u001b[0mfit_params_steps\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_check_fit_params\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m**\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 401\u001b[1;33m         \u001b[0mXt\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_fit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mfit_params_steps\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    402\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0m_print_elapsed_time\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Pipeline\"\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_log_message\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msteps\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m-\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    403\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_final_estimator\u001b[0m \u001b[1;33m!=\u001b[0m \u001b[1;34m\"passthrough\"\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\sklearn\\pipeline.py\u001b[0m in \u001b[0;36m_fit\u001b[1;34m(self, X, y, **fit_params_steps)\u001b[0m\n\u001b[0;32m    357\u001b[0m                 \u001b[0mcloned_transformer\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mclone\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtransformer\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    358\u001b[0m             \u001b[1;31m# Fit or load from cache the current transformer\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 359\u001b[1;33m             X, fitted_transformer = fit_transform_one_cached(\n\u001b[0m\u001b[0;32m    360\u001b[0m                 \u001b[0mcloned_transformer\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    361\u001b[0m                 \u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\joblib\\memory.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    347\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    348\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 349\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfunc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    350\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    351\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mcall_and_shelve\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\sklearn\\pipeline.py\u001b[0m in \u001b[0;36m_fit_transform_one\u001b[1;34m(transformer, X, y, weight, message_clsname, message, **fit_params)\u001b[0m\n\u001b[0;32m    891\u001b[0m     \u001b[1;32mwith\u001b[0m \u001b[0m_print_elapsed_time\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmessage_clsname\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    892\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtransformer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"fit_transform\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 893\u001b[1;33m             \u001b[0mres\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtransformer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit_transform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    894\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    895\u001b[0m             \u001b[0mres\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtransformer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtransform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\sklearn\\utils\\_set_output.py\u001b[0m in \u001b[0;36mwrapped\u001b[1;34m(self, X, *args, **kwargs)\u001b[0m\n\u001b[0;32m    138\u001b[0m     \u001b[1;33m@\u001b[0m\u001b[0mwraps\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mf\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    139\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mwrapped\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 140\u001b[1;33m         \u001b[0mdata_to_wrap\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    141\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata_to_wrap\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtuple\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    142\u001b[0m             \u001b[1;31m# only wrap the first output for cross decomposition\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\sklearn\\base.py\u001b[0m in \u001b[0;36mfit_transform\u001b[1;34m(self, X, y, **fit_params)\u001b[0m\n\u001b[0;32m    876\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0my\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    877\u001b[0m             \u001b[1;31m# fit method of arity 1 (unsupervised transformation)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 878\u001b[1;33m             \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtransform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    879\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    880\u001b[0m             \u001b[1;31m# fit method of arity 2 (supervised transformation)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\feature_engine\\selection\\drop_constant_features.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y)\u001b[0m\n\u001b[0;32m    175\u001b[0m         \u001b[1;31m# check we are not dropping all the columns in the df\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    176\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfeatures_to_drop_\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m==\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 177\u001b[1;33m             raise ValueError(\n\u001b[0m\u001b[0;32m    178\u001b[0m                 \u001b[1;34m\"The resulting dataframe will have no columns after dropping all \"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    179\u001b[0m                 \u001b[1;34m\"constant or quasi-constant features. Try changing the tol value.\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: The resulting dataframe will have no columns after dropping all constant or quasi-constant features. Try changing the tol value."
     ]
    }
   ],
   "source": [
    "d = my_pipi.fit(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 646,
   "id": "fedb0d94-dbc9-4040-9ba0-a3d2f4b2a679",
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'd' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32mC:\\Users\\HPZ2~1\\AppData\\Local\\Temp/ipykernel_9240/3161387801.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0md\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m: name 'd' is not defined"
     ]
    }
   ],
   "source": [
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 664,
   "id": "f8e614cf-517c-4d84-9158-7900626e0551",
   "metadata": {},
   "outputs": [],
   "source": [
    "wwwwww = df.iloc[[0]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 665,
   "id": "cac5786d-4832-4099-bd99-31571b4f7888",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>session_id</th>\n",
       "      <th>client_id</th>\n",
       "      <th>visit_date</th>\n",
       "      <th>visit_time</th>\n",
       "      <th>visit_number</th>\n",
       "      <th>utm_source</th>\n",
       "      <th>utm_medium</th>\n",
       "      <th>utm_campaign</th>\n",
       "      <th>utm_adcontent</th>\n",
       "      <th>utm_keyword</th>\n",
       "      <th>device_category</th>\n",
       "      <th>device_os</th>\n",
       "      <th>device_brand</th>\n",
       "      <th>device_model</th>\n",
       "      <th>device_screen_resolution</th>\n",
       "      <th>device_browser</th>\n",
       "      <th>geo_country</th>\n",
       "      <th>geo_city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>905544597018549464.1636867290.1636867290</td>\n",
       "      <td>210838531.1636867225</td>\n",
       "      <td>2021-11-14</td>\n",
       "      <td>08:21:30</td>\n",
       "      <td>1</td>\n",
       "      <td>MvfHsxITijuriZxsqZqt</td>\n",
       "      <td>cpm</td>\n",
       "      <td>FTjNLDyTrXaWYgZymFkV</td>\n",
       "      <td>xhoenQgDQsgfEPYNPwKO</td>\n",
       "      <td>IGUCNvHlhfHpROGclCit</td>\n",
       "      <td>mobile</td>\n",
       "      <td>Android</td>\n",
       "      <td>Samsung</td>\n",
       "      <td>None</td>\n",
       "      <td>385x854</td>\n",
       "      <td>Samsung Internet</td>\n",
       "      <td>Russia</td>\n",
       "      <td>Moscow</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 session_id             client_id  visit_date  \\\n",
       "0  905544597018549464.1636867290.1636867290  210838531.1636867225  2021-11-14   \n",
       "\n",
       "  visit_time  visit_number            utm_source utm_medium  \\\n",
       "0   08:21:30             1  MvfHsxITijuriZxsqZqt        cpm   \n",
       "\n",
       "           utm_campaign         utm_adcontent           utm_keyword  \\\n",
       "0  FTjNLDyTrXaWYgZymFkV  xhoenQgDQsgfEPYNPwKO  IGUCNvHlhfHpROGclCit   \n",
       "\n",
       "  device_category device_os device_brand device_model  \\\n",
       "0          mobile   Android      Samsung         None   \n",
       "\n",
       "  device_screen_resolution    device_browser geo_country geo_city  \n",
       "0                  385x854  Samsung Internet      Russia   Moscow  "
      ]
     },
     "execution_count": 665,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wwwwww"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 668,
   "id": "8ee293be-6726-41e5-811a-aff1dfe8e5a3",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2vtF_r6P1sen",
    "outputId": "491885c9-2d6a-4a7e-d1a2-1e5325b51986"
   },
   "outputs": [],
   "source": [
    "we = preprocessor.fit_transform(df)\n",
    "# we3 = preprocessor.transform(df)\n",
    "\n",
    "# print(f'X_train.shape = {we.shape}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 669,
   "id": "ef360303-1f9d-4092-b5a8-725c879d99a4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>visit_number</th>\n",
       "      <th>visit_date_weekday</th>\n",
       "      <th>visit_date_weekend</th>\n",
       "      <th>visit_date_day</th>\n",
       "      <th>visit_time_hour</th>\n",
       "      <th>visit_time_minute</th>\n",
       "      <th>utm_medium_added_is_organic</th>\n",
       "      <th>device_screen_resolution_width</th>\n",
       "      <th>utm_source_ZpYIoDJMcFzVoPFsHGJL</th>\n",
       "      <th>utm_campaign_LEoPHuyFvzoNfnzGgfcd</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>session_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>905544597018549464.1636867290.1636867290</th>\n",
       "      <td>-0.5</td>\n",
       "      <td>1.332099</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-0.466530</td>\n",
       "      <td>-0.365225</td>\n",
       "      <td>0.398810</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.422743</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9055446045651783499.1640648526.1640648526</th>\n",
       "      <td>-0.5</td>\n",
       "      <td>-0.174235</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.861966</td>\n",
       "      <td>-1.625726</td>\n",
       "      <td>1.061528</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-0.431235</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9055447192389856083.1622453074.1622453074</th>\n",
       "      <td>-0.5</td>\n",
       "      <td>-1.062745</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.151198</td>\n",
       "      <td>0.862834</td>\n",
       "      <td>-1.194269</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.065987</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9055466554104774132.1629442326.1629442326</th>\n",
       "      <td>2.0</td>\n",
       "      <td>0.967626</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.097909</td>\n",
       "      <td>-0.085101</td>\n",
       "      <td>-1.194269</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.492653</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9055488359664698205.1635763037.1635763037</th>\n",
       "      <td>-0.5</td>\n",
       "      <td>-1.062745</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.644543</td>\n",
       "      <td>1.213219</td>\n",
       "      <td>0.928199</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.550147</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           visit_number  visit_date_weekday  \\\n",
       "session_id                                                                    \n",
       "905544597018549464.1636867290.1636867290           -0.5            1.332099   \n",
       "9055446045651783499.1640648526.1640648526          -0.5           -0.174235   \n",
       "9055447192389856083.1622453074.1622453074          -0.5           -1.062745   \n",
       "9055466554104774132.1629442326.1629442326           2.0            0.967626   \n",
       "9055488359664698205.1635763037.1635763037          -0.5           -1.062745   \n",
       "\n",
       "                                           visit_date_weekend  visit_date_day  \\\n",
       "session_id                                                                      \n",
       "905544597018549464.1636867290.1636867290                  1.0       -0.466530   \n",
       "9055446045651783499.1640648526.1640648526                 0.0        0.861966   \n",
       "9055447192389856083.1622453074.1622453074                 0.0        1.151198   \n",
       "9055466554104774132.1629442326.1629442326                 0.0        0.097909   \n",
       "9055488359664698205.1635763037.1635763037                 0.0       -1.644543   \n",
       "\n",
       "                                           visit_time_hour  visit_time_minute  \\\n",
       "session_id                                                                      \n",
       "905544597018549464.1636867290.1636867290         -0.365225           0.398810   \n",
       "9055446045651783499.1640648526.1640648526        -1.625726           1.061528   \n",
       "9055447192389856083.1622453074.1622453074         0.862834          -1.194269   \n",
       "9055466554104774132.1629442326.1629442326        -0.085101          -1.194269   \n",
       "9055488359664698205.1635763037.1635763037         1.213219           0.928199   \n",
       "\n",
       "                                           utm_medium_added_is_organic  \\\n",
       "session_id                                                               \n",
       "905544597018549464.1636867290.1636867290                           0.0   \n",
       "9055446045651783499.1640648526.1640648526                          0.0   \n",
       "9055447192389856083.1622453074.1622453074                          1.0   \n",
       "9055466554104774132.1629442326.1629442326                          0.0   \n",
       "9055488359664698205.1635763037.1635763037                          0.0   \n",
       "\n",
       "                                           device_screen_resolution_width  \\\n",
       "session_id                                                                  \n",
       "905544597018549464.1636867290.1636867290                         0.422743   \n",
       "9055446045651783499.1640648526.1640648526                       -0.431235   \n",
       "9055447192389856083.1622453074.1622453074                        0.065987   \n",
       "9055466554104774132.1629442326.1629442326                        1.492653   \n",
       "9055488359664698205.1635763037.1635763037                       -1.550147   \n",
       "\n",
       "                                           utm_source_ZpYIoDJMcFzVoPFsHGJL  \\\n",
       "session_id                                                                   \n",
       "905544597018549464.1636867290.1636867290                               0.0   \n",
       "9055446045651783499.1640648526.1640648526                              1.0   \n",
       "9055447192389856083.1622453074.1622453074                              0.0   \n",
       "9055466554104774132.1629442326.1629442326                              1.0   \n",
       "9055488359664698205.1635763037.1635763037                              1.0   \n",
       "\n",
       "                                           utm_campaign_LEoPHuyFvzoNfnzGgfcd  \n",
       "session_id                                                                    \n",
       "905544597018549464.1636867290.1636867290                                 0.0  \n",
       "9055446045651783499.1640648526.1640648526                                1.0  \n",
       "9055447192389856083.1622453074.1622453074                                0.0  \n",
       "9055466554104774132.1629442326.1629442326                                1.0  \n",
       "9055488359664698205.1635763037.1635763037                                0.0  "
      ]
     },
     "execution_count": 669,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "we"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "857824ce-e74b-4318-9956-82345cca2d26",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}