notebooks/parameter_tuning_ex_03.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# \ud83d\udcdd Exercise M3.02\n",
    "\n",
    "The goal is to find the best set of hyperparameters which maximize the\n",
    "generalization performance on a training set.\n",
    "\n",
    "Here again with limit the size of the training set to make computation\n",
    "run faster. Feel free to increase the `train_size` value if your computer\n",
    "is powerful enough."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "adult_census = pd.read_csv(\"../datasets/adult-census.csv\")\n",
    "\n",
    "target_name = \"class\"\n",
    "target = adult_census[target_name]\n",
    "data = adult_census.drop(columns=[target_name, \"education-num\"])\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "data_train, data_test, target_train, target_test = train_test_split(\n",
    "    data, target, train_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "In this exercise, we will progressively define the classification pipeline\n",
    "and later tune its hyperparameters.\n",
    "\n",
    "Our pipeline should:\n",
    "* preprocess the categorical columns using a `OneHotEncoder` and use a\n",
    "  `StandardScaler` to normalize the numerical data.\n",
    "* use a `LogisticRegression` as a predictive model.\n",
    "\n",
    "Start by defining the columns and the preprocessing pipelines to be applied\n",
    "on each group of columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.compose import make_column_selector as selector\n",
    "\n",
    "# Write your code here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# Write your code here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Subsequently, create a `ColumnTransformer` to redirect the specific columns\n",
    "a preprocessing pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.compose import ColumnTransformer\n",
    "\n",
    "# Write your code here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Assemble the final pipeline by combining the above preprocessor\n",
    "with a logistic regression classifier. Force the maximum number of\n",
    "iterations to `10_000` to ensure that the model will converge."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "# Write your code here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use `RandomizedSearchCV` with `n_iter=20` to find the best set of\n",
    "hyperparameters by tuning the following parameters of the `model`:\n",
    "\n",
    "- the parameter `C` of the `LogisticRegression` with values ranging from\n",
    "  0.001 to 10. You can use a log-uniform distribution\n",
    "  (i.e. `scipy.stats.loguniform`);\n",
    "- the parameter `with_mean` of the `StandardScaler` with possible values\n",
    "  `True` or `False`;\n",
    "- the parameter `with_std` of the `StandardScaler` with possible values\n",
    "  `True` or `False`.\n",
    "\n",
    "Once the computation has completed, print the best combination of parameters\n",
    "stored in the `best_params_` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "from scipy.stats import loguniform\n",
    "\n",
    "# Write your code here."
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "encoding": "# -*- coding: utf-8 -*-",
   "main_language": "python"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}