5_Machine_Learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "131d9420-bd7e-4d50-a69c-0208ef3328eb",
   "metadata": {},
   "source": [
    "<img src=\"images/bwHPC_Logo_cmyk.svg\" width=\"200\" /> <img src=\"images/HochschuleEsslingen_Logo_RGB_DE.png\" width=\"200\" /> <img src=\"images/Konstanz_Logo.svg\" width=\"200\" /> <img src=\"images/KIT_Logo.png\" width=\"200\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "066db463-4114-4054-83da-383d07b2e626",
   "metadata": {},
   "source": [
    "# Machine Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a7930f8-ff17-4042-825d-4dffb15b9b42",
   "metadata": {},
   "source": [
    "Algorithmen die sich durch das Nutzen von Daten automatisch optimieren\n",
    "\n",
    "Welche Probleme können mit ML gelöst werden?\n",
    "\n",
    "Kreditwürdigkeit, Preisvorhersagen, Spamfilter, ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54077fe2-aae4-43b6-9ff2-ec35db9b4ba4",
   "metadata": {},
   "source": [
    "## Überwachtes Lernen"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "624a8a86-ca5a-4e2f-a031-4de08460dfbf",
   "metadata": {},
   "source": [
    " - Man kennt Daten aus der Vergangenheit (Quadratmeter von Haus, Anzahl Zimmer, Garten vorhanden, ...)\n",
    " - Labeled: Man kennt den gewuenschten Output (Man kennt z.B. Preis zu dem Haus in Vergangenheit verkauft wurde. Man will neuen Verkaufspreis schaetzen)\n",
    " - Regression Task: Label das man vorhersagen will ist kontinuierlich (z.B. Euro-Preis)\n",
    " - Classification Task: Einordnen in eine bestimmte Kategorie (Handschrifterkennung)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71ca7e66-8b40-4ae4-92dc-f58e76976afe",
   "metadata": {},
   "source": [
    "## Unüberwachtes Lernen"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05e89655-e1d0-4fbf-b215-cc5664580b30",
   "metadata": {},
   "source": [
    " - Label nicht vorhanden\n",
    " - Label muss selbst gefunden werden"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c83b9064-09db-4ce4-956a-8f084e89df66",
   "metadata": {},
   "source": [
    "## Unterteilung der Daten\n",
    "- X: Features (Daten die zur Schaetzung herangezogen werden)\n",
    "- y: Label (Wert den wir vorhersagen wollen, z.B. Preis einer Taxifahrt)\n",
    "\n",
    "### Training-Datensatz vs. Test-Datensatz\n",
    "- Messen der Zuverlaessigkeit des trainierten Modells\n",
    "- Trainieren mit Tainingsdatensatz und Verifizierung durch Testdatensatz\n",
    "\n",
    "![title](images/train_test.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b138265-32fa-4711-982a-f48a91bdffd1",
   "metadata": {},
   "source": [
    "# Lineare Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffa8c55c-615f-4ac5-8ee8-7b33911202e1",
   "metadata": {},
   "source": [
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/3/3a/Linear_regression.svg/1024px-Linear_regression.svg.png\" alt=\"MAE\"\n",
    "\ttitle=\"Linear Regression\" width=\"500\" />\n",
    "\n",
    "$\\hat y = b_{0}x_{0} + ... + b_{n}x_{n}$\n",
    "\n",
    "Ziel: Finden der $b_{n}$'s für bestee Gerade durch alle Punkte.\n",
    "\n",
    "### Gradientenverfahren\n",
    "<img src=\"https://blog.paperspace.com/content/images/2018/05/gd_basic.png\" alt=\"MAE\"\n",
    "\ttitle=\"Linear Regression\" width=\"500\" />\n",
    "\n",
    "Source: https://blog.paperspace.com/content/images/2018/05/gd_basic.png"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e3b91dc-0cb9-44fb-8027-bc395a5093a6",
   "metadata": {
    "tags": []
   },
   "source": [
    "#  Scikit learn\n",
    "- Beihnhaltet viele Machine Learning Algorithmen\n",
    "- Algorithmen koennen schnell getauscht werden\n",
    "- Algorithmen muessen nicht bis in die Tiefe verstanden werden\n",
    "- Pragmatischer Ansatz\n",
    "\n",
    "- X_train; y_train (Trainingsdaten)\n",
    "- X_test; y_test (Testdaten)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be60adb8-7199-4290-b587-7c0638cb754e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split ##Train-Test-Split erstellen\n",
    "import numpy as np\n",
    "\n",
    "X, y = np.arange(12).reshape((6, 2)), range(6)\n",
    "display(X) # Daten zum Lernen (Teil der Trainingsdaten, die zur Vorhersage genutzt werden)\n",
    "display(y) # Werte, die durch X vorhergesagt werden sollen (hier Index aller Elemente in den Test-Daten)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2cc8a330-5476-4276-a40e-e7cde7591e37",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y) ##Auch Prozentuale Angabe möglich\n",
    "# nach dem Split:\n",
    "display(X_train) # 7 Elemente zum Training\n",
    "display(X_test)  # 3 Elemente zum Test\n",
    "display(y_train) # Index der Trainings-Daten (soll vorhergesagt werden)\n",
    "display(y_test)  # Index der Test-Daten (soll vorhergesagt werden)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1454341-3e84-4e4e-af8a-f449cfe8b5e2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5) ##Auch Prozentuale Angabe möglich\n",
    "# nach dem Split:\n",
    "display(X_train) # 5 Elemente zum Training\n",
    "display(X_test)  # 5 Elemente zum Test\n",
    "display(y_train) # Index der Trainings-Daten\n",
    "display(y_test)  # Index der Test-Daten"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abb31955-52b3-4783-ba13-b653af15e310",
   "metadata": {},
   "outputs": [],
   "source": [
    "##Importieren eines ML-Algorithmus der auf Daten angewendet wird\n",
    "from sklearn.linear_model import LinearRegression\n",
    "model = LinearRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04788493-1ac4-4851-b21f-9d1e4c1c8d34",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.fit(X_train, y_train) ##Das Modell wird trainiert"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c69132bc-1229-4877-8565-11011721215f",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = model.predict(X_test) ##Vorhersagen für X_test treffen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1330a449-4d67-40c7-b3b2-49b2c215c122",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "performance = mean_squared_error(y_test, predictions) # Vergleich zwischen y_test und Vorhersagen des Modells mit ausgewählter Error-Metrik\n",
    "display(y_test)\n",
    "display(predictions)\n",
    "display(performance)\n",
    "# aufgrund des direkten Zusammenhangs der Daten und des Index\n",
    "# (der Index ist die abgerundete Hälfte der einzelnen Elemente in den Trainingsdaten)\n",
    "# ist die Performance selbst bei wenigen Daten sehr gut (MSE nahe 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b57e08c7-13e2-46b3-9685-4899102f2148",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split ##Train-Test-Split erstellen\n",
    "import numpy as np\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "X, y = np.arange(4).reshape((2, 2)), range(2)\n",
    "display(X)\n",
    "display(y)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "model = LinearRegression()\n",
    "model.fit(X_train, y_train)\n",
    "predictions = model.predict(X_test)\n",
    "performance = mean_squared_error(y_test, predictions)\n",
    "display(y_test)\n",
    "display(predictions)\n",
    "display(performance) # mit nur einem Datum zum Training wird die Performance sehr schlecht (MSE nahe 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "842f439b-44bc-490b-9aaa-9d4fb00f3bce",
   "metadata": {},
   "source": [
    "## Erstes kleines ML-Beispiel (ohne Parallelisierung)\n",
    "\n",
    "Basiert auf folgendem Datensatz: https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0cd90994-6ba0-4c1d-a9aa-d4f9b50fa3d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import random\n",
    "import sys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8bb176fa-ef97-4d0e-9b87-fdb76a71f74d",
   "metadata": {},
   "outputs": [],
   "source": [
    "#n = 100  # CSV Einlesen und Datensatz verkleinern --> Nicht gut aber haben auf Jupyter-Node nicht viel Speicher da shared.\n",
    "#df = pd.read_csv('s3://nyc-tlc/trip data/green_tripdata_2019-02.csv', parse_dates=['lpep_pickup_datetime', 'lpep_dropoff_datetime'], header=0, skiprows=lambda i: i % n != 0)\n",
    "## Beispiel: i=5 --> 5 mod 100 != 0 --> lambda gibt true zurück --> Zeile wird übersprungen\n",
    "##\n",
    "## S3 benötigt einen AWS account, daher ist der direkte Zugriff nicht mehr möglich\n",
    "## bitte die lokal bereitgestellte parquet-Datei nutzen (siehe unten)\n",
    "\n",
    "# https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page\n",
    "df = pd.read_parquet('./files/green_tripdata_2023-01.parquet', engine='pyarrow')\n",
    "display(df)\n",
    "df = df.sample(1000) # Datensatz verkleinern --> Nicht gut aber haben auf Jupyter-Node nicht viel Speicher da shared\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c363b1dd-488e-4229-baff-0546a38c3cff",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82a1c8c3-a9a6-4f01-8743-fb4edc2d1298",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "404c294d-bc3b-48de-ad3a-fa8f5fdeedc9",
   "metadata": {},
   "outputs": [],
   "source": [
    "df['pickup_hour'] = df['lpep_pickup_datetime'].dt.hour #Spalte mit 24 Kategorien für 24 Stunden hinzufügen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af0444e4-7431-4b31-aa39-505c7224f87e",
   "metadata": {},
   "outputs": [],
   "source": [
    "df['pickup_hour'].tail(500)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80f52c75-9f2a-4b36-be6e-0547761c63d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "df['ride_duration'] = df['lpep_dropoff_datetime'].sub(df['lpep_pickup_datetime'], axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fa571142-0811-4841-8de7-5702facefce9",
   "metadata": {},
   "outputs": [],
   "source": [
    "df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c66b2f11-51d4-426f-afc6-2ef428ca1151",
   "metadata": {},
   "outputs": [],
   "source": [
    "df['ride_duration_minutes'] = df['ride_duration'].dt.total_seconds().div(60).astype(int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf53ebb4-4809-4a09-a9ce-c1606533969a",
   "metadata": {},
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99fca155-c09a-4c3f-bee5-dc5907118869",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df[['passenger_count', 'trip_distance', 'fare_amount', 'total_amount', 'tip_amount','pickup_hour', 'ride_duration_minutes']] #Bereinigung für Pairplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6bd8f92-80a3-483f-b77f-42f1a31c0d89",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = sns.pairplot(df)\n",
    "fig.savefig(\"output.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ead82ff4-a68e-4e19-ba22-2fd41e300044",
   "metadata": {},
   "outputs": [],
   "source": [
    "##Beispiel mit Matplotlib\n",
    "fig, axes = plt.subplots(nrows=1, ncols=3, figsize=(16,6)) ## 1 Zeile mit 3 Spalten\n",
    "axes[0].plot(df['tip_amount'],df['total_amount'],'o')\n",
    "axes[0].set_ylabel(\"Total Amount\")\n",
    "axes[0].set_title(\"Tip Amount\")\n",
    "\n",
    "axes[1].plot(df['fare_amount'],df['total_amount'],'o')\n",
    "axes[1].set_ylabel(\"Total Amount\")\n",
    "axes[1].set_title(\"Fare Amount\")\n",
    "\n",
    "axes[2].plot(df['trip_distance'],df['total_amount'],'o')\n",
    "axes[2].set_ylabel(\"Total Amount\")\n",
    "axes[2].set_title(\"Trip Distance\")\n",
    "\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99c4702c-a975-4b04-8639-c112d0b57eff",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df[['trip_distance', 'pickup_hour', 'ride_duration_minutes']] ##Definieren der Labels\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "168d3fd2-1859-4416-93ca-ac6f77f2ff11",
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df['tip_amount']\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d73783bc-42a9-4d09-9f71-15c087808547",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cddf64e5-2b0a-4ace-9006-219d55d6fe0f",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size= 0.20, random_state=42) #Definieren der Train-Test-Aufteilung"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7c04e34-dc73-452a-bc7b-21f597956d4c",
   "metadata": {},
   "outputs": [],
   "source": [
    "len(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42440ea4-16c3-4100-95cb-da66638b0e84",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "953da337-0181-4f4a-8237-e72b8e8467b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = LinearRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14153c97-59fb-452d-831e-5c37526fb40b",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d25edbf-04cd-401b-8e18-ec202f2a8e89",
   "metadata": {},
   "outputs": [],
   "source": [
    "prediction = model.predict(X_test)\n",
    "type(prediction)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "352034f9-d82e-4a63-acbd-05f46d0992d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.set_printoptions(threshold=sys.maxsize)\n",
    "prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d0a2c8a-8255-41c7-b858-adab96cbf3e3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "26862056-b7c5-42a8-aa2a-77c31181f6eb",
   "metadata": {},
   "source": [
    "## Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ced8554-b833-423a-bff6-02d8b9dbfa74",
   "metadata": {},
   "source": [
    "Wir haben nun ein trainiertes Modell. Aber: Ist es gut??\n",
    "\n",
    "<img src=\"https://i.imgur.com/19LNbyQ.jpg\" alt=\"MAE\"\n",
    "\ttitle=\"MAE\" width=\"500\" />\n",
    "    \n",
    "Source: https://stackoverflow.com/questions/56401346/mean-absolute-error-in-tensorflow-without-built-in-functions/56401550"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b9fa873-49e6-463f-b87c-b20ac3a220af",
   "metadata": {},
   "source": [
    "Problem: Wenn nur einzelne Werte extrem abweichen würde das nicht erkannt werden......"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7c4a134-fdd4-4f62-a441-aae2baae87ad",
   "metadata": {},
   "source": [
    "Besser: Mean Squared Error (MSE): Durch das Quadrat werden einzelne starke Abweichungen stärker \"bestraft\"\n",
    "\n",
    "<img src=\"https://cdn-media-1.freecodecamp.org/images/hmZydSW9YegiMVPWq2JBpOpai3CejzQpGkNG\" alt=\"MAE\"\n",
    "\ttitle=\"MAE\" width=\"500\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb14f6c1-03bd-4938-9a5e-9a9bdc3f87af",
   "metadata": {},
   "source": [
    "<img src=\"https://miro.medium.com/max/483/1*lqDsPkfXPGen32Uem1PTNg.png\" alt=\"MAE\"\n",
    "\ttitle=\"MAE\" width=\"500\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5fdce03-9f6e-4bde-a597-c66f4f503859",
   "metadata": {},
   "source": [
    "Root Mean Square Error: Bestraft einzelne starke Abweichungen stärker und zieht dann die Wurzel um die original Einheiten zu bekommen. Diese Variante wird oft verwendet."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29877688-aef3-48ce-bde3-d5677349169c",
   "metadata": {},
   "source": [
    "## Frage: Was ist ein guter RMSE und was nicht?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b26599a-47c2-4a42-a87b-cf00714f0d57",
   "metadata": {},
   "source": [
    "Antwort: Es kommt drauf an."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9211a36-337b-467d-9ad1-b2885abea1ca",
   "metadata": {},
   "source": [
    "Ein RMSE von 50€ wäre gut um z.B. den Preis einer Immobilie zu schätzen aber nicht für die Schätzung des Preises von Schokolade"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bbf57eb8-a683-4ff6-9540-7949c395644e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_absolute_error, mean_squared_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "507fd59f-c1b8-4eb4-a7e0-b4055d001580",
   "metadata": {},
   "outputs": [],
   "source": [
    "df['tip_amount'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54311243-f966-42b5-9cd7-003c0af59d51",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.sqrt(mean_squared_error(y_test, prediction))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c28fb66-dfaf-496d-b9a1-cb2b8ebb4540",
   "metadata": {},
   "source": [
    "Sehr schlechter Wert im Vergleich zum Durchschnittlichen Wert von tip_amount --> Kein gutes Modell "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b9bcc99-68e4-47e4-be8d-606647f5b537",
   "metadata": {},
   "source": [
    "## Cross Validation, Grid Search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef129fc8-b6a0-4e30-87bc-940edf6186c6",
   "metadata": {},
   "source": [
    "- Viele ML Algorithmen sind mit Parametern anpassbar. Frage: Welche Parameter liefern die besten Ergebnisse?\n",
    "\n",
    "- Idee: Ich stelle einen Parameter ein und trainiere das Modell und teste mit den Testdaten.\n",
    "\n",
    "- Es ist nicht gut wenn z.B. stets die letzten 20% der Daten als Testdaten verwendet werden. Es kann sein, dass das trainierte Modell mit diesen Daten zufällig sehr gut oder sehr schlecht läuft.\n",
    "\n",
    "- Kompletter Datensatz wird aufgeteilt in Training, Validation und Test. Der Test-Datensatz wird erst verwendet, wenn die Finalen Parameter gefunden wurden.\n",
    "\n",
    "### k-fold Cross Validation\n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/b/b5/K-fold_cross_validation_EN.svg/1920px-K-fold_cross_validation_EN.svg.png\" alt=\"MAE\"\n",
    "\ttitle=\"MAE\" width=\"500\" />\n",
    "\n",
    "1. Parameter des ML-Algos werden festgelegt.\n",
    "2. Es werden k-Iterationen ausgeführt mit jeweils verschiedenen Test- und Trainingsdaten.\n",
    "3. Error jeder Iteration wird berechnet.\n",
    "4. Durschnitts-Error aller Iterationen bewertet die aktuelle Parameterkonfiguration des ML-Algos\n",
    "5. Parameter verändern und Schritte 1 bis 4 wiederholen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f1af81b-3eed-45c8-a805-ff5af81b61d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# nur ein Beispiel: MLAlgo muss durch einen echten ML Algorithmus ersetzt werden\n",
    "from sklearn.model_selection import cross_val_score\n",
    "model = MLAlgo(paramter=xx)\n",
    "score = cross_val_score(model, X_train, y_train, scoring='mean_squared_error', cv=5)\n",
    "\n",
    "## Danach Durchschnitt der Fehler berechnen und dann für nächsten Parameter des ML Algos wiederholen"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4ebcff5-8206-43a6-a05e-5b38310213c1",
   "metadata": {},
   "source": [
    "## Grid Search "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31179219-9ec1-49ca-a86a-4b5fd85fd11a",
   "metadata": {},
   "source": [
    "Bisher mussten Parameter des Algorithmus händisch angepasst werden. GridSearchCV automatisiert dies indem eine Liste an Parametern übergeben werden kann. Am Schluss lässt sich die Konfiguration anzeigen, die die beste Schätzungen geliefert hat."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "729b7913-c405-4fe4-9c39-08298d4fca71",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "df = pd.read_parquet('./files/green_tripdata_2023-01.parquet', engine='pyarrow')\n",
    "df = df.sample(1000)\n",
    "df['pickup_hour'] = df['lpep_pickup_datetime'].dt.hour\n",
    "df['ride_duration'] = df['lpep_dropoff_datetime'].sub(df['lpep_pickup_datetime'], axis=0)\n",
    "df['ride_duration_minutes'] = df['ride_duration'].dt.total_seconds().div(60).astype(int)\n",
    "X = df[['trip_distance', 'pickup_hour', 'ride_duration_minutes']]\n",
    "y = df['tip_amount']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size= 0.20, random_state=42)\n",
    "\n",
    "from sklearn.svm import SVR\n",
    "reg = SVR(C=1)\n",
    "#param_grid = {'param1':[0.1, 0.2, ...], 'param2':[0.4, 0.5,...] }\n",
    "#param_grid = {'param1':[0.1], 'param2':[0.4] }\n",
    "param_grid = {'C': [0.001,0.01,0.1,1,10,100,1000]}\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "grid_model = GridSearchCV(reg, param_grid=param_grid, scoring='neg_mean_squared_error', cv=5, verbose=2)\n",
    "grid_model.fit(X_train, y_train)\n",
    "\n",
    "grid_model.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14c2af8e-cd5a-4444-9f1b-9a3f88f3f1cf",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python_workshop_env",
   "language": "python",
   "name": "python_workshop_env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}