ensemble_python_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Ensembling Custom Python Models in TurboML\n",
    "TurboML allows you to create custom ensemble models using Python classes, leveraging the flexibility of Python while benefiting from TurboML's performance and scalability. In this notebook, we'll walk through how to create a custom ensemble model using TurboML's PythonEnsembleModel interface."
   ],
   "id": "0",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Set up the environment and install TurboML's SDK.\n",
    "We use `turboml-installer` to set up the environment for TurboML's SDK."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "!pip install -q turboml-installer\n",
    "import turboml_installer ; turboml_installer.install_on_colab()"
   ],
   "execution_count": null,
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "The kernel should now be restarted with TurboML's SDK installed."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Login to your TurboML instance\n",
    "\n",
    "Note that you can copy and replace this snippet with one from your TurboML homepage."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "import turboml as tb",
    "\ntb.init(backend_url=BACKEND_URL, api_key=API_KEY)"
   ],
   "execution_count": null,
   "id": "1",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "!pip install river"
   ],
   "execution_count": null,
   "id": "2",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "import pandas as pd\n",
    "import turboml.common.pytypes as types\n",
    "import turboml.common.pymodel as model\n",
    "import logging\n",
    "from typing import List\n",
    "import matplotlib.pyplot as plt"
   ],
   "execution_count": null,
   "id": "3",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Prepare an Evaluation Dataset\n",
    "\n",
    "We choose a standard Credit Card Fraud dataset that ships with River to evaluate our models on."
   ],
   "id": "4",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "features = tb.datasets.CreditCardsDatasetFeatures()\n",
    "labels = tb.datasets.CreditCardsDatasetLabels()\n",
    "\n",
    "features"
   ],
   "execution_count": null,
   "id": "5",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "features.df.loc[0]"
   ],
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Load Datasets into TurboML\n",
    "We'll load the features and labels with our `OnlineDataset` interface."
   ],
   "id": "7",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "features = tb.OnlineDataset.from_local_dataset(\n",
    "    features, \"cc_features\", load_if_exists=True\n",
    ")\n",
    "labels = tb.OnlineDataset.from_local_dataset(labels, \"cc_labels\", load_if_exists=True)"
   ],
   "execution_count": null,
   "id": "8",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Isolate features"
   ],
   "id": "9",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "numerical_cols = features.preview_df.columns.tolist()\n",
    "numerical_cols.remove(\"index\")\n",
    "input_features = features.get_model_inputs(numerical_fields=numerical_cols)\n",
    "label = labels.get_model_labels(label_field=\"score\")"
   ],
   "execution_count": null,
   "id": "10",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Structure of Ensemble Models\n",
    "\n",
    "A custom ensemble model in TurboML must implement three instance methods:\n",
    "\n",
    "- `init_imports`: Import any external modules used in the class.\n",
    "- `learn_one`: Receive labeled data for the model to learn from.\n",
    "- `predict_one`: Receive input features for prediction and output the result.\n",
    "Here's the general structure:\n",
    "\n",
    "```python\n",
    "class CustomEnsembleModel:\n",
    "    def __init__(self, base_models: List[types.Model]):\n",
    "        # Ensure at least one base model is provided\n",
    "        if not base_models:\n",
    "            raise ValueError(\"PythonEnsembleModel requires at least one base model.\")\n",
    "        self.base_models = base_models\n",
    "\n",
    "    def init_imports(self):\n",
    "        \"\"\"\n",
    "        Import any external symbols/modules used in this class\n",
    "        \"\"\"\n",
    "        pass\n",
    "\n",
    "    def learn_one(self, input: types.InputData):\n",
    "        \"\"\"\n",
    "        Receives labelled data for the model to learn from\n",
    "        \"\"\"\n",
    "        pass\n",
    "\n",
    "    def predict_one(self, input: types.InputData, output: types.OutputData):\n",
    "        \"\"\"\n",
    "        Receives input features for a prediction, must pass output to the\n",
    "        output object\n",
    "        \"\"\"\n",
    "        pass\n",
    "```\n"
   ],
   "id": "11",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Example - Creating a Custom Ensemble Model\n",
    "We'll create a custom ensemble model that averages the predictions of its base models."
   ],
   "id": "12",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "class MyEnsembleModel:\n",
    "    def __init__(self, base_models: List[model.Model]):\n",
    "        if not base_models:\n",
    "            raise ValueError(\"PythonEnsembleModel requires at least one base model.\")\n",
    "        self.base_models = base_models\n",
    "        self.logger = logging.getLogger(__name__)\n",
    "\n",
    "    def init_imports(self):\n",
    "        pass\n",
    "\n",
    "    def learn_one(self, input: types.InputData):\n",
    "        try:\n",
    "            for model in self.base_models:\n",
    "                model.learn_one(input)\n",
    "        except Exception as e:\n",
    "            self.logger.exception(f\"Exception in learn_one: {e}\")\n",
    "\n",
    "    def predict_one(self, input: types.InputData, output: types.OutputData):\n",
    "        try:\n",
    "            total_score = 0.0\n",
    "            for model in self.base_models:\n",
    "                model_output = model.predict_one(input)\n",
    "                model_score = model_output.score()\n",
    "                total_score += model_score\n",
    "            average_score = total_score / len(self.base_models)\n",
    "            output.set_score(average_score)\n",
    "        except Exception as e:\n",
    "            self.logger.exception(f\"Exception in predict_one: {e}\")"
   ],
   "execution_count": null,
   "id": "13",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Set Up the Virtual Environment\n",
    "We'll set up a virtual environment and add our custom ensemble class to it. Since our class requires arguments in the constructor, we'll disable validation when adding it."
   ],
   "id": "14",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# Set up the virtual environment\n",
    "venv_name = \"my_ensemble_venv\"\n",
    "venv = tb.setup_venv(venv_name, [\"river\"])\n",
    "\n",
    "# Add the ensemble class without validation\n",
    "venv.add_python_class(MyEnsembleModel, do_validate_as_model=False)"
   ],
   "execution_count": null,
   "id": "15",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Create Base Models\n",
    "We'll use TurboML's built-in models as base models for our ensemble."
   ],
   "id": "16",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# Create individual base models\n",
    "model1 = tb.HoeffdingTreeClassifier(n_classes=2)\n",
    "model2 = tb.AMFClassifier(n_classes=2)"
   ],
   "execution_count": null,
   "id": "17",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "# Create the PythonEnsembleModel\n",
    "ensemble_model = tb.PythonEnsembleModel(\n",
    "    base_models=[model1, model2],\n",
    "    module_name=\"\",\n",
    "    class_name=\"MyEnsembleModel\",\n",
    "    venv_name=venv_name,\n",
    ")"
   ],
   "execution_count": null,
   "id": "18",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Deploy the Ensemble Model\n",
    "We'll deploy the ensemble model, providing the input features and labels."
   ],
   "id": "19",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "deployed_ensemble_model = ensemble_model.deploy(\n",
    "    name=\"ensemble_model\", input=input_features, labels=label\n",
    ")"
   ],
   "execution_count": null,
   "id": "20",
   "metadata": {},
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Evaluate the Ensemble Model\n",
    "We'll add a metric to evaluate the model and plot the results."
   ],
   "id": "21",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# Add a metric to the deployed model\n",
    "deployed_ensemble_model.add_metric(\"WindowedRMSE\")\n",
    "\n",
    "# Retrieve the evaluation results\n",
    "model_rmse_scores = deployed_ensemble_model.get_evaluation(\"WindowedRMSE\")\n",
    "\n",
    "# Plot the RMSE scores\n",
    "plt.figure(figsize=(10, 6))\n",
    "plt.plot([score.metric for score in model_rmse_scores], label=\"Ensemble Model RMSE\")\n",
    "plt.xlabel(\"Time Steps\")\n",
    "plt.ylabel(\"RMSE\")\n",
    "plt.title(\"Ensemble Model Evaluation\")\n",
    "plt.legend()\n",
    "plt.show()"
   ],
   "execution_count": null,
   "id": "22",
   "metadata": {},
   "outputs": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}