docs/nb/train_models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "488098e3",
   "metadata": {},
   "source": [
    "# Train models I"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9a9b11a",
   "metadata": {},
   "source": [
    "> Note: This notebook assumes that you have annotated/labelled data within your SimBA project. The notebook also assumes\n",
    "that you have a apriori set the hyperparameters/sampling/evaluation settings either in the SimBA project_config.ini (when training a single model)\n",
    "or have a set of valid CSV config files inside the project_folder/configs directory of your SimBA project (when grid searching models). Also, this notebook is written using a smaller dataset, so you may expect longer runtimes on your end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "cd5ac1b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from simba.model.train_rf import TrainRandomForestClassifier\n",
    "from simba.model.grid_search_rf import GridSearchRandomForestClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3af19fc9",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Define the path to the SimBA project config ini.\n",
    "CONFIG_PATH = '/Users/simon/Desktop/envs/simba/troubleshooting/two_black_animals_14bp/project_folder/project_config.ini'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "204e4ec4",
   "metadata": {},
   "source": [
    "### TRAIN A SINGLE MODEL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cd1d71a8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading in 4 annotated files...\n",
      "Dataset size: 13.876192MB / 0.013876GB\n",
      "Number of features in dataset: 454\n",
      "Number of Attack frames in dataset: 272.0 (3.91%)\n"
     ]
    }
   ],
   "source": [
    "###Create an instance of a model trainer based on the settings in the project config\n",
    "model_trainer = TrainRandomForestClassifier(config_path=CONFIG_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "61bbe016",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training and evaluating model...\n",
      "Fitting Attack model...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend ThreadingBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:    0.1s\n",
      "[Parallel(n_jobs=-1)]: Done 184 tasks      | elapsed:    0.7s\n",
      "[Parallel(n_jobs=-1)]: Done 434 tasks      | elapsed:    1.6s\n",
      "[Parallel(n_jobs=-1)]: Done 784 tasks      | elapsed:    2.9s\n",
      "[Parallel(n_jobs=-1)]: Done 1234 tasks      | elapsed:    4.5s\n",
      "[Parallel(n_jobs=-1)]: Done 1784 tasks      | elapsed:    6.6s\n",
      "[Parallel(n_jobs=-1)]: Done 2000 out of 2000 | elapsed:    7.3s finished\n"
     ]
    }
   ],
   "source": [
    "###Run the model trainer based on the settings in the project config\n",
    "model_trainer.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f3d69c03",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SIMBA COMPLETE: Classifier Attack saved in models/generated_models directory (elapsed time: 10.5654s) \tcomplete\n",
      "SIMBA COMPLETE: Evaluation files are in models/generated_models/model_evaluations folders \tcomplete\n"
     ]
    }
   ],
   "source": [
    "### Save the model\n",
    "model_trainer.save()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bc2d642",
   "metadata": {},
   "source": [
    "### TRAIN MULTIPLE MODELS: ONE FOR EACH SETTINGS FILE PRESENT IN THE PROJECT_FOLDER/CONFIGS DIRECTORY.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4c41114d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading in 4 annotated files...\n",
      "Reading complete Together_2 (elapsed time: 0.3268s)...\n",
      "Reading complete Together_3 (elapsed time: 0.3219s)...\n",
      "Reading complete Together_1 (elapsed time: 0.2962s)...\n",
      "Reading complete Together_4 (elapsed time: 0.2478s)...\n"
     ]
    }
   ],
   "source": [
    "###Create an instance of a grid model trainer.\n",
    "model_trainer = GridSearchRandomForestClassifier(config_path=CONFIG_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "bf348691",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training model 1/1 (Attack)...\n",
      "MODEL 1 settings\n",
      "+------------------------+---------+\n",
      "| Setting                | value   |\n",
      "+========================+=========+\n",
      "| Model name             | Attack  |\n",
      "+------------------------+---------+\n",
      "| Ensemble method        | RF      |\n",
      "+------------------------+---------+\n",
      "| Estimators (trees)     | 2000    |\n",
      "+------------------------+---------+\n",
      "| Max features           | sqrt    |\n",
      "+------------------------+---------+\n",
      "| Under sampling setting | None    |\n",
      "+------------------------+---------+\n",
      "| Under sampling ratio   | nan     |\n",
      "+------------------------+---------+\n",
      "| Over sampling setting  | None    |\n",
      "+------------------------+---------+\n",
      "| Over sampling ratio    | nan     |\n",
      "+------------------------+---------+\n",
      "| criterion              | gini    |\n",
      "+------------------------+---------+\n",
      "| Min sample leaf        | 1       |\n",
      "+------------------------+---------+ \tTABLE\n",
      "# 454 features.\n",
      "Fitting Attack model...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend ThreadingBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:    0.1s\n",
      "[Parallel(n_jobs=-1)]: Done 184 tasks      | elapsed:    0.7s\n",
      "[Parallel(n_jobs=-1)]: Done 434 tasks      | elapsed:    1.7s\n",
      "[Parallel(n_jobs=-1)]: Done 784 tasks      | elapsed:    3.1s\n",
      "[Parallel(n_jobs=-1)]: Done 1234 tasks      | elapsed:    5.3s\n",
      "[Parallel(n_jobs=-1)]: Done 1784 tasks      | elapsed:    8.2s\n",
      "[Parallel(n_jobs=-1)]: Done 2000 out of 2000 | elapsed:    9.5s finished\n",
      "[Parallel(n_jobs=8)]: Using backend ThreadingBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=8)]: Done  34 tasks      | elapsed:    0.0s\n",
      "[Parallel(n_jobs=8)]: Done 184 tasks      | elapsed:    0.1s\n",
      "[Parallel(n_jobs=8)]: Done 434 tasks      | elapsed:    0.1s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating classification report visualization...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=8)]: Done 784 tasks      | elapsed:    0.2s\n",
      "[Parallel(n_jobs=8)]: Done 1234 tasks      | elapsed:    0.3s\n",
      "[Parallel(n_jobs=8)]: Done 1784 tasks      | elapsed:    0.4s\n",
      "[Parallel(n_jobs=8)]: Done 2000 out of 2000 | elapsed:    0.6s finished\n",
      "[Parallel(n_jobs=8)]: Using backend ThreadingBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=8)]: Done  34 tasks      | elapsed:    0.0s\n",
      "[Parallel(n_jobs=8)]: Done 184 tasks      | elapsed:    0.0s\n",
      "[Parallel(n_jobs=8)]: Done 434 tasks      | elapsed:    0.1s\n",
      "[Parallel(n_jobs=8)]: Done 784 tasks      | elapsed:    0.2s\n",
      "[Parallel(n_jobs=8)]: Done 1234 tasks      | elapsed:    0.3s\n",
      "[Parallel(n_jobs=8)]: Done 1784 tasks      | elapsed:    0.4s\n",
      "[Parallel(n_jobs=8)]: Done 2000 out of 2000 | elapsed:    0.4s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving model meta data file...\n",
      "Classifier Attack_0 saved in models/validations/model_files directory ...\n",
      "SIMBA COMPLETE: All models and evaluations complete. The models/evaluation files are in models/validations folders \tcomplete\n"
     ]
    }
   ],
   "source": [
    "###Run the grid search model trainer. Note: Each model is saved without the need to call the save function (as when training a single model above).\n",
    "model_trainer.run()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "simba",
   "language": "python",
   "name": "simba"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}