integrations/lightgbm/showcase/Neptune_lightGBM.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r0_saOeEgZYI",
    "tags": [
     "header"
    ]
   },
   "source": [
    "# Neptune + LightGBM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZuHrZJ17getO",
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Before you start"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Hp7aEYTQiIUf",
    "tags": [
     "installation",
     "header"
    ]
   },
   "source": [
    "### Install Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "EK9lGpTTgUzR",
    "tags": [
     "installation"
    ]
   },
   "outputs": [],
   "source": [
    "! pip install --quiet lightgbm==2.2.3 matplotlib==3.2 neptune-client==0.4.132 neptune-contrib[monitoring]==0.25.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jzggt2fghC4S",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Import Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "uf4a1WxBiRQO",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import lightgbm as lgb\r\n",
    "import neptune\r\n",
    "from neptunecontrib.monitoring.lightgbm import neptune_monitor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TyeP6qI0iT1A",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Initialize Neptune. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Neptune gives you an option of logging data under a public folder as an anonymous user. This is great when you are just trying out the application and don't have a Neptune account yet. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "3-88mnKPib0T",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "neptune.init(api_token='ANONYMOUS', project_qualified_name='shared/LightGBM-integration')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "axBFfUNhwGx5",
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Quickstart"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y_L0HVQ4igs0",
    "tags": [
     "comment"
    ]
   },
   "source": [
    "**Note:** \r\n",
    "\r\n",
    "\r\n",
    "Instead of logging data to the public project 'shared/showroom' as an anonymous user 'neptuner' you can log it to your own project.\r\n",
    "\r\n",
    "To do that:\r\n",
    "\r\n",
    "1. Get your Neptune API token\r\n",
    "\r\n",
    "  ![image](https://neptune.ai/wp-content/uploads/get_token.gif)\r\n",
    "\r\n",
    "2. Pass the token to the `api_token` parameter of `neptune.init()` (learn how to do this securely [here](https://docs.neptune.ai/security-and-privacy/api-tokens/how-to-find-and-set-neptune-api-token.html))\r\n",
    "3. Create a new Neptune Project (learn how to do that [here](https://docs.neptune.ai/workspace-project-and-user-management/projects/create-project.html))\r\n",
    "4. Pass your username and project_name to the `project_qualified_name` parameter of `neptune.init()`\r\n",
    "\r\n",
    "For example:\r\n",
    "\r\n",
    "```\r\n",
    "neptune.init(api_token='eyJhcGlfYW908fsdf23f940jiri0bn3085gh03riv03irn', project_qualified_name='siddhantsadangi/lightgbm')\r\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pzZB67vkjpQP",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 1: Create an Experiment\r\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "This creates an experiment in Neptune.\n",
    "\n",
    "Once you have a live experiment you can log things to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "HdsYsx0zlBNy",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "neptune.create_experiment(name='LightGBM-training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_wQ8_ZfImk3D",
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Click on the link above to open this experiment in Neptune.\r\n",
    "\r\n",
    "For now it is empty but keep the tab with experiment open to see what happens next."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aS6LitM3lF6N",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 2: Pass ``neptune_monitor`` to ``lgb.train``ll"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You can view the logging live in the Neptune tab once you run the below cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GKpVvQUDlPlN",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "# Setting up a samplt lightGBM training job\r\n",
    "from sklearn.model_selection import train_test_split\r\n",
    "from sklearn.datasets import load_wine\r\n",
    "\r\n",
    "data = load_wine()\r\n",
    "X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.1)\r\n",
    "lgb_train = lgb.Dataset(X_train, y_train)\r\n",
    "lgb_eval = lgb.Dataset(X_test, y_test, reference=lgb_train)\r\n",
    "\r\n",
    "params = {'boosting_type': 'gbdt',\r\n",
    "              'objective': 'multiclass',\r\n",
    "              'num_class': 3,\r\n",
    "              'num_leaves': 31,\r\n",
    "              'learning_rate': 0.05,\r\n",
    "              'feature_fraction': 0.9\r\n",
    "              }\r\n",
    "\r\n",
    "# Passing `neptune_monitor` to `lgb.train()`\r\n",
    "gbm = lgb.train(params,\r\n",
    "    lgb_train,\r\n",
    "    num_boost_round=500,\r\n",
    "    valid_sets=[lgb_train, lgb_eval],\r\n",
    "    valid_names=['train','valid'],\r\n",
    "    callbacks=[neptune_monitor()],\r\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yasauNnibbkB",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 3: Stop logging."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "When you track experiments with Neptune in Jupyter notebooks you need to explicitly stop the experiment by running ```neptune.stop()```.\n",
    "\n",
    "If you are running Neptune in regular ```.py``` scripts it will stop automatically when your code stops running."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "__iDgMetbbkC",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "neptune.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7ZUX64yqw0ov",
    "tags": [
     "header"
    ]
   },
   "source": [
    "## More Options"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log hardware consumption"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "To log GPU/CPU and memory during training you just need to have ``psutil`` installed and run your logger as you normally would. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "installation"
    ]
   },
   "outputs": [],
   "source": [
    "! pip install --quiet psutil==5.6.6"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hvv3k4hFya0m",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log Hyperparameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You can log training and model hyperparameters. To do that just pass the parameter dictionary to the `create_experiment()` method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TUw01SiUypNX",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "params = {'boosting_type': 'gbdt',\r\n",
    "              'objective': 'multiclass',\r\n",
    "              'num_class': 3,\r\n",
    "              'num_leaves': 31,\r\n",
    "              'learning_rate': 0.05,\r\n",
    "              'feature_fraction': 0.9\r\n",
    "              }\r\n",
    "\r\n",
    "# Log hyperparameters\r\n",
    "neptune.create_experiment(name='LightGBM-training', params=params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "Train a model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "gbm = lgb.train(params,\n",
    "    lgb_train,\n",
    "    num_boost_round=500,\n",
    "    valid_sets=[lgb_train, lgb_eval],\n",
    "    valid_names=['train','valid'],\n",
    "    callbacks=[neptune_monitor()],\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gtoSCHzO04JY",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Save Model Artifacts."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You can log model weights to Neptune both during and after training.\n",
    "To do that just use the `log_artifact()` method on the saved model file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-FHCBg561K7k",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "gbm.save_model('lightgbm.pkl')\r\n",
    "\r\n",
    "# Log model\r\n",
    "neptune.log_artifact('lightgbm.pkl')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aI5qqqnr5Ttk",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log Interactive Charts."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You can log interactive charts to Neptune.\n",
    "\n",
    "For example, let’s log an interactive ROC AUC curve."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oQzWZQBu5ekM",
    "tags": [
     "header",
     "installation"
    ]
   },
   "source": [
    "#### 1. Install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "EMgwMNHX5tqi",
    "tags": [
     "installation"
    ]
   },
   "outputs": [],
   "source": [
    "! pip install --quiet scikit-plot matplotlib==3.2.0 plotly==4.12.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JURNe3WX6CJs",
    "tags": [
     "header"
    ]
   },
   "source": [
    "#### 2. Create an ROC AUC curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_VqgN7G56J0i",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\r\n",
    "from scikitplot.metrics import plot_roc\r\n",
    "\r\n",
    "y_test_pred = gbm.predict(X_test)\r\n",
    "\r\n",
    "fig, ax = plt.subplots()\r\n",
    "plot_roc(y_test, y_test_pred, ax=ax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mNKk9TIQ7K2s",
    "tags": [
     "header"
    ]
   },
   "source": [
    "#### 3. Log it to Neptune via `log_chart()` function.\r\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GxfGt4Yl7QL-",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "from neptunecontrib.api import log_chart\r\n",
    "\r\n",
    "log_chart(name='ROC curve', chart=fig)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "afTLlUaxlob4",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Stop logging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "DRnw-JeQl4IC",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "neptune.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LMWa-USAl5i9",
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Explore results in the Neptune UI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Neptune-LightGBM.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "py_37",
   "language": "python",
   "name": "py_37"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}