integrations-and-supported-tools/sklearn/notebooks/Neptune_Scikit_learn.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Neptune + scikit-learn](https://neptune.ai/wp-content/uploads/2023/09/sklearn-1.svg)\n",
    "\n",
    "# Neptune + scikit-learn\n",
    "\n",
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/neptune-ai/examples/blob/main/integrations-and-supported-tools/sklearn/notebooks/Neptune_Scikit_learn.ipynb\">\n",
    "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\"/>\n",
    "</a><a target=\"_blank\" href=\"https://github.com/neptune-ai/examples/blob/main/integrations-and-supported-tools/sklearn/notebooks/Neptune_Scikit_learn.ipynb\">\n",
    "  <img alt=\"Open in GitHub\" src=\"https://img.shields.io/badge/Open_in_GitHub-blue?logo=github&labelColor=black\">\n",
    "</a><a target=\"_blank\" href=\"https://app.neptune.ai/o/common/org/sklearn-integration/runs/table?viewId=9b015358-59bd-4c02-a020-3426f5a8f09e\"> \n",
    "  <img alt=\"Explore in Neptune\" src=\"https://neptune.ai/wp-content/uploads/2024/01/neptune-badge.svg\">\n",
    "</a><a target=\"_blank\" href=\"https://docs.neptune.ai/integrations/sklearn/\">\n",
    "  <img alt=\"View tutorial in docs\" src=\"https://neptune.ai/wp-content/uploads/2024/01/docs-badge-2.svg\">\n",
    "</a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Introduction"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Neptune helps you keep track of your machine learning runs and if you are using scikit-learn you can add tracking very easily.\n",
    "\n",
    "This quickstart will show you how to (using just single function) log scikit-learn:\n",
    "\n",
    "* regression summary,\n",
    "* classification summary,\n",
    "* kmeans clustering summary."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Before you start\n",
    "\n",
    "This notebook example lets you try out Neptune as an anonymous user, with zero setup.\n",
    "\n",
    "If you want to see the example logged to your own workspace instead:\n",
    "\n",
    "  1. Create a Neptune account. [Register &rarr;](https://neptune.ai/register)\n",
    "  1. Create a Neptune project that you will use for tracking metadata. For instructions, see [Creating a project](https://docs.neptune.ai/setup/creating_project) in the Neptune docs."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Neptune and dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "installation"
    ]
   },
   "outputs": [],
   "source": [
    "! pip install -U \"neptune[sklearn]\" scikit-learn matplotlib_inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To fix the random RuntimeError: main thread is not in main loop error in Windows running python 3.8\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.switch_backend(\"agg\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Scikit-learn regression"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Create and fit random forest regressor"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Define regressor parameters, that will be later passed to Neptune."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "parameters = {\"n_estimators\": 100, \"max_depth\": 5, \"min_samples_split\": 5}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Create and fit regressor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_california_housing\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "rfr = RandomForestRegressor(**parameters)\n",
    "\n",
    "X, y = fetch_california_housing(return_X_y=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=28743)\n",
    "\n",
    "rfr.fit(X_train, y_train)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Initialize Neptune"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "To create a new run for tracking the metadata, you tell Neptune who you are (`api_token`) and where to send the data (`project`).\n",
    "\n",
    "You can use the default code cell below to create an anonymous run in a public project. **Note**: Public projects are cleaned regularly, so anonymous runs are only stored temporarily.\n",
    "\n",
    "### Log to your own project instead\n",
    "\n",
    "Replace the code below with the following:\n",
    "\n",
    "```python\n",
    "import neptune\n",
    "from getpass import getpass\n",
    "\n",
    "run = neptune.init_run(\n",
    "    project=\"workspace-name/project-name\",  # replace with your own (see instructions below)\n",
    "    api_token=getpass(\"Enter your Neptune API token: \"),\n",
    "    name=\"regression-example\",\n",
    "    tags=[\"RandomForestRegressor\", \"regression\"],\n",
    ")\n",
    "```\n",
    "\n",
    "To find your API token and full project name:\n",
    "\n",
    "1. [Log in to Neptune](https://app.neptune.ai/).\n",
    "1. In the bottom-left corner, expand your user menu and select **Get your API token**.\n",
    "1. To copy the project path, open the settings menu and select **Details & privacy**.\n",
    "\n",
    "For more help, see [Setting Neptune credentials](https://docs.neptune.ai/setup/setting_credentials) in the Neptune docs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptune\n",
    "\n",
    "run = neptune.init_run(\n",
    "    project=\"common/sklearn-integration\",\n",
    "    api_token=neptune.ANONYMOUS_API_TOKEN,\n",
    "    name=\"regression-example\",\n",
    "    tags=[\"RandomForestRegressor\", \"regression\"],\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "**To open the run in the Neptune web app, click the link that appeared in the cell output.**\n",
    "\n",
    "We'll use the `run` object we just created to log metadata. You'll see the metadata appear in the app."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log regressor summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptune.integrations.sklearn as npt_utils\n",
    "\n",
    "run[\"rfr_summary\"] = npt_utils.create_regressor_summary(rfr, X_train, X_test, y_train, y_test)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You just logged information about the regressor, including:\n",
    "\n",
    "* [logged regressor parameters](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2Fall_params),\n",
    "* [logged pickled model](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2F&attribute=pickled_model),\n",
    "* [logged test predictions](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2Ftest&attribute=preds),\n",
    "* [logged test scores](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2Ftest%2Fscores),\n",
    "* [logged regressor visualizations](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2Fdiagnostics_charts&attribute=feature_importance),\n",
    "* [logged metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=sys),\n",
    "* [logged code and git metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/source-code?file=main.py)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stop logging\n",
    "\n",
    "<font color=red>**Warning:**</font><br>\n",
    "Once you are done logging, you should stop tracking the run using the `stop()` method.\n",
    "This is needed only while logging from a notebook environment. While logging through a script, Neptune automatically stops tracking once the script has completed execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run.stop()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Explore results"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You just learned how to log scikit-learn regression summary to Neptune using single function.\n",
    "\n",
    "Click on the link that was outputted to the console or [go here](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all) to explore a run similar to yours. In particular check:\n",
    "\n",
    "* [logged regressor parameters](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2Fall_params),\n",
    "* [logged pickled model](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2F&attribute=pickled_model),\n",
    "* [logged test predictions](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2Ftest&attribute=preds),\n",
    "* [logged test scores](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2Ftest%2Fscores),\n",
    "* [logged regressor visualizations](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=rfr_summary%2Fdiagnostics_charts&attribute=feature_importance),\n",
    "* [logged metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/all?path=sys),\n",
    "* [logged code and git metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-92/source-code?file=main.py)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Scikit-learn classification"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Create and fit gradient boosting classifier"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Define classifier parameters, that will be later passed to Neptune."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "parameters = {\n",
    "    \"n_estimators\": 80,\n",
    "    \"learning_rate\": 0.1,\n",
    "    \"min_samples_split\": 5,\n",
    "    \"min_samples_leaf\": 5,\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Create and fit regressor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_digits\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "gbc = GradientBoostingClassifier(**parameters)\n",
    "\n",
    "X, y = load_digits(return_X_y=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=28743)\n",
    "\n",
    "gbc.fit(X_train, y_train)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Initialize Neptune"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Connect your script to Neptune application and create new run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptune\n",
    "\n",
    "run = neptune.init_run(\n",
    "    project=\"common/sklearn-integration\",\n",
    "    api_token=neptune.ANONYMOUS_API_TOKEN,\n",
    "    name=\"classification-example\",\n",
    "    tags=[\"GradientBoostingClassifier\", \"classification\"],\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Click on the link above to open this run in Neptune. For now it is empty but keep the tab with run open to see what happens next."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You tell Neptune: \n",
    "\n",
    "* **who you are**: your Neptune API token `api_token` \n",
    "* **where you want to send your data**: your Neptune `project`.\n",
    "\n",
    "At this point you have new run in Neptune. For now on you will use `run` to log metadata to it.\n",
    "\n",
    "---\n",
    "\n",
    "**Note**\n",
    "\n",
    "\n",
    "Instead of logging data to the public project `'common/sklearn-integration'` as an anonymous user 'neptuner' you can log it to your own project.\n",
    "\n",
    "To do that:\n",
    "\n",
    "1. Get your [Neptune API token](https://docs.neptune.ai/setup/setting_api_token/)\n",
    "2. Pass the token to ``api_token`` argument of ``neptune.init_run()`` method: ``api_token=YOUR_API_TOKEN``\n",
    "3. Pass your project to the ``project`` argument of ``neptune.init_run()``.\n",
    "\n",
    "For example:\n",
    "\n",
    "```python\n",
    "neptune.init_run(project=\"YOUR_WORKSPACE/YOUR_PROJECT\", \n",
    "             api_token=\"YOUR_API_TOKEN\")\n",
    "```"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log classifier summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptune.integrations.sklearn as npt_utils\n",
    "\n",
    "run[\"cls_summary\"] = npt_utils.create_classifier_summary(gbc, X_train, X_test, y_train, y_test)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You just logged information about the classifier, including:\n",
    "\n",
    "* [logged classifier parameters](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Fall_params),\n",
    "* [logged pickled model](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2F&attribute=pickled_model),\n",
    "* [logged test predictions](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Ftest&attribute=preds),\n",
    "* [logged test predictions probabilities](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Ftest&attribute=preds_proba),\n",
    "* [logged test scores](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Ftest%2Fscores%2F),\n",
    "* [logged classifier visualizations](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Fdiagnostics_charts&attribute=class_prediction_error),\n",
    "* [logged metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=sys),\n",
    "* [logged code and git metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/source-code?file=main.py&filePath=integrations%2Fsklearn%2F)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stop logging\n",
    "\n",
    "<font color=red>**Warning:**</font><br>\n",
    "Once you are done logging, you should stop tracking the run using the `stop()` method.\n",
    "This is needed only while logging from a notebook environment. While logging through a script, Neptune automatically stops tracking once the script has completed execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run.stop()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Explore Results"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You just learned how to log scikit-learn classification summary to Neptune using single function.\n",
    "\n",
    "Click on the link that was outputted to the console or [go here](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/allhttps://ui.neptune.ai/o/shared/org/sklearn-integration/e/SKLEARN-312/charts) to explore a run similar to yours. In particular check:\n",
    "\n",
    "* [logged classifier parameters](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Fall_params),\n",
    "* [logged pickled model](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2F&attribute=pickled_model),\n",
    "* [logged test predictions](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Ftest&attribute=preds),\n",
    "* [logged test predictions probabilities](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Ftest&attribute=preds_proba),\n",
    "* [logged test scores](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Ftest%2Fscores%2F),\n",
    "* [logged classifier visualizations](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=cls_summary%2Fdiagnostics_charts&attribute=class_prediction_error),\n",
    "* [logged metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/all?path=sys),\n",
    "* [logged code and git metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-95/source-code?file=main.py&filePath=integrations%2Fsklearn%2F)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Scikit-learn KMeans clustering"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Create KMeans object and example data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Define KMeans clustering parameters, that will be later passed to Neptune."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "parameters = {\"n_init\": 12, \"max_iter\": 250}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Create and fit KMeans model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "from sklearn.cluster import KMeans\n",
    "from sklearn.datasets import make_blobs\n",
    "\n",
    "km = KMeans(**parameters)\n",
    "\n",
    "X, y = make_blobs(n_samples=579, n_features=17, centers=7, random_state=28743)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Initialize Neptune"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Connect your script to Neptune application and create new run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptune\n",
    "\n",
    "run = neptune.init_run(\n",
    "    project=\"common/sklearn-integration\",\n",
    "    api_token=neptune.ANONYMOUS_API_TOKEN,\n",
    "    name=\"clustering-example\",\n",
    "    tags=[\"KMeans\", \"clustering\"],\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Click on the link above to open this run in Neptune. For now it is empty but keep the tab with run open to see what happens next."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You tell Neptune: \n",
    "\n",
    "* **who you are**: your Neptune API token `api_token` \n",
    "* **where you want to send your data**: your Neptune `project`.\n",
    "\n",
    "At this point you have new run in Neptune. For now on you will use `run` to log metadata to it.\n",
    "\n",
    "---\n",
    "\n",
    "**Note**\n",
    "\n",
    "\n",
    "Instead of logging data to the public project `'common/sklearn-integration'` as an anonymous user 'neptuner' you can log it to your own project.\n",
    "\n",
    "To do that:\n",
    "\n",
    "1. Get your [Neptune API token](https://docs.neptune.ai/setup/setting_api_token/)\n",
    "2. Pass the token to `api_token` argument of the `init_run()` method: `api_token=YOUR_API_TOKEN`\n",
    "3. Pass your project to the `project` argument of the `init_run()` method.\n",
    "\n",
    "For example:\n",
    "\n",
    "```python\n",
    "neptune.init_run(project=\"YOUR_WORKSPACE/YOUR_PROJECT\", \n",
    "             api_token=\"YOUR_API_TOKEN\")\n",
    "```"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log KMeans clustering summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptune.integrations.sklearn as npt_utils\n",
    "\n",
    "run[\"kmeans_summary\"] = npt_utils.create_kmeans_summary(km, X, n_clusters=17)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You just logged information about the KMeans clustering, including:\n",
    "\n",
    "* [logged KMeans parameters](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/all?path=kmeans_summary%2Fall_params),\n",
    "* [logged cluster labels](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/all?path=kmeans_summary%2F&attribute=cluster_labels),\n",
    "* [logged KMeans clustering visualizations](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/all?path=kmeans_summary%2Fdiagnostics_charts&attribute=silhouette),\n",
    "* [logged metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/all?path=sys),\n",
    "* [logged code and git metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/source-code?file=main.py&filePath=integrations%2Fsklearn%2F)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stop logging\n",
    "\n",
    "<font color=red>**Warning:**</font><br>\n",
    "Once you are done logging, you should stop tracking the run using the `stop()` method.\n",
    "This is needed only while logging from a notebook environment. While logging through a script, Neptune automatically stops tracking once the script has completed execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run.stop()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Explore Results"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You just learned how to log scikit-learn KMeans clustering summary to Neptune using single function.\n",
    "\n",
    "Click on the link that was outputted to the console or [go here](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/all) to explore a run similar to yours. In particular check:\n",
    "\n",
    "* [logged KMeans parameters](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/all?path=kmeans_summary%2Fall_params),\n",
    "* [logged cluster labels](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/all?path=kmeans_summary%2F&attribute=cluster_labels),\n",
    "* [logged KMeans clustering visualizations](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/all?path=kmeans_summary%2Fdiagnostics_charts&attribute=silhouette),\n",
    "* [logged metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/all?path=sys),\n",
    "* [logged code and git metadata](https://app.neptune.ai/o/common/org/sklearn-integration/e/SKLEAR-96/source-code?file=main.py&filePath=integrations%2Fsklearn%2F)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Other logging options"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Neptune-Sklearn integration also lets you log only specific metadata of your choice, by using additional methods.\n",
    "\n",
    "Below are few examples, visit the [scikit-learn integration documentation](https://docs.neptune.ai/integrations-and-supported-tools/model-training/sklearn) for the full example."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Before you start: create and fit gradient boosting classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "rfc = RandomForestClassifier()\n",
    "\n",
    "X, y = load_digits(return_X_y=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=28743)\n",
    "\n",
    "rfc.fit(X_train, y_train)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Import scikit-learn integration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptune.integrations.sklearn as npt_utils\n",
    "\n",
    "run = neptune.init_run(\n",
    "    project=\"common/sklearn-integration\",\n",
    "    api_token=neptune.ANONYMOUS_API_TOKEN,\n",
    "    name=\"other-options\",\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Open link above to see the metadata logging results, as we add them below."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log estimator parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "from neptune.utils import stringify_unsupported\n",
    "\n",
    "run[\"estimator/parameters\"] = stringify_unsupported(npt_utils.get_estimator_params(rfc))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "run[\"estimator/pickled-model\"] = npt_utils.get_pickled_model(rfc)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log confusion matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "run[\"confusion-matrix\"] = npt_utils.create_confusion_matrix_chart(\n",
    "    rfc, X_train, X_test, y_train, y_test\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stop logging\n",
    "\n",
    "<font color=red>**Warning:**</font><br>\n",
    "Once you are done logging, you should stop tracking the run using the `stop()` method.\n",
    "This is needed only while logging from a notebook environment. While logging through a script, Neptune automatically stops tracking once the script has completed execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run.stop()"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Neptune_Scikit_learn.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "py38",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.15"
  },
  "vscode": {
   "interpreter": {
    "hash": "a9715cf0b0024f6e1c62cb31a4f1f43970eb41991212681878768b4bfe53050a"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}