r-integration.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Untitled1.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "ir",
      "display_name": "R"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X9Mejpf-WzGO",
        "colab_type": "text"
      },
      "source": [
        "# Installation\n",
        "\n",
        "Since Neptune is available on CRAN you can simpy use `install.packages` function.\n",
        "\n",
        "For this tutorial I'll need a few more packages and those are listed below.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jk1fIqH74qXb",
        "colab_type": "code",
        "outputId": "8e0ccf95-64c9-4771-bc55-dd0217d9e5c3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "source": [
        "# install neptune\n",
        "install.packages('neptune', dependencies = TRUE)\n",
        "\n",
        "# install other packages for this tutorial\n",
        "install.packages(c('digest', 'mlbench', 'randomForest'), dependencies = TRUE)"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Installing package into ‘/usr/local/lib/R/site-library’\n",
            "(as ‘lib’ is unspecified)\n",
            "\n",
            "Installing packages into ‘/usr/local/lib/R/site-library’\n",
            "(as ‘lib’ is unspecified)\n",
            "\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "error",
          "ename": "ERROR",
          "evalue": "ignored",
          "traceback": [
            "Error: Miniconda is already installed at '/root/.local/share/r-miniconda'\nTraceback:\n",
            "1. reticulate::install_miniconda()",
            "2. install_miniconda_preflight(path, force)",
            "3. stopf(\"Miniconda is already installed at %s\", shQuote(path))",
            "4. stop(sprintf(fmt, ...), call. = call.)"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v4ejGMmzoYJX",
        "colab_type": "text"
      },
      "source": [
        "For simplicity install miniconda nad copy the python path of your miniconda environemnt to the `init_neptune` function that we will need later."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "62EEgmJAnG7J",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        },
        "outputId": "0c76c77c-5115-4808-a3d3-841e6b68494d"
      },
      "source": [
        "reticulate:: install_miniconda()\n",
        "reticulate:: py_config()"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "python:         /root/.local/share/r-miniconda/envs/r-reticulate/bin/python\n",
              "libpython:      /root/.local/share/r-miniconda/envs/r-reticulate/lib/libpython3.6m.so\n",
              "pythonhome:     /root/.local/share/r-miniconda/envs/r-reticulate:/root/.local/share/r-miniconda/envs/r-reticulate\n",
              "version:        3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54)  [GCC 7.3.0]\n",
              "numpy:          /root/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/numpy\n",
              "numpy_version:  1.18.1"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "btWLay1NWzrD",
        "colab_type": "text"
      },
      "source": [
        "## Load packages and data\n",
        "\n",
        "Nothing fancy here."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "etd3EwXr4yLl",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        },
        "outputId": "dbd7df09-29c6-4d47-c8a8-3602b3b9a63a"
      },
      "source": [
        "# load libraries\n",
        "library(neptune)\n",
        "library(digest)\n",
        "library(mlbench)\n",
        "library(randomForest)\n",
        "\n",
        "SEED=1234\n",
        "set.seed(SEED)\n",
        "\n",
        "# load dataset \n",
        "data(Sonar)\n",
        "dataset <- Sonar\n",
        "x <- dataset[,1:60]   # predictors\n",
        "y <- dataset[,61]     # labels"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "randomForest 4.6-14\n",
            "\n",
            "Type rfNews() to see new features/changes/bug fixes.\n",
            "\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YAufzFsCaCUK",
        "colab_type": "text"
      },
      "source": [
        "# Initialize Neptune\n",
        "\n",
        "In order to start logging things to Neptune you need to \"connect\" your script to Neptune service. \n",
        "\n",
        "Since Neptune is working on top of \"reticulate\" and Python you may need to specify your Python environment with `python` and `python_path` arguments."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cS-nSl2j42zY",
        "colab_type": "code",
        "outputId": "3ab85150-e013-48c8-c412-746144a8f14d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "init_neptune(project_name = 'shared/r-integration',\n",
        "             api_token = 'ANONYMOUS',\n",
        "             python_path='/root/.local/share/r-miniconda/envs/r-reticulate/bin/python'\n",
        "             )"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Project(shared/r-integration)"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HHA0KjZnai-H",
        "colab_type": "text"
      },
      "source": [
        "# Create Experiment\n",
        "\n",
        "To start tracking your work you need to create a Neptune experiment. \n",
        "\n",
        "You can name your experiment, add tags to organize things and track hyperparameters of your ML models."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "peu1BSjO46td",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "4214be67-9096-4bf0-b374-f6571c130f1a"
      },
      "source": [
        "params = list(ntree=100,\n",
        "              mtry=10,\n",
        "              maxnodes=20\n",
        "              )\n",
        "\n",
        "create_experiment(name='training on Sonar', \n",
        "                  tags=c('random-forest','sonar'),\n",
        "                  params = params\n",
        ")"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Experiment(RIN-164)"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G7Yo4Wi8a0X3",
        "colab_type": "text"
      },
      "source": [
        "# Set properties\n",
        "\n",
        "You can also use `set_property` function to save `key:value` pairs.\n",
        "\n",
        "For example, I'll keep track of the `data version` and random `seed`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "05AzjZO75CyY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "set_property(property = 'data-version', value = digest(dataset))\n",
        "set_property(property = 'seed', value = SEED)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J_87SA8mbGi-",
        "colab_type": "text"
      },
      "source": [
        "# Train your model\n",
        "\n",
        "I defined hyperparameters in the `params` list and I will pass them directly into `randomForest`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kTjXdOwY5GoG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model <- randomForest(x = x, y = y,\n",
        "  ntree=params$ntree, mtry = params$mtry, maxnodes = params$maxnodes,\n",
        "  importance = TRUE\n",
        "  )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jHeWztZpY8uq",
        "colab_type": "text"
      },
      "source": [
        "# Log metrics \n",
        "\n",
        "Logging metrics to Neptune is trivial.\n",
        "\n",
        "For example, I will log mean out-of-bag error and errors per class taken from the confusion matrix.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5j97EL_L5Mc5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "log_metric('mean OOB error', mean(model$err.rate[,1]))\n",
        "log_metric('error class M', model$confusion[1,3])\n",
        "log_metric('error class R', model$confusion[2,3])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G-IAS_d8ZxBK",
        "colab_type": "text"
      },
      "source": [
        "You can also log multiple values to the same channel. \n",
        "\n",
        "If you do that, Neptune will automatically create charts for you."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l-J2zXBXZlbS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for (err in (model$err.rate[,1])) {\n",
        "  log_metric('OOB errors', err)\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "WaLmZXGv-LpH"
      },
      "source": [
        "# Log artifacts\n",
        "\n",
        "You can log any file to Neptune. just use the `log_artifact` function.\n",
        "\n",
        "For example, we can log our `model.Rdata` file."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gKag2lIA5RB-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "save(model, file=\"model.Rdata\")\n",
        "log_artifact('model.Rdata')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IuheqrfVYd9l",
        "colab_type": "text"
      },
      "source": [
        "# Log charts\n",
        "\n",
        "You can log performance charts like ROC AUC, Confusion Matrix or anything else you think is important.\n",
        "\n",
        "You need to specify the log channel to which you want to log it and push a chart file there.\n",
        "\n",
        "For example, I'll log two versions of feature importance visualizations to Neptune."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SBcI4AeI5T7M",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for (t in c(1,2)){\n",
        "  jpeg('temp_plot.jpeg')\n",
        "  varImpPlot(model,type=t)\n",
        "  dev.off()\n",
        "  log_image('feature_importance', 'temp_plot.jpeg')\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X9DSsw_KYM50",
        "colab_type": "text"
      },
      "source": [
        "# Stop the experiment\n",
        "\n",
        "After everything is done you need to stop the experiment.\n",
        "\n",
        "Thanks to `create_experiment` and `stop_experiment` functions you can create multiple experiments in a single script."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HyF7vBss4jOZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "stop_experiment()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "77iN5e1xXrsu",
        "colab_type": "text"
      },
      "source": [
        "# Explore experiments in Neptune\n",
        "\n",
        "Now you can explore everything you logged in Neptune.\n",
        "\n",
        "You can use your link or go check out [this experiment dashboard](https://ui.neptune.ai/o/shared/org/r-integration/experiments?viewId=fa3b57a5-77fb-4edb-83fc-505014d3649d):\n",
        "\n",
        "![image](https://neptune.ai/wp-content/uploads/r-integration-tour.gif)\n",
        "\n",
        "## Create your free account\n",
        "\n",
        "The best part is, Neptune is completely free for individuals and research teams so you can go ahead and [create your free account](https://neptune.ai?utm_source=colab&utm_medium=notebook&utm_campaign=integration-r) and check it out for yourself."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3rHVrgZpYBxa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}