integrations/tensorflow-keras/showcase/Neptune-TensorFlow-Keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "G4ny3mHCFrqG",
    "tags": [
     "header"
    ]
   },
   "source": [
    "# Neptune + TensorFlow / Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Before we start"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xndcBi_EUFVi",
    "tags": [
     "header",
     "installation"
    ]
   },
   "source": [
    "### Install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jl90ahSMvhmV",
    "tags": [
     "installation"
    ]
   },
   "outputs": [],
   "source": [
    "! pip install --quiet tensorflow==2.3.1 neptune-client==0.4.132 neptune-contrib==0.25.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1As3OW3LGqKg",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FufE4GTzvRZG",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Define your model, data loaders and optimizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "mnist = tf.keras.datasets.mnist\n",
    "(x_train, y_train),(x_test, y_test) = mnist.load_data()\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0\n",
    "\n",
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(),\n",
    "  tf.keras.layers.Dense(256, activation=tf.keras.activations.relu),\n",
    "  tf.keras.layers.Dropout(0.5),\n",
    "  tf.keras.layers.Dense(10, activation=tf.keras.activations.softmax)\n",
    "])\n",
    "\n",
    "optimizer = tf.keras.optimizers.SGD(lr=0.005, momentum=0.4,)\n",
    "\n",
    "model.compile(optimizer=optimizer,\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "If9d2A8zUynE",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Initialize Neptune"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FQgpdwXVvcY1",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "import neptune\n",
    "\n",
    "neptune.init(api_token='ANONYMOUS', project_qualified_name='shared/tensorflow-keras-integration')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "auuzaMR3Sei2",
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Neptune gives you an option of logging data under a public folder as an anonymous user. This is great when you are just trying out the application and don't have a Neptune account yet.  \n",
    "\n",
    "---\n",
    "\n",
    "If you already have a [Neptune account](https://neptune.ai/register), you can create your own experiment and start logging to it using your personal API token. Pass your `username` to the `project_qualified_name` argument of the `neptune.init()` method: `project_qualified_name='YOUR_USERNAME/YOUR_PROJECT_NAME`. If you don't have a project yet, keep `/sandbox` at the end. The `sandbox` project is automatically created for you."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Quickstart"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CHp9zfqBToiX",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 1: Create an Experiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Be3xPNTHTsIz",
    "tags": [
     "comment"
    ]
   },
   "source": [
    "This creates an experiment in Neptune.\n",
    "\n",
    "Once you have a live experiment you can log things to it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "t5VBefIDwBjR",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "neptune.create_experiment('tensorflow-keras-quickstart')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_QdNmNLIXYXx",
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Click on the link above to open this experiment in Neptune.\n",
    "\n",
    "For now it is empty but keep the tab with experiment open to see what happens next. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XomuTzlsVILc",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 2: Add NeptuneMonitor Callback to model.fit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "82879lQuVfsJ",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "from neptunecontrib.monitoring.keras import NeptuneMonitor\n",
    "\n",
    "model.fit(x_train, y_train,\n",
    "          epochs=5,\n",
    "          batch_size=64,\n",
    "          callbacks=[NeptuneMonitor()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Explore results in the Neptune UI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Go to the experiment link and explore charts that were created from the metric values you logged. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P04k_nFiXl63",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Step 4: Stop logging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "When you track experiments with Neptune in Jupyter notebooks you need to explicitly stop the experiment by running ```neptune.stop()```.\n",
    "\n",
    "If you are running Neptune in regular ```.py``` scripts it will stop automatically when your code stops running."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "T5teGbiLwuEi",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "neptune.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YHQrdIgOIo_E",
    "tags": [
     "header"
    ]
   },
   "source": [
    "## More Options"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "You can log way more than just metrics to Neptune. \n",
    "\n",
    "Let's go over some of those options here. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q9zxE-EzLAbe",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log hardware consumption"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "To log GPU/CPU and memory during training you just need to have ``psutil`` installed and run your logger as you normally would. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "installation"
    ]
   },
   "outputs": [],
   "source": [
    "! pip install --quiet psutil==5.6.6"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iAJZLVQKL8Dt",
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log hyperparameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "If you want to log hyperparameters of your model training run you just need to pass a parameters dictionary into the ``.create_experiment`` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Hw_lQq9AL5PR",
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "PARAMS = {'lr':0.005, \n",
    "          'momentum':0.9, \n",
    "          'epochs':10,\n",
    "          'batch_size':32}\n",
    "\n",
    "optimizer = tf.keras.optimizers.SGD(lr=PARAMS['lr'], momentum=PARAMS['momentum'])\n",
    "\n",
    "model.compile(optimizer=optimizer,\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# log params\n",
    "neptune.create_experiment('tensorflow-keras-advanced', params=PARAMS)\n",
    "\n",
    "model.fit(x_train, y_train,\n",
    "          epochs=PARAMS['epochs'],\n",
    "          batch_size=PARAMS['batch_size'],\n",
    "          callbacks=[NeptuneMonitor()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log image predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "x_test_sample = x_test[:100]\n",
    "y_test_sample_pred = model.predict(x_test_sample)\n",
    "\n",
    "for image, y_pred in zip(x_test_sample, y_test_sample_pred):\n",
    "    description = '\\n'.join(['class {}: {}'.format(i, pred)\n",
    "                                for i, pred in enumerate(y_pred)])\n",
    "    neptune.log_image('predictions',\n",
    "                      image,\n",
    "                      description=description)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Log model weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "model.save('my_model')\n",
    "\n",
    "# log model\n",
    "neptune.log_artifact('my_model')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Tcq00KslIdDd",
    "tags": [
     "header"
    ]
   },
   "source": [
    "## Explore results in the Neptune UI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "comment"
    ]
   },
   "source": [
    "Go over experiment details and see all the objects that were logged. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "header"
    ]
   },
   "source": [
    "### Stop logging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "code"
    ]
   },
   "outputs": [],
   "source": [
    "neptune.stop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Neptune-TensorFlow-Keras.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "py_37",
   "language": "python",
   "name": "py_37"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}