notebooks/Visdom-With-FastAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Visdom With FastAI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook i will show how to use Visdom https://github.com/facebookresearch/visdom with the FastAI library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visdom is an amazing data visualization tool. What it essentially does is to create a new server (basically opens a new port in the server) and displays all the data in that place. There are a wide range of options and a lot of data type support, lot of plotting library support. The UI is extremely clean. I highly encourage to have a look at the github repository to have a taste of its capabilities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we will try to plot the training loss after every batch and see how that varies. Next, we will plot the training loss and the validation loss after every epoch. This notebook should give a broad and general idea and one can incorporate other types of visualization into this easily."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Confirming Visdom is working properly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As always, we will first ensure that visdom is working properly. Easiest way to install visdom is via `pip install visdom`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, in your terminal open a tmux session or screen session and open a visdom server with suitable port (by default it uses 8097). In this example I use the port 6008. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In terminal session type: `python -m visdom.server -port=6008`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'window_365889b2226a26'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import visdom\n",
    "import numpy as np\n",
    "vis = visdom.Visdom(port=6008)\n",
    "vis.text('Hello, world!')\n",
    "vis.image(np.ones((3, 10, 10)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now head over to `localhost:6008` (or ipaddress-of-server:6008) and select `main` in the environments. You should be able to see \"Hello World\" in a canvas. Once done, do a kernel restart so that the port 6008 is free again"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It might be a good idea to do port forwarding (tuneling) of another port other than the one you use jupyter notebook for this purpose. In my case, I have forward 6007 and 6008 ports for jupyter and visdom respectively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will try to use Visdom to monitor training loss. Again, I will be using dogcats dataset which can be found here http://files.fast.ai/data/."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib\n",
    "matplotlib.use('Agg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastai.imports import *\n",
    "from fastai.transforms import *\n",
    "from fastai.conv_learner import *\n",
    "from fastai.model import *\n",
    "from fastai.dataset import *\n",
    "from fastai.sgdr import *\n",
    "from fastai.plots import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import visdom"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "PATH = \"../data/dogscats/\"\n",
    "sz=224"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "arch=resnet34\n",
    "data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use Visdom properly we will first have look at the concept of CallBacks. I don't think I can explain callbacks better than that explained in this post https://github.com/sgugger/Deep-Learning/blob/master/Using%20the%20callback%20system%20in%20fastai.ipynb by Sylvain Gugger."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is a good idea to create plotter class for plotting onto visdom. The following function is inspired from https://github.com/andreasveit/triplet-network-pytorch/blob/master/train.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "class VisdomLinePlotter:\n",
    "    def __init__(self, env='main'):\n",
    "        self.vis = visdom.Visdom(port=6008)\n",
    "        self.env = env\n",
    "        self.plots = dict()\n",
    "        \n",
    "    def plot(self, window_name, var_name, x, y, xlabel='Epochs', ylabel='Loss'):\n",
    "        if window_name not in self.plots:\n",
    "            self.plots[window_name] = self.vis.line(X=np.array([x]), Y=np.array([y]), \n",
    "                                                    env=self.env, opts=dict(\n",
    "                                                        legend=[var_name],\n",
    "                                                        title=window_name,\n",
    "                                                        xlabel=xlabel,\n",
    "                                                        ylabel=ylabel\n",
    "                                                    ))\n",
    "        else:\n",
    "            self.vis.line(X=np.array([x]), Y=np.array([y]), env=self.env, \n",
    "                          win=self.plots[window_name], name=var_name, update='append')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will define a callback function which inherits the Callback class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "class visdom_callback(Callback):\n",
    "    def __init__(self, plotter):\n",
    "        self.plotter = plotter\n",
    "        self.num_epochs=0\n",
    "        self.num_batches=0\n",
    "    def on_batch_end(self, los):\n",
    "        self.deb_loss = los\n",
    "        self.num_batches+=1\n",
    "        self.plotter.plot('train', 'train_loss', self.num_batches, \n",
    "                          self.deb_loss, xlabel='Batches', ylabel='Training Loss')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "plotter = VisdomLinePlotter()\n",
    "\n",
    "vcb = visdom_callback(plotter)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn = ConvLearner.pretrained(arch, data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the next line executes, you should be able to see at `localhost:6008` or `ipaddress:6008` the training plot being continuously updated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d21c2b1f1a0741779a52910a526d0a51",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Epoch', max=1), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch      trn_loss   val_loss   accuracy                     \n",
      "    0      0.0607     0.028024   0.9895    \n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[array([0.02802]), 0.9895]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learn.fit(1e-2, 1, cycle_len=1, metrics=[accuracy], callbacks=[vcb])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will try to see the training loss and validation loss after each epoch. We will need to modify our callback a bit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "class visdom_callback(Callback):\n",
    "    def __init__(self, plotter):\n",
    "        self.plotter = plotter\n",
    "        self.num_epochs=0\n",
    "        self.num_batches=0\n",
    "    def on_batch_end(self, los):\n",
    "        self.deb_loss = los\n",
    "        self.num_batches+=1\n",
    "    def on_epoch_end(self, metrics):\n",
    "        self.num_epochs += 1\n",
    "        self.plotter.plot('train', 'train_loss', self.num_epochs, \n",
    "                          self.deb_loss, xlabel='Epochs', ylabel='Loss')\n",
    "        self.plotter.plot('train', 'val_loss', self.num_epochs, \n",
    "                          metrics[0], xlabel='Epochs', ylabel='Loss')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "plotter = VisdomLinePlotter()\n",
    "\n",
    "vcb = visdom_callback(plotter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To speed up the process we can try only fine tuning the final layers. For this we can save the activations first, then freeze the layers and set precompute as true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100%|██████████| 360/360 [00:41<00:00,  8.78it/s]\n",
      "100%|██████████| 32/32 [00:03<00:00,  8.61it/s]\n"
     ]
    }
   ],
   "source": [
    "learn.save_fc1()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.freeze()\n",
    "learn.precompute = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again you can see your visdom plots while this is training on the visdom port."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2fcb54c9b97447f0b35b08619f8056ae",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Epoch', max=10), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch      trn_loss   val_loss   accuracy                     \n",
      "    0      0.045204   0.024696   0.9915    \n",
      "    1      0.037385   0.024245   0.992                        \n",
      "    2      0.037161   0.024644   0.9915                       \n",
      "    3      0.041269   0.025129   0.991                        \n",
      "    4      0.040194   0.02618    0.991                        \n",
      "    5      0.043403   0.026201   0.9915                       \n",
      "    6      0.037505   0.028432   0.9895                       \n",
      "    7      0.033672   0.027222   0.9905                       \n",
      "    8      0.026833   0.027428   0.9905                       \n",
      "    9      0.030505   0.026051   0.99                         \n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[array([0.02605]), 0.99]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learn.fit(1e-2, 10, cycle_len=1, metrics=[accuracy], callbacks=[vcb])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cheers. Now you can easily customize visdom and use it in your own project which uses fastai library."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}