nbs/mnist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab Notebook: MNIST with PyTorch\n",
    "\n",
    "This notebook demonstrates how to use labnotebook for MNIST in PyTorch. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import labnotebook\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "import torch.utils\n",
    "from torchvision import datasets, transforms\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from torch.optim import SGD\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's make our train and test `DataLoader` objects using the build in MNIST dataset. We're going to keep *all* our parameters in a dictionary, `model_desc`, so we can easily pass it to `labnotebook`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model_desc = {'batch_size': 128,  #  train and test batch size\n",
    "             'n_filters1': 32,   #  size of first conv layer\n",
    "             'n_filters2': 32,   #  size of second conv layer\n",
    "             'n_fc': 32,        #  size of fully connected layer\n",
    "             'dropout': False,   #  wether to use dropout or not\n",
    "             'n_epochs': 5,     #  number of epochs to train for\n",
    "             'lr': 0.001}        #  learning rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
      "Processing...\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "train_loader = torch.utils.data.DataLoader(\n",
    "    datasets.MNIST('data', train=True, download=True, transform=transforms.ToTensor()),\n",
    "    batch_size=model_desc['batch_size'],\n",
    "    shuffle=True)\n",
    "\n",
    "\n",
    "test_loader = torch.utils.data.DataLoader(\n",
    "    datasets.MNIST('data', train=False, download=True, transform=transforms.ToTensor()),\n",
    "    batch_size=model_desc['batch_size'],\n",
    "    shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Setup \n",
    "We specifiy a traditional convnet with two conv layers, max pool, dropout, the usual. As above, we keep our architecture parameters in the `model_desc` dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MnistModel(nn.Module):\n",
    "    def __init__(self,\n",
    "                n_filters1=32,\n",
    "                n_filters2=32,\n",
    "                n_fc=128,\n",
    "                dropout=False):\n",
    "    \n",
    "        super(MnistModel, self).__init__()\n",
    "\n",
    "        self.n_filters1 = n_filters1\n",
    "        self.n_filters2 = n_filters2\n",
    "        self.n_fc = n_fc\n",
    "        self.dropout = dropout\n",
    "\n",
    "        self.conv1 = nn.Conv2d(1, self.n_filters1, 5, padding=2)\n",
    "        self.conv2 = nn.Conv2d(self.n_filters1, self.n_filters2, 5, padding=2)\n",
    "        self.fc1 = nn.Linear(self.n_filters2*7*7, self.n_fc)\n",
    "        self.fc2 = nn.Linear(self.n_fc, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = F.max_pool2d(F.relu(self.conv1(x)), 2)\n",
    "        x = F.max_pool2d(F.relu(self.conv2(x)), 2)\n",
    "        x = x.view(-1, self.n_filters2*7*7)   # reshape Variable\n",
    "        x = F.relu(self.fc1(x))\n",
    "        if self.dropout: x = F.dropout(x, training=self.training)\n",
    "        x = self.fc2(x)\n",
    "        return F.log_softmax(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = MnistModel(\n",
    "    n_filters1=model_desc['n_filters1'],\n",
    "    n_filters2=model_desc['n_filters2'],\n",
    "    n_fc=model_desc['n_fc'],\n",
    "    dropout=model_desc['dropout'])\n",
    "model = model.cuda() # optional"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LabNotbook Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just like in `basic_usage.ipynb`, the first step is to initialize the package by providing it the address of the database you want to use.\n",
    "\n",
    "It will create three tables: `experiments`, `steps`, and `model_params`.\n",
    "\n",
    "- `experiments` is used to store a list of experiments, along with their hyperparameters and final results.\n",
    "\n",
    "- `steps` is used to store the intermediary results for each step of each experiment. This is what you would want to plot if you're monitoring your experiments.\n",
    "\n",
    "- Finally, `model_params` is used to store your model parameters; what you would use to save the weights of your neural network for later inference. This can get pretty big so it's recommended not to save all the parameters at every step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "db_url = 'postgres://postgres:1418@localhost/experiments'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/henripal/anaconda3/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use \"pip install psycopg2-binary\" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.\n",
      "  \"\"\")\n"
     ]
    }
   ],
   "source": [
    "experiments, steps, model_params = labnotebook.initialize(db_url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training our ConvNet\n",
    "\n",
    "We'll use a normal training loop, but instead of printing results out, we'll just log them with `step_experiment`.\n",
    "\n",
    "There are only three extra lines of code added to permanently record this experiment in your database and plot it using the web app: `start_experiment`, `step_experiment` and `stop_experiment`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = SGD(model.parameters(), lr=model_desc['lr'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch  0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/henripal/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:27: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch  1\n",
      "epoch  2\n",
      "epoch  3\n",
      "epoch  4\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Run 65 on GPU 0 at 2018-03-21 20:27:34.094927"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we start the experiment and output it to an 'experiment' variable\n",
    "# we can then pass this experiment to step_experiment and end_experiment\n",
    "experiment = labnotebook.start_experiment(model_desc = model_desc)\n",
    "timestep = 0\n",
    "\n",
    "for epoch in range(model_desc['n_epochs']):\n",
    "    print('epoch ', epoch)\n",
    "    \n",
    "    model.train()\n",
    "    \n",
    "    for data, target in train_loader:\n",
    "        data, target = Variable(data), Variable(target)\n",
    "        data, target = data.cuda(), target.cuda()\n",
    "        optimizer.zero_grad()\n",
    "        output = model(data)\n",
    "        loss = F.nll_loss(output, target)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        prediction = output.data.max(1)[1]\n",
    "        accuracy = np.mean(prediction.eq(target.data)) * 100\n",
    "        \n",
    "        if timestep%100 == 0:\n",
    "            model.eval()\n",
    "            for data, target in test_loader:\n",
    "                data, target = Variable(data, volatile=True), Variable(target, volatile=True)\n",
    "                data, target = data.cuda(), target.cuda()\n",
    "                output=model(data)\n",
    "                prediction = output.data.max(1)[1]\n",
    "                val_accuracy = np.mean(prediction.eq(target.data))*100\n",
    "                \n",
    "        labnotebook.step_experiment(experiment, timestep,\n",
    "                   trainloss=loss.data[0],\n",
    "                   trainacc=accuracy,\n",
    "                   valacc=val_accuracy,\n",
    "                   epoch=epoch,\n",
    "                    custom_fields={'whatever': val_accuracy})\n",
    "        \n",
    "        timestep += 1\n",
    "        \n",
    "labnotebook.end_experiment(experiment,\n",
    "                            final_trainloss=np.mean(loss.data[0]),\n",
    "                            final_valacc=np.mean(accuracy),\n",
    "                            final_trainacc=np.mean(val_accuracy))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing our experiments\n",
    "### Through the web app\n",
    "\n",
    "Two steps are needed:\n",
    "- Launch the backend flask API by running from the command line:\n",
    "```\n",
    "start_backend <database_url>\n",
    "```\n",
    "- Navigate to the `labnotebook/frontend` directory and serve its contents, for example by running `python -m http.server`\n",
    "\n",
    "You should see something like this after selecting experiments from the left menu:\n",
    "\n",
    "![](./img/mnist.png)\n",
    "\n",
    "You can change what you see, turn live updating on or off, etc... from the `options` menu. \n",
    "\n",
    "## Through direct querying from the database\n",
    "\n",
    "See example queries from the `basic_usage` notebook. \n",
    "\n",
    "For more advanced queries, take a look at [sqlalchemy's documentation](http://docs.sqlalchemy.org/en/latest/orm/tutorial.html#querying) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}