tiny_imagenet_custom_resnet/tiny_imagenet_custom_resnet.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "tiny_imagenet_custom_resnet.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "private_outputs": true,
      "collapsed_sections": [
        "Tce3stUlHN0L"
      ],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Tce3stUlHN0L"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "tuOe1ymfHZPu",
        "colab": {}
      },
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "MfBg1C5NB3X0"
      },
      "source": [
        "# TinyImageNet Custom ResNet\n",
        "\n",
        "\n",
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/examples/blob/master/template/notebook.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/examples/blob/master/template/notebook.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "xHxb-dlhMIzW"
      },
      "source": [
        "## Overview\n",
        "The ImageNet challenge (ILSVRC) is the most common benchmark to evaluate image classification and localization models. Although ImageNet is a good dataset for large scale evaluation, the resources required to build and train a model becomes a limiting factor even with a good GPU. The Tiny ImageNet is a dataset made famous by Stanford University which consists of 200 classes and 100,000 training images. It is a good subset of the actual ImageNet dataset and still manages to capture the variety and 'hardness' of the ILSVRC albeit at a smaller scale.\n",
        "\n",
        "As a researcher with resource constraints, it becomes imperative to adopt smart training strategies and efficiency tactics to effectively solve a hard problem. This notebook shows how building a model with 10x lesser parameters and using good training strategies (with Tensorflow 2.0) can enable one to build a good model with the available resources.\n",
        "\n",
        "This notebook will show the following:\n",
        "- Training with image augmentation on the fly with tf.keras' ImageDataGenerator class\n",
        "- Building an efficient custom ResNet model with 10x lesser parameters\n",
        "- Progressive resizing for the model to learn scene dependent features from diffeent image sizes\n",
        "\n",
        "\n",
        "**Note**: The top accuracies for this dataset found on Kaggle use pre-trained models with freeze layer training. This \n",
        "The top scorers for the 1st Stanford challenge can be found at this link: [Tiny ImageNet Visual Recognition Challenge](https://tiny-imagenet.herokuapp.com/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "vNbeBoBWsDfa"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "DBluBf7GsDfb",
        "colab": {}
      },
      "source": [
        "from __future__ import absolute_import\n",
        "from __future__ import division\n",
        "from __future__ import print_function\n",
        "\n",
        "! pip install -q tensorflow-gpu==2.0.0-beta1\n",
        "import tensorflow as tf\n",
        "assert tf.__version__.startswith('2')\n",
        "\n",
        "print(f'{tf.__version__}')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "gnv2z7-O2clU",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.layers import BatchNormalization, Conv2D, AveragePooling2D, MaxPooling2D\n",
        "from tensorflow.keras.layers import ZeroPadding2D, Activation, Flatten, add\n",
        "from tensorflow.keras.layers import GlobalAveragePooling2D, SeparableConv2D\n",
        "from tensorflow.keras.callbacks import LearningRateScheduler, ModelCheckpoint\n",
        "from tensorflow.keras.optimizers import Adam\n",
        "from tensorflow.keras.regularizers import l2"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "qT8LP0hfsDfi",
        "colab": {}
      },
      "source": [
        "# Import the data\n",
        "\n",
        "import os\n",
        "download_path = os.getcwd()\n",
        "    \n",
        "import pathlib\n",
        "path = tf.keras.utils.get_file('tiny-imagenet-200.zip', extract=True, \n",
        "                               cache_subdir=download_path,\n",
        "                               origin='http://cs231n.stanford.edu/tiny-imagenet-200.zip')\n",
        "\n",
        "data_dir = pathlib.Path(path).with_suffix('')\n",
        "\n",
        "TRAIN = data_dir/\"train\"\n",
        "VAL = data_dir/\"val/images\"\n",
        "VAL_ANNOT = data_dir/'val/val_annotations.txt'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "p-SQU5PLsDfn"
      },
      "source": [
        "## Image augmentation and image generators\n",
        "- The function below returns the generators for the ImageDataGenerator objects we will use to train and validate our ResNet model.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "kGSBMQVqsDfo",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
        "\n",
        "val_data = pd.read_csv(VAL_ANNOT , sep='\\t', names=['File', 'Class', 'X', 'Y', 'H', 'W'])\n",
        "val_data.drop(['X','Y','H', 'W'], axis=1, inplace=True)\n",
        "\n",
        "\n",
        "def train_val_gen(train_target=64, train_batch=64, val_target=64, val_batch=64):\n",
        "\n",
        "        train_datagen = ImageDataGenerator(\n",
        "            rescale=1./255,\n",
        "            rotation_range=18,\n",
        "            zoom_range=0.15,  \n",
        "            width_shift_range=0.2, \n",
        "            height_shift_range=0.2, \n",
        "            shear_range=0.15, \n",
        "            horizontal_flip=True,\n",
        "            fill_mode=\"reflect\", # Fills empty with reflections\n",
        "            brightness_range=[0.4, 1.6]  \n",
        "    )\n",
        "\n",
        "        train_generator = train_datagen.flow_from_directory(\n",
        "                TRAIN,\n",
        "                target_size=(train_target, train_target),\n",
        "                batch_size=train_batch,\n",
        "                class_mode='categorical')\n",
        "\n",
        "        val_datagen = ImageDataGenerator(rescale=1./255)\n",
        "\n",
        "        val_generator = val_datagen.flow_from_dataframe(\n",
        "            val_data, directory=VAL, \n",
        "            x_col='File', \n",
        "            y_col='Class', \n",
        "            target_size=(val_target, val_target),\n",
        "            color_mode='rgb', \n",
        "            class_mode='categorical', \n",
        "            batch_size=val_batch, \n",
        "            shuffle=False, \n",
        "            seed=42\n",
        "        )\n",
        "\n",
        "        return train_generator, val_generator"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "WlmlEVva1lgv"
      },
      "source": [
        "## Defining callbacks to employ different training strategies"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "iSzYk0oH10mv",
        "colab": {}
      },
      "source": [
        "# Creating a custom callback to save the model after every 5 epochs\n",
        "\n",
        "class EpochCheckpoint(tf.keras.callbacks.Callback):\n",
        "    def __init__(self, outputPath, every=5, startAt=0):\n",
        "        super(EpochCheckpoint, self).__init__()\n",
        "        \n",
        "        self.outputPath = outputPath\n",
        "        self.every = every\n",
        "        self.intEpoch = startAt\n",
        "        \n",
        "    def on_epoch_end(self, epoch, log={}):\n",
        "        \n",
        "        if (self.intEpoch+1) % self.every == 0:\n",
        "            path = os.path.sep.join([self.outputPath, \n",
        "                    \"custom_resnet.hdf5\".format(self.intEpoch+1)])\n",
        "            self.model.save(path, overwrite=True)\n",
        "        self.intEpoch+=1"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Aslv1B7gpahK",
        "colab_type": "text"
      },
      "source": [
        "## Learning rate decay function to enable the model to converge better"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OQx6hbJxpOkJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.callbacks import LearningRateScheduler\n",
        "\n",
        "NUM_EPOCHS = 30\n",
        "INIT_LR = 0.01\n",
        "\n",
        "def poly_decay(epoch):\n",
        "    # The epoch value is passed on by LearningRateScheduler\n",
        "    maxEpochs = NUM_EPOCHS\n",
        "    baseLR = INIT_LR\n",
        "    power = 1.0\n",
        "    \n",
        "    alpha = baseLR * (1 - (epoch / float(maxEpochs))) ** power\n",
        "    \n",
        "    return alpha"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "fhrD1MEvsDft"
      },
      "source": [
        "## Custom ResNet that uses Pre-Activation and BottleNeck Blocks with SeparableConv2D\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "-  We use 1x1 to increase the number of channels is to create a wider model with minimum increase in trainable parameters.\n",
        "- This [reserach paper](https://arxiv.org/abs/1812.01187) documents improved accuracy with AveragePooling2D in the shortcut connection. This model showed a performance drop and hence was replaced with a 1x1 convolution.\n",
        "- Uses SeparableConv2D rather than vanilla Conv2D to reduce the nmber of parameters and make the model feasible to train on constrained environments like Google colab."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "cNdKeWjwsDfz",
        "colab": {}
      },
      "source": [
        "class ResNet:\n",
        "\n",
        "    def residual_module(data, K, stride, chanDim, red=False, reg=0.0001, bnEps=2e-5, bnMom=0.9):\n",
        "        shortcut = data\n",
        "\n",
        "        bn1 = BatchNormalization(axis=chanDim, epsilon=bnEps, momentum=bnMom, beta_initializer=\"zeros\", gamma_initializer=\"ones\")(data)\n",
        "        act1 = Activation(\"relu\")(bn1)\n",
        "        conv1 = Conv2D(int(K * 0.25), (1, 1), use_bias=False, kernel_regularizer=l2(reg))(act1)\n",
        "\n",
        "        bn2 = BatchNormalization(axis=chanDim, epsilon=bnEps, momentum=bnMom, beta_initializer=\"zeros\", gamma_initializer=\"ones\")(conv1)\n",
        "        act2 = Activation(\"relu\")(bn2)\n",
        "        conv2 = SeparableConv2D(int(K * 0.25), (3, 3), strides=stride, padding=\"same\", use_bias=False, depthwise_regularizer=l2(reg), depthwise_initializer='glorot_uniform')(act2)\n",
        "\n",
        "        bn3 = BatchNormalization(axis=chanDim, epsilon=bnEps, momentum=bnMom, beta_initializer=\"zeros\", gamma_initializer=\"ones\")(conv2)\n",
        "        act3 = Activation(\"relu\")(bn3)\n",
        "        conv3 = Conv2D(K, (1, 1), use_bias=False, kernel_regularizer=l2(reg))(act3)\n",
        "\n",
        "        if red and stride == (2,2):\n",
        "            shortcut = AveragePooling2D((2,2))(bn1)\n",
        "\n",
        "        shortcut = Conv2D(K, (1,1))(shortcut)\n",
        "        x = add([conv3, shortcut])\n",
        "\n",
        "        return x\n",
        "\n",
        "\n",
        "    def build(width, height, depth, classes, stages, filters, reg=0.0001, bnEps=2e-5, bnMom=0.9):\n",
        "        inputShape = (height, width, depth)\n",
        "        chanDim = -1\n",
        "\n",
        "        inputs = tf.keras.Input(shape=inputShape)\n",
        "        x = BatchNormalization(axis=chanDim, epsilon=bnEps, momentum=bnMom, beta_initializer=\"zeros\", gamma_initializer=\"ones\")(inputs)\n",
        "        x = Activation(\"relu\")(x)\n",
        "        x = SeparableConv2D(64, (3, 3), use_bias=False, padding=\"same\", depthwise_regularizer=l2(reg), depthwise_initializer='glorot_uniform')(x)\n",
        "        x = SeparableConv2D(128, (3, 3), use_bias=False, padding=\"same\", depthwise_regularizer=l2(reg), depthwise_initializer='glorot_uniform')(x)\n",
        "        x = SeparableConv2D(256, (3, 3), use_bias=False, padding=\"same\", depthwise_regularizer=l2(reg), depthwise_initializer='glorot_uniform')(x)\n",
        "        x = BatchNormalization(axis=chanDim, epsilon=bnEps, momentum=bnMom, beta_initializer=\"zeros\", gamma_initializer=\"ones\")(x)\n",
        "        x = Activation(\"relu\")(x)\n",
        "        x = ZeroPadding2D((1, 1))(x)\n",
        "        x = MaxPooling2D((3, 3), strides=(2, 2))(x)\n",
        "\n",
        "        for i in range(0, len(stages)):\n",
        "            stride = (1, 1) if i == 0 else (2, 2)\n",
        "            x = ResNet.residual_module(x, filters[i], stride, chanDim, red=True, bnEps=bnEps, bnMom=bnMom)\n",
        "\n",
        "            for j in range(0, stages[i] - 1):\n",
        "                x = ResNet.residual_module(x, filters[i], (1, 1), chanDim, bnEps=bnEps, bnMom=bnMom)\n",
        "\n",
        "        x = BatchNormalization(axis=chanDim, epsilon=bnEps, momentum=bnMom)(x)\n",
        "        x = Activation(\"relu\")(x)\n",
        "        x = Conv2D(200, (1,1), kernel_regularizer=l2(reg))(x)\n",
        "        x = GlobalAveragePooling2D('channels_last')(x)\n",
        "        x = Activation(\"softmax\")(x)\n",
        "\n",
        "        model = tf.keras.Model(inputs, x, name=\"resnet\")\n",
        "\n",
        "        return model"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d31xznchs1n_",
        "colab_type": "text"
      },
      "source": [
        "## Build the model\n",
        "---\n",
        "We pass in (None, None, 3) for the shape of the image to allow th emodel to take up any image sizes we provide it. This way, we can employ progresive resizing to feed in different sizes of images to get the model to learn scene dependent features as well."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "oKVErJnbsDf5",
        "colab": {}
      },
      "source": [
        "model = ResNet.build(None, None, 3, 200, (3, 4, 6), (64, 128, 256, 512), reg=0.0005)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cEnyfbREsm71",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "callbacks = [EpochCheckpoint(\"/content/\", every=5),\n",
        "            LearningRateScheduler(poly_decay)]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "UD15cK08sDf-",
        "colab": {}
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "H9GdQWfCsDgE"
      },
      "source": [
        "## Compile the model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "kib6v8vhsDgI",
        "colab": {}
      },
      "source": [
        "opt = Adam(learning_rate=0.1, beta_1=0.9, beta_2=0.999, epsilon=0.1, amsgrad=False)\n",
        "model.compile(loss=\"categorical_crossentropy\", optimizer=opt, metrics=[\"accuracy\"])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "5rwwKT2EsDgO"
      },
      "source": [
        "## Using fit_generator to train the model\n",
        "- ImageDataGenerator is best suited for augmenting images on the fly and training the model. Custom image augmentation functions can also be used.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "ztTE-5SJ83jy",
        "colab": {}
      },
      "source": [
        "train_gen, val_gen = train_val_gen(train_target=64, train_batch=64, val_target=64, val_batch=64)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wSqzvttvAGQ1",
        "colab_type": "text"
      },
      "source": [
        "## Training the model on 64x64 sized images\n",
        "- First train the model with 64x64 sized images with a batch size of 64.\n",
        "- **Note**: Although training with higher batch sizes is recommended, ensure that Colab provides 25.51 GB of RAM rather than 12.72 GB. This will happen if colab crashes due to RAM usage and will prompt the user to allow RAM increase. \n",
        "- **Note**: Every epoch should take around 9.5 minutes with Colab's GPU hardware accelerator. Please ensure that the Hardware Accelrator in the Runtime settings is appropriately set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "_paZU0BAsDgP",
        "colab": {}
      },
      "source": [
        "model.fit_generator(\n",
        "  train_gen,\n",
        "  steps_per_epoch=100000 // 64,\n",
        "  validation_data=val_gen,\n",
        "  validation_steps=10000 // 64,\n",
        "  epochs=20,\n",
        "  max_queue_size=64 * 2,\n",
        "  callbacks=callbacks,\n",
        "  verbose=1\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "3YytXkT3CLYI",
        "colab": {}
      },
      "source": [
        "# Save the model\n",
        "filepath = \"/content/epoch_20.hdf5\"\n",
        "\n",
        "model.save(\n",
        "    filepath,\n",
        "    overwrite=True,\n",
        "    include_optimizer=True\n",
        ")\n",
        "\n",
        "# Load it again to continue training\n",
        "model = tf.keras.models.load_model(\n",
        "    filepath,\n",
        "    custom_objects=None,\n",
        "    compile=True\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YzRDgj0f33UJ",
        "colab_type": "text"
      },
      "source": [
        "## Training with 32x32 sized images\n",
        "- Train the model for 20 epochs with 32x32 sized images to enable the model to learn semantic scene dependent features for the same layers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "n-bTMEzKN-g8",
        "colab": {}
      },
      "source": [
        "train_gen, val_gen = train_val_gen(train_target=32, train_batch=64, val_target=64, val_batch=64)\n",
        "\n",
        "model.fit_generator(\n",
        "  train_gen,\n",
        "  steps_per_epoch=100000 // 64,\n",
        "  validation_data=val_gen,\n",
        "  validation_steps=10000 // 64,\n",
        "  epochs=20,\n",
        "  max_queue_size=128,\n",
        "  callbacks=callbacks,\n",
        "  verbose=1\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "1MbQZYeSnJHx",
        "colab": {}
      },
      "source": [
        "filepath = \"/content/epoch_40.hdf5\"\n",
        "\n",
        "model.save(\n",
        "    filepath,\n",
        "    overwrite=True,\n",
        "    include_optimizer=True\n",
        ")\n",
        "\n",
        "# Load it again to continue training\n",
        "model = tf.keras.models.load_model(\n",
        "    filepath,\n",
        "    custom_objects=None,\n",
        "    compile=True\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yisIPXvr4TT8",
        "colab_type": "text"
      },
      "source": [
        "## Training with 16x16 sized images\n",
        "- Train the model for 20 epochs with 32x32 sized images to enable the model to learn semantic scene dependent features for the same layers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "M-6IvpOvJNFC",
        "colab": {}
      },
      "source": [
        "train_gen, val_gen = train_val_gen(train_target=16, train_batch=64, val_target=64, val_batch=64)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "_XvA_nhdnWh9",
        "colab": {}
      },
      "source": [
        "model.fit_generator(\n",
        "  train_gen,\n",
        "  steps_per_epoch=100000 // 64,\n",
        "  validation_data=val_gen,\n",
        "  validation_steps=10000 // 64,\n",
        "  epochs=20,\n",
        "  max_queue_size=64,\n",
        "  callbacks=callbacks,\n",
        "  verbose=1\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "HSq0SvRoJYot",
        "colab": {}
      },
      "source": [
        "# Save the model\n",
        "filepath = \"/content/epoch_60.hdf5\"\n",
        "\n",
        "model.save(\n",
        "    filepath,\n",
        "    overwrite=True,\n",
        "    include_optimizer=True\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EmeG_TK-3rW8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Load it again to continue training\n",
        "model = tf.keras.models.load_model(\n",
        "    filepath,\n",
        "    custom_objects=None,\n",
        "    compile=True\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o0Dkbx024ZTH",
        "colab_type": "text"
      },
      "source": [
        "## Back to training with 32x32 sized images\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "T_poZQLCTkpR",
        "colab": {}
      },
      "source": [
        "train_gen, val_gen = train_val_gen(train_target=32, train_batch=64, val_target=64, val_batch=64)\n",
        "\n",
        "model.fit_generator(\n",
        "  train_gen,\n",
        "  steps_per_epoch=100000 // 64,\n",
        "  validation_data=val_gen,\n",
        "  validation_steps=10000 // 64,\n",
        "  epochs=20,\n",
        "  max_queue_size=64,\n",
        "  verbose=1\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "uvX-lzEMUDHq",
        "colab": {}
      },
      "source": [
        "# Save the model\n",
        "filepath = \"/content/epoch_80.hdf5\"\n",
        "\n",
        "model.save(\n",
        "    filepath,\n",
        "    overwrite=True,\n",
        "    include_optimizer=True\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cYfRItCa_5xM",
        "colab_type": "text"
      },
      "source": [
        "## Back to training with 64x64 sized images\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "QqVK3Hh745Ca",
        "colab": {}
      },
      "source": [
        "# Load it again to continue training\n",
        "model = tf.keras.models.load_model(\n",
        "    filepath,\n",
        "    custom_objects=None,\n",
        "    compile=True\n",
        ")\n",
        "\n",
        "train_gen, val_gen = train_val_gen(train_target=64, train_batch=64, val_target=64, val_batch=64)\n",
        "\n",
        "model.fit_generator(\n",
        "  train_gen,\n",
        "  steps_per_epoch=100000 // 64,\n",
        "  validation_data=val_gen,\n",
        "  validation_steps=10000 // 64,\n",
        "  epochs=20,\n",
        "  max_queue_size=64,\n",
        "  verbose=1\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "mwZkcCnEsDgU"
      },
      "source": [
        "## List of references for easy lookup\n",
        "\n",
        "---\n",
        "\n",
        "1. Building blocks of interpretability: [Link](https://distill.pub/2018/building-blocks/) (Holy Grail of Intuition!)\n",
        "2. Deep Residual Learning for image classification: [Link](https://arxiv.org/abs/1512.03385) (Resnet Paper)\n",
        "3. Bag of tricks for image classification: [Link](https://arxiv.org/abs/1812.01187) (Tweaks and tricks to Resnet for increased performance paper)\n",
        "2. Imbalanced Deep Learning by Minority Class\n",
        "Incremental Rectification: [Link](https://arxiv.org/pdf/1804.10851.pdf) (Selectively Sampling Data paper)\n",
        "2. Improved Regularization of Convolutional Neural Networks with Cutout: [Link](https://arxiv.org/pdf/1708.04552.pdf) (Cutout/Occlusion Augmentation paper)\n",
        "3. Survey of resampling techniques for improving\n",
        "classification performance in unbalanced datasets [Link](https://arxiv.org/pdf/1608.06048v1.pdf) (Resampling paper)"
      ]
    }
  ]
}