00_cifar10_tinyvgg_benchmark.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "00_cifar10_tinyvgg_benchmark.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.7"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mrdbourke/m1-machine-learning-test/blob/main/00_cifar10_tinyvgg_benchmark.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5TSbi6c74A6E"
      },
      "source": [
        "# CIFAR10 TinyVGG Benchmark\n",
        "\n",
        "The following notebook tests the speed at which a given device can perform training iterations on the [CIFAR10 dataset](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/cifar10/load_data) (10 classes, 50,000 training images, 10,000 testing images) training the TinyVGG architecture as a base.\n",
        "\n",
        "It's designed to be a simple test to compare Apple's M1 (normal, Pro, Max) to each other and other sources of compute.\n",
        "\n",
        "| Model | Dataset | Train Size | Test Size |\n",
        "| ----- | ----- | ----- | ----- |\n",
        "| [TinyVGG](https://poloclub.github.io/cnn-explainer/) (trained from scratch) | CIFAR10 (from TensorFlow Datasets) | ~50,000 (32x32x3) images | ~10,000 (32x32x3) images|\n",
        "\n",
        "## Resources\n",
        "* Code on GitHub: https://github.com/mrdbourke/m1-machine-learning-test\n",
        "* Code in this notebook adapted from: https://dev.mrdbourke.com/tensorflow-deep-learning/03_convolutional_neural_networks_in_tensorflow/"
      ],
      "id": "5TSbi6c74A6E"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "06JiS98-5DvF"
      },
      "source": [
        "## Check for GPU\n",
        "\n",
        "If you're using Google Colab, you'll need to activate a GPU:\n",
        "1. Go to \"Runtime\"\n",
        "2. Go to \"Change Runtime Type\"\n",
        "3. Select \"GPU\"\n",
        "4. Click \"Save\" (this will restart the runtime)\n",
        "\n",
        "If you're using a Mac with an M1 chip, the GPU should already be selected if you've installed TensorFlow correctly."
      ],
      "id": "06JiS98-5DvF"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LbLhLej4JbQE",
        "outputId": "96c253b3-fb77-4abe-c06d-a616aef387f5"
      },
      "source": [
        "# This will error if not using a Nvidia GPU (only works on Colab)\n",
        "!nvidia-smi"
      ],
      "id": "LbLhLej4JbQE",
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mon Nov 15 00:23:58 2021       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 495.44       Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   47C    P0    65W / 149W |      0MiB / 11441MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3bHGp6bH5OCm",
        "outputId": "71c42d9d-86f1-4748-fd23-71291ac55259"
      },
      "source": [
        "# Check for GPU using TensorFlow\n",
        "import tensorflow as tf\n",
        "tf.config.list_physical_devices()"
      ],
      "id": "3bHGp6bH5OCm",
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'),\n",
              " PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cQ4yT7yE6heC"
      },
      "source": [
        "## Setup hyperparameters\n",
        "\n",
        "Change these to suit your needs.\n",
        "\n",
        "The main one will be the device you're running code on.\n",
        "\n",
        "E.g. `DEVICE = \"Google Colab (K80 GPU)\"` if using Google Colab."
      ],
      "id": "cQ4yT7yE6heC"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L8nDFuw_6tCE"
      },
      "source": [
        "# Setup hyperparameters\n",
        "BATCH_SIZE = 32 # good for your health: https://twitter.com/ylecun/status/989610208497360896\n",
        "EPOCHS = 10 # only run for a short period of time... we don't have all day\n",
        "DATASET_NAME = \"cifar10\" # change this to try other image datasets from TensorFlow Datasets\n",
        "DEVICE = \"Google Colab (K80 GPU)\" # change this depending on where you're runing the code"
      ],
      "id": "L8nDFuw_6tCE",
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fc0MfFZU5HhS"
      },
      "source": [
        "## Get helper functions and import dependencies\n",
        "\n",
        "The function below downloads the helper functions if necessary (if running this notebook in Google Colab, it's easier to download a single file than clone the whole repo)."
      ],
      "id": "fc0MfFZU5HhS"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UBY_EONu5HQk",
        "outputId": "cf5504aa-2afc-497a-af66-a9e88d98f747"
      },
      "source": [
        "# Get helper functions\n",
        "import os\n",
        "import requests\n",
        "\n",
        "if not os.path.exists(\"helper_functions.py\"):\n",
        "  print(\"Downloading helper functions...\")\n",
        "  r = requests.get(\"https://raw.githubusercontent.com/mrdbourke/m1-machine-learning-test/main/helper_functions.py\")\n",
        "  print(\"Writing helper functions to file...\")\n",
        "  open(\"helper_functions.py\", \"wb\").write(r.content)\n",
        "else:\n",
        "  print(\"Helper functions already downloaded, skipping redownload.\")"
      ],
      "id": "UBY_EONu5HQk",
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Helper functions already downloaded, skipping redownload.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b8f23a80",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "68ecf66a-e18f-475b-b389-4283bd45b861"
      },
      "source": [
        "# Check TensorFlow version\n",
        "import tensorflow as tf\n",
        "print(tf.__version__) # should be 2.5.0+\n",
        "\n",
        "# Get TensorFlow Datasets\n",
        "import tensorflow_datasets as tfds\n",
        "\n",
        "# Get data science libraries\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "from tensorflow.keras import layers\n",
        "from timeit import default_timer as timer \n",
        "from helper_functions import print_train_time"
      ],
      "id": "b8f23a80",
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2.7.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2425d422"
      },
      "source": [
        "## Get data from TensorFlow Keras Datasets\n",
        "\n",
        "Let's download the data and inspect it. "
      ],
      "id": "2425d422"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d1dcec42"
      },
      "source": [
        "# Get data from tf.keras.datasets\n",
        "(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.cifar10.load_data()\n",
        "\n",
        "# Normalize pixel values to be between 0 and 1\n",
        "train_images, test_images = train_images / 255.0, test_images / 255.0"
      ],
      "id": "d1dcec42",
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ea3b59dd"
      },
      "source": [
        "class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',\n",
        "               'dog', 'frog', 'horse', 'ship', 'truck']"
      ],
      "id": "ea3b59dd",
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 367
        },
        "id": "db015720",
        "outputId": "d4e8157f-0f92-434e-d379-94150bf12585"
      },
      "source": [
        "# Inspect image\n",
        "image = train_images[0]\n",
        "label = int(train_labels[0])\n",
        "\n",
        "print(f\"\"\"\n",
        "  Image shape: {image.shape}\n",
        "  Image dtype: {image.dtype}\n",
        "  Target class from Food101: {label}\n",
        "  Class name (str form): {class_names[label]}\n",
        "        \"\"\")\n",
        "plt.imshow(image)\n",
        "plt.title(class_names[label])\n",
        "plt.axis(False);"
      ],
      "id": "db015720",
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "  Image shape: (32, 32, 3)\n",
            "  Image dtype: float64\n",
            "  Target class from Food101: 6\n",
            "  Class name (str form): frog\n",
            "        \n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAAD3CAYAAADmIkO7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAXDklEQVR4nO2dS4wc13WGz62ufk0/5z2cGT5EUqRoQZRsy4LAOLAUb+xFhARZBMnCWWQZJ0tvEyAJkCyyy2sVIwjyWASOXzCQILEFJ5Zh2IKE2KIVmiJFzQzJ4bx6uqu7uqur+mZBBdDi/idikJCHzv8BAow6vjW3q+vvS9z/nnOc914IIfaIHvUECCFhKE5CjEJxEmIUipMQo1CchBiF4iTEKBTnY4Bz7qJz7k3n3MA591uPej7k4RA/6gmQD8UXRORb3vvnHvVEyMODK+fjwWkReSsUcM6VHvJcyEOC4jSOc+6bIvKyiPyxcy5xzv2Nc+7PnHPfcM4NReRl59wl59yrzrmec+4t59wrHxi/6Jz7mnOu75z7vnPu95xz//bIPhD50FCcxvHe/5yI/KuIfN573xSRTER+VUR+X0RaIvI9EfmaiPyTiKyIyG+KyF875y6+f4s/EZGhiKyJyK+9/x95DKA4H0++4r3/jvd+JiLPiUhTRP7Ae595778pIl8XkV95/5+8vyQiv+29H3nvr4rIXz66aZMHgeJ8PNn6wP9eF5Gt94X6X9wSkQ0RWZb7m35bYCwxDMX5ePLBVKLbInLSOffB7/KUiOyIyJ6I5CKy+YHYyf/76ZH/DSjOx5/vichIRL7gnCs7514SkZ8Xkb/z3hci8iUR+R3n3Jxz7ikR+dyjmyp5ECjOxxzvfSb3xfhZEdkXkT8Vkc95799+///yeRHpiMhdEfkrEflbEZk8gqmSB8Qx2fr/F865PxSRNe89d22Nw5Xzpxzn3FPOucvuPi+IyK+LyD886nmR/x4e3/vppyX3/ym7LiK7IvJHIvKVRzoj8qHgP2sJMQr/WUuIUdR/1n7yUy/BZbXXO4TjqtEseH2hglfpU4tzMLa80ICxpW4TxiqlcvB6XK3DMVLCj+TwqAdjWY4/23y3A2NRMQ1en0zwhup4PIaxWr0GY4UUMDZKk+D1TrcNx4jH98smGYyVJPy9iIiUSuFz/K0m/p4bDfx+lMv4eaTKHL1T1q0o/I5onzn3DsZ+43f/PBjkykmIUShOQoxCcRJiFIqTEKNQnIQYheIkxCiqlfLW1WDZGhER6e3vw9gC2L12i3hbe6lowZirr8DYcIYtnaQI2xveVeCY0Rhvh49SbG9Mi7B9JCKyX8Lb6LU4PMc8x/crga18EZFqtQpjo/EQxvJZ+HO78SIcEynVi6aKFVSP8XuQADvisMjhmLk5bKW4CNs2DlhtIiIS4XVrNA7bX/k0fF1EpBTj7wVO4YFHEEIeChQnIUahOAkxCsVJiFEoTkKMQnESYhTVSqnH2AIQZWf4NLBMzqzi7IyV5QU8D22r3OE5ppNw9sZ4irf5vXK/Sl3JZlGyUvwM/73OQjgbJ5/i+1XKeB4FThSRUgV/aZMs/KymOX4ec8r94gaeY00Zl7uw3RN5bC3lgueouFjSbOBMqGQ4grFpHrZMIuVvDfrHOAjgykmIUShOQoxCcRJiFIqTEKNQnIQYRd2trTl82LjVwkMvbMwHry/W8Unp8gzXxUkO8WH0YoZ/X9JReP4RPvcubaUmUazsMvaOB3ic8pQXWuEdw0EfH1LPlAPsKTiULSLilV3NJqjDM81SOCYq8AcrKwfwC1A3SUQkBturkwkeUynjLzSa4Xd4khzBmICkCRGRKniN8xneUT4ePniRfa6chBiF4iTEKBQnIUahOAkxCsVJiFEoTkKMolop81Ucritb5R1w6Hm5jWu2FDN8Yls5yy2lWClkA+rATGbKVr7ie8TK4etigi0HX8K/gffuhVs8FFP8qQcjfCh7VGDbqVlXWitMwn+vJPgzRw7bDaWq0gZhiG2zuXJ4jrHScGus1H1Kp9hKmQm+Zy/Bc+yNwu9PAqw7EZHx9MHXQa6chBiF4iTEKBQnIUahOAkxCsVJiFEoTkKMolopy128Hd4qYwujVgvHohLeuq4r9XmmObYVZkqmhffhLXatC3WRYZtl5pWMD8XC8DHOmhhk4QyTosDPd6S0fsiV2GCI579zGJ5HGXQpFxFpJ/jZT+/idh3pMbaCTi2dD15fWdmEY1wL1+eZHB3AWJLg7J7jAbZS9o/Dttm7W3gehdIxHcGVkxCjUJyEGIXiJMQoFCchRqE4CTEKxUmIUdT93fVl3AahXcEn8JtzYevAKVaEKBkCTskGmaR4Wz4CNstiC7eFaDSwfdQ/xvZAp40zPgZK0a1bO+F7JhNspVTw45CNOSWrpowzZ949CGfHTLxSlE3JSum0cafyKx95Hsb6d8K2mR8pf2sJZztNRvh5JAlem6plfM+Ta+HPtrKyCsfs9rE1g+DKSYhRKE5CjEJxEmIUipMQo1CchBiF4iTEKKqVstDCmSJxFt56FxGplsO3naviTsKTFNsNU6XfRbcb7ssiIuJBUaiswL9J06lSfKqJ+6jc3sO9MN65hbMV9gbhz6bUipLTSs+ZX/jZ52Bs8wSe/9+/fiN4/bvX78Ix+Qxn4sQRtj4GvT0YGyXh59hqYWtDCpwdU6vhcRWQPSUiMufwuLwIfzmnTq7DMa1D3EsHwZWTEKNQnIQYheIkxCgUJyFGoTgJMYq6W7uysAhj6SHe1Yxc+LYJKGMvIpJmeHsydko9HaVtAfrlSad4l7E7jw+wZ0q34xvbt2HssI/niOoLlZQWDu0avt9KjHcFa4d4R/nJ9lrw+p0FPI/d3j0Ym4zwM37j2jUYi/Lwqf5pQ2kl0cEHziXCr3ing92D1kxp/wDqTPmsD8ecUZJIEFw5CTEKxUmIUShOQoxCcRJiFIqTEKNQnIQYRe9svbSMY018KD6KwoeGe/0jOGY6TPD9Cq0dAy6o48EB/GYT1wmaCo79+Aa2AIYTXNq/VsNdwGuV8BzrDbzNP1/CttPr13dhLM/w1z3phK2U5Xn8PJxge2OaY6ttlOFaRkNQKyjL8Wd2ijWmdOuQcqS08oiU2kmg+3k+wVaVV2w4BFdOQoxCcRJiFIqTEKNQnIQYheIkxCgUJyFG0dvtAktERMQp5eoRVaWey5zgU/ux8hsSRUo9IGCzVOu4HcP+XZzVMdrHVtDZBWw5TJRK/DVgmVw8twHHRMoN8xJ+xn3FyopL4TpHrQr+Xhbnz8HYuSdPwdjN974PY29f2wler8SKTeGxDZfn+BWPlI7j5Qp+jrNZ+L3Suqw79+DrIFdOQoxCcRJiFIqTEKNQnIQYheIkxCgUJyFGUa2UVOnI7KY4s0AknEEwHOICSNkU/07kEbYpkhG2PvogtnESf2yf4/udXsJb5efW8db7aIzHbVx4Nni94rFdcnSMv5d6FxdlkwOcaXFy7UTwem+Is23OPvUkjLXncVZNe/4SjB3thZ//0TFuaVFW7J7I44yg6UzJdlK6hxfT8PutJLnA1iAaXDkJMQrFSYhRKE5CjEJxEmIUipMQo1CchBhFtVIKp/T4AN19RfC2cb2Gi4I1W3jr/fYetm1ubuMuyXE5PI/KLu5rMt7F93tyBdsln34J2wrv7BzCWGsjXERtaTFccEtE5N4eLuLV7Sq2wkzp8gwKWt3bC2eJiIjENdzdfK93B8Z27uAsknI5/B5029jbSFNsU/gYrz9O8T5mis0SufA4p2RI/Q/qe3HlJMQqFCchRqE4CTEKxUmIUShOQoxCcRJiFNVK6XabMJbH2EpJknBGhVdaxB8PcNbBrfewdZAkeFu+Xgv/9ty5ibNjVmu46NPGxmkY664/AWPlgZLiAIqebT77Ah5yF9sb9RxbQYXgTJfhMBw7MYf75WQF/lyugd+dzcY6jLW6YQtpcHAXjrm3ewBjU4fto3GGi4ZJhL2PRjWcJZWlikWkFAyDU3jgEYSQhwLFSYhRKE5CjEJxEmIUipMQo6i7tYMe3gWLM1xrp4xKz+MSNhKXcHCU4J3c+RY+6N1thHfV0iO8W7uyjmvwbFz+FIz9aBt3V752HceunFgIXu/18JjVc+G6QyIikYxgLJvgndyuD++89u/hd6Ce4VpGJxbCn0tEpFfguj7ly/PB66lykP473/gqjG1v4c9cUndQ8aF4dM5+qrUNmeJnBcc88AhCyEOB4iTEKBQnIUahOAkxCsVJiFEoTkKMolopJaW8fKEc8vVgGzoCbRpERAqHrZQjZRe631fqx0zCdsSJDrZfPvHyyzC2efFFGPvSF/8CxtaUQ+ClLFwfaefGO/h+Zz8CY7XF8zDW8ErX7sN7wev1WdjaEBHJUmzb7A9wrLuMkwQW184Er6dJG46JcEiKCj7sr9UQmk6xleXycAKH8zixQ+uwjeDKSYhRKE5CjEJxEmIUipMQo1CchBiF4iTEKOr+rlNKyBfKKXtUll6pjC8+Ve6nlOBZWMRtHNbmwtbNx56/AMdcuoLtkqN72D6q5jhz5uzmJozNwIdbW8G1e/IxtqRGSjZLluNx0zT8KhSCbaB3drZh7Ic/+gGMXXkRz3FxLZwV1B+ErR4REdDBQUREls5g22ymtU/IFFsEWHTHe7g9xWSgTBLAlZMQo1CchBiF4iTEKBQnIUahOAkxCsVJiFFUK2UGTt+LiKQT7G9UQBZGHOOCSqUIb6+fX8OZEbU6/n05c/pk8Pqzn8SZJycuXoaxN7/7RRg7dRLPce3pZ2CssnwueD2e68AxozG2dNI+zjzZvb0FY0e7YVukmOLsknorXEBNRGRpCX/XW7ffgLHVExvB6/lIyYJKcVsFNzyCscLjjule8RHr1fBnq6zhz9yvKileAK6chBiF4iTEKBQnIUahOAkxCsVJiFEoTkKMolop5RIOHykFnIpxeNu4PleHY0pKJ+EVJfNk6w7OBDj3sc8Er28+E75+H2yJTAdDGOu0sPWxfOE5GBvG4Z4ib73xfThmkuJ59Pv4eezvvAdjpSJsZdVq+B3YeCJse4iIXL6AC43lJZwpUi51w9crOGspHuMiXqNbuAu4ZhXmyrKVgL4+c4v4c60qPXgQXDkJMQrFSYhRKE5CjEJxEmIUipMQo6i7tZMU74LNVfFQVwvvZpUjXMPGFzhWb+JWDa/88iswduWznw5eby+twjG7N34MYyVl/r0BriG09+5/wNjtQXjH8NUvfxmOadbxAevxBB8QX1vFO8pt0CH85jY+LJ8pz2Nh/QyMXXjm4zAmoOv1YQ/XKxoBd0BE5CjFc3Qev8PjFCd2JD7sLPgE6+VSeBNahSsnIUahOAkxCsVJiFEoTkKMQnESYhSKkxCj6DWEPK7rIzN8aNjl4W3o3CstF5SaLbUqbl383Mfxtny1HLYcrr6Ja9gc3cYdpScTvFU+ODqEsa3rV2Es8eFkgHKB/1YzxtZSu4YPXy/PYyvlzu7d4PVcabsxGmDbZusmPmQv8haMJEm4BlItxu9HXl2BsYMcvzv1Oq6BNNfCSRr1OGz3DEZ9OCafYUsHwZWTEKNQnIQYheIkxCgUJyFGoTgJMQrFSYhRVCtFBJ/Mn+XYZolBq+FCqdmSCd5qXu3guj7/+NWvw9jCanjLfuVEuE2DiEg2wtkl5XJ4C11EpNnAW/ZxhK2PBrB71lZwzZl0gFsM1Et4jgd7+zA2BZ2cWzVsKWQJtlJ+8gbubH3n7WswNslBi4QyfoaF9nw3sbUkDfwOR1VsZdWALTIv+FldevoJPA80hwceQQh5KFCchBiF4iTEKBQnIUahOAkxCsVJiFH0rJQZLpxUUTIjajGwYCJ8P6+U6J9lODNifz+cTSEikuyFY/Upzh6YCf5cC/PY3uiuL8NYXuDOyzu3w3P0grMwogh/bVmOLamSw4XBGrWw/QUSjO7fTwsqWUZFhu2qCLxz/RG2j7Iq7lDdWsfPfljHrSsGM2yzjIfhNW2xfRaOWVKsMQRXTkKMQnESYhSKkxCjUJyEGIXiJMQoFCchRlGtlMjhDIdaFZ/A9yDDpFHHHaobrSUYG01xhsBiqwJjMZhHdrwLx8wifL9RGVsHq6s462CW4W35i5c3g9df+9a/wDGZx13Fyw7bVWmCx7Vb4ayaSoxfkZJT+oko3aZv3sG2SK8X/s4mDnfzXr6A15iNrpJV4/F3fbSPn1VlHLakGhtKJtEIZ2QhuHISYhSKkxCjUJyEGIXiJMQoFCchRlF3aysx1u5ogg8Ul0BLgJlS32Y0xYeXS2V8iLpawbtx5XJ4HpU53Jag08YH8O/u4V3e0UZ411VEZOXkeRjbuReu6/P0J34Gjkn2bsPYjWu41cEwwQe941L4+Xc6uDaSU2pM3dnBc3zvlnLwvRp+/u1VvNO/vKDMUdk1dof4u54/wtLYWFkIXt/s4nfg+lWcoPHyL4avc+UkxCgUJyFGoTgJMQrFSYhRKE5CjEJxEmIU1UpZXcbanR4cwFhahLfYh/jssvgIHwyOlcPX7TY+bFwBrQ7SIa4hVC8rjyTDsR+89hqMnb2ILZjt7fAWe6TUW5qr4lpAJcWuqtexdTBMwlZKmmKLK1dacjTreB5XPnoBxmrgAH5ewrWRiik+pJ5uYSslGuDO1itzLRj76IWnw2O6q3DM63duwhiCKychRqE4CTEKxUmIUShOQoxCcRJiFIqTEKOoVsqpk7jGSsfhbejrW+Gt7d09nF2SFUrX6Cae5lDpRF3Mwp2XS8pv0uEetogGCd7OH0/xPEoex1rNcNfu3buHcMz2ENsDM48tmNVlbDu5WbjlxVEP1/upNvB31u1gK6JSws9/AjpsS4zto+EE3y9LlBYUMzzu/Mk1GFtfCz/HrW1smR3sYbsHwZWTEKNQnIQYheIkxCgUJyFGoTgJMQrFSYhRVCulPY+3oVNla3h+BXSHbuAiTfu7uGDYWGlnEFdwcSc0bDbFGTBTpQv1cYpthYaShTEeYesjHYcLfGXKHAsl5j3uzJ30lXYM7XChtHYbF0NLU3y//QP8rJpNnB3jovB64XJsw1ViXOStih0/qVTwszpz/gyMpaPwXL797atwzL9fu4cnAuDKSYhRKE5CjEJxEmIUipMQo1CchBiF4iTEKKqVEtdwuNbGGSsLzbDm4xTbFOU67rvRV/pWSIF/X+q1lfAQpUN1McH9RCpzeB7lGD+PUglbSBMfnks2xfaRVzJPHHYcxGfY0ilAqKxkg0gF20e9I2ylpFk4A0ZEpNMNW2MxsFhERCLl2Y9Ad3MRkd39AYwdKRlIg2E4y+ifX30b/60HT0rhykmIVShOQoxCcRJiFIqTEKNQnIQYheIkxCiqlZIoxZGk1IShZiO8L1+u433+hpI+0Olg6yPp414eST9ccCkZKVkpYxxrVXCBrBroyyIikk+whRTH4d/HivKzWa7ibArn8MA5pVBaBEJ5gS2FSl3pYdPF9tHhIbYwBsBaai/gZz9Serb85F1csO3tH27B2KrSyn51E3y2CL+nS0rBMwRXTkKMQnESYhSKkxCjUJyEGIXiJMQo6m7t9i0cm/Tw7mprObzDV6srB57x5q8sLOBpJkN8orjXC8eODvBB6SO8uSelGd4lnXm8E10UeAdYZuGY9qvplK7XJaULeKokCXiwKVsGbRpERPIRbhlRKPWFCuUwfS8Jj0NdGkREDpUd+3ev4y+0d4BbrWdD/AfXOuFWDZdOb8AxyhQhXDkJMQrFSYhRKE5CjEJxEmIUipMQo1CchBhFtVKK8hKMTSvPw9hkFj7oHeXh1gMiIrUOtge6y9i2mY/wweyFUfggcu8Ql+/v7WO7JB3ix1Xk2J4Rj38DZ3l4juMU1/upVJR6RTGe/2CMD2anCUhW8PhQeSvCh7lnUR/GplP8HKuNsCVVKytdtCt4jmelC2PPPIvbQly8/CyMnTl/Pnj9hRexfbR9O9xlXYMrJyFGoTgJMQrFSYhRKE5CjEJxEmIUipMQozivZFMQQh4dXDkJMQrFSYhRKE5CjEJxEmIUipMQo1CchBjlPwH8eVQ6V51vFgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uif5hgCa5tpp"
      },
      "source": [
        "## Prepare data\n",
        "\n",
        "The data looks good. But before we model it, we're going to prepare it to run as fast as possible using [TensorFlow data loading best practices](https://www.tensorflow.org/guide/data_performance)."
      ],
      "id": "uif5hgCa5tpp"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3c862c1e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2ad14326-e58d-4fea-9e37-04a8c3761495"
      },
      "source": [
        "# Create datasets \n",
        "train_data = tf.data.Dataset.from_tensor_slices((train_images, train_labels))\n",
        "test_data = tf.data.Dataset.from_tensor_slices((test_images, test_labels))\n",
        "\n",
        "# Make datasets faster\n",
        "train_data = train_data.batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)\n",
        "test_data = test_data.batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)\n",
        "\n",
        "train_data, test_data, len(train_data), len(test_data)"
      ],
      "id": "3c862c1e",
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<PrefetchDataset shapes: ((None, 32, 32, 3), (None, 1)), types: (tf.float64, tf.uint8)>,\n",
              " <PrefetchDataset shapes: ((None, 32, 32, 3), (None, 1)), types: (tf.float64, tf.uint8)>,\n",
              " 1563,\n",
              " 313)"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "25d58b30"
      },
      "source": [
        "## Setup and fit model\n",
        "\n",
        "To keep things simple, we're going to use the TinyVGG architecture from the [CNN explainer website](https://poloclub.github.io/cnn-explainer/)."
      ],
      "id": "25d58b30"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7938027f",
        "outputId": "87e95c58-acb6-46ad-a2c7-e373daf25add"
      },
      "source": [
        "# Set random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# Start time\n",
        "start_time = timer()\n",
        "\n",
        "# Create a CNN model (same as Tiny VGG - https://poloclub.github.io/cnn-explainer/)\n",
        "model = tf.keras.models.Sequential([\n",
        "  tf.keras.layers.Conv2D(filters=10, \n",
        "                         kernel_size=3, # can also be (3, 3)\n",
        "                         activation=\"relu\", \n",
        "                         input_shape=(32, 32, 3)), # first layer specifies input shape (height, width, colour channels)\n",
        "  tf.keras.layers.Conv2D(10, 3, activation=\"relu\"),\n",
        "  tf.keras.layers.MaxPool2D(pool_size=2, # pool_size can also be (2, 2)\n",
        "                            padding=\"valid\"), # padding can also be 'same'\n",
        "  tf.keras.layers.Conv2D(10, 3, activation=\"relu\"),\n",
        "  tf.keras.layers.Conv2D(10, 3, activation=\"relu\"), # activation='relu' == tf.keras.layers.Activations(tf.nn.relu)\n",
        "  tf.keras.layers.MaxPool2D(2),\n",
        "  tf.keras.layers.Flatten(),\n",
        "  tf.keras.layers.Dense(10, activation=\"softmax\") # multi-class activation output\n",
        "], name=\"TinyVGG\")\n",
        "\n",
        "# Compile the model\n",
        "model.compile(loss=\"sparse_categorical_crossentropy\", # since labels aren't one-hot, use sparse_categorical_crossentropy\n",
        "              optimizer=tf.keras.optimizers.Adam(),\n",
        "              metrics=[\"accuracy\"])\n",
        "\n",
        "# Fit model \n",
        "history = model.fit(train_data,\n",
        "                    epochs=EPOCHS,\n",
        "                    steps_per_epoch=len(train_data),\n",
        "                    validation_data=test_data,\n",
        "                    validation_steps=len(test_data))\n",
        "\n",
        "# Track time \n",
        "end_time = timer()\n",
        "train_time = print_train_time(start_time, \n",
        "                              end_time, \n",
        "                              device=DEVICE)"
      ],
      "id": "7938027f",
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "1563/1563 [==============================] - 17s 7ms/step - loss: 1.7134 - accuracy: 0.3780 - val_loss: 1.5412 - val_accuracy: 0.4528\n",
            "Epoch 2/10\n",
            "1563/1563 [==============================] - 10s 7ms/step - loss: 1.4335 - accuracy: 0.4883 - val_loss: 1.4051 - val_accuracy: 0.5016\n",
            "Epoch 3/10\n",
            "1563/1563 [==============================] - 12s 8ms/step - loss: 1.3289 - accuracy: 0.5282 - val_loss: 1.2800 - val_accuracy: 0.5406\n",
            "Epoch 4/10\n",
            "1563/1563 [==============================] - 12s 8ms/step - loss: 1.2496 - accuracy: 0.5575 - val_loss: 1.2405 - val_accuracy: 0.5590\n",
            "Epoch 5/10\n",
            "1563/1563 [==============================] - 11s 7ms/step - loss: 1.1967 - accuracy: 0.5764 - val_loss: 1.2138 - val_accuracy: 0.5705\n",
            "Epoch 6/10\n",
            "1563/1563 [==============================] - 14s 9ms/step - loss: 1.1566 - accuracy: 0.5905 - val_loss: 1.1849 - val_accuracy: 0.5810\n",
            "Epoch 7/10\n",
            "1563/1563 [==============================] - 13s 8ms/step - loss: 1.1246 - accuracy: 0.6025 - val_loss: 1.1616 - val_accuracy: 0.5893\n",
            "Epoch 8/10\n",
            "1563/1563 [==============================] - 13s 8ms/step - loss: 1.0975 - accuracy: 0.6130 - val_loss: 1.1231 - val_accuracy: 0.6066\n",
            "Epoch 9/10\n",
            "1563/1563 [==============================] - 14s 9ms/step - loss: 1.0766 - accuracy: 0.6206 - val_loss: 1.1160 - val_accuracy: 0.6067\n",
            "Epoch 10/10\n",
            "1563/1563 [==============================] - 14s 9ms/step - loss: 1.0596 - accuracy: 0.6271 - val_loss: 1.1083 - val_accuracy: 0.6095\n",
            "\n",
            "Train time on Google Colab (K80 GPU): 166.911 seconds\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qymGk8mw6VaD"
      },
      "source": [
        "## Track results and save to file"
      ],
      "id": "qymGk8mw6VaD"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81
        },
        "id": "cferSb088tne",
        "outputId": "d88d79a3-6079-485f-ad98-bca241cf5359"
      },
      "source": [
        "results = {\n",
        "    \"device\": DEVICE,\n",
        "    \"dataset_name\": DATASET_NAME,\n",
        "    \"epochs\": EPOCHS,\n",
        "    \"batch_size\": BATCH_SIZE,\n",
        "    \"num_train_samples\": len(train_data)*BATCH_SIZE,\n",
        "    \"num_test_samples\": len(test_data)*BATCH_SIZE,\n",
        "    \"total_train_time\": round(train_time, 3),\n",
        "    \"time_per_epoch\": round(train_time/EPOCHS, 3),\n",
        "    \"model\": model.name\n",
        "    }\n",
        "results_df = pd.DataFrame(results, index=[0])\n",
        "results_df"
      ],
      "id": "cferSb088tne",
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>device</th>\n",
              "      <th>dataset_name</th>\n",
              "      <th>epochs</th>\n",
              "      <th>batch_size</th>\n",
              "      <th>num_train_samples</th>\n",
              "      <th>num_test_samples</th>\n",
              "      <th>total_train_time</th>\n",
              "      <th>time_per_epoch</th>\n",
              "      <th>model</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Google Colab (K80 GPU)</td>\n",
              "      <td>cifar10</td>\n",
              "      <td>10</td>\n",
              "      <td>32</td>\n",
              "      <td>50016</td>\n",
              "      <td>10016</td>\n",
              "      <td>166.911</td>\n",
              "      <td>16.691</td>\n",
              "      <td>TinyVGG</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                   device dataset_name  ...  time_per_epoch    model\n",
              "0  Google Colab (K80 GPU)      cifar10  ...          16.691  TinyVGG\n",
              "\n",
              "[1 rows x 9 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Z4oPZYi8y1Y"
      },
      "source": [
        "# Write CSV to file\n",
        "if not os.path.exists(\"results/\"):\n",
        "  os.makedirs(\"results/\")\n",
        "\n",
        "results_df.to_csv(f\"results/{DEVICE}_{DATASET_NAME}.csv\")"
      ],
      "id": "9Z4oPZYi8y1Y",
      "execution_count": 12,
      "outputs": []
    }
  ]
}