Donkey_Car_Training_using_Google_Colab.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "X93SodzAv9hV"
      ],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/robocarstore/donkey-car-training-on-google-colab/blob/master/Donkey_Car_Training_using_Google_Colab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BlmQIFSLZDdc"
      },
      "source": [
        "# Donkey Car Training using Google Colab\n",
        "\n",
        "Train your donkey car model using accelerated GPU for FREE on Google Colab"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "arsH-DhLcihq"
      },
      "source": [
        "## Check GPU and Tensorflow version\n",
        "If \"Found GPU at: / device: GPU: 0\" is displayed, the GPU is ready to use."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!nvidia-smi"
      ],
      "metadata": {
        "id": "ORYT2sEa9Vx1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Verified tensorflow version: > 2.9 and <= 2.15"
      ],
      "metadata": {
        "id": "Xf2CZgLm9gGk"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uQgEhuoTcg0N"
      },
      "source": [
        "import tensorflow\n",
        "print(tensorflow.__version__)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ba2oPDIrsDFg"
      },
      "source": [
        "## Git Clone the donkey repository"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oOxd9PFUyNxI"
      },
      "source": [
        "!git clone https://github.com/robocarstore/donkeycar.git\n",
        "%cd /content/donkeycar"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!git checkout v5"
      ],
      "metadata": {
        "id": "u6nZMEOSmBy3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9TkkcF-gsAnx"
      },
      "source": [
        "## Install donkey car"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# !pip3 install --upgrade pip setuptools wheel\n",
        "!pip3 install -e .[pc]"
      ],
      "metadata": {
        "id": "bhfs7OmMlXYT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "syCctLq2r4Wk"
      },
      "source": [
        "## Create Project"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1xjJBSITyXy2"
      },
      "source": [
        "!donkey createcar --path /content/mycar"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W47gmXA0O4Lo"
      },
      "source": [
        "### Download your dataset\n",
        "\n",
        "You can download your dataset to this Colab instance by the following command. You might obtain the uuid of the dataset from the meta.json under each tub."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%cd /content/mycar/data\n",
        "!curl  -O -J \"https://robocarstore-dc-sg.s3.amazonaws.com/tubs/63783f2f-9497-460b-a6ad-3607c20dcbff/data.tar.gz\"\n"
      ],
      "metadata": {
        "id": "22t7txjxCKld"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IfSDpXj9x16v"
      },
      "source": [
        "Use the following command to extract the dataset into a folder named tub_xxxxxxx"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "urhFWBkUyGf0"
      },
      "source": [
        "!tar -xzf data.tar.gz"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Alternatively, use the \"Getting UUID\" notebook to generate the list of tubs uuid for training\n",
        "\n",
        "https://github.com/sctse999/donkeycar-advance-course/blob/master/Getting%20tub's%20UUID.ipynb"
      ],
      "metadata": {
        "id": "st8m6bQ1JCQW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tub_uuids = ['b7da4c7a-d868-46e1-a2e7-1b02c472c8ef', 'a38a0958-baba-4b65-aded-e60e2f22e265']"
      ],
      "metadata": {
        "id": "zoXqqmPiJOTw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for tub_uuid in tub_uuids:\n",
        "  !curl -J \"https://robocarstore-dc-sg.s3.amazonaws.com/tubs/{tub_uuid}/data.tar.gz\" -o /tmp/data.tar.gz\n",
        "  !tar -zxf /tmp/data.tar.gz -C /content/mycar/data\n"
      ],
      "metadata": {
        "id": "dyyFNr_KC8xp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!ls"
      ],
      "metadata": {
        "id": "f4GCu9keDzmV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Assuming the latest created folder is the tub_name."
      ],
      "metadata": {
        "id": "4F40oA10DSnL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%cd /content/mycar/data\n",
        "\n",
        "tub_name=!ls -td -- */ | head -n 1\n",
        "tub_name = tub_name[0].strip('/')\n",
        "tub_name"
      ],
      "metadata": {
        "id": "QCNXdA7GCaKg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eyPsgV5KyFkg"
      },
      "source": [
        "Check whether the data is there"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KsItvBTkzWcH"
      },
      "source": [
        "!ls {tub_name}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Check number of images in this tub"
      ],
      "metadata": {
        "id": "gyskUezgrSUH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%cd /content/mycar/data/{tub_name}\n",
        "!find . -type f -name \"*.jpg\" | wc -l"
      ],
      "metadata": {
        "id": "UjFMR-rRrC-m"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Generate a movie for preview"
      ],
      "metadata": {
        "id": "oOnUjZaBELYY"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4AZvWSeiyqto"
      },
      "source": [
        "!pip uninstall keras-vis\n",
        "!pip install git+https://github.com/sctse999/keras-vis\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "%cd /content/mycar\n",
        "!donkey makemovie --tub ./data/tub* --out {tub_name}.mp4 --start 1 --end 2000"
      ],
      "metadata": {
        "id": "_G0VMYJ_tCCP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Download this movie"
      ],
      "metadata": {
        "id": "4d0ZP9JMESFC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import files\n",
        "\n",
        "files.download(f\"{tub_name}.mp4\")"
      ],
      "metadata": {
        "id": "Ca2ON1a5ERxU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T3Ya8qEUAfOv"
      },
      "source": [
        "## Train your model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "edH3xO_AVWXu"
      },
      "source": [
        "%cd /content/mycar\n",
        "\n",
        "model_name = \"model1\"\n",
        "\n",
        "!donkey train --tub ./data/tub* --model models/{model_name}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Quiz\n",
        "1. How many records are used for training?\n",
        "2. How many records are used for validation?\n",
        "3. Does the sum of this number adds up to the number of images in the dataset?\n",
        "4. What is the final lost value?\n",
        "5. How long does this training last?\n",
        "6. How many epoches in this training?\n",
        "7. How many steps in this epoch?\n",
        "8. What's the value of `MAX_EPOCHS` in `myconfig.py`?\n",
        "9. Why the training stop before it reaches the 100th epoch?"
      ],
      "metadata": {
        "id": "HeHjt60xKphz"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rXzn1noJz5MQ"
      },
      "source": [
        "Check if the model is generated\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5b9vJV4EzlO8"
      },
      "source": [
        "!ls -alh /content/mycar/models/{model_name}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AixQrFy_z3vv"
      },
      "source": [
        "%cd /content/mycar/models\n",
        "\n",
        "import glob\n",
        "file = glob.glob(f\"{model_name}.png\")\n",
        "\n",
        "from IPython.display import Image\n",
        "Image(file[0])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6BEOJYH601O0"
      },
      "source": [
        "## Copy the trained model back to Donkey Car (Pi)\n",
        "\n",
        "Once the training is complete on colab, download the model file under /content/mycar/models/ folder location."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YtvyJpOdocjb"
      },
      "source": [
        "from google.colab import files\n",
        "\n",
        "files.download(f\"{model_name}.tflite\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hfERkGy821Xy"
      },
      "source": [
        "## Start Autopilot on Pi\n",
        "\n",
        "\n",
        "```bash\n",
        "cd ~/mycar\n",
        "python manage.py drive --model models/mypilot.h5 --js\n",
        "```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Changing training parameters"
      ],
      "metadata": {
        "id": "7vBkLcqCKMHv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Change number of epoch"
      ],
      "metadata": {
        "id": "7itOFJ_8KP8N"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "1. Edit `myconfig.py`\n",
        "2. Change `MAX_EPOCHS` to 5, 10 , 20\n",
        "3. Training the model again by executing the training script above\n",
        "4. What is `MAX_EPOCHS` used for?\n",
        "5. Why do we want to change `MAX_EPOCHS`?"
      ],
      "metadata": {
        "id": "ZgnRGiMtLSHv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Change TRAIN_FILTER"
      ],
      "metadata": {
        "id": "jy5ax0YlKTyq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Change TRAIN_FILTER in `myconfig.py` to > 0.1 and check the number of images used for training and validation"
      ],
      "metadata": {
        "id": "CCnJoXqLKeDo"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Change Batch Size"
      ],
      "metadata": {
        "id": "J7Nn7rjkLVRb"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "1. Edit `myconfig.py`\n",
        "2. Change `BATCH_SIZE` to 256\n",
        "3. Training the model again by executing the training script above\n",
        "4. Does the training go faster? Why?\n"
      ],
      "metadata": {
        "id": "Zv_4oluqLXa4"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X93SodzAv9hV"
      },
      "source": [
        "## Bonus - Salient Object Visualization"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install imageio-ffmpeg\n"
      ],
      "metadata": {
        "id": "1nGatrwZiTEt"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kKI37gVrv9Q8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "72188c04-bdde-4428-f546-31a109d9fb14"
      },
      "source": [
        "%cd /content/mycar\n",
        "!donkey makemovie --tub data/{tub_name} --model models/{model_name} --type linear --salient  --start 1 --end 1000"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "t:  59% 3290/5592 [02:04<01:18, 29.38it/s, now=None]"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IcUrgOq_pePV"
      },
      "source": [
        "from google.colab import files\n",
        "\n",
        "%cd /content/mycar\n",
        "!ls -ahl\n",
        "files.download('tub_movie.mp4')"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}