{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Race track simulation\n",
    "\n",
    "by Uki D. Lucas \n",
    "\n",
    "February 13, 2017\n",
    "\n",
    "Udacity Car Nanodergree \n",
    "\n",
    "Project 3: Behavioral Clonning\n",
    "\n",
    "## Overview\n",
    "\n",
    "In this project I will be **training and then driving a race car game/simulator** thru couple of different terrains. No other cars are present and no street signs have to be obeyed, in fact in the simplest form of the exercise the speed is constant."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notes before start of the project\n",
    "\n",
    "## Observations\n",
    "\n",
    "- Large amount of training data, large files: \n",
    "    - possibly 100,000+ images \n",
    "    - 320x160 color images \n",
    "    - 1.5GB total\n",
    "- Various types of the road:\n",
    "    - yellow solid line with white-red stripes on the corners\n",
    "    - yellow solid line with tall metal guard\n",
    "    - big rock surface on bridge with black walls\n",
    "    - brown sand embankment\n",
    "- Camera angles are such that the sides of the road go off to the sides\n",
    "- Top of the images are just distractions, sky, trees, hills\n",
    "- There is a lot of noice (pixelation on the road)\n",
    "- I have limited computer processing resources:\n",
    "    - expensive AWS GPU instances\n",
    "    - MacBook Pro with (at this time) only CPU support\n",
    "    - Ubuntu with GTX 1050Ti GPU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusions and assumptions\n",
    "\n",
    "- Bottom part of the image (including car hood) can be cropped to a point where lines touch the margin\n",
    "- Top of the image can be cropped at about 80% (at sky horizon)\n",
    "- The neural network does not need all the details in the picture:\n",
    "    - this problem is more similar to hard written character recognition, then to color street sign recognition, in any case I beleive I can scale down inages to 32x32, or even **28x28** and drastically decrese processing time (including image transfer time).\n",
    "- Because of **ALL POSSIBLE ROAD TYPES** and because in this part of the code I just predict the steering angles (not speeed) I believe:\n",
    "    - colors are not important, but rather the **SHAPE** of the side of the road\n",
    "    - the **side-to-side spacing** of the car on the road is important, but as true race car I would like to **favor inside turns**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "AlexNet \n",
    "\n",
    "data    (3, 227, 227)\n",
    "conv1   (96, 55, 55) \n",
    "norm1   (96, 55, 55) \n",
    "pool1   (96, 27, 27) \n",
    "conv2   (256, 27, 27) \n",
    "norm2   (256, 27, 27) \n",
    "pool2   (256, 13, 13) \n",
    "conv3   (384, 13, 13) \n",
    "conv4   (384, 13, 13) \n",
    "conv5   (256, 13, 13) \n",
    "pool5   (256, 6, 6)\n",
    "fc6     (4096,)\n",
    "fc7     (4096,)\n",
    "fc8     (8,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Udacity suggestions:\n",
    "\n",
    "## Framework to use\n",
    "\n",
    "Udacity wants me to use Keras with TensorFlow.\n",
    "    \n",
    "## Driving off the track\n",
    "\n",
    "- If your model has low mean squared error on the training and validation sets but is driving off the track, this could be because of the data collection process. \n",
    "\n",
    "- It's important to feed the network examples of good driving behavior so that the vehicle stays in the center and recovers when getting too close to the sides of the road.\n",
    "\n",
    "- If your training data is all focused on driving down the middle of the road, your model won’t ever learn what to do if it gets off to the side of the road.\n",
    "\n",
    "## Recovery to center of the line\n",
    "\n",
    "- One approach might be to constantly wander off to the side of the road and then steer back to the middle. That’s not great, though, because now your model is just learning to wander all over the road. \n",
    "- A better approach is to only record data when the car is driving from the side of the road back toward the center line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Implementation\n",
    "\n",
    "## Preprocessing\n",
    "\n",
    "Please review \"**preporcessing.py**\" in this directory."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "\n",
    "Below are known online references, however the concept and code implementation are **solely by Uki D. Lucas**.\n",
    "\n",
    "- https://classroom.udacity.com/nanodegrees/nd013/parts/fbf77062-5703-404e-b60c-95b78b2f3f9e/modules/6df7ae49-c61c-4bb2-a23e-6527e69209ec/lessons/46a70500-493e-4057-a78e-b3075933709d/concepts/d76f87f5-f9d9-4d63-968a-99606fe90749\n",
    "\n",
    "- http://images.nvidia.com/content/tegra/automotive/images/2016/solutions/pdf/end-to-end-dl-using-px.pdf\n",
    "\n",
    "- https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model\n",
    "\n",
    "- https://github.com/ancabilloni/SDC-P3-BehavioralCloning\n",
    "- https://medium.com/@fromtheast/implement-fit-generator-in-keras-61aa2786ce98#.jbdqrylpu\n",
    "- https://medium.com/@fromtheast/you-dont-need-lots-of-data-udacity-behavioral-cloning-6d2d87316c52#.szgsmco00\n",
    "- https://github.com/DavidAwad/behavioral_cloning\n",
    "- https://medium.com/@MSqalli/teach-a-car-how-to-drive-b54e921f64d2#.n54nbz3lp\n",
    "- https://medium.com/@tjosh.owoyemi/#.9y96vw8w5\n",
    "- https://chatbotslife.com/learning-human-driving-behavior-using-nvidias-neural-network-model-and-image-augmentation-80399360efee#.ho8262tv4\n",
    "- https://github.com/agileminor/carnd-cloning\n",
    "- https://github.com/toluwajosh/CarND-Behavioural-Cloning\n",
    "- tensorflow/tensorflow#7025 (comment)",
    "- aymericdamien/TensorFlow-Examples#38",
    "- http://davidaventimiglia.com/carnd_behavioral_cloning_part1.html\n",
    "- https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model\n",
    "- https://classroom.udacity.com/nanodegrees/nd013/parts/fbf77062-5703-404e-b60c-95b78b2f3f9e/modules/83ec35ee-1e02-48a5-bdb7-d244bd47c2dc/lessons/8c82408b-a217-4d09-b81d-1bda4c6380ef/concepts/4f1870e0-3849-43e4-b670-12e6f2d4b7a7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda env tensorflow_cpu",
   "language": "python",
   "name": "tensorflow_cpu"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}