Project was inspired by "Deeptesla" (https://github.com/lexfridman/deeptesla.git)
Use TensorFlow to Inference the steering wheel angle only based on the single front image
- Nvidia Jetson TX2 (256 Pascal CUDA Cores, 8GB shared RAM)
- Texas Instrument SN65HVD230 (Can bus transceiver)
- Toyota 2016 Camry LE
- Wires and Electricity (12V DC)
- Logitech Webcam (Cheapest possible from Micro Center)
- Tensorfow v1.0.1
- Python3
- OpenCV 3.2
- Numpy
I drove my car by myself and at the same time collecting the data. It is possible to connect the keyboard to the Jetson TX2 and run the Python code, but I found that it is more convenient to use the serial port in the GPIO port and connect to my MacBook and run the script.
Network: Modified AlexNet to fit my model
Loss: Mean Squared Error
I resized the frame into size of [66,256,1] which I personally thought is good enough for the CNN.
I could find that the Toyota uses the "ID : 0x025" as the steering wheel angle. I found that they are using the signed data, fixed point decimal. For more detail please check the code.
Steering wheel usually don't change a lot in the real world driving environment. Therefore, the data was extremely imbalanced. I spend lots of time balancing the data.
CNN requires big amount of memory, if the dataset don't fit into the memory, then cut it down to smaller batch.
Before I start this project I thought the network will make huge difference in linear regression problem, because it does make huge difference in categorical problem. However, I found that the linear regression problem doesn't hugely affected by the network
It was challenging to make the data into the dataset and balance the data prevent from biased weight.
