Reinforcement learning

[RossMelbourne]

Hi Ryan,
Great project and thank you for sharing it with us. I am trying to learn reinforcement learning and I wondered if you considered using RL instead of supervised learning? Since you are doing inference (predicting the cars next steering action) wirelessly from a laptop I assume you could also train a neural net using Deep Q Learning the same way. Just curious about what you think of that approach and would it work. Also, do you think a Jetson board would be able to do inference on the robot car itself assuming a bigger car and a bigger budget for the Jetson board vs. Raspberry Pi.
Thank again,
Ross

[RyanZotti]

@RossMelbourne thanks for the note. I've considered RL, but decided against it because I figured that the car would crash too much in the beginning. RL is primarily about learning by intelligent trial and error and I figured there would be a lot of bad crashes if I started RL with weights that were randomly initialized. It might make sense to try RL after pre-training in a supervised fashion, similar to how Google built Alpha Go (first trained through supervision on expert moves and then through RL).

I haven't used a Jetson board, but after doing a quick Google search it looks like it should be fine. The Raspberry Pi is super slow during inference, that's why I had to stream from the laptop. The Jetson board would no doubt perform better than the Pi.

[fakhirali]

Doing just RL would not be practical as you would have to reset the position of the car each time and it would take hours to train. I was thinking if there is a way to make a simulated environment in Unity3D and then train it in the environment and then test it in the real world. Though the environment would have to behave very similar to the real world. What do you think?