Credit for the file organization and the real time JavaScript engine goes to Frank Poth https://youtu.be/w-OKdSHRlfA. For other credits see credits.md
Since last summer I've been learning about AI and testing my understanding by coding little projects with neural networks from scratch. I tried to make the projects as visual and interactive as possible so others can engage with the material like I did.
Play with the projects for yourself here
In the first project I trained a neural network to recognize handwritten digits from the famous MNIST dataset. The most exciting bit is that you can draw your own digits and see how the network activations change. In other words, you can see how the network "thinks" in real time.
Network activations changing whilst drawing a digit.
I also added the ability to drag your digits around to see how that changes the network's prediction. This has a significant effect on whether the digit is correctly identified.
Network activations changing whilst moving a digit.
In the second project I tackled the well known environment where a cart has to balance a pole by moving left or right.
The white cart trying to balance the blue pole.
I chose to use a Double DQN, reusing the previous neural network as policy and target network. With the right parameters it can solve the environment in around 150 episodes (meaning: it achieves a rolling average of 195.0 over a hundred episodes), which translates to a few minutes depending on your hardware. Larger networks like the one you see below were prone to exploding gradients so I chose a network with one hidden layer of 16 neurons.
Exploding gradients visualized in the cartpole project.
For the third project I wanted to stick with reinforcement learning and was inspired by CodeBullet. I wanted to test my DDQN on something more complicated than cartpole and saw the racing idea as a fun way to make it interactive.
I added the feature of drawing your own tracks and letting the racer train on them. Alternatively you can click on the "Info" button and select "Load Best Parameters" to see the trained model try your track. Do note that this last option turns off learning so you'll have to go back to the default settings to let it learn from there.
Drawing a new racetrack and testing the trained network on it.