Test charts, analysis & conclusion
Now that I ran all the tests (more than I needed to), it's time to make proper charts for them and analyze them. Just to clarify and as a reminder to myself: only comparisons for the same type of reinforcement learning will be done. For example:
I will only compare data that was trained on the same type of 'reinforcement learning':
- negative reinforcement
- positive reinforcement
- mixed reinforcement
Setup explanation:
-
in a 28x28 unit environment, collectibles (coins) are spawned at the beginning of an episode. Every time a new episode begins, the position of all coins is randomized. This ensures the agent learns to adapt to a dynamic environment, rather than learning one path and keep perfecting its efficiency.
-
the coins are detected using the
RayPerceptionSensor3D, with a max vision distance of 15. -
rewards are set to 0.1, 0.5 and 0.9. I used 3 different values to be able to determine if there is a linear relation between a single coin reward vs cumulative reward.
-
Amount of episodes: 800
-
Amount of collectibles at any given time: 12
Before performing the tests, I expected the agent wouldn't care about how high the reward would be since the collectibles were the only object it could interact with and therefore, would quickly learn to start gathering them. However, analyzing these charts it seems that regardless of collectibles being the sole purpose in the environment, a higher reward results in quicker learning