this project began when I saw an article showing Unity's machine learning toolkit and thought to myself, "Wow, that looks really cool! I wanna try it out for myself!" That was really my whole thought process. This is the article for anyone who's interested: https://blogs.unity3d.com/2017/12/11/using-machine-learning-agents-in-a-real-game-a-beginners-guide/.
And thus began my journey into creating the Aquarium. What started off as one A.I. searching for a little green pellet soon became a version of Agar.IO where the player is pit against several highly intelligence (a bit of an alliteration) fighting to see who can become the largest. How on earth did it come to that? Well keep reading and you'll find out. (Go to the "Fish v3" section to read about the final product)
I began by creating a very basic reinforcement learning brain that is rewarded whenever it finds a food. This in itself was a long learning process as I prefer jumping straight into a project before I learn what each of the buttons and switches do. But after several YouTube series and skimming through the documentation, I created my first Fluid Impulsive Simulated Host, or F.I.S.H. for short. FISH v1 was simple minded. It spawned in a random location in a small "tank" (bounded area) and was given one values: the distance between it and the closest piece of food. These pieces of food were scattered around the tank in random locations as well. When it touched the food, it was rewarded. As time went on, the fish was punished for taking longer to locate the food. The fish then took the input, and returned two values: the movement in the x-axis and the movement in the y-axis. Ideally FISH should have quickly swam from food to food, until there were no more. But of course, what code works on the first try? It swam randomly, most of the time moving in one direction. I then did what any good programmer does, and sleep until I have a revelation while trying to fall asleep. And right on que, the revelation came. I realized two solutions. One, give FISH a more accurate description of the location of the food. That is, give it two values: the coordinate difference between its location and the food's location. Two, give it a small reward whenever it moves closer to the food. Kind of like a kick-start to the learning process. These changes produced a speedy FISH who quickly zoomed from food to food. Nice.
"What's next", I asked myself. While FISH v1 was great for being a self-learned object, it's functionality could have been way more efficiently solved with a couple of for loops and if-statements. So what better way of testing their brains by pitting them against each other? I quickly added more FISH in each tank and added a function where if they collided, the larger one would be rewarded and grow even larger, while the smaller one would become small again, move to a random location, and be punished (if you've played Agar.io, it's basically the same concept). To detect the other FISH, another 3 input were added giving the x and y difference in position between the FISH and the nearest FISH in the tank, and the difference in size between these two FISH. Now, if you were paying close attention, you may have realized I didn't say the nearest FISH other than itself in the tank. Rookie mistake. That issue caused me way more precious time than it should have. Once I patched that issue out, the fish were nearing completion. However, they still ignored the other FISH in the tank for the most part and occasionally ran straight into the mouth of the other larger fish. But in terms of frustration, this was probably the easiest fix as all that was required was to tinker with the rewards and punishments. Now we arrive at FISH v3.
These fish are the smartest of all all previous iterations combined. They actively go for the food, and attempt to eat the other FISH if they are larger. They still sometimes are eaten by others when it could be easily avoided. I also made small changes such as changing the movement from a change in velocity to a change in acceleration. This way the movement is more fluid and less jittery. Finally, I created a larger Tank which I dubbed the Pool, and an even larger version of that named the Pond (maybe an ocean is next?). Now here's a closer look under the hood.