Using a layered network of predictions, based on raw sensorimotor data, to represent knowledge. These ``predictions'' are represented using general value functions - a function common in reinforcement learning, which allows the agent to ask a question about its raw sensorimotor data, and then based on experience, begin to learn estimates for these questions. One RL researcher conducted a thought experiment using a robot which can sense a wall, and move forward or turn, to begin to ask questions about its environment using the simple sensorimotor data. The questions respective answers begin to become more and more abstract and sophisticated as the GVSs are layered on top of each other.
This repository extends the thought experiment by implementing the first few layers of it, using the 3D virtual environment of Minecraft. This implementation demonstrates how an artificial intelligence agent can build useful abstract knowledge from its raw sensorimotor experience alone using a network of General Value functions. This paper shares the design of such a learning agent and preliminary results.
A significant challenge in Artificial Intelligence is connecting a representation with what it represents. No representation has truly emerged capable of capturing the richness of a child`s understanding. As human agents interacting with the world, we draw upon our experience with the world to form our knowledge. We know that bananas are yellow, pillows are soft, and that glass will shatter. This knowledge is learned through our interactions, rather than being explicitly told to us by a teacher. These bits of knowledge are represented in an accessible way and can be used for more abstract and deeper reasoning.
AI researchers tend to consider
knowledge'' as symbols bound together by a network of relationships, and reasoning'' is an application which leverages these symbols to make decisions. Yet, these symbols tend to not be rooted in the most universal and basic of human knowledge: our sensorimotor interaction with the world.
General Value Function (GVFs) may be the right representation for a knowledge. These GVFs are rooted in the most primitive sensorimotor observations from the environment, and can be layered to form increasingly more abstract knowledge. Such a view of learning is often referred to as constructivist. In this constructivist life long learning agent architecture, that the GVF is the single mechanism to bridge the gap from low level sensorimotor interaction to abstract knowledge.