SystemOverview

OpenNERO is a federation of several mods - nearly independent collections of resources and scripts stored in individual directories. For example, there are the Maze mod, the NERO mod and the Roomba mod - see Running OpenNERO for more details.

Since OpenNERO is all about designing intelligent agents, it has an API for adding new agents and new environments to test them in. An agent can interact with an environment by receiving observations from it, sending it actions, and receiving rewards for those actions. Agents in OpenNERO extend the class AgentBrain. Environments extend the Environment class. Both agents and environments can be written in Python or in C++.

Agent-Environment interaction diagram

Agent-Environment interface classes

The main interfaces (which are defined in C++ and exported into Python) look like this:

Environment

• sense(agent): Observations // produce the observations that the agent 'sees'
• step(agent, action): Reward // perform the action requested by the agent and return the reward
• get_agent_info(agent): AgentInitInfo // return the initialization information describing the state and action spaces for the agent
• is_episode_over(agent): boolean // return true iff the episode is over for the agent
• cleanup(agent) // clean up the environment before exiting
• reset(agent) // return the environment to its initial state

AgentBrain

• initialize(agent_init_info) // initialize the agent with the AgentInitInfo from the environment
• start(time, observations): Actions // determine the first action the agent should take given its initial observations
• act(time, observations, reward): Actions // determine the consequent actions the agent should take given the observations
• end(time, observations, reward) // get information about the final observation and reward from the environment
• destroy() // destroy the agent and release its resources

AgentInitInfo, Actions and Observations

The basic types OpenNERO uses to pass around information between Environment and Agent are defined in AI.h. Both Actions and Observations are actually a FeatureVector - an array of one or more double values. Conceptually, a FeatureVector carries discrete or continuous values within a certain range. In order to describe the possible values for a particular FeatureVector, a FeatureVectorInfo is used:

AgentInitInfo

• sensors: FeatureVectorInfo // description of the state (observation) space
• actions: FeatureVectorInfo // description of the action space
• rewards: FeatureVectorInfo // description of the rewards

FeatureVectorInfo

• size(): integer // number of elements in the vector
• addDiscrete(min, max): integer // add a discrete feature
• addContinuous(min, max): integer // add a continuous feature
• getBounds(i): Bounds // get the bounds of the ith feature
• getMin(i): double // min of ith element
• getMax(i): double // max of ith element
• isDiscrete(i): boolean // return true iff the ith feature is discrete
• isContinuous(i): boolean // return true iff the ith feature is continous
• validate(feature_vector): boolean // check that feature_vector is in compliance with these bounds
• getInstance(): FeatureVector // get a zero instance of this feature vector
• getRandom(): FeatureVector // get a random instance of this feature vector

Components

OpenNERO is built using open source components, including:

• Irrlicht 3D Engine - released under the Irrlicht Engine License
• Boost C++ libraries - governed by the Boost Software License
• Python scripting language - governed by the Python License
• rtNEAT algorithm - created by Ken Stanley and Risto Miikkulainen at UT Austin.