Collaborating with Fraunhofer-IKS, this project is dedicated to investigating Emergent Dysfunctions (EDYs) within multi-agent environments. In multi-agent reinforcement learning (MARL), a population of agents learns by interacting with each other in a shared environment and adapt their behavior based on the feedback they receive from the environment and the actions of other agents.
In this context, emergent behavior describes spontaneous behaviors resulting from interactions among agents and environmental stimuli, rather than explicit programming. This promotes natural, adaptable behavior, increases system unpredictability for dynamic learning , enables diverse strategies, and encourages collective intelligence for complex problem-solving. However, the complex dynamics of the environment also give rise to emerging dysfunctions—unexpected issues from agent interactions. This research aims to enhance our understanding of EDYs and their impact on multi-agent systems.
-
Create an environment that provokes emerging dysfunctions.
- This is achieved by creating a high level of background noise in the domain, where various entities perform diverse tasks, resulting in a deliberately chaotic dynamic.
- The goal is to observe and analyze naturally occurring emergent dysfunctions within the complexity generated in this dynamic environment.
-
Observational Framework:
- The project introduces an environment that is designed to capture dysfunctions as they naturally occur.
- The environment allows for continuous monitoring of agent behaviors, actions, and interactions.
- Tracking emergent dysfunctions in real-time provides valuable data for analysis and understanding.
-
Compatibility
- The Framework allows learning entities from different manufacturers and projects with varying representations of actions and observations to interact seamlessly within the environment.
-
Placeholders
- One can provide an agent with a placeholder observation that contains no information and offers no meaningful insights.
- Later, when the environment expands and introduces additional entities available for observation, these new observations can be provided to the agent.
- This allows for processes such as retraining on an already initialized policy and fine-tuning to enhance the agent's performance based on the enriched information.
Install this environment using pip install marl-factory-grid. For more information refer
to 'installation'.
Refer to quickstart for specific scenarios.
The majority of environment objects, including entities, rules, and assets, can be loaded automatically. Simply specify the requirements of your environment in a yaml-config file.
If you only plan on using the environment without making any modifications, use quickstart_use.
This creates a default config-file and another one that lists all possible options of the environment.
Also, it generates an initial script where an agent is executed in the specified environment.
For further details on utilizing the environment, refer to 'usage'.
Existing modules include a variety of functionalities within the environment:
- Agents implement either static strategies or learning algorithms based on the specific configuration.
- Their action set includes opening door entities, cleaning dirt, picking up items and delivering them to designated drop-off locations.
- Agents are equipped with a battery that gradually depletes over time if not charged at a chargepod.
- The maintainer aims to repair machines that lose health over time.
If you plan on modifying the environment by for example adding entities or rules, use quickstart_modify.
This creates a template module and a script that runs an agent, incorporating the generated module.
More information on how to modify the levels, entities, groups, rules and assets
goto modifications.
Varying levels are created by defining Walls, Floor or Doors in .txt-files (see levels for examples). Define which level to use in your configfile as:
General:
level_name: rooms # 'double', 'large', 'simple', ...... or create your own , maybe with the help of asciiflow.com.
Make sure to use # as Walls, - as free (walkable) floor, D
for Doors.
Other Entites (define you own) may bring their own Symbols
Entites are Objects that can additionally be assigned a position. Abstract Entities are provided.
Groups are entity Sets that provide administrative access to all group members. All Entites are available at runtime as EnvState property.
Rules define how the environment behaves on microscale.
Each of the hookes (on_init, pre_step, on_step, 'post_step', on_done)
provide env-access to implement custom logic, calculate rewards, or gather information.
Results provide a way to return rule evaluations such as rewards and
state reports back to the environment.
Make sure to bring your own assets for each Entity living in the Gridworld as the Renderer relies on it.
PNG-files (transparent background) of square aspect-ratio should do the job, in general.
