Coordinating Search with Foundation Models and Multi-Agent Reinforcement Learning in Complex Environments

Direct Lab 
Department of Computer Science, Utah State University, Nvidia, Autonomous Systems Branch, Army Research Lab (ARL), Department of Computer Science, University of Washington

This repository contains the codebase for our research on coordinating search operations using multi-agent reinforcement learning (MARL) and foundation models in complex, unstructured environments.

Overview

Our system simulates a fleet of heterogeneous robots designed for search operations in challenging outdoor scenarios. The approach combines reinforcement learning (RL) and a centralized Multi-Agent Transformer (MAT) to coordinate autonomous agents, integrating sensory data into a structured knowledge graph for efficient decision-making.

Key Features

• Multi-Agent Reinforcement Learning: Autonomous coordination among diverse robots using MARL.
• Multi-Agent Transformer (MAT): Centralized decision-making model for high-level command interpretation.
• Simulation Environment: Developed with NVIDIA Isaac Sim and GRUtopia for high-fidelity and scalable robot simulations.

Installation

CODE

Usage

COMING SOON!

Complex Environments

We evaluate our approach in complex, unstructured environments to test the system's robustness and scalability. The environments include:

• Rock fields (Nvidia Blast)

  A giude on how to create and use a destructible Environments with Blast is available here

  

• Forest (Nvidia optimized vegetation assets)

  You can download a forest environment here

  

• Night and Day Lighting (Nvidia Sun Study)

These environments are open-source and available for download SOON

Downloading the Environments

Contributing

Contributions are welcome! Please read our CONTRIBUTING.md for details on the code of conduct and submission process.

License

The project License is found in the LICENSE

Acknowledgments

This work was supported by the U.S. Army Research Fellowship Program and the Department of Computer Science, Utah State University.

• MAT: We use MAT for centralized decision-making in the multi-agent system.
• GRUtopia: We use GRUtopia for simulation of the robots.

  GRUtopia Acknowledgments

  • OmniGibson: GRUtopia refers to OmniGibson for designs of oracle actions.
  • RSL_RL: GRUtopia utilized rsl_rl library to train the control policies for legged robots.
  • ReferIt3D: GRUtopia refers to the Sr3D's approach to extract spatial relationship.
  • Isaac Lab: GRUtopia utilized some utilities from Orbit (Isaac Lab) for driving articulated joints in Isaac Sim.

Citation

If you find this work helpful, please cite it as follows:

@inproceedings{allred2024coordinating,
  title={Coordinating Search with Foundation Models and Multi-Agent Reinforcement Learning in Complex Environments},
  author={Allred, Christopher and Haight, Jacob and Justice, Chandler and Peterson, Isaac and Scalise, Rosario and Hromadka, Ted and Pusey, Jason and Harper, Mario},
    year={2024},
}