DualMap

Project Page | Paper |

DualMap is an online open-vocabulary mapping system that enables robots to understand and navigate dynamic 3D environments using natural language.

The system supports multiple input sources, including offline datasets (Dataset Mode), ROS streams & rosbag files (ROS Mode), and iPhone video streams(Record3d Mode). We provide examples for each input type.

Updates

[2025.06] We released the Offline Query Examples — check it out here

Release Plan

• Environment setup & dataset links
• Full system code (Mapping + Navigation)
• Evaluation Code
• [Apps] Running with Datasets
• [Apps] Running with ROS
• [Apps] Running with iPhone
• [Apps] Offline Query
• Realworld Deployment Guidance

Installation

✅ Tested on Ubuntu 22.04 with ROS 2 Humble and Python 3.10

1. Clone the Repository (with submodules)

git clone --recurse-submodules git@github.com:Eku127/DualMap.git
cd DualMap

Make sure to use --recurse-submodules to get mobileclip.

2. Create Conda Environment

conda env create -f environment.yml
conda activate dualmap

3. Install MobileCLIP

cd 3rdparty/mobileclip
pip install -e . --no-deps
cd ../..

(Optional) Setup ROS 2 Environment

Setting up ROS2 environment for ROS support and applications. We recommend ROS 2 Humble. Once installed, activate the environment:

source /opt/ros/humble/setup.bash

DualMap’s navigation functionality and real-world integration are based on ROS 2. Installation is strongly recommended.

ROS1 noetic is also supported, you can setup the ROS 1 by follow this guide.

(Optional) Setup Habitat Data Collector

Habitat Data Collector is a tool built on top of the Habitat-sim. It supports agent control, object manipulation, dataset and ROS bag recording, as well as navigation through external ROS topics. DualMap subscribes to live ROS topics from the collector for real-time mapping and language-guided querying, and publishes navigation trajectories for the agent to follow.

For the best DualMap experience (especially navigation tests), we strongly recommend setting up the Habitat Data Collector. See the repo for installation and usage details.

Dataset

Replica & ScanNet

Please follow this guide to download and prepare the public Replica and ScanNet datasets for use with DualMap.

HM3D Dynamic Scene Data

We release both static and dynamic HM3D scene data to support faithful reproduction of the results presented in the "Navigation in Simulation" section. Researchers can use this dataset to replicate both the static and dynamic environments evaluated in our experiments.

The dataset includes three HM3D scenes tested in the paper.

You can download the data here: 🔗 OneDrive

Per-Scene Directory Structure

Each scene folder follows the structure below:

00829-QaLdnwvtxbs/                    # Example scene ID
├── data.zip                          # RGB-D + pose + intrinsic offline dataset (5.08 GB)
├── static_scene_config.json          # Static scene configuration
├── dynamic_scene_config/             # Dynamic scene configurations
│   ├── cross_anchor/                       
│   │   ├── 0128-1.json
│   │   └── ...
│   └── in_anchor/                          
│       ├── 0128-1.json
│       └── ...
├── global_map/                       # Prebuilt abstract map from rosbag and dataset
├── rosbag2_odom/                     # ROS2 bag of static scene traversal
├── class_bbox.json                   # Object bounding boxes (for evaluation)
├── class_num.json                    # Object class count summary (for evaluation)

See the Applications section for detailed dataset usage. The usage of scene configuration is documented in Habitat Data Collector documentation.

Applications

Run with Datasets [TBD]

Coming soon: Reproduce offline mapping results with Replica, ScanNet and self collected datasets.

Run with ROS [TBD]

Coming soon: Online mapping and navigation via live ROS topics from the Habitat Data Collector and real world robot. Both ROS1 and ROS2 are supported

ROS1 Support

This system can also work with ROS1 Noetic by using the ros1_bridge.

To set up the bridge between ROS1 Noetic and ROS2 Humble on Ubuntu 22.04, follow the instructions in this guide. All commands have been tested and confirmed to work.

Run with iPhone [TBD]

Coming soon: Real-time data streaming from Record3D to DualMap with open vocabulary semantic mapping.

Offline Map Query

We provide two prebuilt map examples for offline querying: one from iPhone data and one from Replica Room 0.

👉 Follow this guide to run the query application.

Visualization

Coming soon: The system supports both Rerun and Rviz visualization

Evaluation [TBD]

Evaluation code and result reproduction tools will be released in future updates. We ensure that you can reproduce all the results in our paper.

Citation

If you find our work helpful, please consider starring this repo 🌟 and cite:

@article{jiang2025dualmap,
  title={DualMap: Online Open-Vocabulary Semantic Mapping for Natural Language Navigation in Dynamic Changing Scenes},
  author={Jiang, Jiajun and Zhu, Yiming and Wu, Zirui and Song, Jie},
  journal={arXiv preprint arXiv:2506.01950},
  year={2025}
}

Acknowledgment

We are grateful to the authors of HOVSG and ConceptGraphs for their contributions and inspiration.

Special thanks to @TOM-Huang for his valuable advice and support throughout the development of this project.

We also thank the developers of MobileCLIP, CLIP, Segment Anything (SAM), MobileSAM, FastSAM, and YOLO-World for their excellent open-source work, which provided strong technical foundations for this project.