SplatSim: Photo-Realistic Simulation with 3D Gaussian Splatting (OSS, coming soon)

[hakuturu583]

Overview

We are currently preparing to open-source SplatSim, a photo-realistic simulation tool based on 3D Gaussian Splatting (3DGS).

3D Gaussian Splatting enables real-time, high-fidelity rendering of real-world environments — making it possible to generate camera sensor data that closely resembles actual driving scenes, without relying on artist-created 3D assets.

3DGS.mp4

Key Features

Standards-Based 3D Gaussian Splatting Data Pipeline

We adopt Cesium (a widely-used GIS platform) and the Khronos Group glTF Gaussian Splatting extension as the data format for 3DGS assets. By using these industry-standard specifications, SplatSim can seamlessly integrate with other geospatial tools and visualization platforms.

CARLA Co-Simulation

SplatSim supports co-simulation with CARLA. Traffic participants (vehicles, pedestrians, etc.) are simulated in CARLA, while the camera rendering is performed by SplatSim using 3DGS. This combines CARLA's mature traffic/scenario simulation capabilities with SplatSim's photo-realistic rendering.

DDS / ROS 2 Integration

SplatSim publishes rendered images and camera info via CycloneDDS, making it directly compatible with ROS 2 and Autoware without additional bridges.

Autoware Integration (In Progress)

We are currently working on integrating SplatSim into the Autoware ecosystem. There are some open design questions — for example, whether to adopt colcon-uv for Python package management — so we are still designing the best approach for contributing this to Autoware.

For context on the ongoing discussion, see: https://github.com/orgs/autowarefoundation/discussions/7117#discussioncomment-16998457

We welcome any feedback or suggestions from the community on how to best integrate this kind of Python/GPU-heavy simulation tool into the Autoware build ecosystem.

Links

• Repository (now under tier4 org, but it will be moved to AWF soon): tier4/splatsim

[Kaweees]

Interesting, is physics decoupled from SplatSim in such a way that you can bring your own physics engine?

[hakuturu583]

Interesting, is physics decoupled from SplatSim in such a way that you can bring your own physics engine?

Yes, splatsim is implemented upon gsplat.
It is just a rendering engine, so you can integrate own physics engine.
By default, we support carla integration.

[mitsudome-r]

There are some open design questions — for example, whether to adopt colcon-uv for Python package management — so we are still designing the best approach for contributing this to Autoware.

Does the current SplatSim have any conflicts with default packages installed by Autoware? I just want to know the discussion with colcon-uv will be a blocker for integrating this into Autoware.

[hakuturu583]

Does the current SplatSim have any conflicts with default packages installed by Autoware? I just want to know the discussion with colcon-uv will be a blocker for integrating this into Autoware.

There may be conflicts, but since colcon-uv resolves dependencies within the virtual environment, it won’t break your local environment.