BrickSim

A Physics-Based Simulator for Manipulating Interlocking Brick Assemblies

Paper • Quickstart • Build & Test • Demos • Video: coming soon

Quickstart

git clone --recursive https://github.com/intelligent-control-lab/BrickSim BrickSim
cd BrickSim

sudo apt install build-essential wget python3.11-full xz-utils zstd

python3.11 -m venv --symlinks --prompt bricksim --upgrade-deps .venv
source .venv/bin/activate

pip install "isaacsim[all,extscache]==5.1.0" --extra-index-url https://pypi.nvidia.com
CMAKE_POLICY_VERSION_MINIMUM=3.5 ./IsaacLab/isaaclab.sh --install

scripts/build.sh
pip install -e exts/bricksim -v

./scripts/launch_isaacsim.sh demos/demo_assembly.py

Prerequisites

• BrickSim currently supports the x86-64 Linux platform. Support for other platforms is coming.
• Ubuntu 22.04+ or another Linux distribution with GLIBC >= 2.35, GLIBCXX >= 3.4.30, and CXXABI >= 1.3.13
• Python 3.11
• A working NVIDIA driver compatible with Isaac Sim RTX requirements

Installation

1. Clone the repository

git clone --recursive https://github.com/intelligent-control-lab/BrickSim BrickSim
cd BrickSim

2. Create the virtual environment

python3.11 -m venv --symlinks --prompt bricksim --upgrade-deps .venv
source .venv/bin/activate

3. Install system packages

sudo apt install build-essential wget python3.11-full xz-utils zstd

4. Install Isaac Sim

Only Isaac Sim 5.1 is currently supported.

pip install "isaacsim[all,extscache]==5.1.0" --extra-index-url https://pypi.nvidia.com

5. Install Isaac Lab

CMAKE_POLICY_VERSION_MINIMUM=3.5 ./IsaacLab/isaaclab.sh --install

6. Build and install BrickSim

scripts/build.sh
pip install -e exts/bricksim -v

Build & Test

Build the native extension:

scripts/build.sh

Build and run the C++ sanity checks:

RUN_TESTS=1 scripts/build.sh

The build copies the compiled _native module into exts/bricksim/bricksim/.

Run Demos

Launch the assembly demo:

./scripts/launch_isaacsim.sh demos/demo_assembly.py

Other useful entry points:

• demos/demo_inhand.py for in-hand manipulation experiments
• demos/demo_keyboard_teleop.py for keyboard-driven interaction
• demos/demo_teleop.py for teleoperation, recording, and replay workflows

The teleoperation demo expects the lerobot package plus a configured leader device path inside demos/demo_teleop.py.

Repository Layout

Path	Purpose
native/	C++26 core
exts/bricksim/	Python extension and API
demos/	End-to-end examples
resources/	USD assets, robot assets, and brick datasets
scripts/	Utility scripts

Development Setup

Generate dependency-aware Pyright configuration for VS Code completion:

source .venv/bin/activate
python scripts/generate_pyrightconfig.py

This creates pyrightconfig.deps.json.