v0.0.15

The Agentive Operating System for Physical Space

https://discord.gg/dimos

Highlights

This is the Local Planning, Live Loop Closure, Module Configuration, and WebRTC Video release.

231 commits, 18 contributors, 1,529 files changed since v0.0.14.

The Go2 closes loops live, and unitree-go2-map keeps a permanent map across runs. A new SE(2) local planner and trajectory follower, both Rust native modules tuned per robot by an Embodiment, run under the MLS global planner on the real Go2 over DDS (go2-nav), on Alfred (alfred-nav), and in MuJoCo (go2-sim-nav). Module settings leave GlobalConfig for --module.field flags, MODULE__FIELD variables and per-module config sections. Startup is faster, shutdown no longer hangs, and the shared-memory transports are gone. The web stack sends video as H.264 WebRTC tracks through a Cloudflare SFU, authenticates robots and viewers, ships a relay Docker image, and adds a Map3D cockpit panel with click-to-go. dimos evals gains baselines without dimOS, Habitat and MuJoCo environments, and --docker for parallel runs. Experimental distributed runs place modules on other machines with dimos host, the dimOS Desktop gateway lands, and scripts/install.sh is the recommended installer.

⚠️ Breaking Changes

This release has many breaking changes. Read this section before upgrading.

Configuration

• 44 GlobalConfig fields are gone, with their flags, env vars and JSON g keys. Each setting belongs to the module that owns it: --<module>.<field> after the blueprint name, MODULE__FIELD in the environment, or a "<module>": {...} section in --config, and dimos run <blueprint> --help lists every field. dimos run refuses to start while an old variable such as UNITREE_AES_128_KEY, XARM7_IP or DIMOS_API_KEY is set, and names the replacement. Renames you will hit first: --mcp-port -> --mcpserver.port, --rerun-web -> --rerunbridgemodule.rerun-web true, --mujoco-* -> --go2connection.mujoco.*, and the arm address flags -> one controlcoordinator.addresses map. The full table is in docs/usage/configuration.md. (#4610) by @paul-nechifor

Navigation & mapping

• The go2web Zenoh bridge is gone: GO2Zenoh and the go2-zenoh-* blueprints are replaced by the native DDS stack (go2-dds or go2-nav on the robot's Jetson, go2-nav-viewer on the laptop), and vis_module() no longer bundles the port 7779 WebsocketVisModule unless passed websocket_vis=True. (#4449) by @leshy
• unitree-go2-relocalization is removed, and --map-file=<map> loads a <map>.map.db keyframe graph instead of a .pc2.lcm cloud: re-export recordings with dimos map global <memory.db> --export. PoseArray is now an LCM message, so its pickle encode() / decode() and wire format changed. (#4754, #4331) by @leshy
• Navigation import paths moved with no aliases: nav_3d.mls_planner -> global_planner.mls_planner, basic_path_follower -> trajectory_follower.basic, replanning_a_star -> go2.replanning_a_star, and dannav, frontier_exploration and patrolling -> experimental. (#4203) by @leshy
• Lidar addresses are module settings: the DIMOS_MID360_*, DIMOS_POINTLIO_* and DIMOS_FASTLIO_* address env vars are no longer read, use MID360__LIDAR_IP (--mid360.lidar-ip), FASTLIO2__LIDAR_IP or VIRTUALMID360__PCAP. Mid360Config.lidar_ip has no default any more, so a live Mid-360 blueprint fails at start until it is set. (#4203, #4416) by @leshy, @aclauer
• Point-LIO is the Rust port only: the C++ module and its embedded Livox SDK are deleted, PointLioConfig loses host_ip, lidar_ip, frequency and the SDK port fields, the imu input is imu_raw, and the driver runs as its own module (mid360_for_pointlio()). (#4311) by @aclauer
• mid360-fastlio-ray-trace is removed (use mid360-pointlio-ray-trace), and dimos nav-eval is deleted (evaluate in sim with dimos run go2-sim-nav). (#4411, #4438) by @aclauer

Robot support

• Galaxea R1 Pro: R1ProConnection no longer bridges the vendor chassis lidar (the cloud comes from our Mid-360 driver through Point-LIO), publish_odom is renamed publish_odom_tf and only gates the odom -> base_link tf edge, and the head colour streams moved to the new HeadLeftCamera / HeadRightCamera modules. (#4360, #4388) by @jeff-hykin
• The control extra no longer installs a750-control: run uv pip install a750-control yourself to drive a real A-750. (#4494) by @Dreamsorcerer
• WebXR arm teleop engages with the grip instead of the face buttons, and manual tasks now preempt trajectory tasks in the OpenYAM and OpenArm teleop blueprints. OpenYAM joint constants moved to dimos.robot.manipulators.dual_openyam.joints, the home pose is OPENYAM_HOME_JOINTS instead of all zeros, and the teleop IK gain went from 0.25 to 1.0. (#3315, #3853) by @TomCC7

Manipulation, control & agents

• SkillResult reports facts and leaves the verdict to the agent: success, error_code, is_success(), .ok(), .fail(), CommonSkillError and dimos.manipulation.skill_errors are gone, and a failing skill raises, which McpServer returns as an MCP tool result with isError: true. (#4404, #4401) by @mustafab0
• The planning extra requires roboplan>=0.7.0,<0.8.0 (run uv sync), and PointCloudSelfFilter is abstract: use RoboPlanPointCloudSelfFilter. The graspgenx extra is removed because GraspGenX provisions its own Python 3.12 environment on first run, and custom IsolatedPythonModule subclasses must declare a repo-relative project_dir. (#4420, #4102) by @TomCC7

Web & cockpit

• Relay wire protocol v6 -> v7: a relay and the robots and pages behind it must run the same version (a v0.0.14 relay answers version_mismatch), so update a hosted relay together with its robots. The relay also refuses --serve-dir on a non-loopback --host, so a public relay serves only the built cockpit. (#4264, #3953) by @paul-nechifor

Core, build & packaging

• Shared-memory transports are deleted (SHMTransport, pSHMTransport, JpegShmTransport, ShmRPC, the shm / pshm / jpeg_shm URI protos and the unitree-g1-shm blueprint, use unitree-g1-full). Streams run on Zenoh or LCM only, so drop pSHMTransport(...) pins. (#4550) by @paul-nechifor
• Nix is required to auto-build the Rust native modules: build_command is nix build -L path:. and executable is result/bin/<name>, so without Nix build the crate yourself and override both on the module config. The root Cargo.toml workspace is deleted, bin/build-native-modules is bin/native-modules --build, and NativeModuleConfig.cwd is replaced by source_dir plus a relative executable, which also works from a pip install. (#4187, #3970) by @jeff-hykin, @TomCC7
• Dead code that imported undeclared packages is removed: the Redis pubsub and RPC backends, the Genesis and Isaac simulator stubs, PyTTSNode, dimos.types.sample.Sample, the MobileCLIP model and PersonTrackPublisher. (#4576, #4569) by @paul-nechifor
• dimos.perception.memory is replaced by dimos.perception.localize: localize() and embed_index() have new signatures built on Rig and LocalizeParams, the offline tool is python -m dimos.perception.localize.tools.tool_localize, and inventory.py and gates.py are removed. (#4271, #4582) by @bogwi
• The dimensional/ros* Docker images move from Ubuntu 24.04 with ROS 2 Jazzy to Ubuntu 26.04 with ROS 2 Lyrical, and the documented macOS minimum moves from 12.6 to 14. (#4496, #4073) by @Dreamsorcerer, @TomCC7

✨ New Features

Navigation

• A new SE(2) local planner and trajectory follower, both Rust native modules, sit under the MLS global planner: LocalPlannerNative replans a lattice route over the ray tracer's local_map towards a carrot on the global planner_path, and TrajectoryFollowerNative turns it into nav_cmd_vel at 10 Hz and latches goal_reached. Each robot body is described once as an Embodiment (GO2, GO2_PAYLOAD, ALFRED). (#4203, #4608) by @leshy, @aclauer
• The Go2 runs its navigation natively over DDS: go2-nav (Mid-360 + Point-LIO, ray tracing, MLS and local planners, trajectory follower) runs headless on the robot's Jetson, and go2-nav-viewer on a laptop dials the robot's router and subscribes to just the nav topics. With --map-file the robot relocalizes in a saved premap, which seeds the map and the planner region by region so planning starts before the whole map has loaded. alfred-nav runs the same stack on Alfred. (#4449, #4382, #4744) by @leshy, @aclauer

Mapping, SLAM & relocalization

• unitree-go2 closes loops live: the new PGOVoxelMapper replaces the plain voxel mapper in the default Go2 stack, keeps a keyframe pose graph, and rebuilds the global map after each loop closure (about 60 ms per rebuild on the CPU). (#4331) by @leshy
• Permanent maps: unitree-go2-map loads map_go2.map.db, relocalizes against it, extends it as the robot explores, and saves it back with the last ten versions as backups. dimos run unitree-go2 --map-file=<map> relocalizes without updating, and dimos map global <memory.db> --export writes a map from a recording. (#4754) by @leshy

Robot support

• Go2 over DDS as a native Rust module: GO2DDS (go2-dds) runs on a Jetson wired to the robot, terminates cmd_vel as sport Move, and streams the robot's own odometry and tf, the head L1 cloud, the front camera and the body state as standard ROS types on robot timestamps (joint_state, imu, lidar_raw, battery, joy), with a deadman_ms stop. The default unitree-go2* blueprints still connect over WebRTC. (#4332, #4596) by @leshy
• Galaxea R1 Pro: the head cameras come straight off V4L2 through a native V4L2Camera module, both eyes fired from the FSYNC hardware trigger, and base_link is placed by Point-LIO on the chassis Mid-360 in r1pro-coordinator, r1pro-teleop and r1pro-nav. (#4388, #4360) by @jeff-hykin

Simulation

• A MuJoCo Go2 with a simulated Mid-360: dimos run go2-sim generates a seeded procedural office, walks the legged Go2 with the bundled rl_sar locomotion policy, and publishes Point-LIO's output contract from ground truth. dimos run go2-sim-nav runs the whole Go2 navigation stack against it from the same _go2_nav fragment that go2-nav runs on the robot. (#4441, #4481) by @aclauer
• The Go2 MuJoCo simulation defaults to a new office_555 scene, pulled from LFS on first use (--go2connection.mujoco.room office1 restores the old room). (#4542) by @jeff-hykin

Web, cockpit & relay

• Video flows as H.264 WebRTC tracks through a Cloudflare Realtime SFU instead of JPEG frames through the relay, so video stays fast behind a hosted relay far from both ends. Start the relay with --rtc-file rtc.json and install dimos[webrtc] on the robot: every jpeg.v1 channel becomes a video.webrtc.v1 track and no media passes through the relay. Without aiortc it stays on JPEG. (#4264, #4267) by @paul-nechifor
• Map3D cockpit panel: the mapper's global_map as one point per voxel coloured by height, with orbit, zoom, pan and a follow toggle, sent as voxel occupancy bits (about 100 KB per frame instead of 6 MB). Map3D(click="clicked_point", path="path") publishes a goal where you click and draws the planner's route. (#4251, #4637) by @paul-nechifor
• Relay authentication with --auth-file auth.json (robot keys bound to robot ids, viewer tokens): a robot sends its key from RELAYBRIDGEMODULE__RELAY_KEY, the cockpit asks for a viewer token, and your own page passes connect({ url, token }). docker/relay/ runs a hosted relay, and docs/web/relay_hosting.md covers DNS, certbot, Cloudflare video and a LAN recipe. (#3953, #3954) by @paul-nechifor

Distributed runs, cloud & desktop

• Experimental distributed runs: dimos host install --start turns a machine into a Host with its own Zenoh router and detected tags (gpu, gpu_mem_gb, jetson and more), and .hosted(host=..., tags={"gpu": True}, minimum={Tag.gpu_mem: 11}) places a module or sub-blueprint on a matching Host. dimos run compiles one fragment per Host, streams their logs back, and stops them on Ctrl-C. Host RPC is pickle, so open listen= only on networks you trust. (#3967, #4457) by @poorwym, @leshy
• The dimOS Desktop gateway, the HTTP API Desktop builds on (GET /openapi.json): /catalog (blueprints, modules, RPCs, config schemas), /active (launch, stop, logs, RPC calls, stream peeks), /history, /extras and /jobs. It ships as experimental.gateway, and dimos.yaml at the repo root is the contract Desktop reads per tag. (#4429, #4428) by @jeff-hykin

Evals

• Baselines: --agent dimos.evals.agents.pi runs the Pi coding agent against any suite with every model call traced, and --set no_dimos=true strips dimOS off PATH and gives robot cases only the new raw-robot-bridge (plain Zenoh topics for camera, lidar, odometry and cmd_vel). --exclude dimos,dimensionalos denies tool calls that mention a keyword. (#3879, #4137) by @hvent90, @spomichter
• Live-sim environments: HabitatEnvironment launches a Habitat stack per case, DimSimEnvironment replaces Sim(simulator="dimsim"), and MujocoEnvironment launches a MuJoCo blueprint headless and tracks object bodies on tf so graders read real poses. dimos evals run --docker runs the command in a detached container with its own network namespace, so any number of evals run side by side. (#4126, #4297, #4225) by @ruthwikdasyam
• Suites: dimsim_apartment_qa (23 questions, with new numeric, ranking and rank_order scorers) and belief_apartment_qa (41 more), mujoco_xarm (pick up the cylinder, stack the ball), robosuite (six scenes with pose-based graders), and habitat_cones (find every traffic cone in an HSSD house). (#4117, #4348, #4413, #4414) by @ruthwikdasyam, @moshiii-wei, @hvent90

Imitation & datasets

• LeRobotPolicyModule runs a trained LeRobot checkpoint against live color_image and coordinator_joint_state and submits timestamped action chunks to the coordinator's joint_trajectory task, with LeRobot and Torch in their own isolated Python project. dimos dataprep aligns recordings against explicit feature schemas and records alignment and quality reports in dimos_meta.json. (#3315, #4394) by @TomCC7

Perception

• PerceptionLocalize: text queries in, verified 3D object instances out, from a camera posed by tf and lifted through an aligned depth image or the lidar. It answers the localize(objects, start, duration, params) skill over MCP from a time window of its memory and publishes detections: Detection3DArray, with swappable embedders, detectors and segmenters. dimos run go2-localize adds it to the real Go2 stack. (#4271, #4582) by @bogwi

Build & installation

• scripts/install.sh is repaired and is the recommended install path (curl -fsSL https://raw.githubusercontent.com/dimensionalOS/dimos/main/scripts/install.sh | bash), with a library mode (published package in a project venv) and a dev mode (clone with the test and lint groups), verified in CI on Ubuntu 22.04 and 24.04, x86_64 and ARM64. (#4073) by @TomCC7

🐛 Bug Fixes

• Teleop overriding a navigation goal now cancels it, so the robot no longer resumes driving at the wall you just backed it away from. (#4748) by @aclauer
• Picks no longer get stuck when the planner's voxel map is on: a new PointCloudBoxFilter keeps a box around the target out of the voxel map during the pick and rides it on the gripper once the jaws close. (#4519) by @mustafab0
• Eval cases whitelist the artifacts the evaluated agent receives so files that hold answers no longer reach it, and dimos evals run exits with status 1 when a case errored or nothing matched. (#4384, #4824) by @hvent90, @paul-nechifor
• dimos stop and Ctrl-C no longer hang, need SIGKILL, or print tracebacks after the workers exit: workers own module.stop(), a worker that ignores SIGTERM is killed, and an LCM service can be stopped from inside a dispatch callback. (#4820, #4515) by @paul-nechifor
• backpressure() no longer leaks memory with GPU subscribers: the unitree-go2-detection worker grew from 1.9 GB to 4.3 GB in 4.5 minutes on a Jetson Orin Nano before, and stays near 1.7 GB after. (#4796, #4819) by @alxbouchard, @paul-nechifor
• --record keeps depth images lossless: DEPTH, DEPTH16 and GRAY16 streams use the lz4+lcm codec instead of JPEG, which turned 16-bit depth into 8-bit RGB. (#4554) by @alxbouchard
• SQLite memory stores serialize every use of a shared connection, so concurrent reads no longer return empty results, and stopping a --replay run early no longer raises closed-database errors. (#4405, #4810) by @bogwi, @paul-nechifor
• dimos run --daemon on macOS no longer dies with daemon exited during startup: the daemon re-execs the interpreter after the double fork. (#4422) by @aa-MONTS

⚡ Performance

• Startup: pickled RPC proxies no longer carry the module class, so a worker that receives a proxy stops importing that module's dependencies (a SpatialMemory proxy pulled in transformers and torch), and heavy libraries are imported only where they are used. On unitree-go2-agentic-cockpit in MuJoCo, launch to running went from 42 s to 27 s from the proxy change alone. Editable installs must run uv sync after upgrading. (#4283, #4282) by @paul-nechifor
• MuJoCo simulation starts faster (mujoco_playground is imported only when a menagerie clone is needed, and the simulator subprocess launches during construction), and CLIP embeddings for SpatialMemory no longer import transformers or torch: launch to running 25 s -> 13 s on unitree-go2-agentic-cockpit. (#4281, #4286) by @paul-nechifor
• Live lidar clouds take priority over premap tiles in the ray tracer and the MLS planner: on a Jetson Orin Nano the planner's live cloud wait went from 49 / 105 ms (mean / p95) to 10 / 28 ms, premap load from 76 s to 9 s, and the share of live clouds applied from 46% to 99%. (#4454) by @aclauer
• Web stack: a 30 fps camera passes a 30 Hz cap at 30 fps instead of 20, costmap and voxel frames compress in 4 ms instead of 18 ms, JPEG encoding skips an RGB copy, a dead relay is noticed in about 9 s, and the 3D map panel no longer leaks its scene on every page switch. (#4620) by @paul-nechifor

👥 New Contributors

• @aa-MONTS made their first contribution in #4422
• @alxbouchard made their first contribution in #4554
• @moshiii-wei made their first contribution in #4517

Full Changelog: v0.0.14...v0.0.15