feat:rust native modules

[aclauer]

Problem

We need support for Rust native modules to offload performance critical modules.

Closes DIM-XXX

Solution

See native/rust/examples for a Rust native ping pong example

Added NativeModule struct to allow easy publishing and subscribing to topics. NativeModules can use any transport that implements the Transport trait (currently only LCM).

Rust NativeModules also support configuration through stdin instead of cli args. Cli args are kept for backwards compatibility.

Breaking Changes

None

How to Test

For unit tests
cd dimos/native/rust
cargo test

For end to end test with two Rust modules talking over LCM
python3 rust_ping_pong.py

Contributor License Agreement

• I have read and approved the CLA.

[greptile-apps]

Greptile Summary

This PR introduces a Rust native module SDK (native/rust) with a transport-agnostic NativeModule struct, LCM transport implementation, and a ping-pong end-to-end example. The Python NativeModule coordinator is extended with an optional stdin_config mode that writes a JSON blob (topics + config) to the child process's stdin before closing the pipe.

• P1 breaking change: stdin=subprocess.PIPE is now set unconditionally and the pipe is always closed, even when stdin_config=False. Pre-existing native modules that previously inherited the parent's stdin now receive an immediately-closed pipe and will see EOF on any stdin read, silently altering their startup behaviour.

Confidence Score: 4/5

Safe to merge after fixing the unconditional stdin pipe change, which silently breaks existing non-stdin-config native modules.

One confirmed P1 defect: stdin=subprocess.PIPE is applied and closed unconditionally, making the change behaviorally breaking for any existing native module that reads from stdin, despite the PR claiming no breaking changes. The Rust SDK itself is well-structured and has solid unit-test coverage. Once the stdin guard is added, all remaining notes are P2 style/improvement suggestions already covered in previous threads.

dimos/core/native_module.py — the subprocess stdin pipe handling

Important Files Changed

Filename	Overview
dimos/core/native_module.py	Adds stdin_config flag and to_config_dict() for Rust module bootstrapping; unconditionally setting stdin=PIPE and closing it silently breaks existing non-stdin-config modules
native/rust/src/module.rs	Core Rust SDK: NativeModule struct with transport-agnostic pub/sub, stdin JSON config parsing, and well-tested unit suite; previously flagged blocking I/O and error busy-loop are addressed in the current diff
native/rust/src/transport.rs	Defines async Transport trait with publish and recv — minimal and clean
native/rust/src/lcm.rs	Thin LcmTransport wrapper around dimos_lcm::Lcm implementing the Transport trait — straightforward delegation
examples/native-modules/rust/src/native_ping.rs	Ping example: publishes Twist at 5 Hz and logs echoes; correctly uses from_stdin::<()> with unit config
examples/native-modules/rust/src/native_pong.rs	Pong example: receives Twist and echoes it back with sample_config embedded; correctly uses typed PongConfig deserialization
examples/native-modules/rust_ping_pong.py	Python orchestrator for the end-to-end ping-pong demo; _RUST_DIR and _EXAMPLES paths are correct relative to the script location
native/rust/Cargo.toml	Library manifest with correct Apache-2.0 license and git-pinned dimos-lcm dependency; lock file pins exact commits for reproducibility
examples/native-modules/rust/Cargo.toml	Example workspace manifest with two binary targets (native_ping, native_pong) and correct path dependency on the SDK

Sequence Diagram

sequenceDiagram
    participant PY as Python NativeModule
    participant PROC as Native Process (Rust)
    participant LCM as LCM Multicast Bus

    PY->>PROC: Popen(stdin=PIPE, stdout=PIPE, stderr=PIPE)
    alt stdin_config=True
        PY->>PROC: write JSON {"topics":{...},"config":{...}}\n
    end
    PY->>PROC: stdin.close() [EOF]
    PROC->>PROC: from_stdin() reads line, parses JSON
    PROC->>PROC: module.input("data", decode) / module.output("confirm", encode)
    PROC->>PROC: module.spawn() → background tokio task

    loop publish loop
        PROC->>LCM: transport.publish(topic, data)
    end
    loop recv loop
        LCM-->>PROC: transport.recv() → (channel, data)
        PROC->>PROC: route.try_dispatch(data) → Input mpsc channel
    end

Loading

_{Reviews (2): Last reviewed commit: "Merge branch 'dev' into andrew/feat/rust..." | Re-trigger Greptile}

[leshy]

small tweaks