website 2.0: add inference tutorial#490
Closed
abetomo wants to merge 1 commit into
Closed
Conversation
|
Preview: https://abetomo.github.io/openarm See https://github.com/enactic/openarm/blob/main/website/README.md#pull-request-and-preview how to configure preview on fork. |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new tutorial page documenting the inference pipeline for OpenArm: the policy server contract (UNIX socket transport, Arrow IPC observation input, JSON action output), an architecture diagram of the Dora dataflow, an annotated YAML snippet of the key nodes, and basic prerequisites/run commands.
Changes:
- New
Inferencetutorial page describing the policy server I/O contract (observation Arrow IPC + JSON action chunk format). - Documented dataflow architecture and key Dora nodes (
observer,quittable-observer,policy-server,actions-executor). - Added prerequisites and local-run commands using
uvanddora-rs0.5.0.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| ## The Policy Server Contract | ||
|
|
||
| This is the what one needs to implement when adapting to a new model. |
| | Field | Type | Meaning | | ||
| |---|---|---| | ||
| | `interval` | int (ns) | Time between consecutive position steps. `int(1e9 / 30)` ≈ 33 ms for 30 Hz | | ||
| | `cutoff_hz` | number, optional | Execute only the first N steps of the chunk before the next inference for receding horizon control | |
| For local development, the server listens on the socket: | ||
|
|
||
| ``` | ||
| SOCKET=/dev/shm/policy-server.socket dora run dataflow-inference.yaml --uv |
Comment on lines
+157
to
+202
| ```yaml | ||
| nodes: | ||
| # --- observer: collects arms + cameras into one Arrow IPC bundle --- | ||
| - id: observer | ||
| path: dora-openarm-observer | ||
| inputs: | ||
| tick: quittable-tick-observer/tick # 30 Hz gate | ||
| arm_right: arm-right/position | ||
| arm_left: arm-left/position | ||
| camera_wrist_right: camera-wrist-right/image | ||
| camera_wrist_left: camera-wrist-left/image | ||
| camera_head: camera-head/image | ||
| camera_ceiling: camera-ceiling/image | ||
| phase_classifier_result: phase-classifier/result | ||
| outputs: | ||
| - observation # Arrow IPC file path + metadata, written to /dev/shm | ||
|
|
||
| # --- quittable-observer: gate controlled by controller --- | ||
| - id: quittable-observer | ||
| path: dora-openarm-quitter | ||
| inputs: | ||
| command: controller/command | ||
| observation: observer/observation | ||
| outputs: | ||
| - observation | ||
|
|
||
| # --- policy-server: YOUR node --- | ||
| - id: policy-server | ||
| path: dora-openarm-local-policy-server | ||
| env: | ||
| SOCKET: /dev/shm/policy-server.socket # must match your running server | ||
| inputs: | ||
| observation: quittable-observer/observation | ||
| outputs: | ||
| - actions # JSON: {interval, cutoff_hz, positions: [[16D], ...]} | ||
|
|
||
| # --- actions-executor: unpacks the chunk and drives the arms --- | ||
| - id: actions-executor | ||
| path: dora-openarm-actions-executor | ||
| args: "--upsample-on --filter-on" | ||
| inputs: | ||
| actions: policy-server/actions | ||
| outputs: | ||
| - move_position_right # → arm-right | ||
| - move_position_left # → arm-left | ||
| ``` |
Comment on lines
+84
to
+117
| ```python | ||
| import pyarrow as pa | ||
| import numpy as np | ||
|
|
||
| with pa.OSFile(request["data_path"], "rb") as f: | ||
| with pa.ipc.open_file(f) as reader: | ||
| observations = reader.get_batch(0).to_struct_array() | ||
|
|
||
| last = observations[-1] | ||
| metadata = request["metadata"] | ||
|
|
||
| # Camera frames — shape (H, W, 3), uint8 | ||
| def read_camera(name): | ||
| return ( | ||
| last[name].values.to_numpy(zero_copy_only=False) | ||
| .reshape(metadata[f"{name}.height"], metadata[f"{name}.width"], 3) | ||
| ) | ||
|
|
||
| frames = { | ||
| "head": read_camera("camera_head"), | ||
| "ceiling": read_camera("camera_ceiling"), | ||
| "right_wrist": read_camera("camera_wrist_right"), | ||
| "left_wrist": read_camera("camera_wrist_left"), | ||
| } | ||
|
|
||
| # Joint positions — float32, shape (16,) | ||
| # Layout: right_arm[7] | right_gripper[1] | left_arm[7] | left_gripper[1] | ||
| pos_dim = len(last["position"]) | ||
| qpos = ( | ||
| last["position"].values.to_numpy(zero_copy_only=False) | ||
| .reshape(pos_dim) | ||
| .astype(np.float32) | ||
| ) | ||
| ``` |
Comment on lines
+111
to
+114
| pos_dim = len(last["position"]) | ||
| qpos = ( | ||
| last["position"].values.to_numpy(zero_copy_only=False) | ||
| .reshape(pos_dim) |
Contributor
|
duplicate with #491 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.