feat: add OMX-AI robot arm support alongside SO-101#81
Open
robertchoi wants to merge 11 commits into
Open
Conversation
[260711] fix to recognize camera at chrome and fix to 3d model limit …
…uration management and status reporting
…w fixes Abstract the hardcoded SO-101 wiring so the web UI can drive OMX-AI arms: - make_device_config/make_device in utils/devices.py dispatch config and device classes by robot_type (so101 -> SO101*, omx -> Omx*) - teleoperate/record/calibrate build devices through the new helpers; rollout resolves --robot.type dynamically - calibration paths, robot records, and all frontend start requests (teleop, recording, inference, calibration) carry robot_type - frontend calibration targets gain OMX-AI joint range maps Review fixes on top of the feature: - calibrate: pick the POSITION operating-mode value per bus protocol; the Feetech value (0) means CURRENT mode on Dynamixel (OMX) motors - server: delete_calibration_config resolves paths via get_calibration_path so OMX calibration files can be deleted - drop the test-mock shim from make_device; tests now patch make_device - lint cleanup (unused imports, import sorting, whitespace) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…onfiguration UI components
…viewer OMX-AI arms self-calibrate on first connect and never went through LeLab's SO-101-only calibration wizard, so their robot records could never satisfy the "calibration file exists" check that gated the Teleoperation button, and `lerobot[dynamixel]` was missing from the pinned extras so OMX motors couldn't even connect. teleoperate.py also pushed an empty calibration to the bus instead of letting OMX self-calibrate. Also wires the Teleoperation page's 3D viewer to show the actual OMX-AI model (ROBOTIS open_manipulator_description URDF/meshes, Apache-2.0) instead of always rendering the SO-101 arm, with a matching robot-type-aware joint-name/unit mapping for the live joint broadcast. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
fix: make OMX-AI robots teleoperable without calibration, add OMX 3D viewer
There was a problem hiding this comment.
Pull request overview
Adds OMX-AI support alongside SO-101 across hardware configuration, calibration, recording, inference, and visualization.
Changes:
- Introduces robot-type-aware backend device creation and calibration paths.
- Adds frontend model selection, persistence, and OMX 3D visualization.
- Expands tests, dependencies, documentation, and generated frontend assets.
Reviewed changes
Copilot reviewed 38 out of 57 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_teleoperate.py |
Updates device mocking. |
tests/test_rollout.py |
Tests shared error hints. |
tests/test_record.py |
Tests recording error status. |
tests/test_calibrate.py |
Tests protocol-specific position mode. |
study/install.md |
Adds Windows installation guide. |
study/day0_overview.md |
Adds Korean project overview. |
pyproject.toml |
Enables Dynamixel support. |
lelab/utils/devices.py |
Adds dynamic device factories and hints. |
lelab/utils/config.py |
Adds model-specific calibration persistence. |
lelab/teleoperate.py |
Parameterizes devices and visualization. |
lelab/server.py |
Parameterizes calibration routes and Linux camera names. |
lelab/scripts/lelab.py |
Extends Windows startup timeout. |
lelab/rollout.py |
Parameterizes inference robot type. |
lelab/record.py |
Parameterizes recording devices and errors. |
lelab/calibrate.py |
Adds OMX device and Dynamixel handling. |
frontend/src/pages/Teleoperation.tsx |
Selects the active robot model. |
frontend/src/pages/Recording.tsx |
Displays recording failures. |
frontend/src/pages/Landing.tsx |
Sends recording robot type. |
frontend/src/pages/Calibration.tsx |
Adds robot model configuration. |
frontend/src/lib/inferenceApi.ts |
Adds inference robot type. |
frontend/src/lib/defaultUrdfModels.ts |
Maps built-in URDF models. |
frontend/src/lib/calibrationTargets.ts |
Adds OMX calibration targets. |
frontend/src/hooks/useRobots.ts |
Persists robot model selection. |
frontend/src/contexts/UrdfContext.tsx |
Tracks the default URDF type. |
frontend/src/components/UrdfViewer.tsx |
Loads model-specific URDF assets. |
frontend/src/components/landing/RobotSelector.tsx |
Adds model chips at creation. |
frontend/src/components/landing/RobotConfigManager.tsx |
Sends teleoperation robot type. |
frontend/src/components/landing/InferenceModal.tsx |
Sends inference robot type. |
frontend/public/omx-urdf/urdf/omx_f.urdf |
Adds the OMX follower model. |
frontend/public/omx-urdf/meshes/omx_f/follower_01_base.stl |
Adds OMX base mesh. |
frontend/public/omx-urdf/meshes/omx_f/follower_02_base_tilt_Revised.stl |
Adds OMX base-tilt mesh. |
frontend/public/omx-urdf/meshes/omx_f/follower_03_middle_verticle.stl |
Adds OMX middle-link mesh. |
frontend/public/omx-urdf/meshes/omx_f/follower_04_middle_horizontal.stl |
Adds OMX horizontal-link mesh. |
frontend/public/omx-urdf/meshes/omx_f/follower_05_tip.stl |
Adds OMX tip mesh. |
frontend/public/omx-urdf/meshes/omx_f/follower_06_pan_Revised.stl |
Adds OMX wrist-pan mesh. |
frontend/public/omx-urdf/meshes/omx_f/follower_07_gripper_motorized.stl |
Adds OMX motorized-gripper mesh. |
frontend/public/omx-urdf/meshes/omx_f/follower_08_gripper_gear.stl |
Adds OMX gripper-gear mesh. |
frontend/public/omx-urdf/LICENSE |
Adds asset licensing. |
frontend/package-lock.json |
Refreshes lockfile metadata. |
frontend/dist/omx-urdf/urdf/omx_f.urdf |
Ships the built OMX URDF. |
frontend/dist/omx-urdf/LICENSE |
Ships the asset license. |
frontend/dist/index.html |
References rebuilt assets. |
frontend/dist/assets/index-DDnZ8aJk.css |
Adds rebuilt stylesheet. |
frontend/dist/assets/index-Da-rRmcX.css |
Removes the previous stylesheet. |
Files not reviewed (1)
- frontend/package-lock.json: Generated file
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+124
to
+132
| else: | ||
| angle_degrees = raw_value | ||
| #correction = _SO101_URDF_CORRECTIONS.get(motor_name) | ||
| #if correction is not None and motor_name in calibration: | ||
| # sign, urdf_zero_ticks = correction | ||
| # cal = calibration[motor_name] | ||
| # mid = (cal.range_min + cal.range_max) / 2 | ||
| # motor_at_urdf_zero = (urdf_zero_ticks - mid) * 360 / _STS3215_MAX_RES | ||
| # angle_degrees = sign * (raw_deg - motor_at_urdf_zero) |
Comment on lines
+335
to
+337
| position_mode = _position_operating_mode(self.device.bus) | ||
| for motor in self.device.bus.motors: | ||
| self.device.bus.write("Operating_Mode", motor, OperatingMode.POSITION.value) | ||
| self.device.bus.write("Operating_Mode", motor, position_mode) |
Comment on lines
+51
to
+52
| else: | ||
| folder = f"so_{side}" |
Comment on lines
+257
to
+258
| else: | ||
| robot_type_arg = "--robot.type=so101_follower" |
| <visual> | ||
| <origin rpy="0 0 0" xyz="0 0 0"/> | ||
| <geometry> | ||
| <mesh filename="package://open_manipulator_description/meshes/omx_f/follower_01_base.stl" scale="0.001 0.001 0.001"/> |
Comment on lines
+103
to
+111
| elif "omx" in model: | ||
| if side == "follower": | ||
| from lerobot.robots.omx_follower import OmxFollowerConfig | ||
|
|
||
| return OmxFollowerConfig(port=port, id=config_id, cameras=cameras or {}) | ||
| else: | ||
| from lerobot.teleoperators.omx_leader import OmxLeaderConfig | ||
|
|
||
| return OmxLeaderConfig(port=port, id=config_id) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #79
Summary
This PR adds native support for the ROBOTIS OMX-AI robotic arm to LeLab's web GUI, alongside the existing SO-101 leader/follower arms. All four hardcoded areas identified in #79 are now abstracted behind a per-robot
robot_typesetting.Changes
Backend: dynamic robot configuration loading
make_device_config()/make_device()helpers inlelab/utils/devices.pydispatch LeRobot config and device classes byrobot_type(so101→SO101Follower/SO101Leader,omx_ai→OmxFollower/OmxLeader).teleoperate.py,record.py, andcalibrate.pybuild devices through these helpers instead of importing SO-101 classes directly; every request schema gains arobot_typefield (default"so101", fully backward compatible).utils/config.py) resolve per robot type (so_leader/so_followervsomx_leader/omx_follower), matching the directories LeRobot derives from each device'sname.POSITIONoperating-mode value per bus protocol: Feetech uses0, but on Dynamixel (OMX)0means CURRENT mode — writing the Feetech value would have put OMX motors in current-control mode.Rollout parameterization
rollout.pyresolves--robot.typefrom the request (so101_follower/omx_follower) instead of hardcoding it.Frontend
robot_type; it is sent with every teleoperation, recording, inference, and calibration start request.calibrationTargets.tsgains OMX-AI per-joint target ranges, selected by the active robot's type.Testing
0vs Dynamixel3).make_deviceseam, so both SO-101 and OMX paths are testable.ruff checkclean.robot_type: "omx_ai", and SO-101 flows are unchanged.Notes for reviewers
robot_typedefaults to"so101"everywhere, so existing robot records and API clients keep working unchanged.make_device_config/make_device, a calibration-path folder mapping, and (optionally) frontend target ranges.