Skip to content

Support custom robot URDFs in simulation environments and layout generation #85

Description

@Charcot-NUDT

Summary

First of all, thank you for this impressive project! I would like to use
EmbodiedGen to collect training data for my own robot, but the current
pipeline appears to only support robots registered in ManiSkill's
REGISTERED_AGENTS (defaulting to Franka Panda), with no documented way
to plug in a custom robot URDF.

Use case

I have my own robot (URDF file) and would like to:

  1. Load my robot into the simulation environments instead of the default
    panda — e.g., in PickEmbodiedGen-v1 and the sim-cli / parallel_sim.py
    pipelines.
  2. Generate custom scenes and task layouts around my robot (via layout-cli
    or a hand-written layout.json).
  3. Collect demonstration / trajectory data with my robot for policy training.

Current behavior

From reading the code:

  • embodied_gen/utils/enum.pyRobotItemEnum only enumerates
    franka / ur5 / piper.
  • embodied_gen/utils/simulation.pyload_mani_skill_robot instantiates
    robots via ManiSkill's REGISTERED_AGENTS; if the robot name in the layout
    is not registered, it silently falls back to panda, which is easy to miss.
  • embodied_gen/envs/pick_embodiedgen.pyPickEmbodiedGen-v1 is the only
    task environment, hard-coded with robot_uids="panda".
  • The documentation (tutorials) does not cover how to integrate a custom robot.

Feature requests

  1. A documented, first-class way to register a custom robot URDF
    (e.g., a --robot_urdf argument, a robot config file, or a tutorial on
    writing a minimal ManiSkill BaseAgent wrapper), usable across
    sim-cli, parallel_sim.py, and the gym environments.
  2. An explicit error or warning instead of silently falling back to
    panda when the requested robot is not found.
  3. Optionally: guidance on how to define custom task environments
    (beyond pick-and-place) that work with the layout/data-collection pipeline.

Alternatives considered

I understand I could register my robot as a ManiSkill agent myself and extend
BaseEnv, but an official mechanism + documentation would make EmbodiedGen
much more useful as a data-generation platform for arbitrary robots.

Environment

  • EmbodiedGen version: v2.0.0
  • OS: Linux

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions