Create an actuated single free-floating rigid body

[hongkai-dai]

I would like to control the motion of a single free-floating actor (rigid body). As far as I understand it, similar to controlling a revolute joint by torque, now I should control this free-floating object by a wrench (including both a force and a torque vector). Is there an example of adding such free-floating actor with actuation in ManiSkill2/SAPIEN?

My main question is what sapien.Joint should I use in my controller? I was trying to subclass MyPoseController from BaseController. But BaseController would need a list of Joint https://github.com/haosulab/ManiSkill2/blob/5178d73929076e1174d934565a8a11b339aae890/mani_skill2/agents/base_controller.py#L19. Currently Sapien only supports revolute, prismatic and fix joint. How should I handle this free-floating joint with actuation?

[Jiayuan-Gu]

Since we focus on realistic settings, we do not take free-floating actors into consideration when we design ManiSkill2. It is tricky to support sapien.Actor rather than sapien.Articulation, since most designs in BaseAgent and BaseController are based on sapien.Articulation.

The easiest way I have come up with is to wrap your actor in a URDF as pybullet does. And then you can use the ManiSkill2 framework seamlessly. But could you provide more details or any reference implementation in other frameworks? For example, whether it should be controlled by force and torque, or by target velocity/position.

If you build an environment with SAPIEN from scratch instead of using the ManiSkill2 framework, then you can just control actor by actor.add_force_torque in env.step. Some examples of building RL envs from scratch can be found at here

[hongkai-dai]

Thanks @Jiayuan-Gu for the reply, that is super helpful!

As you said, I can build the environment with SAPIEN. At the same time, I also like to simulate deformable dough. So I think I will need to do the coupling myself, that the actor in SAPIEN will add boundary condition to the dough in WARP simulator. (I don't care about the coupling from dough back to the actor, my use case is more similar to PlasticineLab, that I can suppose the rolling pin just follows the prescribed trajectory).

Specifically it looks to me that I will copy MPMBaseEnv class, but modify the step_action function, that I do not need to compute the wrench in this block
https://github.com/haosulab/ManiSkill2/blob/9885db8cf1df374277ca74a9d6234087ceb269f2/mani_skill2/envs/mpm/base_env.py#L604-L618

Does this sound right to you?

[fbxiang]

Your approach sounds right. However, I suggest using PlasticineLab directly. Compared to PlasticineLab, ManiSkill features articulation (instead of rigid actor), dynamic control (instead of fixed or kinematic trajectory), and 2-way coupling (instead of using rigid body as boundary condition only). I also find 1-way coupling a really bad idea for our dynamic simulation. For example, if a rigid body squishes a soft body without the soft body pushing back, the soft body can be completely flattened and it crashes the simulation easily. However, if you are using a prescribed trajectory, it is probably fine.

So for me, it sounds like Plasticine Lab suits your use case perfectly and ManiSkill will only give you additional overhead in learning the framework. If there is some reason you really cannot use PasticineLab, you can just modify the MPM base class and not apply the rigid body wrench to achieve 1-way coupling.

[hongkai-dai]

Thanks @Jiayuan-Gu and @fbxiang for the help (and sorry for my belated reply).

I implemented both the one-way and two-way coupling in ManiSkill2. As you mentioned, the one-way coupling is dangerous. The dough might apply a large reaction force to resist the motion of the rolling pin on the actual robot, such that we cannot follow a prescribed trajectory.

Close this issue as it has been resolved.