Task scenes for the v2 model #39
Replies: 11 comments 3 replies
|
@Manas-arumalla, just out of curiosity, have you tried MjLab for RL training with OpenArm? |
|
@k1000dai Not yet my RL stack so far is Gymnasium + Stable-Baselines3, single-env, which was fine for the small tasks I've trained (reach/insertion/balance). But I'm about to start policy training on these task scenes, and I have an RTX 5060 on my laptop, so I was actually thinking of trying MjLab's parallel envs for it. Have you used it with OpenArm yourselves? |
|
@Manas-arumalla Not yet tested on MjLab. But planning to test! Please let us know if you tried some experiment on MjLab ( of course, anything else related !) |
|
Following up on previous mergers. I also have a few dynamic scenes on my side that go beyond the tabletop set: catching a ball thrown through the air (single, bimanual, and two-ball) and throwing a ball into a bin. They're a differen from the merged ones no fixed table, a projectile launched at runtime rather than a static keyframe, and the catch scenes carry a couple of RGB-D cameras for tracking. Would these be useful to have in the repo? If so, I'm happy to adapt them to whatever structure you'd prefer for dynamic scenes. And if they're out of scope for the model repo, no problem at all just wanted to offer |
|
Yeah, i want to see if possible ! I think dynamic tasks are really good extention for Openarm ! |
|
@Manas-arumalla I tried the MjLab experiments on OpenArm Cell Environments! ( easy pick and place task, but it works!) . |
|
@k1000dai Yeah, please do. I have been working on some complex tasks using Mjlab as well. I have tried the tasks that I did, such as opening a drawer, opening a door, and rotating stuff. These are working, but not to the extent I would like. I will continue improving them and post them soon. |
|
@k1000dai I kept going with MjLab after my last update. Built a full contact-only RL suite (no grasp welds) covering reach, valve, puck, door, drawer, and lift, all evaluated with \hardened metrics . All of those came out good but i want you to check them if possible. Also mid-way through bimanual coordination (both arms lifting one object together. happy to send these if that's useful, or just share it as a companion repo if you'd rather build out your own pick-and-place work first. Let me know which is more useful. |
|
Sorry! @k1000dai did it but I was a blocker. I was preparing his work to make it public. I've done it: https://github.com/enactic/openarm_mjlab |
|
Thanks @kou Will pull it and give it a try. And just to close the loop from my last message happy to share the reach, valve, door, drawer, puck, lift with MJlab whenever useful, plus a robustness pass with domain randomization and a first vision-based policy (depth camera, no privileged state) I've been running. Let me know if should open a PR in that repo |
|
Yes! But could you open one PR per task for easy to review? |
|
okay i will |
Uh oh!
There was an error while loading. Please reload this page.
Hi! I've been building a control stack on top of the v2 MJCF for the past few months (https://github.com/Manas-arumalla/openarm-control) - IK, motion planning, bimanual skills, some RL, and a benchmark suite over all of it.
Along the way I ended up writing a bunch of self-contained task scenes around the official model: peg-in-hole insertion, articulated objects (a drawer, a cabinet door, a valve), ball-on-plate balancing, cloth folding, catching/throwing. They're plain MJCF, no python dependencies - each one loads directly with
python -m mujoco.viewerand comes with a "ready" keyframe.Would you be open to a PR contributing some of these under v2/? I'd start with three (insertion, articulated, balance) and add more later if people find them useful. The branch is ready, so I'll open the PR now - happy to rework or drop anything that doesn't fit where you want the repo to go.
One small thing I noticed while adapting them to the current model: the material names in openarm_bimanual.xml changed at some point, so the scenes define their own pedestal material the same way pedestal.xml does. They don't touch the model file at all.
All reactions