New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support pybullet-based Gym Environments #87
Conversation
Hi @gauravmm! Thank you for your pull request. We require contributors to sign our Contributor License Agreement, and yours needs attention. You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks! |
Should there be a separate mbrl/util/pybullet.py instead of adding pybullet envs to mujoco.py? Or, perhaps, a general interface that's not (explicitly or implicitly) tied to a specific engine? Looks good, though, glad you're doing this! |
Thanks for the contribution @gauravmm, this is great! The quote above was going to be my first suggestion. But, in general, do you want me to start reviewing the code as it is, or want to finish other changes first? (asking given your first sentence in the description that says code not ready yet). |
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks! |
Let me finish the rest of the code. It seems to me like the existing |
Sounds good, I'll wait. Regarding |
Hi @gauravmm, just checking if you had any questions about this or generally checking if there is anything I can do to help. |
Oh, oops. I haven't had the time to look into this further. To check the initial code (before the planned refactor), I ran the PyBullet "MuJoCo-clone" environment and I could not get your vanilla MBPO to learn well. This is most likely because there's some part of the state that I am not freezing correctly, or there's some issue with running multiple PyBullet instances in parallel. I'm in the thick of working on a paper right now, and I'll get back to this when I have some free time. Until then, please feel free to delegate this to someone else or see if you can spot the issue. |
Hi @gauravmm thanks for the update, and no worries. To clarify, do you perhaps mean CEM controller instead of MBPO? So far, I've use the freeze functionality mostly in the Incidentally, I happened to start looking at MBPO on PyBullet, as another student reported issues with learning and I'm trying to debug learning in the Hopper and HalfCheetah versions, where the SAC agent diverges. |
Don't accept this yet -- this is still a work-in-progress. Remaining work:
General-purpose environment loader:
mujoco.py
Add support for freezing environments:
Add documentation for:
mujoco.py
Tests:
Other:
pybullet-gym
setup.py
needs to copy 3d assets as well.Types of changes
Motivation and Context / Related issue
This adds support for PyBullet, an open-source alternative to MuJoCo. MuJoCo-compatible and RobotSchool environments are supported via
pybullet-gym
.How Has This Been Tested (if it applies)
Using this for research.
Checklist