GPU acceleration for pybullet3?

[Simon-Steinmann]

Hello,
since pybullet is now on bullet3, is gpu compute supported? The documentation is not clear on this. We are working on a scientific paper about a simulation benchmark, comparing different simulation environments (Gazebo, Mujoco, Webots, Pybullet) with a focus on industrial application and reinforcement learning. Currently the gpu is not used at all on pybullet. Can gpu acceleration be enabled? The scientific paper should reflect the state of the art.

We oriented ourselves on the quickstart guide, however, if you have other suggestions, that would improve pybullets performance, feel free to mention anything. Currently the used settings are the following:

self._p = bullet_client.BulletClient(connection_mode=pybullet.DIRECT)
self._p.setPhysicsEngineParameter(contactERP=0.2)
self._p.setPhysicsEngineParameter(globalCFM=2e-2)

and for objects: (pick and place scenario):

for o in self.objects:            
            self._p.changeDynamics(o, -1, lateralFriction=0.45)
            self._p.changeDynamics(o, -1, spinningFriction=0.05)
            self._p.changeDynamics(o, -1, rollingFriction=0.001)
            self._p.changeDynamics(o, -1, restitution=0.0)
            self._p.changeDynamics(o, -1, linearDamping=0.04)
            self._p.changeDynamics(o, -1, angularDamping=0.04)
            self._p.changeDynamics(o, -1, frictionAnchor=1)

[erwincoumans]

Heh, I'm a bit sceptical about benchmark papers, since most of the 'scientific papers about physics engine benchmarks' so far turn out to be advertisements: they are written by an author who has a physics engine in the race, and magically the physics engine by the author always comes out best. Usually they 'thank' the other physics engine creators at the end, as if it is an endorsement.

Bullet still has an OpenCL GPU backend, we haven't enabled it at the moment in PyBullet, but we may do so in the future.

Tuning parameters depends on the scenario, there is not one 'best' set of parameters (otherwise we could get rid of them). It takes time to tune a physics engine to work best for a scenario or benchmark.

Some nice features of PyBullet:

1. it can be trivially installed on Windows, Mac OSX, Linux using

pip3 install pybullet --upgrade --user

2. PyBullet is 100% deterministic, if used properly (of course it is easy to violate, if desired for a benchmark)
3. PyBullet runs headless in the cloud, on machines without GPU or even X11, since it has a simple CPU renderer. It can also run headless in the cloud using EGL hardware rendering
4. PyBullet has been used for many sim-to-real robot hardware experiments
5. PyBullet has forward kinematics, inverse kinematics (faster and more reliable than IKFast if tuned well, in our experiments), inverse dynamics, virtual reality (using OpenVR)
6. PyBullet has efficient shared memory API (for client server) and can run remote using TCP/TP using a graphics client/server of physics client/server.
7. PyBullet support URDF, SDF and MJCF (subset of its features)
8. You can run PyBullet in a web page using Google Colab, also with hardware OpenGL rendering
9. PyBullet has support for many collision shape types (sphere, box, capsule, cylinder, convex hull, concave triangle mesh, and height field (animated) and signed distance field (SDF)
10. PyBullet uses a Projected Gauss Seidel solver by default, but you can select an alternative constraint solver (Dantzig) which could be better for some scenarios
11. PyBullet uses reduced coordinates (Featherstone style) by default, but also supports maximal coordinates rigid body with constraints, and two-way coupling between those. You can close loops of reduced coordinate methods using constraints.
12. PyBullet support deformable simulation using finite element method, mass-spring or position based dynamics, including two-way interaction with multibodies and rigid bodies

[tlemarec]

Bullet still has an OpenCL GPU backend, we haven't enabled it at the moment in PyBullet, but we may do so in the future.

Any news on that ? @erwincoumans

[weixuanzhang]

Any news on that in 2023? @erwincoumans Thanks!

[Simon-Steinmann]

Heh, I'm a bit sceptical about benchmark papers, since most of the 'scientific papers about physics engine benchmarks' so far turn out to be advertisements: they are written by an author who has a physics engine in the race, and magically the physics engine by the author always comes out best

This became very apparent to us as well. We have no affiliation with any engine / simulation author. We are a consortium of several research institutes (DLR, Fraunhofer) and a few robotics and aerospace companies. The goal is to figure out, which engine and simulation is best suited for certain tasks. Thank you for your list of information. In a feature overview, some of these things will certainly be listed.

While it is not in the scope of this paper, I would be very interested in the implemented IK solution. Especially for RL, very fast IK is critical. So far I use IKFast, as it is several orders of magnitude faster than numerical solvers. Can you link me to some documentation?

• Is it analytical or numeric?
• Does it support any number of DOF?
• Does pybullet include a motion planning framework akin to OMPL or MoveIt?

[erwincoumans]

PyBullet's IK is numerical, using selectively damped least squares.

> Does it support any number of DOF?

Yes, PyBullet supports any number of DOF, you just load the URDF (or SDF, MJCF etc). We also use IK for motion retargeting of a quadruped robot (tree structure), see https://github.com/google-research/motion_imitation/tree/master/retarget_motion

> So far I use IKFast, as it is several orders of magnitude faster than numerical solvers.

No, that is not my experience at all, IKFast was must slower than PyBullet. Timings were in the file: IKFast was 300us versus PyBullet 150us, for the IK on Xarm6. But I'm not a IKFast expert, so perhaps a matter of tuning? Aside of being slower, IKFast also often didn't produce a solution, while PyBullet always provided a reasonable solution. See https://github.com/erwincoumans/xArm-Python-SDK/blob/master/example/wrapper/xarm6/xarm_sim.py S-DLS method is described here: https://www.math.ucsd.edu/~sbuss/ResearchWeb/ikmethods/SdlsPaper.pdf We use our custom code for computing the Jacobians, and made some improvements of S-DSL, using velocity IK, using null-space and a few other bits. At the moment, no motion planning published, some projects using learning for motion planning. We also used Reflexxes 2, by Torsten Kroeger, for motion planning before, also not released. PyBullet does have various collision queries that can help motion planning. We didn't expose the swept volumes in Python yet, should be easy to do. We use PyBullet primarily for quadruped locomotion research @ Google Robotics, sim-to-real, motion imitation etc, reinforcement learning etc, and for arm grasping research and robot development. We also did some projects for miniature car driving (MIT racecar etc). There are also related projects, such as Stanford Gibson 1/2, and Facebook Habitat that have nicer rendering with PyBullet. https://pybullet.org/wordpress/index.php/2019/11/22/facebook-habitat-v0-1-3-adds-bullet-physics/ http://gibsonenv.stanford.edu/ We have some nicer render plugins that we didn't open source yet. Thanks! Erwin

…

On Fri, 9 Oct 2020 at 02:22, Simon Steinmann ***@***.***> wrote: Heh, I'm a bit sceptical about benchmark papers, since most of the 'scientific papers about physics engine benchmarks' so far turn out to be advertisements: they are written by an author who has a physics engine in the race, and magically the physics engine by the author always comes out best This became very apparent to us as well. We have no affiliation with any engine / simulation author. We are a consortium of several research institutes (DLR, Fraunhofer) and a few robotics and aerospace companies. The goal is to figure out, which engine and simulation is best suited for certain tasks. Thank you for your list of information. In a feature overview, some of these things will certainly be listed. While it is not in the scope of this paper, I would be very interested in the implemented IK solution. Especially for RL, very fast IK is critical. So far I use IKFast, as it is several orders of magnitude faster than numerical solvers. Can you link me to some documentation? - Is it analytical or numeric? - Does it support any number of DOF? - Does pybullet include a motion planning framework akin to OMPL or MoveIt? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3105 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFRDXHWCDQUPT4FULXOKALSJ3I4ZANCNFSM4SI5QCUA> .

[erwincoumans]

(re-open to keep some discussion going)

[BluePrintRandom]

There is a motion plan system for game agents called phase function neural network, I believe making a plan with this and simulating it could be supperior to IK and faster.

…

On Sun, Oct 11, 2020, 10:05 PM erwincoumans ***@***.***> wrote: (re-open to keep some discussion going) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3105 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABW3SWKKOZQVIM5TTRER2YLSKKFBPANCNFSM4SI5QCUA> .

[WilliamTambellini]

The question is very legit and early closing the ticket would be questionable.
Please find a recent paper confirming the need for GPU accelerated physics engines :
"Local Optimization for Robust Signed Distance Field Collision"
https://mmacklin.com/sdfcontact.pdf
and a quick summary video:
https://www.youtube.com/watch?v=ooZ9rUYOFI4
First, has bullet3 an implementation of SDF (Signed Distance Field)?
Kind regards

[tlemarec]

@Simon-Steinmann I am interested in reading your conclusions, where can I find your paper ?

[1heart]

It's probably this one

[Simon-Steinmann]

That is the one. Sorry, I missed the question. @tlemarec Let me know if you have any questions.