Bullet Physics Integration #1 #91

hzyjerry · 2019-06-28T16:32:14Z

Motivation and Context

First PR towards physical interactions with Habitat 😊
Early feedbacks are severely welcomed 🎉
Laying the foundation for upcoming development (articulated agents & objects, robotics, etc) requires moderate refactoring of existing habitat code and data organizations.

Types of changes

Introduced esp/assets/PhysicsManager (similar to esp/assets/ResourceManager in style) to track & manage states of physics simulation, such as bulletWorld, FPS, timestep, objects etc
Introduced esp/physics/BulletObject as an extended SceneNode. BulletObject interacts with PhysicsManager to create collision mesh, set object properties, etc.
Supporting APIs: generic object state update, instancing
- loadObject(xyz, rotation)
- setObject(ID, xyz, rotation)
- removeObject(ID)
- Move functions from viewer.cpp to habitat core
- Support object instancing by ID
Testing

Challenges 💥

File formats: getting GLB/obj/ply objects to all work correctly (E.g. Bullet mesh uses PHY_INTEGER, incompatible with Magnum::UnsignedInt)
Object: requires preprocessing before we can use them
Speed: physics simulation slows down significantly when there are >20 objects
Physical metadata: how to store physical properties (friction, restitution, mass)

Checklist

PR Milestones

July 2 GLB Scene
July 4 Surreal Scene
July 5 Replica Scene
Tested on Gibson & MP3D

… castle & teleport with key

… slates)

… mesh)

Pull from fb master

…ld error)

erikwijmans

Left some high-level comments. This should be rebased on master ASAP as that will make seeing how it interfaces with the current code easier.

src/esp/assets/FRLInstanceMeshData.h

src/esp/assets/PhysicsManager.h

src/esp/assets/ResourceManager.cpp

erikwijmans · 2019-06-28T22:56:18Z

src/esp/assets/ResourceManager.cpp

@@ -227,15 +461,27 @@ bool ResourceManager::loadGeneralMeshData(const AssetInfo& info,
    MeshMetaData metaData;
    loadTextures(*importer, &metaData);
    loadMaterials(*importer, &metaData);
-    loadMeshes(*importer, &metaData);
+    loadMeshes(*importer, &metaData, shiftOrigin);


Is the origin shift needed? That will probably break all of our current episode datasets. We will also have to make sure we propagate the new origin to the annotations for MP3D.

Default shift is false. If shifted, the transform is now stored for query. This is only expected to be used for object meshes, not static scenes.

mosra

Just quickly went through -- please have a look at the reinterpret_casts ;)

src/cmake/FindMagnumIntegration.cmake

src/esp/assets/CMakeLists.txt

src/esp/assets/FRLInstanceMeshData.cpp

src/esp/assets/FRLInstanceMeshData.h

mosra · 2019-07-01T12:34:11Z

src/esp/assets/PhysicsManager.cpp

+  }
+
+  // TODO (JH) Bottom up search gives bus error because scene"s rootnode does
+  // not point to nullptr, but something strange


... or maybe it points to Scene from Magnum's SceneGraph and so static_cast<const scene::SceneNode*> will likely not work?

src/esp/gfx/CMakeLists.txt

…tat-sim into bullet-integration

addressed

…gs, fixed dependency.

src/esp/gfx/Simulator.cpp

mathfac

As soon it's passing CI it looks good to be merged.

mathfac · 2019-07-09T17:09:25Z

src/esp/assets/CMakeLists.txt

    tinyply
  PRIVATE
    Assimp::Assimp
    geo
    io
 )

+set_directory_properties(PROPERTIES CORRADE_USE_PEDANTIC_FLAGS ON)


why we need this?

mathfac · 2019-07-09T17:10:47Z

src/esp/assets/FRLInstanceMeshData.cpp

+    (*cpu_vbo_3_)[i] = cpu_vbo_[i].head<3>();
+  }
+
+  // create ibo converting quads to tris [0, 1, 2, 3] -> [0, 1, 2],[0,2,3]


Suggested change

// create ibo converting quads to tris [0, 1, 2, 3] -> [0, 1, 2],[0,2,3]

// create ibo converting quads to tris [0, 1, 2, 3] -> [0, 1, 2], [0, 2, 3]

mathfac · 2019-09-24T21:45:50Z

examples/example.py

-    "FPS is below regression threshold: %0.1f < %0.1f"
-    % (perf["fps"], args.test_fps_regression)
-)
+    # assert perf["fps"] > args.test_fps_regression, (


@aclegg3, maybe I missed the comment. Can you remind me why we commented out the speed regression check??

Should be there when not running with physics. May have been commented out in the rush to verify physics pybind, sorry for that. We will need a separate threshold for the new physics benchmarks, but we can add this back when enable_physics is false.

@aclegg3, no worries, just wanted to check there is no serious reason to disable it.

* Introduced esp/assets/PhysicsManager to track & manage states of kinematic and physically simulated objects in Habitat-sim. Functionality includes adding/removing objects, kinematic updates, applying forces/torques, getters/setters for physical properties. * Introduced esp/physics/BulletObject as an extended SceneNode. BulletObject interacts with PhysicsManager to create collision mesh, set object properties, and interface with specific physics engine implementations. * Introduced global and object specific physics config files (JSON), parsers, containers, and resource libraries in esp/assets/ResourceManager. * Introduced dynamic simulation of rigid objects through optional build with Bullet physics engine (build argument --bullet with Bullet3 installed). * Implemented python bindings for basic physics functionality exposed through simulator.py. * Implemented physics functionality demos in C++ (Viewer.cpp) and python (example.py).

Jerry He and others added 18 commits June 3, 2019 15:27

Object loading minimal demo v0.1: loading cheezit box into skokloster…

e4e6ad4

… castle & teleport with key

Merge branch 'bullet-integration' of github.com:hzyjerry/habitat-sim

3d647e3

Update physics example with build error

0f6c81b

SceneManager debugging checkpoint: segfault

c6f3709

Refactor physics manager; gravity working; occasional segfault

786081a

Minimal Friday demo; with segfault; position & mass hardcoded

617f25c

Fix segfault issue by refactoring

06fe5a1

Collision working for GIC demo; use simplified collision shape (box &…

ec445a1

… slates)

Surreal apt checkpoint (not working)

b97ad69

Static mesh collision working (applied correct transform to collision…

236ea39

… mesh)

Convex hull collision shape working; middle point as center of mass

6f9d891

Segfault debugging checkpoint

cc1a6b2

Erik's fixes

1d6a215

Merge pull request #1 from facebookresearch/master

831f7cd

Pull from fb master

Resolve merge conflict

7406de7

Resolve merge conflicts

40c5fb3

Resolve merge conflict to build

04258e7

Clean up BulletObject and Physics Manager interface (with Corrade bui…

df3b311

…ld error)

facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Jun 28, 2019

mathfac requested review from msavva, mathfac, erikwijmans, mosra and abhiskk June 28, 2019 17:37

abhiskk requested a review from bigbike June 28, 2019 18:49

erikwijmans reviewed Jun 28, 2019

View reviewed changes

mosra reviewed Jul 1, 2019

View reviewed changes

Jerry He added 3 commits July 1, 2019 13:46

Fix meshData interface issue, compile ready

7e64cd8

Update

dcb7697

Merge branch 'master' of https://github.com/facebookresearch/habitat-sim

e9a44d5

aclegg3 added 5 commits August 12, 2019 15:23

Setup pysics interface in Simulator class for pybinding.

06d95d4

Merge branch 'bullet-integration' of https://github.com/hzyjerry/habi…

913b98f

…tat-sim into bullet-integration

isort change

75febd5

Fix reference to optional setting bug

3968ca6

Merge 'master' changes into bullet-integration

2cbba91

aclegg3 added 2 commits August 15, 2019 20:04

Added physics profiling to example.py, updated readme, added pybindin…

ba8b915

…gs, fixed dependency.

Remove DebugTools

61e84e9

mosra reviewed Aug 16, 2019

View reviewed changes

src/esp/gfx/Simulator.cpp Show resolved Hide resolved

aclegg3 added 4 commits August 16, 2019 08:24

Fixed Simulator applyTorque copy paste bug.

0c7b8e7

Merge remote-tracking branch 'public/master' into bullet-integration

3dc693e

Merge remote-tracking branch 'public/master' into bullet-integration

dcda359

merge and deps/magnum,corrade

3245e7a

mathfac approved these changes Aug 16, 2019

View reviewed changes

aclegg3 merged commit 9575dcd into facebookresearch:master Aug 16, 2019

erikwijmans mentioned this pull request Aug 25, 2019

Get rid of the warnings added in PR91 #173

Merged

mosra mentioned this pull request Aug 27, 2019

Lots of warnings and all warnings as errors for esp code #174

Merged

11 tasks

mathfac added this to 2019 Q3 in Habitat Roadmap Sep 5, 2019

mathfac moved this from 2019 Q3 to 2019 Q4 in Habitat Roadmap Sep 5, 2019

mathfac removed this from 2019 Q4 in Habitat Roadmap Sep 5, 2019

mathfac deleted the bullet-integration branch September 5, 2019 22:24

mathfac added this to the Bullet-based rigid body physics milestone Sep 5, 2019

aclegg3 mentioned this pull request Sep 6, 2019

Physics namespace docs and resulting code cleanup. #195

Merged

11 tasks

This was referenced Sep 13, 2019

Physics tests: kinematics #211

Merged

Physics benchmarking #227

Merged

mathfac reviewed Sep 24, 2019

View reviewed changes

aclegg3 mentioned this pull request Oct 16, 2019

PhysicsManager CHECK cleanup #305

Merged

11 tasks

eundersander pushed a commit to eundersander/habitat-sim that referenced this pull request Aug 6, 2020

Account for paused envs in num_envs (facebookresearch#91)

60653f7

ChanganVR mentioned this pull request May 6, 2022

[Audio] - importing habitat_sim crashes without first importing quaternion #1747

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bullet Physics Integration #1 #91

Bullet Physics Integration #1 #91

hzyjerry commented Jun 28, 2019 •

edited

erikwijmans left a comment

erikwijmans Jun 28, 2019

aclegg3 Aug 12, 2019

mosra left a comment

mosra Jul 1, 2019

mathfac left a comment

mathfac Jul 9, 2019

mathfac Jul 9, 2019

mathfac Sep 24, 2019

aclegg3 Sep 24, 2019

mathfac Sep 25, 2019

	// create ibo converting quads to tris [0, 1, 2, 3] -> [0, 1, 2],[0,2,3]
	// create ibo converting quads to tris [0, 1, 2, 3] -> [0, 1, 2], [0, 2, 3]

Bullet Physics Integration #1 #91

Bullet Physics Integration #1 #91

Conversation

hzyjerry commented Jun 28, 2019 • edited

Motivation and Context

Types of changes

Challenges 💥

Checklist

PR Milestones

erikwijmans left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mosra left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mathfac left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hzyjerry commented Jun 28, 2019 •

edited