Physics body not updating with changes in bone rotation of mesh?

[dante-biase]

I have a rigged mesh loaded via:

    var hand
    const loader = new THREE.GLTFLoader().load('./hand.glb', function (gltf) {
        const Object3D = gltf.scene.children[0]

        hand = {
            Object3D: Object3D,
            skinned_mesh: Object3D.children[1],
            skeleton: Object3D.children[1].skeleton,
            position: Object3D.position,
            rotation: Object3D.rotation,
            quaternion: Object3D.quaternion,
        }

        const bones = hand.skeleton.bones;
        hand.forearm = bones[0];
        hand.wrist = bones[1];
        hand.palm = bones[2];
        hand.thumb = bones.slice(3, 7);
        hand.index = bones.slice(7, 11);
        hand.middle = bones.slice(11, 15);
        hand.ring = bones.slice(15, 19);
        hand.pinky = bones.slice(19, 23);

        hand.Object3D.traverse(child => {
            if (child.isMesh) {
                child.castShadow = child.receiveShadow = false
                child.material.metalness = 0
                child.material.roughness = 1
            }
        })
 
        scene.add(hand.Object3D)
        scene.add(new THREE.SkeletonHelper(hand.Object3D))
        physics.add.existing(hand.Object3D, { shape: 'convex', collisionFlags: 2 })
    })

it loads as expected into the scene:

However, when I try to rotate an individual bone on the mesh, the mesh and bones rotate as expected, but the physics body does not remain in sync:

hand.palm.rotation.y += Math.PI / 2

Is there a way to achieve this?

---

The documentation says for Kinematic bodies, you need to set needUpdate to true when changing rotation or position. I tried setting that after any rotations or transformations via hand.Object3D.body.needUpdate = true and this does not seem to fix it either.

I also tried adding Object3D to an new ExtendedObject3D() as in the headless GLTF example to see if this would make a difference via

Object3D = new ExtendedObject3D() Object3D.add(gltf.scene.children[0])

This does not work and also did not seem necessary anyways because once hand.Object3D is added to physics, it appears it is automatically converted to an ExtendedObject3D.

I'm not sure what else to try besides removing the physics body and then trying to re-add it frame by frame after any transformations? However, this seems horribly inefficient, there must be an already defined approach for doing this that I'm missing.

[yandeu]

Does the physics apply correctly if you first rotate the palm?

I have never worked with bones this way.
Can you share your hand model?

---

Maybe all it needs is a simple update somewhere. You should also post this question in the three.js forum. I'm sure they know how to work with bones.

[yandeu]

You're welcome :)

You would simply adjust the shapes for each bone. Make it fit as much as possible, and no object will pass through.

Btw, check this out: https://youtu.be/e68pu26skLo 😃

[dante-biase]

That's awesome. Seems like you've added a huge amount of functionality to your library in such a short time!

Hmm, so you're basically saying manipulate the shape of the bones to fill the mesh with these physics bodies instead of wrapping the entire mesh in physics body because thats only way to achieve the ability to rotate the physics body and mesh body in sync. I wish there was a complete solution, but I will not say no to a workaround for now!

Great library overall by the way. I'd be surprised if this does not pick up more traction than it already has in the near future.

[yandeu]

manipulate the shape of the bones to fill the mesh with these physics bodies instead of wrapping the entire mesh in physics body

No, both work.
See https://discourse.threejs.org/t/29612/2
But the softBody code is not yet integrated into the framework. It will be in the next release.

I will make this topic as solved since it will be available soon.

[dante-biase]

great! do you mind updating this discussion when it is? This is a critical part of my application

[AliImrangang]

we are waiting for this release