SkinnedMesh: Incorrect bounding box and culling

[soadzoor]

Description of the problem

Hi,

I've been playing around with a model for a while, and noticed that if I load it in three.js and zoom closely to the head, it disappears, along with the helmet.

I'm sure it's not the clipping field, because they disappear suddenly, from one frame to another.

I'm also sure this bug is not limited to @donmccurdy 's gltf viewer ( https://gltf-viewer.donmccurdy.com/ ), as I tested it in a project made from scratch recently, and the bug appears there, too.

It works flawlessly in the babylon sandbox: http://sandbox.babylonjs.com/, so I believe this has something to do with three.js.

Also, I think it has something to do with the fact that the mesh is skinned. If I apply the armature pose in blender before exporting, then the model appears correctly in three.js as well.

I attach a zip with 2 model files, one with skin, the other one with the applied pose:
player.zip

Three.js:

Babylon.js:

Three.js version

• Dev
• r94
• ...

Browser

• All of them
• Chrome
• Firefox
• Internet Explorer

OS

• All of them
• Windows
• macOS
• Linux
• Android
• iOS

[looeee]

Tested it in my loader and it's working fine. So whatever the issue is, it's not a three.js bug.

EDIT: oops, spoke too soon. It's possible to rotate and zoom the model so that the visor or helmet disappear. However from other angles I can zoom right into the interior of the head without clipping occurring.

EDIT EDIT: (sorry should have tested more thoroughly before I started typing 😆 ) It seems the clipping is occurring with one file but not the other - the one with the pose applied displays correctly. @soadzoor that's what you were describing as well, right?

[soadzoor]

Yes, right. I wouldn't call it "clipping" though, but "disappearing", since the whole "mesh" disappears suddenly.

I also had the head and the helmet joined together with the rest of the body, so they would be one single mesh, but that didn't really make any difference, probably because it has a different material, so it becomes a different mesh in three.js. As far as I know, in glTF, it always becomes a separate mesh, when it has different materials, at least in three.js

[donmccurdy]

As far as I know, in glTF, it always becomes a separate mesh, when it has different materials, at least in three.js

This depends on how the exporter creates the file — three.js does support multi-material meshes, but the exporter might not write data that way, or there might be some technical reason the exporter indicates a multi-material mesh but three.js has to split it (e.g. we can't have one mesh containing both points and triangles).

[WestLangley]

Try mesh.frustumCulled = false. Check the bounding boxes. Skinning can translate the geometry.

[soadzoor]

@WestLangley yes, frustumCulled = false works, I can confirm. But I don't understand. Skinning can translate the geometry, that's ok, but the bounding boxes should be translated accordingly, shouldn't they?

[WestLangley]

Bounding boxes exist only on the CPU, and that is where the frustum culling is occurring. Skinning occurs on the GPU.

So in your case, either the skinned vertices are translated away from the bounding box, or the bounding box is not correct for some reason. See if you can track that down.

[soadzoor]

I see. Thanks!

[soadzoor]

Yea, it seems like the skinned vertices are translated away from the bounding box:

So I guess there's no "easy" solution for this, but to recalculate the position of the bounding box manually, or set frustumCulled = false, right? I mean, I'm not sure if it's worth the processing time of recalculating the bounding box, when the model is animated, for example.. Or..?

[WestLangley]

You can enlarge the head's bound box, manually, or perhaps set it to be the union of the bounding boxes of the head and body.

boundingBox.union( boundingBox2 );

Whatever you do, do it just once, not every frame.

[soadzoor]

@WestLangley thanks for the tip, seem like a pretty nice workaround. I'll close the thread I guess

[looeee]

I've been testing this in quite a few models, and it seems like every skinned model that consists of multiple meshes exhibits this behaviour. Testing the same models in other system (Babylon.js, FBX review,sketchfab), they display correctly. So I think we do need to consider this a three.js bug.

Here's a video of the most obvious example I could find (model was shared in #14520). Kind of NSFW I guess.
skin_bug.zip

[benolayinka]

Has there been any progress on this? I have a scene with multiple animated characters, and the bounding boxes for the skinned meshes are way off the models.

Maybe unrelated, but I also noticed that the way the animated models are created in Blender, I end up with an "Object3D" parent that has a Skeleton child and a SkinnedMesh child. SkinnedMesh expects to have a Skeleton child, but my animations are working so maybe this doesn't matter anyway. I only noticed because I can't add a SkeletonHelper to the SkinnedMesh - I have to create it from the Parent.

[donmccurdy]

I also noticed that the way the animated models are created in Blender, I end up with an "Object3D" parent that has a Skeleton child and a SkinnedMesh child.

This is just how the Blender exporter creates models.

SkinnedMesh expects to have a Skeleton child, but my animations are working so maybe this doesn't matter anyway.

SkinnedMesh doesn't depend on its bones being children; it is common but not a requirement.

I only noticed because I can't add a SkeletonHelper to the SkinnedMesh - I have to create it from the Parent.

I've been doing:

const helper = new SkeletonHelper( mesh.skeleton.bones[ 0 ].parent );

This should work regardless of whether the mesh is the parent or not. Perhaps we could set up SkeletonHelper to figure this out automatically. :)

[iar5]

Using three^0.124.0 and Blender 2.9
In my case the BoundigBox was calculated correctly but the values get scaled by its parent resulting in a too small BoundigBox. Same thing with the rotation. In case someone else is running into this problem I want to share my workaround to fix this. Btw: using Blender 2.8, I did not have the bounding box issue but some problems with importing/exporting animation themselves (maybe a similar scaling problem but at a different place)

let mesh = gltf.scene.children[0] 
let geometry = mesh.children[1].geometry
let m = new THREE.Matrix4()
m.makeRotationX(THREE.MathUtils.degToRad(90))
m.scale(new THREE.Vector3(100, 100, 100))
geo.boundingBox.applyMatrix4(m)
geo.boundingSphere.applyMatrix4(m)

[hybridherbst]

I don't have control over threejs in most cases, e.g. when using model-viewer or other existing viewers, so code workarounds are not possible.

For reference, @mrdoob had mentioned here

• Resting Bounds for skinned meshes are calculated without taking initial bind pose into account google/model-viewer#3189 (comment)

that he came up with an efficient solution; but I'm not sure that's already implemented anywhere?

[Mugen87]

Related #21507.

[puxiao]

I encountered the same problem:

1. Some skinnedMesh will disappear inexplicably when orbitControls changes.

   Letting all skinnedMesh .frustumCulled = false is the only solution found that works so far.

2. Unable to correctly obtain Box3 of gltf.scene

   I tried traversing and merging the boundingBox of all skinnedMesh, but the result was not as expected

Is there currently a correct solution?

[Mugen87]

Unable to correctly obtain Box3 of gltf.scene

Make sure you call gltf.scene.updateMatrixWorld() before computing the bounding box. It does not work without up-to-date world matrices.

[puxiao]

Unable to correctly obtain Box3 of gltf.scene

Make sure you call gltf.scene.updateMatrixWorld() before computing the bounding box. It does not work without up-to-date world matrices.

It really works for me. Thanks.

[Mugen87]

There is actually issues tracking the missing world matrix computation in Box3: #21404

I have tried to fix this issue in #21411 since developers frequently report incorrect bounding volumes but unfortunately the PR did not get enough support.

@donmccurdy I have noticed that many users struggle with Box3 when they compute bounding volumes in the onLoad() callback of GLTFLoader for gltf.scene. We could mitigate these issues by calling gltf.scene.updateMatrixWorld() inside the loader right before the onLoad() callback fires. If we do not update matrices in Box3.setFromObject(), can we at least ensure computed world matrices in GLTFLoader?

[donmccurdy]

@Mugen87 if we can do scene.updateMatrixWorld() just once inside the loader before returning the data, that does sound OK to me!