Function to get a humanoid's bone

[reminjp]

VRM format uses node indices to specify the bone but loaded Scene doesn't hold indices info. So we have to add a new function to get a humanoid's bone by name (string or enum?) or index.

• モデルのボーンマッピング(json.extensions.VRM.humanoid)
• https://threejs.org/docs/#api/objects/Bone
• https://threejs.org/docs/#api/objects/Skeleton

[reminjp]

Creating an array of references to Object3Ds parsed from nodes may be better.

[stonecauldron]

So just to be make sure I am not mistaken here, in the vrm.humanoid.humanbones array we have objects with the following properties.

{
  "bone": "leftUpperLeg",
  "node": 5,
  "useDefaultValues": true
}

The node value represents the index of the node containing the designated joint; this index is in relation to the nodes array of the glTF file. So in order to get the leftUpperLeg we need to look up the third node in the glTF file. Is the reasoning above correct?

---

I've looked at the vrm.scene object and I am not entirely sure how to link the VRMHumanBone objects with the actual meshes that are on the three.js scene, the names of the scene objects do not seem to match those of the VRMHumanBone objects.

So it seems like you said that getting the references directly from the parsed nodes might be the better option here.

Is this the way to get the flattened hierarchy of nodes from the glTF file?
https://github.com/rdrgn/three-vrm/blob/c6650a700d9374a724fc043b2866731490a3858a/src/data/VRM.ts#L123

[reminjp]

Is the reasoning above correct?

That's correct. I want to get a THREE.Object3D in vrm.scene by node index of glTF (e.g. vrm.humanoid.humanbones[i].node). However, parsed vrm.scene don't have any node info. Please read mrdoob/three.js#14739 .

Currently I'm using GLTFLoader to load VRM. Basically GLTFLoader builds Object3D hierarchy according to glTF node hierarchy. However, the loader sometimes appends multiple Object3Ds for a single node, to reproduce glTF model in the three.js world.

I have worked on this issue in local. Using name is one of possible solutions. (I just pushed the following. WIP.)

https://github.com/rdrgn/three-vrm/blob/310227822cd360cdc074e6dc06ac0d487feae4e4/src/data/VRM.ts#L118-L124

I've been busy recently. 😢

[stonecauldron]

Perfect, thank you for taking the time to write this detailed reply, the issue you've linked in particular is very helpful.

I did not see how to use the names to link the nodes to 3d objects but I was confusing the names from the vrm.humanoid.humanbones[i].bone and the objects in the scene themselves.

I imagine the names in vrm.humanoid.humanbones[i].bone are part of the VRM standard and have no incidence on the names of the 3d objects in the scene?

I've been busy recently. 😢

The code you pushed looks good, if you think I can help out with this issue in any way do not hesitate to tell me. I would have no objection in submitting a PR for example, but only if you don't prefer doing it yourself of course.

In any case, I am not blocked in my work by this issue so no worries 😉

[reminjp]

I imagine the names in vrm.humanoid.humanbones[i].bone are part of the VRM standard and have no incidence on the names of the 3d objects in the scene?

Yes. vrm.humanoid.humanbones[i].bone is a string defined in VRM specifications. It is used to standardize bones in glTF models. Bone's actual name of the left hand, for example, varies depending on the 3D model: LeftHand, Hand_L, 左手, etc.

Currently, actual name of the bone (that is node of glTF) is at

vrm.parser.json.nodes[vrm.humanoid.humanbones[i].node].name

---

The name of Object3D in the vrm.scene is basically same as the name of its original node. However, this depends on the implementation of GLTFLoader. That's why I used loadNode(i) there:

https://github.com/rdrgn/three-vrm/blob/310227822cd360cdc074e6dc06ac0d487feae4e4/src/data/VRM.ts#L121-L124

I want to map nodes more smartly...

---

In any case, I am not blocked in my work by this issue so no worries 😉

😄

[stonecauldron]

I see, thank you for your in-depth reply 👍

One thing I don't understand is why we cannot write this directly:

this.nodes[i] = await this.parser.loadNode(i);

Is it because there is no guarantee that loadNode will give us an Object3D object?

[reminjp]

loadNode returns a newly constructed Object3D. (But it doesn't have children and a parent.)

I thought I needed to get a reference to the Object3D in the scene to apply a motion to the avatar (scene).

I was concerned about the performance of loadNode... However, it seems to be no problem at the moment because GLTFLoader caches dependencies.

If GLTFLoader is updated, I will change the implementation.