Unity Character Rig Avatar and Avatar Mask creation

[diegodelarocha]

Not sure if this is possible with the GLTF description, but It would be quite useful if the GLTF importer had a feature to add an Character Avatar like the FBX importer has, so that we can use the animation states with Avatar Masks on the Unity Animation Controller 🤓

Here's an image of the FBX importer tab where it deals with creating an Avatar for the imported character's hierarchy of bones.

[diegodelarocha]

The code on the second comment of this thread, does exactly that 🤓, but it would be obviously nice to have this option on the importer from the get go for users, that might not know how to do this? It took me a full day to figure out this, that would've been super appreciated if it came included on the importer <3

[Siccity]

I added a branch with avatar creation. I don't have much experience with Avatars, so please check it out and tell me if it works correctly for you :)
https://github.com/Siccity/GLTFUtility/tree/feature/avatars

[diegodelarocha]

Good day Siccity ^^ Took a look at it and tested with a bunch of characters including the cactus from the PR file.

I think you're running into the same issue I was running with, where your importer is creating an Avatar for the character, but the actual Avatar doesn't carry all the hierarchy definitions of the bones/object hierarchy. Hence when you create an Avatar Mask (right click on Project folders>Create>AvatarMask) and you assign the imported Avatar from the GLTF imported character, it just spits out a blank skeleton.

Here's an example with the same cactus one with GLTFUtility, and the other one with Blender's FBX importer:

No skeleton/hierarchy found Using GLTFUtility's Avatar

Expected skeleton/hierarchy found Using Blender's FBX importer Avatar

So far the only way I was able to solve that, was by using this code referenced on this issue on our fork of the Utility. But it's an Editor command, though maybe there's a way to include that on import level of the Utility, so it assigns the skeleton of the avatar into the Avatar Mask directly, then the user could just duplicate the included Avatar Mask if more than one was required?

The main reason why you'd want to use Avatars is either to be able to add partial animation states on top of an existing anim clip. (ie: you have a running state, and you add an avatar mask on the face, to add an extra animation clip running at the samee time with some head animation so your character looks to the left)

Another use which I'm not quite experienced with either, is re-targeting animation transform data to other rigs with the same avatar, but I wouldn't know how to test that 😭

[Siccity]

I tried by myself, then tried finding help on google. I asked on Unity Answers, Unity Forums, and some different discord servers. Hopefully someone in this world knows how to do this :V

[diegodelarocha]

It feels like something without too much documentation available since it's embeded on the FBX exporter, and its just expected to always work? 😭

[Siccity]

Alright a helpful person on the GDL discord pointed this bit of code out to me. I think this might be it.

The problem is not generating the Avatar, it's showing it in the avatar mask. Apparently, the class in charge of showing the data does a check to see if the imported model's importer derives from ModelImporter. In this case, GLTFImporter does not derive from ModelImporter, but derives from ScriptedImporter instead. Perhaps it's just a matter of switching out the base class. Will check it out tomorrow.

https://github.com/Unity-Technologies/UnityCsReference/blob/master/Editor/Mono/Inspector/AvatarMaskInspector.cs

[diegodelarocha]

Interesting 🤓😮! That makes sense?, I'm hoping it works out 🤓 🤞 !

[Siccity]

Tested a bit. It seems like deriving from ModelImporter is gonna be a challenge, as the class has no overrides and Unity crashes whenever I try anything with it. I'm guessing it's deeply rooted in the c++ side that handles natively supported formats, and I don't know if there is any way to trick Unity into thinking of .gltf as a natively supported format.

[diegodelarocha]

Oh my, yeh that's going in deep, eh? Well for now we have the Editor way of making it at least, but yeh, maybe on import it's not supported?

[jdselig]

Hey guys, sorry to bump an old thread, but I'm struggling to get humanoid animations working on a GLTF character. I imported it into Unity from Sketchfab, but I'm not seeing a way to set up a humanoid avatar for reuse. Is this possible? Am I just missing something? I wasn't sure if this was still being worked on or not, so I figured this was maybe the right place to check. Thanks so much for all the work on this!

[diegodelarocha]

Hi @jdselig, we are still struggling with this I believe, buuuut, you can always just add the logic referenced on this link and that'd create an Editor tool for you to create avatars off them, hope this helps 🤓 .

[jdselig]

ah okay, i'll try that. thanks so much!

[johanhelsing]

Has anyone got this working with humanoid characters? I tried the linked script, but it's just for generic avatars it seems. I tried changing to BuildHumanAvatar, but I don't understand how to use it. It's always complaining about missing hips :/

[johanhelsing]

I kind of figured it out... I opened the avatars generated for the same model when imported with FBX, and found a section like this:

  m_HumanDescription:
    serializedVersion: 3
    m_Human:
    - m_BoneName: DEF-spine
      m_HumanName: Hips
      m_Limit:
        m_Min: {x: 0, y: 0, z: 0}
        m_Max: {x: 0, y: 0, z: 0}
        m_Value: {x: 0, y: 0, z: 0}
        m_Length: 0
        m_Modified: 0
    - m_BoneName: DEF-thigh.L
      m_HumanName: LeftUpperLeg
      m_Limit:
        m_Min: {x: 0, y: 0, z: 0}
        m_Max: {x: 0, y: 0, z: 0}
        m_Value: {x: 0, y: 0, z: 0}
        m_Length: 0
        m_Modified: 0
        ...

I then wrote a script to turn this into a C# dictionary, which I used to generate the entries HumanDescription.human.

Resulting script:

        [MenuItem("CustomTools/MakeHumanAvatar")]
        private static void MakeHumanAvatar()
        {
            var activeGameObject = Selection.activeGameObject;

            if (activeGameObject == null) return;

            var boneMapping = new Dictionary<string, string>
            {
                {"Hips", "DEF-spine"},
                {"LeftUpperLeg", "DEF-thigh.L"},
                {"RightUpperLeg", "DEF-thigh.R"},
                {"LeftLowerLeg", "DEF-shin.L"},
                {"RightLowerLeg", "DEF-shin.R"},
                {"LeftFoot", "DEF-foot.L"},
                {"RightFoot", "DEF-foot.R"},
                {"Spine", "DEF-spine.001"},
                {"Chest", "DEF-spine.002"},
                {"Neck", "DEF-neck"},
                {"Head", "DEF-head"},
                {"LeftShoulder", "DEF-shoulder.L"},
                {"RightShoulder", "DEF-shoulder.R"},
                {"LeftUpperArm", "DEF-upper_arm.L"},
                {"RightUpperArm", "DEF-upper_arm.R"},
                {"LeftLowerArm", "DEF-forearm.L"},
                {"RightLowerArm", "DEF-forearm.R"},
                {"LeftHand", "DEF-hand.L"},
                {"RightHand", "DEF-hand.R"},
                {"LeftToes", "DEF-toe.L"},
                {"RightToes", "DEF-toe.R"},
                {"UpperChest", "DEF-spine.003"},
            };

            var humanDescription = new HumanDescription
            {
                human = boneMapping.Select(mapping =>
                {
                    var bone = new HumanBone {humanName = mapping.Key, boneName = mapping.Value};
                    bone.limit.useDefaultValues = true;
                    return bone;
                }).ToArray(),
            };

            var avatar = AvatarBuilder.BuildHumanAvatar(activeGameObject, humanDescription);
            avatar.name = activeGameObject.name;

            if (!avatar.isValid)
            {
                Debug.LogError("Invalid avatar");
                return;
            }

            Debug.Log(avatar.isHuman ? "is human" : "is generic");

            var path = $"Assets/{avatar.name.Replace(':', '_')}.ht";
            AssetDatabase.CreateAsset(avatar, path);
        }

This was for the basic human generated by blender, might be different for you.

Some weird stuff happens with offsets for rotation and positions, though, but hopefully this will help someone else :)

EDIT: The rotation was due to something else. Seems to work perfectly for basic blender humans

[yh418807968]

I kind of figured it out... I opened the avatars generated for the same model when imported with FBX, and found a section like this:

  m_HumanDescription:
    serializedVersion: 3
    m_Human:
    - m_BoneName: DEF-spine
      m_HumanName: Hips
      m_Limit:
        m_Min: {x: 0, y: 0, z: 0}
        m_Max: {x: 0, y: 0, z: 0}
        m_Value: {x: 0, y: 0, z: 0}
        m_Length: 0
        m_Modified: 0
    - m_BoneName: DEF-thigh.L
      m_HumanName: LeftUpperLeg
      m_Limit:
        m_Min: {x: 0, y: 0, z: 0}
        m_Max: {x: 0, y: 0, z: 0}
        m_Value: {x: 0, y: 0, z: 0}
        m_Length: 0
        m_Modified: 0
        ...

I then wrote a script to turn this into a C# dictionary, which I used to generate the entries HumanDescription.human.

Resulting script:

        [MenuItem("CustomTools/MakeHumanAvatar")]
        private static void MakeHumanAvatar()
        {
            var activeGameObject = Selection.activeGameObject;

            if (activeGameObject == null) return;

            var boneMapping = new Dictionary<string, string>
            {
                {"Hips", "DEF-spine"},
                {"LeftUpperLeg", "DEF-thigh.L"},
                {"RightUpperLeg", "DEF-thigh.R"},
                {"LeftLowerLeg", "DEF-shin.L"},
                {"RightLowerLeg", "DEF-shin.R"},
                {"LeftFoot", "DEF-foot.L"},
                {"RightFoot", "DEF-foot.R"},
                {"Spine", "DEF-spine.001"},
                {"Chest", "DEF-spine.002"},
                {"Neck", "DEF-neck"},
                {"Head", "DEF-head"},
                {"LeftShoulder", "DEF-shoulder.L"},
                {"RightShoulder", "DEF-shoulder.R"},
                {"LeftUpperArm", "DEF-upper_arm.L"},
                {"RightUpperArm", "DEF-upper_arm.R"},
                {"LeftLowerArm", "DEF-forearm.L"},
                {"RightLowerArm", "DEF-forearm.R"},
                {"LeftHand", "DEF-hand.L"},
                {"RightHand", "DEF-hand.R"},
                {"LeftToes", "DEF-toe.L"},
                {"RightToes", "DEF-toe.R"},
                {"UpperChest", "DEF-spine.003"},
            };

            var humanDescription = new HumanDescription
            {
                human = boneMapping.Select(mapping =>
                {
                    var bone = new HumanBone {humanName = mapping.Key, boneName = mapping.Value};
                    bone.limit.useDefaultValues = true;
                    return bone;
                }).ToArray(),
            };

            var avatar = AvatarBuilder.BuildHumanAvatar(activeGameObject, humanDescription);
            avatar.name = activeGameObject.name;

            if (!avatar.isValid)
            {
                Debug.LogError("Invalid avatar");
                return;
            }

            Debug.Log(avatar.isHuman ? "is human" : "is generic");

            var path = $"Assets/{avatar.name.Replace(':', '_')}.ht";
            AssetDatabase.CreateAsset(avatar, path);
        }

This was for the basic human generated by blender, might be different for you.

Some weird stuff happens with offsets for rotation and positions, though, but hopefully this will help someone else :)

EDIT: The rotation was due to something else. Seems to work perfectly for basic blender humans

It can work, Thank you!

[haosizheng]

@johanhelsing @yh418807968

---

do not know why.