WeightedMaskMixer feature.

[tatoforever]

Use Case

Hi, I need a way to properly set Humanoid Avatar masks. Instead of the whole trunk (or the whole head) I need to be able to selectivelly check what humanoid bones can affect the mask. We have a set of custom animations for our characters. Such as In-place reloading/shoting. We do have running, walk, strafing, crouching animations that are mixed with shoting/meleeing and reloading animations. The problem is that those animations are meant to be mixed not from the pelvis (otherwise the characters start shottings while swinging their bodies all over the place very weird), but instead from spine2 bone (the equivalent of the upper chest in Humanoid rig). But the mask doesn't allow us to chose individual bones inside the mask, is either the whole trunk or nothing.
As you suggested in this thread, there's a solution to control individual bones mask and weight.

Solution

Implement WeightedMaskMixer in Animancer

Alternatives

There's currently no other known alternative other than create an explosive combination of animations which forces us to be stuck with clunky unwanted mix of masked animations without fine grain control.

[KybernetikGames]

I don't have much time to develop new features at the moment, but here's a quick implementation of the weighted mixer sample.

Before you can use it you'll need to make the following changes:

// AnimancerPlayable.cs

// Allow the Layers to be set.
// public LayerList Layers { get; private set; }
public LayerList Layers { get; set; }

// AnimancerPlayable.LayerList.cs

// Remove the sealed keyword.
// public sealed class LayerList : ...
public class LayerList : ...

// Add a new constructor:
            /// <summary>Creates a new <see cref="LayerList"/>.</summary>
            protected LayerList(AnimancerPlayable root)
            {
                Root = root;
                _Layers = new AnimancerLayer[DefaultCapacity];
            }

// Add a safety check in IsAdditive so the Inspector doesn't cause exceptions:
            public bool IsAdditive(int index)
            {
#if UNITY_EDITOR
                if (!LayerMixer.IsValid())
                    return false;
#endif
                return LayerMixer.IsLayerAdditive((uint)index);
            }

Then you can import Animancer Weighted Mask Mixer v1.zip (latest version is further down). The actual implementation is in Assets/Plugins/Animancer/Internal/Mixer States/WeightedMaskLayerList.cs and an example of how to use it is in Assets/Experimental.

It only supports 2 layers and has a few other things that might be inconvenient to use, so please do let me know if you make any changes to it.

[tatoforever]

Thanks for the heads up. I will integrate it later tonight and will let you know how it goes.

[KybernetikGames]

Animancer v7.2 is now available with the necessary changes to simply add a weighted mask mixer. But I didn't actually add the mixer since it's not really suitable for general use with its current limitations.

If this gets more interest I'll look into it more.

[baroquedub]

For info, you mentioned that;

Animancer v7.2 is now available with the necessary changes to simply add a weighted mask mixer.

But I found this is still needed to be done:

// AnimancerPlayable.cs

// Allow the Layers to be set.
// public LayerList Layers { get; private set; }
public LayerList Layers { get; set; }

and;

#if UNITY_EDITOR
if (!LayerMixer.IsValid())
return false;
#endif
return LayerMixer.IsLayerAdditive((uint)index);

Although you do have the IsAdditive method. Without the #if UNITY_EDITOR directive I was getting spammed with errors in Editor play mode

[KybernetikGames]

Looks like I have actually made some changes to the Weighted Mask Mixer since I first posted it so here's the new version compatible with Animancer v7.2: Animancer Weighted Mask Mixer v2.zip (latest version is further down)

Exposing the Layers setter publicly was a bad idea because there are several other things that also need to be set for it to work, so instead the base LayerList class has an Activate method for inheriting classes to use.

[baroquedub]

Thanks :) I can confirm that the new import now works out of the box. The WeightedMaskMixerExample script has lost
_Layers.Layer1Weight = _Weight;
(at the end of the script) which I think is useful.

I've added it back in using
_Animancer.Layers[1].Weight = _Weight;

Also the new example scene has a gameObject named 'DefaultHumanoid Regular Layers' which has a missing script on it. Not sure if it was intended as another example?

[KybernetikGames]

Here's a re-upload with the weight field back in and removed that extra character. Not sure how it go in there, it was something completely unrelated. Animancer Weighted Mask Mixer v2.1.zip

[KybernetikGames]

Animancer v7.3 is now available and has those changes in it.

[7Bpencil]

It has required a little bit of fiddling to get what I actually want, but I am impressed that it's even possible in unity. Things that I noticed:

• WeightedMaskLayerList does not like additional transforms in hierarchy (which I use to attach equipment models to character)
• After setup it looked exactly the same as Avatar Mask, so I had to make some modifications

So result looks like this (closest is naive, second is AvatarMask, furthest is customized WightedMaskLayer):

The main idea of my modification is this:

[Serializable]
public struct TransformWeight
{
    public Transform transform;
    public BoneState state;
}

public enum BoneState : byte
{
    UseBaseLayerAnimation,
    UseTopLayerAnimation,
    Boundary,
}

public struct WeightedMaskMixerJob : IAnimationJob
{
    void IAnimationJob.ProcessAnimation(AnimationStream stream)
    {
        var stream0 = stream.GetInputStream(0);
        var stream1 = stream.GetInputStream(1);

        if (stream1.isValid)
        {
            var handleCount = handles.Length;
            for (var i = 0; i < handleCount; i++)
            {
                var handle = handles[i];
                var state = boneWeights[i];
                switch (state)
                {
                    case BoneState.UseTopLayerAnimation:
                    {
                        var positionB = handle.GetLocalPosition(stream1);
                        var rotationB = handle.GetLocalRotation(stream1);
                        handle.SetLocalPosition(stream, positionB);
                        handle.SetLocalRotation(stream, rotationB);
                        break;
                    }
                    case BoneState.UseBaseLayerAnimation:
                    {
                        var positionA = handle.GetLocalPosition(stream0);
                        var rotationA = handle.GetLocalRotation(stream0);
                        handle.SetLocalPosition(stream, positionA);
                        handle.SetLocalRotation(stream, rotationA);
                        break;
                    }
                    case BoneState.Boundary:
                    {
                        var positionA = handle.GetLocalPosition(stream0);
                        var rotationB = handle.GetRotation(stream1);
                        handle.SetLocalPosition(stream, positionA);
                        handle.SetRotation(stream, rotationB);
                        break;
                    }
                }
            }
        }
        else
        {
            var handleCount = handles.Length;
            for (var i = 0; i < handleCount; i++)
            {
                var handle = handles[i];
                handle.SetLocalPosition(stream, handle.GetLocalPosition(stream0));
                handle.SetLocalRotation(stream, handle.GetLocalRotation(stream0));
            }
        }
    }
}

Now I need to figure out how to do transitions, so animations do not pop.
Link to demo in gif if you are interested

[KybernetikGames]

To do smooth transitions you would need to go back to using float weights where 0 is equivalent to your UseBaseLayerAnimation, 1 is equivalent to your UseTopLayerAnimation and other values interpolate between them so you can interpolate the weights between the values you want.

If you still need Boundary mode, then you would need separate floats for position and rotation for each bone, but most animations only animate rotations anyway so there might not be any need for that mode.

I'm considering making an Editor Window for configuring the system similar to the one I made for FlexiMotion where it shows all the Transforms under the character so you can tick the ones you want. Then each column on the right represents a group of weights so you can call SetWeights(2) to apply the weights of group 2 to all bones or FadeWeights(2, 0.25f) to smoothly transition to them. Does that sound like it would meet your needs?

[7Bpencil]

Added transitions, thanks!
Before:

After:

Code:

void IAnimationJob.ProcessAnimation(AnimationStream stream)
{
    var stream0 = stream.GetInputStream(0);
    var stream1 = stream.GetInputStream(1);

    if (stream1.isValid)
    {
        var layerWeight = stream.GetInputWeight(1);
        var handleCount = handles.Length;
        for (var i = 0; i < handleCount; i++)
        {
            var handle = handles[i];
            var state = boneWeights[i];
            switch (state)
            {
                case BoneState.UseTopLayerAnimation:
                {
                    var localPositionA = handle.GetLocalPosition(stream0);
                    var localPositionB = handle.GetLocalPosition(stream1);
                    var localRotationA = handle.GetLocalRotation(stream0);
                    var localRotationB = handle.GetLocalRotation(stream1);

                    handle.SetLocalPosition(stream, Vector3.LerpUnclamped(localPositionA, localPositionB, layerWeight));
                    handle.SetLocalRotation(stream, Quaternion.SlerpUnclamped(localRotationA, localRotationB, layerWeight));
                    break;
                }
                case BoneState.UseBaseLayerAnimation:
                {
                    var localPositionA = handle.GetLocalPosition(stream0);
                    var localRotationA = handle.GetLocalRotation(stream0);

                    handle.SetLocalPosition(stream, localPositionA);
                    handle.SetLocalRotation(stream, localRotationA);
                    break;
                }
                case BoneState.Boundary:
                {
                    var localPositionA = handle.GetLocalPosition(stream0);
                    var localPositionB = handle.GetLocalPosition(stream1);
                    var rotationA = handle.GetRotation(stream0);
                    var rotationB = handle.GetRotation(stream1);

                    handle.SetLocalPosition(stream, Vector3.LerpUnclamped(localPositionA, localPositionB, layerWeight));
                    handle.SetRotation(stream, Quaternion.SlerpUnclamped(rotationA, rotationB, layerWeight));
                    break;
                }
            }
        }
    }
    else
    {
        var handleCount = handles.Length;
        for (var i = 0; i < handleCount; i++)
        {
            var handle = handles[i];
            handle.SetLocalPosition(stream, handle.GetLocalPosition(stream0));
            handle.SetLocalRotation(stream, handle.GetLocalRotation(stream0));
        }
    }
}

In my case boundary node is Spine0, which is right after pelvis. This node's task is to forcefully rotate upper body to match top layer animation, so legs rotation wont have influence on it.

[KybernetikGames]

I see you're using world rotation on the Boundary rather than the local version, I missed that before. It would be good if I can get feedback from some more people to know if that's commonly a useful feature that would be worth trying to standardise somehow.

[7Bpencil]

Night later I realized that it's possible to chim in boundary node correction after WeightedMaskMixerJob did its job:

public struct WeightedMaskMixerJob : IAnimationJob
{
    void IAnimationJob.ProcessAnimation(AnimationStream stream)
    {
        // ...
        // Blending code
        // ...

        ApplyBoundaryNodeCorrection(ref stream, 11);  // hardcoded boundaryNodeIndex
    }

    private void ApplyBoundaryNodeCorrection(ref AnimationStream stream, int boundaryNodeIndex)
    {
        var stream0 = stream.GetInputStream(0);
        var stream1 = stream.GetInputStream(1);

        if (!stream1.isValid)
        {
            return;
        }

        var layerWeight = stream.GetInputWeight(1);
        var handle = handles[boundaryNodeIndex];

        var localPositionA = handle.GetLocalPosition(stream0);
        var localPositionB = handle.GetLocalPosition(stream1);
        var rotationA = handle.GetRotation(stream0);
        var rotationB = handle.GetRotation(stream1);

        handle.SetLocalPosition(stream, Vector3.LerpUnclamped(localPositionA, localPositionB, layerWeight));
        handle.SetRotation(stream, Quaternion.SlerpUnclamped(rotationA, rotationB, layerWeight));
    }
}

Which produces indistinguishable results.
Which also means it should be possible to offload this correction to separate Animation Job:

public class CharacterAnimator : MonoBehaviour
{
    // ...
    [SerializeField] private AnimancerComponent _animancer;
    [SerializeField] private Transform _boundaryBone;
    // ...

    private AnimancerLayer _locomotionLayer;
    private AnimancerLayer _actionLayer;
    private WeightedMaskLayerList _weightedLayers;

    private void Awake()
    {
        _weightedLayers = new WeightedMaskLayerList(_animancer);
        _locomotionLayer = _animancer.Layers[0];
        _actionLayer = _animancer.Layers[1];
        SetupWeights();

        var job = new BoundaryNodeCorrectionJob
        {
            BoundaryNodeHandle = _animancer.Animator.BindStreamTransform(_boundaryBone)
        };
        _animancer.Playable.InsertOutputJob(job);
    }
    
    //...
}

public struct BoundaryNodeCorrectionJob : IAnimationJob
{
    public TransformStreamHandle BoundaryNodeHandle;

    public void ProcessAnimation(AnimationStream stream)
    {
        var stream0 = stream.GetInputStream(0);
        var stream1 = stream.GetInputStream(1);

        if (!stream1.isValid)
        {
            return;
        }

        var layerWeight = stream.GetInputWeight(1);

        var localPositionA = BoundaryNodeHandle.GetLocalPosition(stream0);
        var localPositionB = BoundaryNodeHandle.GetLocalPosition(stream1);
        var rotationA = BoundaryNodeHandle.GetRotation(stream0);
        var rotationB = BoundaryNodeHandle.GetRotation(stream1);

        BoundaryNodeHandle.SetLocalPosition(stream, Vector3.LerpUnclamped(localPositionA, localPositionB, layerWeight));
        BoundaryNodeHandle.SetRotation(stream, Quaternion.SlerpUnclamped(rotationA, rotationB, layerWeight));
    }

    public void ProcessRootMotion(AnimationStream stream) { }
}

The problem is... It doesn't work, stream1 is always invalid. I guess job like this requires more complicated setup. How can I do it?

[KybernetikGames]

The mixer job combines its input streams into a single stream so a second job won't also be able to mix them differently. Best you could do would probably just be to have one struct use the other for the sake of encapsulation.