Using Moetion pose output in Unity

[BrandonBartram98]

Hello, I finally got around to attempting to convert your solver to work in Unity. However I am at a bit of a wall now and I do not know how to correctly apply the outputted Vector3 rotations to the 3D rig.

For example I have a solver script calculating the arms then adding these to a pose struct. I then attempt to apply the pose to the 3D models bones. See the example below where I give the NormalizedLandmarkList to my solve function and return the pose. I then attempt to apply the rotations by setting the bone euler rotations to them. From the result im getting this is clearly not correct so if you could provide any guidance that would be great thanks.

        var pose = _poseSolver.Solve(list);

        LeftHand.eulerAngles = pose.LeftArm.Hand;
        RightHand.eulerAngles = pose.RightArm.Hand;

        LeftLowerArm.eulerAngles = pose.LeftArm.Lower;
        RightLowerArm.eulerAngles = pose.RightArm.Lower;

        LeftUpperArm.eulerAngles = pose.LeftArm.Upper;
        RightUpperArm.eulerAngles = pose.RightArm.Upper;

[BrandonBartram98]

To provide some extra context, I have now modified how I set the bone rotations to the same way its done in KalidoKit but get the same results (twitching arms that barely move). I feel like I am missing something I need to do before applying the rotation to the bone.

private void SetPose(NormalizedLandmarkList list)
{
    if (list != null)
    {
        var pose = _poseSolver.Solve(list);

        SetEuler(LeftHand, pose.LeftArm.Hand, 1f, 0.3f);
        SetEuler(RightHand, pose.RightArm.Hand, 1f, 0.3f);

        SetEuler(LeftLowerArm, pose.LeftArm.Lower, 1f, 0.3f);
        SetEuler(LeftUpperArm, pose.LeftArm.Upper, 1f, 0.3f);

        SetEuler(RightLowerArm, pose.RightArm.Lower, 1f, 0.3f);
        SetEuler(RightUpperArm, pose.RightArm.Upper, 1f, 0.3f);
    }
}

private void SetEuler(Transform rigPart, Vector3 target, float dampener = 1, float lerpAmount = 0.3f)
{
    Vector3 euler = new(target.x * dampener, target.y * dampener, target.z * dampener);

    Quaternion quat = Quaternion.Euler(euler);

    rigPart.rotation = Quaternion.Slerp(rigPart.rotation, quat, lerpAmount);
}

[Speykious]

"twitching arms that barely move"? Are you sure it's not a problem of angle units (degrees vs radians)?
The NormalizedLandmarkList has euler angles in radians.
Apart from that, I'm not sure I can help...

[BrandonBartram98]

I think you are correct, I need to convert the radians to degrees.

Appreciate the reply.

[sr229]

Marking this resolved, but I'll rename the issue so it's more discoverable for those that might have the same problem.