Real-Time Mesh Skinning with Direct Delta Mush (Unity)

University of Pennsylvania, CIS 565: GPU Programming and Architecture, Final Project

• Xuntong Liang
  • xuntong@seas.upenn.edu
  • LinkedIn
  • GitHub
  • twitter
• Bowen Deng
  • dengbw@seas.upenn.edu
  • LinkedIn
• Beini Gu
  • gubeini@seas.upenn.edu
  • LinkedIn
  • personal website
  • twitter

Project Description

This is a Unity project implementing real-time mesh skinning using GPU-based Direct Delta Mush algorithm. This algorithm and its variants enable us to compute mesh skinning and animations with efficiency and quality, even with simply authored skinned meshes.

With Direct Delta Mush (left), we get less bulging effect than what we get with built-in skinning (right).

We can get smooth skinning result even with rigid binding.

How to Build

Requirement

• Unity 2020.3.13
• Visual Studio 2019

Build

Add MeshDeformUnity to Unity Hub and select unity version. Then you can open this project.

Features Overview

• Delta Mush with CPU or GPU.
• Direct Delta Mush with GPU (from variant 0 to variant 4).
• Precomputation of Direct Delta Mush with GPU.

0 iter (LBS)	2 iters	4 iters	8 iters	16 iters

The table above shows how the number of iterations affect the visual effect of skinning. With more and more iterations, the elbow shows smoother, and less bulging effect.

Variants of Direct Delta Mush

The paper also shows some variants which are equivalent to special cases of several previous skinning algorithms.

• The variant 0 is the full DDM model.
• The variant 1 is an approximation by taking the inverse transpose followed by determinant normalization. The visual effect is equivalent to the implementation of the original DM. However, the rotation matrix is approximated, so the deformation is distorted.
• The variant 2 and variant 3 use less precomputation data, directly computing the rotation matrix by blending the rotation components of bone transformations, but those representations cannot propagate changes from bone translations to the local skin rotation. For DDM v2, the rotation matrix is blended in quaternion space, and for DDM v3, the rotation is blended in linear space.
• The variant 4 makes the translation blended directly, based on the variant 2, thus using even less precomputation data. In some cases, it is equal to skinning with optimized centers of rotation (CoR).
• The practical application of the variant 5 is replacing the skinning weight solver using the original DM as the training data. While we are using the variant 5, we are actually using the linear blend skinning in runtime, so we don't implement the variant 5.

Keep in mind that using less precomputation data doesn't mean the performance would be improved. The visual effect of different variants are shown below. For DDM v0 and DDM v2, there are less collapsing artifact, and v0 keeps the shape mostly, thus performs the best.

LBS	v0	v1

v2	v3	v4

See technical notes for technical details.

How to Use

1. Load models, and toggle Read/Write Enabled.

   

2. Drag the model into the scene, or select the object with this model in the scene.

   

3. Expand and find the mesh object of the model. Add component to the mesh object. Make sure that there is a Skinned Mesh Render component in this object.

   

4. Take a look at the component DDM skinned Mesh GPU Var 0 for example. There are several attributes.

   • Iterations represents the iteration count of the precomputation.
   • Smooth Lambda determines the smoothing result for each step.
   • Use Compute determines whether you use GPU skinning or CPU skinning, but currently we only implemented GPU skinning for most of the variants.
   • Adjacency Matching Vertex Tolerance can be set with a small positive float number if you need to merge the adjacency data of the vertices which are very close to each other, but enabling this process may cause longer precomputations.
   • Debug Mode is for comparison to the visual effect of the built-in skinning if you assign Compare With Linear Blend to this attribute.

   You can modify Iterations and Smooth Lambda to change the visual effect of the runtime skinning.

   

5. Set Iterations to 30, for example. For this model, set the Adjacency Matching Vertex Tolerance to a positive number to enable vertex matching. Then click the Play button, and switch to the Scene view. Expand the skeleton in the Hierarchy window and you can select which joint to edit.

   

6. Rotate joints to deform mesh.

   

7. If you want to play animation on the mesh, you can create an Animator Controller and set the animation like the figure below. You can also set the speed as you want. Then choose the root of the model, and add component Animator, and set the animator controller mentioned before to the Controller attribute.

   

   

   After you play, you can see the animation. Some of the models can be found at mixamo.

Performance Analysis

Tested on: Windows 10, i7-10750H @ 2.60GHz 16GB, RTX 2070 Super with Max-Q 8192MB

Precomputation

We implemented precomputation of DDM in both CPU and GPU, and compare their performances on Wahoo model with 3809 vertices and 35 bones. Since building the adjacency matrix should be done in CPU, we don't compare this process.

It can be clearly seen that the GPU implementation significantly improves the performance. The runtime of it grows almost linearly as the number of iterations.

On the other hand, we test the GPU based precomputation on different models, with a larger range of iterations.

The figure above shows a nearly linear relation between iterations and the precomputation time, even on models of different sizes and within a larger iteration range. Also, as the number of vertices in the model becomes larger, the performance drops dramatically.

Animation

We test the runtime performance of different variants of DDM, as well as the original DM and the built-in linear blend skinning. These are tested with two models: Wahoo with 66654 vertices and 45 bones, and Ninja with 13560 vertices and 52 bones.

It is clear that even though we only set 10 iterations, the Delta Mush is much costlier than any variants of the Direct Delta Mush. The variants of the DDM performs different, with v0, v1, and v4 faster than v2 and v3. The built-in linear blend skinning is the fastest, which is very straightforward and reasonable.

The presented results may include significant overhead that would need to be investigated with low-level programming, as the paper says in section 4, so they might be much faster if we carefully optimize v2, v3, and v4.

Future Works

• Implement different Laplacian matrix.
• Refer to the next paper in SIGGRAPH 2021.
• Support more models and scaling.
• Optimize implementation for variants.

Credit & Reference

1. Unity
2. Math.NET Numerics
3. Delta Mush: smoothing deformations while preserving detail
4. Direct Delta Mush Skinning and Variants
5. Direct Delta Mush Skinning Compression with Continuous Examples
6. Mixamo

Presentations

1. Pitch
2. Milestone 1
3. Milestone 2
4. Milestone 3
5. Final Presentation
6. Sample Video

Bloopers