how did you annotate vertices of 3d-shape?

[linpeisensh]

Thank you for your excellent work!

I have read your paper and have questions: how did you annotate vertices of 3d-shape, e.g. horse?

[nileshkulkarni]

Thanks for your interest!

We manually labeled an approximate location of the vertex corresponding to the key point.

Here are our labels for the few categories we used.

[linpeisensh]

Thank you for your reply!

My expression may make you misunderstood.

Could you tell me, how would you group the vertices of a template shape into functional parts?

What's the meaning of the color in Figure 2? Does it mean that the darker the color is, the closer the assignment to 1?

And is it enough for each part only one transformation? As you can see in the following picture, with a simple rotation, the neck is disconnected with the body.

[nileshkulkarni]

Hi,
I'm sorry for the delayed response.

I used Autodesk mesh-mixer to create vertex groups. Yes, your interpretation of figure 2 is correct.

We found that one transformation per part is sufficient to model the major articulations in quadrupeds. We also perform blending along with applying per-part transformations.

Sample pseudo-code:
Let's assume the horse has 7 parts (4 legs, 1 head, 1 neck, and the body). Also we have a hierarchy between parts.
(Parent --> Child)
Body --> Legs
Body --> Neck
Neck --> Head

1. For every vertex assign an \alpha (1 x 7 vector) that maintains the membership of the vertex to the group. sum(alpha) = 1
2. Create 7 copies of the vertices corresponding to each part and then apply the transformation for the children followed by the transformation of the parent.
3. Combine all the vertices using the \alpha from 1.

I hope this helps!

[linpeisensh]

Thank you for your reply! It really helps.

Another question:
How did you get every vertex assignment?
In my opinion, it's from the distance from some picked vertices and sample it with Gaussian, but it's very hard to divide hind leg and tail.
Do you have any suggestion?

[nileshkulkarni]

Hi,
I used Autodesk MeshMixer to basically label the vertices into different categories. If you have access to that software it is very simple to use it.
In summary, it allows you to manually select the vertices you want to label as that part. This annotation process takes about 15mins per model. I tried using some automatic techniques but it is hard to get them separated. If it helps, here are my parts and the code that can generate the alpha.

Please edit the L559 and run the code as python convert_to_parts.py

horse_parts.obj contains all parts as dis-connected components, and the part_names.txt provides the names for those disconnected components.

Best,
Nilesh

[linpeisensh]

Thank you for your reply!

I will try it.

[linpeisensh]

Hello, I have run your code and get the soften assignment.
But as you can see, the result is still weird.

1.For example, neck rotates pi/6, head rotates -pi/2, for head Tl = pi/6. And head rotates first Tk' then Tl. Am I right?

2.And I also want to know, what about the Tk of rotation center of head in my example? And what is your rotation center for each part?

I look forward to your answer!

[nileshkulkarni]

1. I think you are interpreting it right.

2. The rotation center is the average of all the vertices on the boundary connecting the two parts. In the case of head, consider all the vertices that are connecting the head to the neck and choose the mean vertex as the rotation center about which the y-axis passes and rotations are performed.

Best,
Nilesh

[linpeisensh]

Thank you for your reply!

Another questions:
How can we use this alpha?

1.Does this alpha work on the rotation angle or the whole Tk (simply to rotation) ?

2. Does this sum notation means this vertex rotates 7 times or sum up the weighted rotation matrix and rotates one time?

3. Do we need to check the rotation center every time after a rotation?

Best,
Peisen Lin

[nileshkulkarni]

1. Alpha N_verts x N_parts size matrix. So it basically linearly weighs the location of the same the vertex resulting from different transformations applied to it
2. Yes, you are right. Every vertex is rotated 7 times, and then a weighted average is taken using alpha.
3. No, the rotation center is defined by the base template (since the base template does not change you only need to compute it once)

[linpeisensh]

1. For 3. I first rotate neck pi/6 then head pi/2, the rotation center of head should rotates pi/6 before head rotates, is it right?

2. I want to check again, in my example, head rotates pi/2 then rotates pi/6 with neck, is it right?

[linpeisensh]

3. for your 2., are 7 times rotations from the original vertex position then weighted them and get new position of this vertex?

[nileshkulkarni]

First, you rotate the head by pi/2 and then you rotate the by the necks rotations., yes every vertex is rotated by the transformation of the part (which is T_{k} as in equation 4) remember we do not predict T_{k} directly we predict the local transforms for the parts.

I think in your example pi/2 and pi/6 represent the local transformation for neck and head. So you should compute the global transforms for the head as R_{head_global} = R_{neck}* R_{head}. Similarly, create another set of vertices by rotating using just the R_{neck} and then do a linear combination of them using the alpha.

[linpeisensh]

I am still unclear, should I R_{neck}* R_{head} first or should I get mean of 7 rotation first and then rotate with R_{neck}?

[nileshkulkarni]

Here is the idea. Let's assume we have three parts body, neck, and head.
So T' _neck and T'_head represent local part transforms. Let us assume that T' _body is Identity.

Now let T_head = T' _neck * T' _head represents the global transform, and T_neck = T'_neck since neck's parent has identity transform. T _body = T' _body

With these parts, we can create a new set of vertices as follows

v_head = T_head * v
v_neck = T_head * v
v_body = T_body * v

v_blended = alpha_head * v_head + alpha_neck * v_neck + alpha_body * v_body

[linpeisensh]

Is this vblended the final vertex? Because I get a very weird mesh with this function(neck rotates pi/3):

here is my code:

R = []
for i in range(len(current_corr)): #current_corr rotation center
    part_chain = [i]
    while part_parent[part_chain[0]] != "#":
        part_chain.insert(0, part_parent[part_chain[0]])
    cR = np.eye(3)
    for j in part_chain:
        cR = cR.dot(quaternion_to_rotation(transform[j]["R"]))
    R.append(cR)
R = np.array(R)
for i in range(len(verts)):
    nverts = []
    for k in range(len(current_corr)):
        nvert = np.dot(R[k], verts[i] - current_corr[k]) + current_corr[k] + transform[k]['t']
        nverts.append(ass[i][k] * nvert)
    nverts = np.array(nverts)
    verts[i] = np.sum(nverts, axis=0)
Thank you very much!

[linpeisensh]

I think I need to translate the head after neck rotation, but the rotation center changed and not equal to Rneck.dot(center_head). How can get this new center?

[linpeisensh]

This v = vertex - rotation center of each part, is it right?

[nileshkulkarni]

Yes, If you subtract the rotation center you do have to add it back after you perform the rotation. I'm abstracting out everything into the transform T

[linpeisensh]

so in practice we learn only rotation and translation of each part and get a final vertex like

v_blended = alpha_head * v_head + alpha_neck * v_neck + alpha_body * v_body

v_head = R_head * v + t

this v is vertex that is not subtracted by rotation center. Is it right?

[linpeisensh]

If I am not misunderstood, there is no need to get rotation center of each part? Since you put all into t.

[linpeisensh]

Thank you for your advice. I finally finish this part.