the location WFLW five keypoints of is very worse!

[wuyuanmm]

thanks to this nice work, I try to train autoLink on wflw with five keypoints because of low cuda memory. but I don't get the desired result. Any advice is helpful,this is my result.

this is desired location of keypoints.

[wuyuanmm]

I remove pytorch lightning and transform the code to pure pytorch, and I add resume checkpoints function, I have trained 40 epoches in fact.

[xingzhehe]

Hi wuyuanmm,

The heatmap result and its overlap on images seem fine to me, as it covers the face. I cannot be 100% sure if is good because I need more examples to see if it indeed models the shape variation in the dataset.

Note that it is an unsupervised method. Thus the results would not be the same as human annotation. The model would choose those keypoints that are enough to explain the shape variation in the dataset.

If you want the detailed keypoints and edges as those in the paper, I suggest more keypoints (~16) with smaller edge thickness (~1e-3). If memory is the issue, you could try generate smaller heatmaps (~64x64) and then resize them. You could also try to use smaller batch size (~8) as face is not a difficult dataset.

Finally, if you really want keypoints perform like those human-annotated ones, you could try few-shot learning as describe here: https://xingzhehe.github.io/FewShot3DKP/
where you sample partially from annotated examples and partially from unannotated examples. You could supervise both with AutoLink and supervise those with annotated ones with human annotations.

I hope these could help you. If not, ping me again and we can discuss in details how to deal with low-memory issues.

Bests,
Xingzhe

[wuyuanmm]

thanks for your reply, I solve this issue through setting smaller edge thickness and more keypoints！I will close this issue.