Question on fine-tuning for face pose evaluation

[FunkyKoki]

As your paper declares, for face pose evaluation, you fine-tune the model on 300W-LP dataset. However, I cannot find the corresponding code in your repository, did I miss it?

[vitoralbiero]

The code to fine-tune on 300W-LP is the same as in here, we just didn't release the annotations.
If you want to fine-tune yourself, you will need to create 300W-LP ground-truth data, load the pre-trained WIDER-FACE weights when training, and fine-tune on 300W-LP.

If you prefer, the fine-tuned pre-trained model can be found here.

[FunkyKoki]

Sorry, you missed the link in the first sentence. I cannot see it.

[vitoralbiero]

Edited.

[FunkyKoki]

So would you release the annotations?

As I see, without fine-tuning, the network performance is very poor.

Performance comparison on AFLW2000:

	yaw	pitch	roll	MAE
without fine-tuning	4.541	8.322	5.586	6.150
with fine-tuning	3.426	5.034	3.278	3.913

[vitoralbiero]

We have no plans on releasing these annotations at this time.
You can use our instructions to annotate it if you would like.

The performance only appears to be poor because of a problem with Euler angles.
This causes samples that have small qualitative and other format error to be computed as a large error.
You can see an example here.

Apart from evaluating with Euler angles, we recommend using the model without fine-tuning.

[FunkyKoki]

Thank you so much. ❤️

[FunkyKoki]

The performance only appears to be poor because of a problem with Euler angles.

So what if we use 'xyz' and 'zxy' to decode the output at the same time, and choose the best as the final result? (I mean, for each face.)

[vitoralbiero]

We could do that, but to be fair with other models compared to, we only use xyz.

[FunkyKoki]

Alright, thank you.

[FunkyKoki]

Happy new year!

As I have tried. Even though I choose the minimum error between xyz and zxy decoding, the redults are not very well as well.

	yaw	pitch	roll	MAE
without fine-tuning	4.541	8.322	5.586	6.150
without fine-tuning (choose minimum of xyz and zxy decoding)	4.830	7.816	5.517	6.054
with fine-tuning	3.426	5.034	3.278	3.913

So, the fine-tuning plays an important role in the pose evalution.

[vitoralbiero]

Thanks, to you too!

Have you also converted the GT to zxy? Just swapping axis won't work, and the errors will be big.
You'll need to do something like:
gt_zxy = Rotation.from_euler("xyz", pose_target[:3], degrees=True).as_euler("zxy", degrees=True)
Also make sure when converting the prediction to Euler angles in zxy format that z and y have a negative sign and x doesn't.

Regardless, the fine-tuning tries to constrain the poses learned to less than 90 degrees, that's why it performs better when tested with Euler angles.

[FunkyKoki]

OK, I retest the model without fine-tuning, the results are as following:

	yaw	pitch	roll	MAE
without fine-tuning	4.541	8.322	5.586	6.150
without fine-tuning (choose minimum of xyz and zxy decoding) corrected	4.751	5.788	3.898	4.812
with fine-tuning	3.426	5.034	3.278	3.913

[FunkyKoki]

I still consider this result as a big gap. 🐰

[vitoralbiero]

If what you care most is pose evaluation in Euler angles, then sure, use the fine-tuned model.