how to compute feature map variances?

[LostXine]

hello,

Thank you for your great work!

I wonder how you get the feature map variances. According to my understanding, you first need to extract representations of all the samples, which should give us a vector with a length of D (let's just fatten the 2d tensor or concatenate all tokens). Then you calculate the variance of each element in this vector over all the samples, which should give us D variances. Finally, you take the mean value of all D variances and get the variance ready to report.

Did I get you correctly? Sorry if I didn't catch up with your existing documentation or description.

Thank you and I'm looking forward to your reply.

Best,

[xxxnell]

Hi, thank you for your support!

Yes, you explained correctly. I calculated the variance of the tokens per channel and then averaged them.

I will release the code after re-implementing it. I can't give a definite timeline, but I’ll try hard to release the whole code for the feature map variance by this Friday!

[LostXine]

Thanks for the prompt response! Have a nice day

[xxxnell]

I just released featuremap_variance.ipynb (Colab notebook). I'm sorry for the late release.

[LostXine]

Thank you so much!