About the loss the inconsistency between the loss function of the code and the paper

[cjissmart]

Congratulations on the great paper and on being accepted to NIPS! I have a problem with the loss function.
The code corresponding to the self-supervised learning loss function is on lines 75 and 88 of 'main.py'. The formulation of the code is as follows.


However, the formulation in the paper should be:


They are not consistent. Why is this the case?

[hengruizhang98]

Thanks for your interest in our work!
These two implementations are equivalent. Note that Z is standardized (0-mean and 1/\sqrt{N} variance), we have || Z_A ||_2^2 = || Z_B ||_2^2 = D. And then L_Inv = || Z_A - Z_B ||_2^2 = || Z_A ||_2^2 + || Z_B ||_2^2 - 2 * Z_A^T Z_B = 2D - 2* Z_A^T Z_B, so we minimize Z_A^T * Z_B in the codes instead.

[cjissmart]

I find the reason in E.1 of Appendix.

[hengruizhang98]

Yes, we also explain that in the Appendix.

[cjissmart]

Thanks for the reply !!!