Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about normalization constant Z_v1 and Z_v2 in the ContrastMemory #51

Open
YujieZheng99 opened this issue Jul 3, 2022 · 0 comments

Comments

@YujieZheng99
Copy link

YujieZheng99 commented Jul 3, 2022

What an excellent work! I have learned so much from this paper.
But I have a question about normalization constant Z_v1 and Z_v2 in the ContrastMemory.
The Z_v1 and Z_v2 is initialized to -1,and Z_v1/Z_v2 is calculated based on out_v1/out_v2.
As shown in the following code, Z_v1/Z_v2 is updated only if Z_v1<0 and Z_v2<0,so Z_v1/Z_v2 will only be updated in the first batch.

if Z_v1 < 0:  
    self.params[2] = out_v1.mean() * outputSize  
    Z_v1 = self.params[2].clone().detach().item()  
    print("normalization constant Z_v1 is set to {:.1f}".format(Z_v1))  
if Z_v2 < 0:  
    self.params[3] = out_v2.mean() * outputSize  
    Z_v2 = self.params[3].clone().detach().item()  
    print("normalization constant Z_v2 is set to {:.1f}".format(Z_v2))  

But I think Z_v1 and Z_v2 should be updated in every batch.
What is your opinion on this,what are the benefits of this design?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant