Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

orthogonal regularization loss useless? #12

Closed
GallagherCommaJack opened this issue Dec 17, 2021 · 2 comments
Closed

orthogonal regularization loss useless? #12

GallagherCommaJack opened this issue Dec 17, 2021 · 2 comments

Comments

@GallagherCommaJack
Copy link

because the codebooks are not registered as trainable parameters, and the orthogonal loss is only a function of the codebooks, is the orthogonal loss entirely useless?
@lucidrains
Copy link
Owner

@GallagherCommaJack Hey Jack, thank you for catching this! Can you let me know if ebce893 works?

@GallagherCommaJack
Copy link
Author

yep that seems to be working

@liangcl0928 liangcl0928 mentioned this issue Jun 13, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lucidrains @GallagherCommaJack