Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track gradients of activations in HistogramLogger #6

Open
callumm-graphcore opened this issue Mar 6, 2024 · 1 comment
Open

Track gradients of activations in HistogramLogger #6

callumm-graphcore opened this issue Mar 6, 2024 · 1 comment

Comments

@callumm-graphcore
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
I don't think we track these at the moment

Describe the solution you'd like
We can do it using register_backward_hook, presumably

Describe alternatives you've considered
We could just not do it and hope for the best idk

Additional context
No

@callumm-graphcore
Copy link
Collaborator Author

There might be complications arising from the fact that we take gradients of gradients during training (calculate gradients to compute the forces, then compare predicted forces to actual forces, then calculate gradients with respect to the forces loss)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant