Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
fix reduction modes and misc issues #7082
This is the follow up to:
There are still a few issues there with gradient checks:
AlexDBlack left a comment
This makes sense. We can add a "gradient check mask" to allow per-element skipping.
The loss function itself should be gradient checkable (with labels being either one-hot or a probability distribution) though I would have to check the math and the implementation to comment further... maybe open an issue so we don't forget to look into it further?