Core/Rules: Better/Exposed Stabilizer #153
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
exposes a
stabilizer
parameter for all built-in rules other thanepsilon
implements Stabilizer as a class
create a stabilizer function from the stabilizer/epsilon value, for
which Stabilizer.ensure is implemented, for both the
Epsilon
ruleand all other rules that apply
stabilizer/epsilon can be a float/int, in which case a Stabilizer
object will be instantiated with the appropriate epsilon value
stabilizer/epsilon can be callable, in which case the callable itself
will be used as a stabilizer
add batchnorm to the
layer_map_base
, and makelayer_map_base
afunction such that
stabilizer
for Norm() may be passedadded tests for
Stabilizer
added explanations for the use of the new
stabilizer
andepsilon
arguments to
how-to/use-rules-composites-and-canonizers.rst
BatchNorm is now ignored by default instead of using its gradient (!!)
the reason for this choice is that it is a better choice than using
the gradient