Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core/Rules: Better/Exposed Stabilizer #153

Merged
merged 1 commit into from
Jul 20, 2022
Merged

Core/Rules: Better/Exposed Stabilizer #153

merged 1 commit into from
Jul 20, 2022

Conversation

chr5tphr
Copy link
Owner

@chr5tphr chr5tphr commented Jun 28, 2022

  • exposes a stabilizer parameter for all built-in rules other than
    epsilon

  • implements Stabilizer as a class

  • create a stabilizer function from the stabilizer/epsilon value, for
    which Stabilizer.ensure is implemented, for both the Epsilon rule
    and all other rules that apply

  • stabilizer/epsilon can be a float/int, in which case a Stabilizer
    object will be instantiated with the appropriate epsilon value

  • stabilizer/epsilon can be callable, in which case the callable itself
    will be used as a stabilizer

  • add batchnorm to the layer_map_base, and make layer_map_base a
    function such that stabilizer for Norm() may be passed

  • added tests for Stabilizer

  • added explanations for the use of the new stabilizer and epsilon
    arguments to how-to/use-rules-composites-and-canonizers.rst

  • BatchNorm is now ignored by default instead of using its gradient (!!)
    the reason for this choice is that it is a better choice than using
    the gradient

@chr5tphr chr5tphr force-pushed the expose-stabilizer branch 6 times, most recently from 50c014c to ae27d02 Compare July 19, 2022 16:51
- exposes a `stabilizer` parameter for all built-in rules other than
  epsilon
- implements Stabilizer as a class
- create a stabilizer function from the stabilizer/epsilon value, for
  which Stabilizer.ensure is implemented, for both the `Epsilon` rule
  and all other rules that apply
- stabilizer/epsilon can be a float/int, in which case a Stabilizer
  object will be instantiated with the appropriate epsilon value
- stabilizer/epsilon can be callable, in which case the callable itself
  will be used as a stabilizer
- add batchnorm to the `layer_map_base`, and make `layer_map_base` a
  function such that `stabilizer` for Norm() may be passed
- added tests for `Stabilizer`
- added explanations for the use of the new `stabilizer` and `epsilon`
  arguments to `how-to/use-rules-composites-and-canonizers.rst`

- BatchNorm is now ignored by default instead of using its gradient (!!)
  the reason for this choice is that it is a better choice than using
  the gradient
@chr5tphr chr5tphr changed the title Draft: Core/Rules: Better/Exposed Stabilizer Core/Rules: Better/Exposed Stabilizer Jul 20, 2022
@chr5tphr chr5tphr marked this pull request as ready for review July 20, 2022 10:50
@chr5tphr chr5tphr merged commit 60537d8 into master Jul 20, 2022
@chr5tphr chr5tphr deleted the expose-stabilizer branch July 20, 2022 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant