Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster AlphaBetaRule #108

Merged
merged 1 commit into from
Nov 15, 2022
Merged

Faster AlphaBetaRule #108

merged 1 commit into from
Nov 15, 2022

Conversation

adrhill
Copy link
Member

@adrhill adrhill commented Nov 15, 2022

The AlphaBetaRule used to compute pullback functions on all four modified layers (layerᵅ⁺, layerᵅ⁻, layerᵝ⁻ and layerᵝ⁺) using Zygote.
However, since all layers are linear and layerᵅ⁺ & layerᵝ⁺ as well as layerᵅ⁻ & layerᵝ⁻ only differ in the bias terms, two pullbacks can be skipped as they are duplicates.

Local benchmarks

                                          ID                 time ratio memory ratio
  –––––––––––––––––––––––––––––––––––––––––– –––––––––––––––––––––––––– ––––––––––––
    ["apply rule", "AlphaBetaRule", "Dense"] 0.68 (5%) :whitecheckmark:    1.00 (1%)

@codecov-commenter
Copy link

Codecov Report

Merging #108 (781b21a) into master (9695738) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #108   +/-   ##
=======================================
  Coverage   94.70%   94.70%           
=======================================
  Files          16       16           
  Lines         529      529           
=======================================
  Hits          501      501           
  Misses         28       28           
Impacted Files Coverage Δ
src/lrp/rules.jl 94.95% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@adrhill adrhill merged commit d199d40 into master Nov 15, 2022
@adrhill adrhill deleted the ah/faster-alphabeta branch November 15, 2022 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants