Skip to content

Commit

Permalink
add developer comment
Browse files Browse the repository at this point in the history
  • Loading branch information
tomerarnon committed Jul 23, 2020
1 parent d412314 commit d66a8fc
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions src/optimization/duality.jl
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,12 @@ function activation_value(σ::ReLU, μᵢ, λᵢ, l̂ᵢ, ûᵢ)
gᵢûᵢ = @. μᵢ*ûᵢ - λᵢ*σ(ûᵢ)

max = symbolic_max

# NOTE: ifelse evaluates all of its arguments in advance.
# This may seem like a bug since symbolic_max mutates the model
# and introduces new variables, but it is fine. Only those
# variables that are part of the objective will affect the problem,
# and those will only the the ones summed *after* taking the ifelse
return sum(@. ifelse(l̂ᵢ < 0 < ûᵢ,
max(gᵢl̂ᵢ, gᵢûᵢ, 0),
max(gᵢl̂ᵢ, gᵢûᵢ)))
Expand Down

0 comments on commit d66a8fc

Please sign in to comment.