Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gradient bug with ForwardDiff over Enzyme.Forward for size 1 only #192

Closed
gdalle opened this issue Apr 18, 2024 · 1 comment
Closed

Gradient bug with ForwardDiff over Enzyme.Forward for size 1 only #192

gdalle opened this issue Apr 18, 2024 · 1 comment
Labels
backend Related to one or more autodiff backends bug Something isn't working

Comments

@gdalle
Copy link
Owner

gdalle commented Apr 18, 2024

julia> import ForwardDiff, Enzyme

julia> using DifferentiationInterface

julia> b = SecondOrder(AutoForwardDiff(), AutoEnzyme(Enzyme.Forward));

julia> hessian(sum, b, [1.0, 2.0])
2×2 Matrix{Float64}:
 0.0  0.0
 0.0  0.0

julia> hessian(sum, b, [1.0])
ERROR: MethodError: no method matching EnzymeCore.Duplicated(::Vector{ForwardDiff.Dual{ForwardDiff.Tag{…}, Float64, 1}}, ::Tuple{Vector{ForwardDiff.Dual{…}}})
julia> show(err)
1-element ExceptionStack:
MethodError: no method matching EnzymeCore.Duplicated(::Vector{ForwardDiff.Dual{ForwardDiff.Tag{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, Float64}, Float64, 1}}, ::Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, Float64}, Float64, 1}}})

Closest candidates are:
  EnzymeCore.Duplicated(::T1, ::T1) where T1
   @ EnzymeCore ~/.julia/packages/EnzymeCore/MB3tJ/src/EnzymeCore.jl:64
  EnzymeCore.Duplicated(::T1, ::T1, ::Bool) where T1
   @ EnzymeCore ~/.julia/packages/EnzymeCore/MB3tJ/src/EnzymeCore.jl:64

Stacktrace:
  [1] (::Enzyme.var"#81#82"{Tuple{Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, Float64}, Float64, 1}}}}, typeof(sum), Vector{ForwardDiff.Dual{ForwardDiff.Tag{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, Float64}, Float64, 1}}})(i::Int64)
    @ Enzyme ~/.julia/packages/Enzyme/l4FS0/src/Enzyme.jl:909
  [2] ntuple
    @ ./ntuple.jl:19 [inlined]
  [3] #gradient#80
    @ ~/.julia/packages/Enzyme/l4FS0/src/Enzyme.jl:908 [inlined]
  [4] gradient
    @ ~/.julia/packages/Enzyme/l4FS0/src/Enzyme.jl:907 [inlined]
  [5] gradient(f::Function, backend::AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}, x::Vector{ForwardDiff.Dual{ForwardDiff.Tag{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, Float64}, Float64, 1}}, extras::DifferentiationInterfaceEnzymeExt.EnzymeForwardGradientExtras{1, Tuple{Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, Float64}, Float64, 1}}}}})
    @ DifferentiationInterfaceEnzymeExt ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/ext/DifferentiationInterfaceEnzymeExt/forward_onearg.jl:49
  [6] gradient(f::Function, backend::AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}, x::Vector{ForwardDiff.Dual{ForwardDiff.Tag{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, Float64}, Float64, 1}})
    @ DifferentiationInterface ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/gradient.jl:73
  [7] inner_gradient_closure
    @ ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/hvp.jl:69 [inlined]
  [8] value_and_pushforward(f::DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, ::AutoForwardDiff{nothing, Nothing}, x::Vector{Float64}, dx::FillArrays.OneElement{Float64, 1, Tuple{Int64}, Tuple{Base.OneTo{Int64}}}, ::DifferentiationInterface.NoPushforwardExtras)
    @ DifferentiationInterfaceForwardDiffExt ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/ext/DifferentiationInterfaceForwardDiffExt/onearg.jl:8
  [9] pushforward
    @ ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/pushforward.jl:128 [inlined]
 [10] hvp_aux
    @ ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/hvp.jl:120 [inlined]
 [11] hvp
    @ ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/hvp.jl:115 [inlined]
 [12] (::DifferentiationInterface.var"#30#31"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}, Vector{Float64}, DifferentiationInterface.HVPHessianExtras{DifferentiationInterface.ForwardOverForwardHVPExtras{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, DifferentiationInterface.NoPushforwardExtras}}})(j::CartesianIndex{1})
    @ DifferentiationInterface ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/hessian.jl:55
 [13] #186
    @ ./generator.jl:0 [inlined]
 [14] iterate
    @ ./generator.jl:47 [inlined]
 [15] collect(itr::Base.Generator{CartesianIndices{1, Tuple{Base.OneTo{Int64}}}, Base.var"#186#187"{DifferentiationInterface.var"#30#31"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}, Vector{Float64}, DifferentiationInterface.HVPHessianExtras{DifferentiationInterface.ForwardOverForwardHVPExtras{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, DifferentiationInterface.NoPushforwardExtras}}}}})
    @ Base ./array.jl:834
 [16] _stack(dims::Function, ::Base.HasShape{1}, iter::Base.Generator{CartesianIndices{1, Tuple{Base.OneTo{Int64}}}, Base.var"#186#187"{DifferentiationInterface.var"#30#31"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}, Vector{Float64}, DifferentiationInterface.HVPHessianExtras{DifferentiationInterface.ForwardOverForwardHVPExtras{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, DifferentiationInterface.NoPushforwardExtras}}}}})
    @ Base ./abstractarray.jl:2809
 [17] _stack
    @ ./abstractarray.jl:2799 [inlined]
 [18] stack(f::DifferentiationInterface.var"#30#31"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}, Vector{Float64}, DifferentiationInterface.HVPHessianExtras{DifferentiationInterface.ForwardOverForwardHVPExtras{DifferentiationInterface.var"#inner_gradient_closure#24"{typeof(sum), SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}}, DifferentiationInterface.NoPushforwardExtras}}}, iter::CartesianIndices{1, Tuple{Base.OneTo{Int64}}}; dims::Function)
    @ Base ./abstractarray.jl:2796
 [19] hessian
    @ ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/hessian.jl:54 [inlined]
 [20] hessian(f::Function, backend::SecondOrder{AutoForwardDiff{nothing, Nothing}, AutoEnzyme{EnzymeCore.ForwardMode{EnzymeCore.FFIABI}}}, x::Vector{Float64})
    @ DifferentiationInterface ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/hessian.jl:54
 [21] top-level scope
    @ REPL[6]:1
 [22] eval
    @ ./boot.jl:385 [inlined]
 [23] eval
    @ ./Base.jl:88 [inlined]
 [24] repleval(m::Module, code::Expr, ::String)
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.76.2/scripts/packages/VSCodeServer/src/repl.jl:229
 [25] (::VSCodeServer.var"#112#114"{Module, Expr, REPL.LineEditREPL, REPL.LineEdit.Prompt})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.76.2/scripts/packages/VSCodeServer/src/repl.jl:192
 [26] with_logstate(f::Function, logstate::Any)
    @ Base.CoreLogging ./logging.jl:515
 [27] with_logger
    @ ./logging.jl:627 [inlined]
 [28] (::VSCodeServer.var"#111#113"{Module, Expr, REPL.LineEditREPL, REPL.LineEdit.Prompt})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.76.2/scripts/packages/VSCodeServer/src/repl.jl:193
 [29] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
 [30] invokelatest(::Any)
    @ Base ./essentials.jl:889
 [31] (::VSCodeServer.var"#64#65")()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.76.2/scripts/packages/VSCodeServer/src/eval.jl:34
@gdalle gdalle added bug Something isn't working backend Related to one or more autodiff backends labels Apr 18, 2024
@gdalle
Copy link
Owner Author

gdalle commented Jun 20, 2024

Error no longer occurs with current release

@gdalle gdalle closed this as completed Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Related to one or more autodiff backends bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant