ADVI example fails with Zygote AD backend #1988

hdelecki · 2023-05-05T05:24:00Z

Hello,

The ADVI tutorial example fails with Zygote AD. Running the following example

using Random
using Turing, Zygote
using Turing: Variational

Turing.setadbackend(:zygote)

x = randn(2000)

@model function model(x)
    s ~ InverseGamma(2, 3)
    m ~ Normal(0.0, sqrt(s))
    for i = 1:length(x)
        x[i] ~ Normal(m, sqrt(s))
    end
end

m = model(x)

# ADVI
advi = ADVI(10, 1000)
q = vi(m, advi)

Results in the following

┌ Info: [ADVI] Should only be seen once: optimizer created for θ
└   objectid(θ) = 0x9d4e4cf6ed01f357
ERROR: LoadError: MethodError: no method matching iterate(::Nothing)

I'm using Julia 1.8 with Zygote v0.6.60 and Turing v0.25.0

sunxd3 · 2023-05-05T10:41:08Z

I recreated the issue but haven't figure out what's wrong, seems like a Zygote issue.

It does work with ReverseDiff.jl as a backend, you can give it another try by:

...
using ReverseDiff # instead of Zygote
...
Turing.setadbackend(:reversediff)
...

hdelecki · 2023-05-05T16:59:20Z

Thanks for taking a look! ReverseDiff works well, but unfortunately I need to use Zygote in a specific application.

Red-Portal · 2023-05-06T21:49:39Z

Hi,

This issue is possibly related. I've just tried the quick-and-dirty fix I posted there, but I now get a different error:

julia> @eval Bijectors begin
       function with_logabsdet_jacobian(sb::Stacked, x::AbstractVector)
           ys = mapreduce(vcat, sb.bs, sb.ranges) do b, r
               y, _ = with_logabsdet_jacobian(b, x[r])
               y
           end
           logjac = mapreduce(+, sb.bs, sb.ranges) do b, r
               _, l = with_logabsdet_jacobian(b, x[r])
               first(l)
           end
           return (ys, logjac)
       end
       end
with_logabsdet_jacobian (generic function with 51 methods)

julia> q = vi(m, advi)
┌ Info: [ADVI] Should only be seen once: optimizer created for θ
└   objectid(θ) = 0x22a19793ad541501
ERROR: MethodError: no method matching length(::ChainRulesCore.ZeroTangent)
Closest candidates are:
  length(::Union{Base.KeySet, Base.ValueIterator}) at abstractdict.jl:58
  length(::Union{LinearAlgebra.Adjoint{T, <:Union{StaticArraysCore.StaticArray{Tuple{var"#s2"}, T, 1} where var"#s2", StaticArraysCore.StaticArray{Tuple{var"#s3", var"#s4"}, T, 2} where {var"#s3", var"#s4"}}}, LinearAlgebra.Diagonal{T, <:StaticArraysCore.StaticArray{Tuple{var"#s13"}, T, 1} where var"#s13"}, LinearAlgebra.Hermitian{T, <:StaticArraysCore.StaticArray{Tuple{var"#s10", var"#s11"}, T, 2} where {var"#s10", var"#s11"}}, LinearAlgebra.LowerTriangular{T, <:StaticArraysCore.StaticArray{Tuple{var"#s18", var"#s19"}, T, 2} where {var"#s18", var"#s19"}}, LinearAlgebra.Symmetric{T, <:StaticArraysCore.StaticArray{Tuple{var"#s7", var"#s8"}, T, 2} where {var"#s7", var"#s8"}}, LinearAlgebra.Transpose{T, <:Union{StaticArraysCore.StaticArray{Tuple{var"#s2"}, T, 1} where var"#s2", StaticArraysCore.StaticArray{Tuple{var"#s3", var"#s4"}, T, 2} where {var"#s3", var"#s4"}}}, LinearAlgebra.UnitLowerTriangular{T, <:StaticArraysCore.StaticArray{Tuple{var"#s24", var"#s25"}, T, 2} where {var"#s24", var"#s25"}}, LinearAlgebra.UnitUpperTriangular{T, <:StaticArraysCore.StaticArray{Tuple{var"#s21", var"#s22"}, T, 2} where {var"#s21", var"#s22"}}, LinearAlgebra.UpperTriangular{T, <:StaticArraysCore.StaticArray{Tuple{var"#s15", var"#s16"}, T, 2} where {var"#s15", var"#s16"}}, StaticArraysCore.StaticArray{Tuple{var"#s25"}, T, 1} where var"#s25", StaticArraysCore.StaticArray{Tuple{var"#s1", var"#s3"}, T, 2} where {var"#s1", var"#s3"}, StaticArraysCore.StaticArray{<:Tuple, T}} where T) at ~/.julia/packages/StaticArrays/jO3cC/src/abstractarray.jl:1
  length(::Union{LinearAlgebra.Adjoint{T, S}, LinearAlgebra.Transpose{T, S}} where {T, S}) at /usr/share/julia/stdlib/v1.8/LinearAlgebra/src/adjtrans.jl:172
  ...
Stacktrace:
  [1] length(g::Base.Generator{ChainRulesCore.ZeroTangent, ChangesOfVariables.ChangesOfVariablesChainRulesCoreExt.var"#1#2"{Tuple{Float64, Float64}, Float64}})
    @ Base ./generator.jl:50
  [2] _similar_shape(itr::Base.Generator{ChainRulesCore.ZeroTangent, ChangesOfVariables.ChangesOfVariablesChainRulesCoreExt.var"#1#2"{Tuple{Float64, Float64}, Float64}}, #unused#::Base.HasLength)
    @ Base ./array.jl:663
  [3] collect(itr::Base.Generator{ChainRulesCore.ZeroTangent, ChangesOfVariables.ChangesOfVariablesChainRulesCoreExt.var"#1#2"{Tuple{Float64, Float64}, Float64}})
    @ Base ./array.jl:786
  [4] map(f::Function, A::ChainRulesCore.ZeroTangent)
    @ Base ./abstractarray.jl:2961
  [5] (::ChangesOfVariables.ChangesOfVariablesChainRulesCoreExt.WithLadjOnMappedPullback{Tuple{Float64, Float64}})(thunked_ΔΩ::ChainRulesCore.Tangent{Any, Tuple{ChainRulesCore.ZeroTangent, Float64}})
    @ ChangesOfVariables.ChangesOfVariablesChainRulesCoreExt ~/.julia/packages/ChangesOfVariables/qC6bf/ext/ChangesOfVariablesChainRulesCoreExt.jl:12
  [6] ZBack
    @ ~/.julia/packages/Zygote/SuKWp/src/compiler/chainrules.jl:211 [inlined]
  [7] Pullback
    @ ~/.julia/packages/ChangesOfVariables/qC6bf/src/with_ladj.jl:121 [inlined]
  [8] (::Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), Base.Fix1{typeof(broadcast), typeof(exp)}, Vector{Float64}}, Tuple{Zygote.Pullback{Tuple{Type{Base.Fix1}, typeof(with_logabsdet_jacobian), typeof(exp)}, Tuple{Zygote.var"#2176#back#309"{Zygote.Jnew{Base.Fix1{typeof(with_logabsdet_jacobian), typeof(exp)}, Nothing, false}}, Zygote.Pullback{Tuple{typeof(convert), Type{typeof(exp)}, typeof(exp)}, Tuple{}}, Zygote.ZBack{ChainRules.var"#fieldtype_pullback#420"}, Zygote.ZBack{ChainRules.var"#fieldtype_pullback#420"}, Zygote.Pullback{Tuple{typeof(convert), Type{typeof(with_logabsdet_jacobian)}, typeof(with_logabsdet_jacobian)}, Tuple{}}, Zygote.Pullback{Tuple{typeof(Base._stable_typeof), typeof(exp)}, Tuple{Zygote.ZBack{ChainRules.var"#typeof_pullback#45"}}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Base.Fix1{typeof(broadcast), typeof(exp)}, typeof(exp)}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Base.Fix1{typeof(broadcast), typeof(exp)}, typeof(broadcast)}}, Zygote.ZBack{ChangesOfVariables.ChangesOfVariablesChainRulesCoreExt.WithLadjOnMappedPullback{Tuple{Float64, Float64}}}, Zygote.var"#4135#back#1387"{Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcasted), Base.Fix1{typeof(with_logabsdet_jacobian), typeof(exp)}, Vector{Float64}}, Tuple{Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcastable), Vector{Float64}}, Tuple{}}, Zygote.var"#2841#back#683"{Zygote.var"#map_back#677"{typeof(Base.Broadcast.broadcastable), 1, Tuple{Tuple{}}, Tuple{Val{0}}, Tuple{}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.var"#4103#back#1363"{Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Tuple{Float64, Float64}, Zygote.var"#2379#back#440"{Zygote.Pullback{Tuple{Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, Float64}, Tuple{Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), typeof(exp), Float64}, Tuple{Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.ZBack{ChainRules.var"#exp_pullback#1319"{Float64, ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, typeof(exp)}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, typeof(with_logabsdet_jacobian)}}}}}}}, Val{2}}}}}, Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.var"#combine_styles_pullback#1169"{Tuple{Nothing, Nothing}}}}}}}}})(Δ::Tuple{Nothing, Float64})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
  [9] Pullback
    @ ./REPL[11]:8 [inlined]
 [10] (::Zygote.Pullback{Tuple{Bijectors.var"#340#342"{Vector{Float64}}, Base.Fix1{typeof(broadcast), typeof(exp)}, UnitRange{Int64}}, Tuple{Zygote.var"#back#242"{Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 2, Zygote.Context{false}, Float64}}}, Zygote.var"#back#241"{Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Vector{Float64}}}}, Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), Base.Fix1{typeof(broadcast), typeof(exp)}, Vector{Float64}}, Tuple{Zygote.Pullback{Tuple{Type{Base.Fix1}, typeof(with_logabsdet_jacobian), typeof(exp)}, Tuple{Zygote.var"#2176#back#309"{Zygote.Jnew{Base.Fix1{typeof(with_logabsdet_jacobian), typeof(exp)}, Nothing, false}}, Zygote.Pullback{Tuple{typeof(convert), Type{typeof(exp)}, typeof(exp)}, Tuple{}}, Zygote.ZBack{ChainRules.var"#fieldtype_pullback#420"}, Zygote.ZBack{ChainRules.var"#fieldtype_pullback#420"}, Zygote.Pullback{Tuple{typeof(convert), Type{typeof(with_logabsdet_jacobian)}, typeof(with_logabsdet_jacobian)}, Tuple{}}, Zygote.Pullback{Tuple{typeof(Base._stable_typeof), typeof(exp)}, Tuple{Zygote.ZBack{ChainRules.var"#typeof_pullback#45"}}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Base.Fix1{typeof(broadcast), typeof(exp)}, typeof(exp)}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Base.Fix1{typeof(broadcast), typeof(exp)}, typeof(broadcast)}}, Zygote.ZBack{ChangesOfVariables.ChangesOfVariablesChainRulesCoreExt.WithLadjOnMappedPullback{Tuple{Float64, Float64}}}, Zygote.var"#4135#back#1387"{Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcasted), Base.Fix1{typeof(with_logabsdet_jacobian), typeof(exp)}, Vector{Float64}}, Tuple{Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcastable), Vector{Float64}}, Tuple{}}, Zygote.var"#2841#back#683"{Zygote.var"#map_back#677"{typeof(Base.Broadcast.broadcastable), 1, Tuple{Tuple{}}, Tuple{Val{0}}, Tuple{}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.var"#4103#back#1363"{Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Tuple{Float64, Float64}, Zygote.var"#2379#back#440"{Zygote.Pullback{Tuple{Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, Float64}, Tuple{Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), typeof(exp), Float64}, Tuple{Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.ZBack{ChainRules.var"#exp_pullback#1319"{Float64, ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, typeof(exp)}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, typeof(with_logabsdet_jacobian)}}}}}}}, Val{2}}}}}, Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.var"#combine_styles_pullback#1169"{Tuple{Nothing, Nothing}}}}}}}}}, Zygote.var"#2571#back#528"{Zygote.var"#538#540"{1, Float64, Vector{Float64}, Tuple{UnitRange{Int64}}}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 2, Zygote.Context{false}, Int64}}, Zygote.Pullback{Tuple{typeof(first), Float64}, Tuple{}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Float64}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Bijectors.var"#340#342"{Vector{Float64}}, Vector{Float64}}}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [11] (::Zygote.var"#675#680")(::Tuple{Float64, Zygote.Pullback{Tuple{Bijectors.var"#340#342"{Vector{Float64}}, Base.Fix1{typeof(broadcast), typeof(exp)}, UnitRange{Int64}}, Tuple{Zygote.var"#back#242"{Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 2, Zygote.Context{false}, Float64}}}, Zygote.var"#back#241"{Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Vector{Float64}}}}, Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), Base.Fix1{typeof(broadcast), typeof(exp)}, Vector{Float64}}, Tuple{Zygote.Pullback{Tuple{Type{Base.Fix1}, typeof(with_logabsdet_jacobian), typeof(exp)}, Tuple{Zygote.var"#2176#back#309"{Zygote.Jnew{Base.Fix1{typeof(with_logabsdet_jacobian), typeof(exp)}, Nothing, false}}, Zygote.Pullback{Tuple{typeof(convert), Type{typeof(exp)}, typeof(exp)}, Tuple{}}, Zygote.ZBack{ChainRules.var"#fieldtype_pullback#420"}, Zygote.ZBack{ChainRules.var"#fieldtype_pullback#420"}, Zygote.Pullback{Tuple{typeof(convert), Type{typeof(with_logabsdet_jacobian)}, typeof(with_logabsdet_jacobian)}, Tuple{}}, Zygote.Pullback{Tuple{typeof(Base._stable_typeof), typeof(exp)}, Tuple{Zygote.ZBack{ChainRules.var"#typeof_pullback#45"}}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Base.Fix1{typeof(broadcast), typeof(exp)}, typeof(exp)}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Base.Fix1{typeof(broadcast), typeof(exp)}, typeof(broadcast)}}, Zygote.ZBack{ChangesOfVariables.ChangesOfVariablesChainRulesCoreExt.WithLadjOnMappedPullback{Tuple{Float64, Float64}}}, Zygote.var"#4135#back#1387"{Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcasted), Base.Fix1{typeof(with_logabsdet_jacobian), typeof(exp)}, Vector{Float64}}, Tuple{Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcastable), Vector{Float64}}, Tuple{}}, Zygote.var"#2841#back#683"{Zygote.var"#map_back#677"{typeof(Base.Broadcast.broadcastable), 1, Tuple{Tuple{}}, Tuple{Val{0}}, Tuple{}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.var"#4103#back#1363"{Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Tuple{Float64, Float64}, Zygote.var"#2379#back#440"{Zygote.Pullback{Tuple{Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, Float64}, Tuple{Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), typeof(exp), Float64}, Tuple{Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.ZBack{ChainRules.var"#exp_pullback#1319"{Float64, ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, typeof(exp)}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, typeof(with_logabsdet_jacobian)}}}}}}}, Val{2}}}}}, Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.var"#combine_styles_pullback#1169"{Tuple{Nothing, Nothing}}}}}}}}}, Zygote.var"#2571#back#528"{Zygote.var"#538#540"{1, Float64, Vector{Float64}, Tuple{UnitRange{Int64}}}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 2, Zygote.Context{false}, Int64}}, Zygote.Pullback{Tuple{typeof(first), Float64}, Tuple{}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Float64}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Bijectors.var"#340#342"{Vector{Float64}}, Vector{Float64}}}}}}, δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/lib/array.jl:215
 [12] (::Base.var"#4#5"{Zygote.var"#675#680"})(a::Tuple{Tuple{Float64, Zygote.Pullback{Tuple{Bijectors.var"#340#342"{Vector{Float64}}, Base.Fix1{typeof(broadcast), typeof(exp)}, UnitRange{Int64}}, Tuple{Zygote.var"#back#242"{Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 2, Zygote.Context{false}, Float64}}}, Zygote.var"#back#241"{Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Vector{Float64}}}}, Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), Base.Fix1{typeof(broadcast), typeof(exp)}, Vector{Float64}}, Tuple{Zygote.Pullback{Tuple{Type{Base.Fix1}, typeof(with_logabsdet_jacobian), typeof(exp)}, Tuple{Zygote.var"#2176#back#309"{Zygote.Jnew{Base.Fix1{typeof(with_logabsdet_jacobian), typeof(exp)}, Nothing, false}}, Zygote.Pullback{Tuple{typeof(convert), Type{typeof(exp)}, typeof(exp)}, Tuple{}}, Zygote.ZBack{ChainRules.var"#fieldtype_pullback#420"}, Zygote.ZBack{ChainRules.var"#fieldtype_pullback#420"}, Zygote.Pullback{Tuple{typeof(convert), Type{typeof(with_logabsdet_jacobian)}, typeof(with_logabsdet_jacobian)}, Tuple{}}, Zygote.Pullback{Tuple{typeof(Base._stable_typeof), typeof(exp)}, Tuple{Zygote.ZBack{ChainRules.var"#typeof_pullback#45"}}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Base.Fix1{typeof(broadcast), typeof(exp)}, typeof(exp)}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Base.Fix1{typeof(broadcast), typeof(exp)}, typeof(broadcast)}}, Zygote.ZBack{ChangesOfVariables.ChangesOfVariablesChainRulesCoreExt.WithLadjOnMappedPullback{Tuple{Float64, Float64}}}, Zygote.var"#4135#back#1387"{Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcasted), Base.Fix1{typeof(with_logabsdet_jacobian), typeof(exp)}, Vector{Float64}}, Tuple{Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcastable), Vector{Float64}}, Tuple{}}, Zygote.var"#2841#back#683"{Zygote.var"#map_back#677"{typeof(Base.Broadcast.broadcastable), 1, Tuple{Tuple{}}, Tuple{Val{0}}, Tuple{}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.var"#4103#back#1363"{Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Tuple{Float64, Float64}, Zygote.var"#2379#back#440"{Zygote.Pullback{Tuple{Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, Float64}, Tuple{Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), typeof(exp), Float64}, Tuple{Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.ZBack{ChainRules.var"#exp_pullback#1319"{Float64, ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, typeof(exp)}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Zygote.var"#fallback_Fix1#439"{typeof(exp), typeof(with_logabsdet_jacobian)}, typeof(with_logabsdet_jacobian)}}}}}}}, Val{2}}}}}, Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.var"#combine_styles_pullback#1169"{Tuple{Nothing, Nothing}}}}}}}}}, Zygote.var"#2571#back#528"{Zygote.var"#538#540"{1, Float64, Vector{Float64}, Tuple{UnitRange{Int64}}}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 2, Zygote.Context{false}, Int64}}, Zygote.Pullback{Tuple{typeof(first), Float64}, Tuple{}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Float64}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:x, Zygote.Context{false}, Bijectors.var"#340#342"{Vector{Float64}}, Vector{Float64}}}}}}, Float64})
    @ Base ./generator.jl:36
 [13] iterate
    @ ./generator.jl:47 [inlined]
 [14] collect_to!(dest::Vector{Tuple{Nothing, Nothing, Nothing}}, itr::Base.Generator{Base.Iterators.Zip{Tuple{Vector{Tuple{Float64, Zygote.Pullback}}, Vector{Float64}}}, Base.var"#4#5"{Zygote.var"#675#680"}}, offs::Int64, st::Tuple{Int64, Int64})
    @ Base ./array.jl:845
 [15] collect_to_with_first!(dest::Vector{Tuple{Nothing, Nothing, Nothing}}, v1::Tuple{Nothing, Nothing, Nothing}, itr::Base.Generator{Base.Iterators.Zip{Tuple{Vector{Tuple{Float64, Zygote.Pullback}}, Vector{Float64}}}, Base.var"#4#5"{Zygote.var"#675#680"}}, st::Tuple{Int64, Int64})
    @ Base ./array.jl:823
 [16] collect(itr::Base.Generator{Base.Iterators.Zip{Tuple{Vector{Tuple{Float64, Zygote.Pullback}}, Vector{Float64}}}, Base.var"#4#5"{Zygote.var"#675#680"}})
    @ Base ./array.jl:797
 [17] map
    @ ./abstractarray.jl:3055 [inlined]
 [18] (::Zygote.var"#map_back#677"{Bijectors.var"#340#342"{Vector{Float64}}, 2, Tuple{Vector{Function}, Vector{UnitRange{Int64}}}, Tuple{Tuple{Base.OneTo{Int64}}, Tuple{Base.OneTo{Int64}}}, Vector{Tuple{Float64, Zygote.Pullback}}})(Δ::Vector{Float64})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/lib/array.jl:215
 [19] #2841#back
    @ ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71 [inlined]
 [20] #287
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/lib.jl:206 [inlined]
 [21] (::Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing, Nothing}}, Zygote.var"#2841#back#683"{Zygote.var"#map_back#677"{Bijectors.var"#340#342"{Vector{Float64}}, 2, Tuple{Vector{Function}, Vector{UnitRange{Int64}}}, Tuple{Tuple{Base.OneTo{Int64}}, Tuple{Base.OneTo{Int64}}}, Vector{Tuple{Float64, Zygote.Pullback}}}}}})(Δ::Vector{Float64})
    @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71
 [22] Pullback
    @ ./reducedim.jl:359 [inlined]
 [23] (::Zygote.Pullback{Tuple{Base.var"##mapreduce#766", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(mapreduce), Bijectors.var"#340#342"{Vector{Float64}}, typeof(+), Vector{Function}, Vector{UnitRange{Int64}}}, Any})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [24] (::Zygote.var"#287#288"{Tuple{NTuple{4, Nothing}, Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{Base.var"##mapreduce#766", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(mapreduce), Bijectors.var"#340#342"{Vector{Float64}}, typeof(+), Vector{Function}, Vector{UnitRange{Int64}}}, Any}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/lib/lib.jl:206
 [25] #2138#back
    @ ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71 [inlined]
 [26] Pullback
    @ ./reducedim.jl:359 [inlined]
 [27] (::Zygote.Pullback{Tuple{typeof(mapreduce), Bijectors.var"#340#342"{Vector{Float64}}, typeof(+), Vector{Function}, Vector{UnitRange{Int64}}}, Tuple{Zygote.Pullback{Tuple{Type{NamedTuple}}, Tuple{}}, Zygote.var"#2331#back#414"{Zygote.var"#pairs_namedtuple_pullback#413"{(), NamedTuple{(), Tuple{}}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{NTuple{4, Nothing}, Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{Base.var"##mapreduce#766", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(mapreduce), Bijectors.var"#340#342"{Vector{Float64}}, typeof(+), Vector{Function}, Vector{UnitRange{Int64}}}, Any}}}, Zygote.var"#1982#back#200"{typeof(identity)}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [28] Pullback
    @ ./REPL[11]:7 [inlined]
 [29] (::Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{Float64}}, Tuple{Zygote.var"#2176#back#309"{Zygote.Jnew{Bijectors.var"#340#342"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:ranges, Zygote.Context{false}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{UnitRange{Int64}}}}, Zygote.Pullback{Tuple{typeof(mapreduce), Bijectors.var"#340#342"{Vector{Float64}}, typeof(+), Vector{Function}, Vector{UnitRange{Int64}}}, Tuple{Zygote.Pullback{Tuple{Type{NamedTuple}}, Tuple{}}, Zygote.var"#2331#back#414"{Zygote.var"#pairs_namedtuple_pullback#413"{(), NamedTuple{(), Tuple{}}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{NTuple{4, Nothing}, Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{Base.var"##mapreduce#766", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(mapreduce), Bijectors.var"#340#342"{Vector{Float64}}, typeof(+), Vector{Function}, Vector{UnitRange{Int64}}}, Any}}}, Zygote.var"#1982#back#200"{typeof(identity)}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:bs, Zygote.Context{false}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{Function}}}, Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.var"#2176#back#309"{Zygote.Jnew{Bijectors.var"#339#341"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:ranges, Zygote.Context{false}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{UnitRange{Int64}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:bs, Zygote.Context{false}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{Function}}}, Zygote.Pullback{Tuple{typeof(mapreduce), Bijectors.var"#339#341"{Vector{Float64}}, typeof(vcat), Vector{Function}, Vector{UnitRange{Int64}}}, Tuple{Zygote.Pullback{Tuple{Type{NamedTuple}}, Tuple{}}, Zygote.var"#2331#back#414"{Zygote.var"#pairs_namedtuple_pullback#413"{(), NamedTuple{(), Tuple{}}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{NTuple{4, Nothing}, Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{Base.var"##mapreduce#766", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(mapreduce), Bijectors.var"#339#341"{Vector{Float64}}, typeof(vcat), Vector{Function}, Vector{UnitRange{Int64}}}, Any}}}, Zygote.var"#1982#back#200"{typeof(identity)}}}}})(Δ::Tuple{Vector{Float64}, Float64})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [30] Pullback
    @ ~/.julia/packages/AdvancedVI/qluPz/src/utils.jl:13 [inlined]
 [31] (::Zygote.Pullback{Tuple{typeof(AdvancedVI.rand_and_logjac), TaskLocalRNG, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}}, Tuple{Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:transform, Zygote.Context{false}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}}, Zygote.Pullback{Tuple{typeof(rand), TaskLocalRNG, DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}}, Tuple{Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:m, Zygote.Context{false}, DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:m, Zygote.Context{false}, DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Vector{Float64}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.materialize), Vector{Float64}}, Tuple{}}, Zygote.var"#3734#back#1184"{Zygote.var"#1178#1182"{Tuple{Vector{Float64}, Vector{Float64}}}}, Zygote.var"#3782#back#1208"{Zygote.var"#1204#1207"{Vector{Float64}, Vector{Float64}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:σ, Zygote.Context{false}, DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.ZBack{DistributionsAD.var"#adapt_randn_pullback#7"{Tuple{TaskLocalRNG, Vector{Float64}, Int64}}}}}, Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.ZBack{ChainRules.var"#length_pullback#746"}}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Float64}}, Zygote.var"#back#242"{Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 2, Zygote.Context{false}, Float64}}}, Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 2, Zygote.Context{false}, Int64}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:dist, Zygote.Context{false}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}}}, Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.var"#back#241"{Zygote.var"#1998#back#209"{Zygote.var"#back#207"{2, 1, Zygote.Context{false}, Vector{Float64}}}}, Zygote.Pullback{Tuple{typeof(with_logabsdet_jacobian), Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{Float64}}, Tuple{Zygote.var"#2176#back#309"{Zygote.Jnew{Bijectors.var"#340#342"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:ranges, Zygote.Context{false}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{UnitRange{Int64}}}}, Zygote.Pullback{Tuple{typeof(mapreduce), Bijectors.var"#340#342"{Vector{Float64}}, typeof(+), Vector{Function}, Vector{UnitRange{Int64}}}, Tuple{Zygote.Pullback{Tuple{Type{NamedTuple}}, Tuple{}}, Zygote.var"#2331#back#414"{Zygote.var"#pairs_namedtuple_pullback#413"{(), NamedTuple{(), Tuple{}}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{NTuple{4, Nothing}, Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{Base.var"##mapreduce#766", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(mapreduce), Bijectors.var"#340#342"{Vector{Float64}}, typeof(+), Vector{Function}, Vector{UnitRange{Int64}}}, Any}}}, Zygote.var"#1982#back#200"{typeof(identity)}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:bs, Zygote.Context{false}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{Function}}}, Zygote.var"#1982#back#200"{typeof(identity)}, Zygote.var"#2176#back#309"{Zygote.Jnew{Bijectors.var"#339#341"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:ranges, Zygote.Context{false}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{UnitRange{Int64}}}}, Zygote.var"#2149#back#299"{Zygote.var"#back#298"{:bs, Zygote.Context{false}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}, Vector{Function}}}, Zygote.Pullback{Tuple{typeof(mapreduce), Bijectors.var"#339#341"{Vector{Float64}}, typeof(vcat), Vector{Function}, Vector{UnitRange{Int64}}}, Tuple{Zygote.Pullback{Tuple{Type{NamedTuple}}, Tuple{}}, Zygote.var"#2331#back#414"{Zygote.var"#pairs_namedtuple_pullback#413"{(), NamedTuple{(), Tuple{}}}}, Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{NTuple{4, Nothing}, Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{Base.var"##mapreduce#766", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, typeof(mapreduce), Bijectors.var"#339#341"{Vector{Float64}}, typeof(vcat), Vector{Function}, Vector{UnitRange{Int64}}}, Any}}}, Zygote.var"#1982#back#200"{typeof(identity)}}}}}}})(Δ::Tuple{Vector{Float64}, Float64})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [32] Pullback
    @ ~/.julia/packages/AdvancedVI/qluPz/src/advi.jl:94 [inlined]
 [33] (::Zygote.Pullback{Tuple{AdvancedVI.ELBO, TaskLocalRNG, ADVI{AdvancedVI.ZygoteAD}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, Turing.Variational.var"#logπ#2"{DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, DynamicPPL.TypedVarInfo{NamedTuple{(:s, :m), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:s, Setfield.IdentityLens}, Int64}, Vector{InverseGamma{Float64}}, Vector{AbstractPPL.VarName{:s, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:m, Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:m, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, Int64}, Any})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [34] Pullback
    @ ~/.julia/packages/AdvancedVI/qluPz/src/objectives.jl:4 [inlined]
 [35] (::Zygote.Pullback{Tuple{AdvancedVI.var"##_#27", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, AdvancedVI.ELBO, ADVI{AdvancedVI.ZygoteAD}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, Turing.Variational.var"#logπ#2"{DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, DynamicPPL.TypedVarInfo{NamedTuple{(:s, :m), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:s, Setfield.IdentityLens}, Int64}, Vector{InverseGamma{Float64}}, Vector{AbstractPPL.VarName{:s, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:m, Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:m, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, Int64}, Any})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [36] Pullback
    @ ~/.julia/packages/AdvancedVI/qluPz/src/objectives.jl:3 [inlined]
 [37] (::Zygote.Pullback{Tuple{AdvancedVI.ELBO, ADVI{AdvancedVI.ZygoteAD}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, Turing.Variational.var"#logπ#2"{DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, DynamicPPL.TypedVarInfo{NamedTuple{(:s, :m), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:s, Setfield.IdentityLens}, Int64}, Vector{InverseGamma{Float64}}, Vector{AbstractPPL.VarName{:s, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:m, Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:m, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, Int64}, Tuple{Zygote.var"#2331#back#414"{Zygote.var"#pairs_namedtuple_pullback#413"{(), NamedTuple{(), Tuple{}}}}, Zygote.Pullback{Tuple{Type{NamedTuple}}, Tuple{}}, Zygote.Pullback{Tuple{AdvancedVI.var"##_#27", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, AdvancedVI.ELBO, ADVI{AdvancedVI.ZygoteAD}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, Turing.Variational.var"#logπ#2"{DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, DynamicPPL.TypedVarInfo{NamedTuple{(:s, :m), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:s, Setfield.IdentityLens}, Int64}, Vector{InverseGamma{Float64}}, Vector{AbstractPPL.VarName{:s, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:m, Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:m, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, Int64}, Any}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [38] #287
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/lib.jl:206 [inlined]
 [39] (::Zygote.var"#2138#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{AdvancedVI.ELBO, ADVI{AdvancedVI.ZygoteAD}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, Turing.Variational.var"#logπ#2"{DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, DynamicPPL.TypedVarInfo{NamedTuple{(:s, :m), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:s, Setfield.IdentityLens}, Int64}, Vector{InverseGamma{Float64}}, Vector{AbstractPPL.VarName{:s, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:m, Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:m, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, Int64}, Tuple{Zygote.var"#2331#back#414"{Zygote.var"#pairs_namedtuple_pullback#413"{(), NamedTuple{(), Tuple{}}}}, Zygote.Pullback{Tuple{Type{NamedTuple}}, Tuple{}}, Zygote.Pullback{Tuple{AdvancedVI.var"##_#27", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, AdvancedVI.ELBO, ADVI{AdvancedVI.ZygoteAD}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, Turing.Variational.var"#logπ#2"{DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, DynamicPPL.TypedVarInfo{NamedTuple{(:s, :m), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:s, Setfield.IdentityLens}, Int64}, Vector{InverseGamma{Float64}}, Vector{AbstractPPL.VarName{:s, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:m, Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:m, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, Int64}, Any}}}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71
 [40] Pullback
    @ ~/.julia/packages/AdvancedVI/qluPz/src/AdvancedVI.jl:45 [inlined]
 [41] (::Zygote.Pullback{Tuple{AdvancedVI.var"#f#32"{AdvancedVI.ELBO, ADVI{AdvancedVI.ZygoteAD}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, Turing.Variational.var"#logπ#2"{DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, DynamicPPL.TypedVarInfo{NamedTuple{(:s, :m), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:s, Setfield.IdentityLens}, Int64}, Vector{InverseGamma{Float64}}, Vector{AbstractPPL.VarName{:s, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:m, Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:m, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, Tuple{Int64}}, Vector{Float64}}, Any})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [42] (::Zygote.var"#75#76"{Zygote.Pullback{Tuple{AdvancedVI.var"#f#32"{AdvancedVI.ELBO, ADVI{AdvancedVI.ZygoteAD}, MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, Turing.Variational.var"#logπ#2"{DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, DynamicPPL.TypedVarInfo{NamedTuple{(:s, :m), Tuple{DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:s, Setfield.IdentityLens}, Int64}, Vector{InverseGamma{Float64}}, Vector{AbstractPPL.VarName{:s, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}, DynamicPPL.Metadata{Dict{AbstractPPL.VarName{:m, Setfield.IdentityLens}, Int64}, Vector{Normal{Float64}}, Vector{AbstractPPL.VarName{:m, Setfield.IdentityLens}}, Vector{Float64}, Vector{Set{DynamicPPL.Selector}}}}}, Float64}}, Tuple{Int64}}, Vector{Float64}}, Any}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface.jl:45
 [43] grad!(vo::AdvancedVI.ELBO, alg::ADVI{AdvancedVI.ZygoteAD}, q::MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, model::Function, θ::Vector{Float64}, out::DiffResults.MutableDiffResult{1, Float64, Tuple{Vector{Float64}}}, args::Int64)
    @ AdvancedVI ~/.julia/packages/AdvancedVI/qluPz/src/AdvancedVI.jl:50
 [44] macro expansion
    @ ~/.julia/packages/AdvancedVI/qluPz/src/AdvancedVI.jl:245 [inlined]
 [45] macro expansion
    @ ./timing.jl:382 [inlined]
 [46] optimize!(vo::AdvancedVI.ELBO, alg::ADVI{AdvancedVI.ZygoteAD}, q::MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}, model::Function, θ::Vector{Float64}; optimizer::AdvancedVI.TruncatedADAGrad)
    @ AdvancedVI ~/.julia/packages/AdvancedVI/qluPz/src/AdvancedVI.jl:244
 [47] vi(model::DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, alg::ADVI{AdvancedVI.ZygoteAD}, q::MultivariateTransformed{DistributionsAD.TuringDiagMvNormal{Vector{Float64}, Vector{Float64}}, Stacked{Vector{Function}, Vector{UnitRange{Int64}}}}; optimizer::AdvancedVI.TruncatedADAGrad)
    @ Turing.Variational ~/.julia/packages/Turing/WXGfh/src/variational/advi.jl:154
 [48] vi(model::DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, alg::ADVI{AdvancedVI.ZygoteAD}; optimizer::AdvancedVI.TruncatedADAGrad)
    @ Turing.Variational ~/.julia/packages/Turing/WXGfh/src/variational/advi.jl:139
 [49] vi(model::DynamicPPL.Model{typeof(model), (:x,), (), (), Tuple{Vector{Float64}}, Tuple{}, DynamicPPL.DefaultContext}, alg::ADVI{AdvancedVI.ZygoteAD})
    @ Turing.Variational ~/.julia/packages/Turing/WXGfh/src/variational/advi.jl:133
 [50] top-level scope
    @ REPL[12]:1

A few things seem to be broken with mapreduce's chain rule. Can you open an issue on ChainRulesCore? mapreduce is quite the bugger because it breaks quite often.

hdelecki · 2023-05-12T19:39:50Z

Hi @Red-Portal I'm having a hard time finding a good MWE for mapreduce that would trigger the same behavior. Do you have any insight?

Red-Portal · 2023-06-14T23:35:30Z

@hdelecki Sorry for the late response. I'm not an AD expert either, but it does seen that mapreduce is still broken. Asking for help! @devmotion @torfjelde

Red-Portal · 2024-06-15T03:29:43Z

Finally fixed in Bijectors@0.13.14 !

torfjelde · 2024-06-17T20:04:00Z

Lovely stuff @Red-Portal :)

Red-Portal mentioned this issue Aug 17, 2023

mapreduce with accumulation inside is broken FluxML/Zygote.jl#1448

Open

gdalle mentioned this issue Apr 2, 2024

Can DifferentiationInterface be useful for Turing? #2187

Open

Red-Portal mentioned this issue Jun 5, 2024

fix Stacked to work with Zygote TuringLang/Bijectors.jl#315

Merged

Red-Portal closed this as completed Jun 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ADVI example fails with Zygote AD backend #1988

ADVI example fails with Zygote AD backend #1988

hdelecki commented May 5, 2023

sunxd3 commented May 5, 2023

hdelecki commented May 5, 2023

Red-Portal commented May 6, 2023 •

edited

hdelecki commented May 12, 2023

Red-Portal commented Jun 14, 2023

Red-Portal commented Jun 15, 2024 •

edited

torfjelde commented Jun 17, 2024

ADVI example fails with Zygote AD backend #1988

ADVI example fails with Zygote AD backend #1988

Comments

hdelecki commented May 5, 2023

sunxd3 commented May 5, 2023

hdelecki commented May 5, 2023

Red-Portal commented May 6, 2023 • edited

hdelecki commented May 12, 2023

Red-Portal commented Jun 14, 2023

Red-Portal commented Jun 15, 2024 • edited

torfjelde commented Jun 17, 2024

Red-Portal commented May 6, 2023 •

edited

Red-Portal commented Jun 15, 2024 •

edited