Zygote AD: Cannot convert Array{Float64,1} to Float64 #43
Comments
|
This is an issue with using Distributions,DynamicHMC,LogDensityProblems,TransformVariables,Parameters
import Zygote
struct GaussianProb{TY <: AbstractVector}
"Observations."
y::TY
end
function (problem::GaussianProb)(θ)
@unpack y = problem # extract the data
@unpack mu, sigma = θ
loglikelihood(Normal(mu, sigma), y) + logpdf(Normal(0,1), mu) +
logpdf(Truncated(Cauchy(0,5),0,Inf), sigma)
end
function GaussianGen(;μ=0,σ=1,Nd,kwargs...)
data=(y=rand(Normal(μ,σ),Nd),N=Nd)
return data
end
data = first(GaussianGen(;Nd=100))
p = GaussianProb(data);
t = as((mu = as(Real, -25, 25), sigma = asℝ₊), )
P = TransformedLogDensity(t, p)
∇P_FD = ADgradient(:ForwardDiff, P)
∇P_ZY = ADgradient(:Zygote, P)
LogDensityProblems.logdensity(LogDensityProblems.Value, P, zeros(2))
LogDensityProblems.logdensity(LogDensityProblems.ValueGradient, ∇P_FD, zeros(2)) # works
LogDensityProblems.logdensity(LogDensityProblems.ValueGradient, ∇P_ZY, zeros(2)) # errorsI will look into it soon. |
|
@itsdfish: I fixed this in #44, will merge the branch soon and then release a minor version upgrade. Note that Zygote is still experimental and has bugs like FluxML/Zygote.jl#271 (which I just discovered while fixing this), but reports against this package are most welcome in any case, and if they are an issue in Zygote I will just help make an MWE for an issue there. |
|
Thanks for fixing this so quickly. I look forward to trying it out. I will definitely report bugs as I come across them and will try identify MWEs where I can. |
|
I switched to master on LogDensityProblems and still receive the same error, which traces back to this line. I just want to clarify whether this change was supposed to fix the problem or whether FluxML/Zygote.jl#271 needs to be resolved first. |
|
No, that bug is unrelated. Can you paste an pkg> st --manifestand the error message? |
|
Sure thing. Package info: Error message: julia> LogDensityProblems.logdensity(LogDensityProblems.ValueGradient, ∇P_ZY, zeros(2)) # errors |
|
Thanks. This is another bug, possibly in Zygote. I made an MWE at tpapp/TransformVariables.jl#48. I will isolate it and either fix it (if its mine) or report an issue for Zygote. In the meantime, I would recommend that you consider Flux for AD, it is much more mature. Eg the following works fine: using Distributions, LogDensityProblems, TransformVariables, Parameters, Flux
struct GaussianProb{TY <: AbstractVector}
"Observations."
y::TY
end
function (problem::GaussianProb)(θ)
@unpack y = problem # extract the data
@unpack mu, sigma = θ
loglikelihood(Normal(mu, sigma), y) + logpdf(Normal(0,1), mu) +
logpdf(Truncated(Cauchy(0,5),0,Inf), sigma)
end
function GaussianGen(;μ=0,σ=1,Nd,kwargs...)
data=(y=rand(Normal(μ,σ),Nd),N=Nd)
return data
end
data = first(GaussianGen(;Nd=100))
p = GaussianProb(data);
t = as((mu = as(Real, -25, 25), sigma = asℝ₊), )
P = TransformedLogDensity(t, p)
∇P_FL = ADgradient(:Flux, P)
LogDensityProblems.logdensity(LogDensityProblems.Value, P, zeros(2))
LogDensityProblems.logdensity(LogDensityProblems.ValueGradient, ∇P_FL, zeros(2)) # works |
|
Thanks Tamas. I will switch to Flux AD in the meantime. |
|
I was hoping that I could ask you a quick question that might help me make some project decisions. I'm interested in trying Zygote because ForwardDiff hits a performance wall with larger problems, such as a Poisson hierarchical regression model. I have found that it runs ~ 20 slower than Stan, with an increasing disparity as more data are added. A quick benchmark reveals that DynamicHMC with Flux AD is ~3 times slower than DynamicHMC with ForwardDiff AD. Aside from sporadic bugs, have you found that Zygote performs closer to Stan? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi Tamas-
Below I have a simple model that runs with forward diff but throws a conversion error when I switch to Zygote. I will omit the error message because it is quite long. Here is some info about my setup: Julia 1.1.0 and
The simple two parameter Gaussian model I tried is listed below.
The text was updated successfully, but these errors were encountered: