No method matching for mapreduce?
#539
Assignees
Comments
|
Might be a bug here. Could you extend your example to produce it? I.e. presumably it needs |
using Turing
using KernelFunctions
using Zygote
using LinearAlgebra
ard_kernel(σ², ℓ) =
σ²*KernelFunctions.TransformedKernel(
KernelFunctions.Matern52Kernel(),
KernelFunctions.ARDTransform(1 ./ ℓ))
Turing.@model function logisticgp(X, y, jitter=1e-6)
n_features = size(X, 1)
logα ~ Normal(0, 1)
logσ ~ Normal(0, 1)
logℓ ~ MvNormal(n_features, 1)
α² = exp(logα*2)
σ² = exp(logσ*2)
ℓ = exp.(logℓ)
kernel = ard_kernel(α², ℓ)
K = KernelFunctions.kernelmatrix(kernel, X)
K += I*(σ² + jitter)
f ~ MvNormal(zeros(size(X,2)), K)
y .~ Turing.BernoulliLogit.(f)
end
Turing.setadbackend(:zygote)
model = logisticgp(randn(10, 100), rand(100) .> 0.5)
sample(model, NUTS(), 10)Hi @mcabbott , |
|
Ok, can reproduce (on 1.6, but not can't install on 1.7?). And ForwardDiff looks like it runs, with no other changes: But have no idea what's going on yet. The error still happens with JuliaDiff/ChainRulesCore.jl#488 . This must come from But if you try that in isolation, it seems fine: The rule for |
This is known (and expected) due to Libtask_jll: TuringLang/Turing.jl#1713 (comment) |
|
I have a quick question though, the model in question is currently crazy slow with a 60 dimension dataset with like 150 data points. It's quite surprising that a few tens of data points can make such a nightmare of a difference. Would Zygote help in this case? Or is the performance of |
|
First of all, I would recommend that you use AbstractGPs or, usually easier and more user-friendly, one of the downstream packages such as Stheno instead of working with KernelFunctions directly. It seems there are some efficiency improvements possible though. E.g., I would suggest to use kernel(α², logℓ, σ²) = α² * (Matern52Kernel() ∘ ARDTransform(@. exp(-logℓ))) + σ² * WhiteKernel()
@model function logisticgp(X, y, jitter=1e-6)
n_features, n_observations = size(X)
logα ~ Normal(0, 1)
logσ ~ Normal(0, 1)
# MvNormal(::Int, ::Real) is deprecated
logℓ ~ MvNormal(zeros(n_features), I) # or a bit more efficient: logℓ ~ MvNormal(Zeros(n_features), I)
α² = exp(logα*2)
σ² = exp(logσ*2)
K = kernelmatrix(kernel(α², logℓ, σ² + jitter), X)
...
endDue to all the matrix operations involved here and since |
|
@devmotion Thanks for the tips. I'm indeed using AbstractGPs, I swapped it for |
|
I believe that https://discourse.julialang.org/t/ffts-in-probabilistic-models/69775/2 might be a similar issue. You can trigger that like this: julia> using ChainRulesCore
julia> ProjectTo(Ref(1))(Ref{Any}(2)) # this is what's expected
Tangent{Base.RefValue{Int64}}(x = 2.0,)
julia> ProjectTo(Ref(3))(Ref{Any}((x=4,))) # both a Ref and a NamedTuple
ERROR: MethodError: no method matching (::ProjectTo{Float64, NamedTuple{(), Tuple{}}})(::NamedTuple{(:x,), Tuple{Int64}})
...
Stacktrace:
[1] (::ProjectTo{Ref, NamedTuple{(:type, :x), Tuple{DataType, ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}})(dx::Base.RefValue{Any})
@ ChainRulesCore ~/.julia/packages/ChainRulesCore/1L9My/src/projection.jl:275And for mutable structs, Zygote wraps the NamedTuple it would make for a julia> pullback(x -> x[]^2, Ref(3))[2](1) # no projection
(Base.RefValue{Any}((x = 6,)),)
julia> gradient(x -> x[]^2, Ref(3)) # with projection
ERROR: MethodError: no method matching (::ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}})(::NamedTuple{(:x,), Tuple{Int64}}) |
This needs to be handled in: |
|
Done. @Red-Portal see if this is solved by Zygote.0.6.28 |
|
Hi @mcabbott Unfortunately, the example still doesn't run. I'm still on Julia 1.6 |
|
Do you get the same error or a different one? |
|
@mcabbott seems more or less identical. Here's the error. The line numbers are slightly different. |
|
Ok, thanks for checking! I got everything loaded and get the same now, on 1.6. The line Maybe I already said this but a few lines down it has |
|
Here's a more minimal example: julia> using Zygote
julia> y, back = pullback(x -> sum(getindex, x), Ref.(1:3))
(6, Zygote.var"#52#53"{typeof(∂(#11))}(∂(#11)))
julia> back(1)
ERROR: MethodError: no method matching (::ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}})(::NamedTuple{(:x,), Tuple{Int64}})
...
Stacktrace:
[1] (::ChainRulesCore.ProjectTo{ChainRulesCore.Tangent{Base.RefValue{Int64}}, NamedTuple{(:x,), Tuple{ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}})(dx::Base.RefValue{Any})
@ ChainRulesCore ~/.julia/packages/ChainRulesCore/bxKCw/src/projection.jl:284
[2] #56
@ ~/.julia/packages/ChainRulesCore/bxKCw/src/projection.jl:237 [inlined]
[3] #4
@ ./generator.jl:36 [inlined]
[4] iterate
@ ./generator.jl:47 [inlined]
[5] collect(itr::Base.Generator{Base.Iterators.Zip{Tuple{Vector{ChainRulesCore.ProjectTo{ChainRulesCore.Tangent{Base.RefValue{Int64}}, NamedTuple{(:x,), Tuple{ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}}}, Vector{Base.RefValue{Any}}}}, Base.var"#4#5"{ChainRulesCore.var"#56#57"}})
@ Base ./array.jl:710
[6] map
@ ./abstractarray.jl:2860 [inlined]
[7] (::ChainRulesCore.ProjectTo{AbstractArray, NamedTuple{(:elements, :axes), Tuple{Vector{ChainRulesCore.ProjectTo{ChainRulesCore.Tangent{Base.RefValue{Int64}}, NamedTuple{(:x,), Tuple{ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}}}, Tuple{Base.OneTo{Int64}}}}})(dx::Vector{Base.RefValue{Any}})
@ ChainRulesCore ~/.julia/packages/ChainRulesCore/bxKCw/src/projection.jl:237
[8] (::ChainRules.var"#sum_pullback#1375"{Colon, typeof(getindex), ChainRulesCore.ProjectTo{AbstractArray, NamedTuple{(:elements, :axes), Tuple{Vector{ChainRulesCore.ProjectTo{ChainRulesCore.Tangent{Base.RefValue{Int64}}, NamedTuple{(:x,), Tuple{ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}}}, Tuple{Base.OneTo{Int64}}}}}, Vector{Zygote.var"#ad_pullback#47"{Tuple{typeof(getindex), Base.RefValue{Int64}}, typeof(∂(getindex))}}})(ȳ::Int64)
@ ChainRules ~/.julia/packages/ChainRules/RyXef/src/rulesets/Base/mapreduce.jl:88
[9] ZBack
...
(@v1.7) pkg> st Zygote
Status `~/.julia/environments/v1.7/Project.toml`
[e88e6eb3] Zygote v0.6.28And a version with broadcasting, without the julia> y, back = pullback(x -> sum(getindex.(x)), Ref.(1:3))
(6, Zygote.var"#52#53"{typeof(∂(#15))}(∂(#15)))
julia> back(1)
ERROR: MethodError: no method matching (::ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}})(::NamedTuple{(:x,), Tuple{Int64}})
...
Stacktrace:
[1] (::ChainRulesCore.ProjectTo{ChainRulesCore.Tangent{Base.RefValue{Int64}}, NamedTuple{(:x,), Tuple{ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}})(dx::Base.RefValue{Any})
@ ChainRulesCore ~/.julia/packages/ChainRulesCore/bxKCw/src/projection.jl:284
...
[7] (::ChainRulesCore.ProjectTo{AbstractArray, NamedTuple{(:elements, :axes), Tuple{Vector{ChainRulesCore.ProjectTo{ChainRulesCore.Tangent{Base.RefValue{Int64}}, NamedTuple{(:x,), Tuple{ChainRulesCore.ProjectTo{Float64, NamedTuple{(), Tuple{}}}}}}}, Tuple{Base.OneTo{Int64}}}}})(dx::Vector{Base.RefValue{Any}})
@ ChainRulesCore ~/.julia/packages/ChainRulesCore/bxKCw/src/projection.jl:237
[8] _project
@ ~/.julia/packages/Zygote/fDJjj/src/compiler/chainrules.jl:142 [inlined]
[9] unbroadcast(x::Vector{Base.RefValue{Int64}}, x̄::Vector{Base.RefValue{Any}})
@ Zygote ~/.julia/packages/Zygote/fDJjj/src/lib/broadcast.jl:51
julia> @eval Zygote @inline wrap_chainrules_input(xs::AbstractArray{<:Ref}) = wrap_chainrules_input.(xs);
julia> Zygote.refresh()
julia> y, back = pullback(x -> sum(getindex.(x)), Ref.(1:3)) # now try again, same thing
(6, Zygote.var"#52#53"{typeof(∂(#21))}(∂(#21)))
julia> back(1)[1] # ChainRules types escape, but otherwise OK.
3-element Vector{ChainRulesCore.Tangent{Base.RefValue{Int64}, NamedTuple{(:x,), Tuple{Float64}}}}:
Tangent{Base.RefValue{Int64}}(x = 1.0,)
Tangent{Base.RefValue{Int64}}(x = 1.0,)
Tangent{Base.RefValue{Int64}}(x = 1.0,) |
|
Example from #539 (comment) is still broken on latest Zygote + 1.6, although at least the error is different now: |
|
This now seems to be fixed, with the versions shown: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I'm having the following issue
which pos up when differentiating through the following Turing model
Is this issue related to
ChainRules.jl? Not entirely sure.The text was updated successfully, but these errors were encountered: