Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Using Zygote failed on Julia v1.0.0, MacOS #7

Closed
johnnychen94 opened this issue Aug 20, 2018 · 2 comments
Closed

Using Zygote failed on Julia v1.0.0, MacOS #7

johnnychen94 opened this issue Aug 20, 2018 · 2 comments

Comments

@johnnychen94
Copy link
Contributor

The following procedure works on Ubuntu 16.04, Julia v1.0.0.
But fails on MacOS 10.13.6, Julia v1.0.0

(v1.0) pkg> activate test
[ Info: new environment will be placed at /Users/jc/test

(test) pkg> add Zygote
 Resolving package versions...
 [ ......some packages added]

julia> using Zygote
ERROR: schedule: Task not runnable
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] enq_work at ./event.jl:88 [inlined]
 [3] #schedule#428 at ./event.jl:134 [inlined]
 [4] schedule at ./event.jl:129 [inlined]
 [5] notify(::Condition, ::Nothing, ::Bool, ::Bool) at ./event.jl:68
 [6] put!(::Channel{Any}, ::Tuple{Expr,Int64}) at ./channels.jl:261
 [7] send_to_backend(::Any, ::Any, ::Any) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:687
 [8] send_to_backend at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:683 [inlined]
 [9] (::getfield(REPL, Symbol("#do_respond#40")){Bool,getfield(REPL, Symbol("##50#59")){REPL.LineEditREPL,REPL.REPLHistoryProvider},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:702
 [10] #invokelatest#1 at ./essentials.jl:686 [inlined]
 [11] invokelatest at ./essentials.jl:685 [inlined]
 [12] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:2261
 [13] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:1029
 [14] run_repl(::REPL.AbstractREPL, ::Any) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:191
 [15] (::getfield(Base, Symbol("##720#722")){Bool,Bool,Bool,Bool})(::Module) at ./logging.jl:311
 [16] #invokelatest#1 at ./essentials.jl:686 [inlined]
 [17] invokelatest at ./essentials.jl:685 [inlined]
 [18] macro expansion at ./logging.jl:308 [inlined]
 [19] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:330
 [20] exec_options(::Base.JLOptions) at ./client.jl:242
 [21] _start() at ./client.jl:421
@MikeInnes
Copy link
Member

Do you still get this reliably? Seems like it's probably not related to Zygote itself.

@johnnychen94
Copy link
Contributor Author

Forget it, this seems to be the problem of my dirty General Registry.

Keno added a commit that referenced this issue Feb 24, 2019
@Keno
WIP: Don't use stacks for simple control flow
b909b9b 
Right now Zygote inserts stacks whenever it needs to use an ssa value
not defined in the first basic block. This is of course unnecessary.
The condition for needing stacks is that the basic block that defines
it is self-reachable (i.e. in a loop). Otherwise, we can simply insert
phi nodes to thread the desired SSA value through to the exit block
(we don't need to do anything in the adjoint, since the reversal of
the CFG ensures dominance). Removing stacks allows for both more
efficient code generation and enables higher order auto-diff (since
we use control flow in Zygote, but can't handle differentiating code
that contains stacks). The headline example is something like the following:

```
function foo(b, x)
    if b
        sin(x)
    else
        cos(x)
    end
end
```

Then looking at `@code_typed derivative(x->foo(true, x), 1.0)`, we get:

Before:
```
CodeInfo(
1 ── %1  = $(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{Int8,1}, svec(Any, Int64), :(:ccall), 2, Array{Int8,1}, 0, 0))::Array{Int8,1}
│    %2  = $(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{Any,1}, svec(Any, Int64), :(:ccall), 2, Array{Any,1}, 0, 0))::Array{Any,1}
│    %3  = $(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{Any,1}, svec(Any, Int64), :(:ccall), 2, Array{Any,1}, 0, 0))::Array{Any,1}
│    %4  = Base.sin::typeof(sin)
│          invoke %4(_3::Float64)::Float64
│    %6  = %new(##334#335{Float64}, x)::##334#335{Float64}
│    %7  = %new(##758#back#336{##334#335{Float64}}, %6)::##758#back#336{##334#335{Float64}}
[snip]
23 ─ %52 = invoke %47(1::Int8)::Tuple{Nothing,Nothing,Any}
│    %53 = Base.getfield(%52, 3, true)::Any
└───       goto #24
24 ─       return %53
) => Any
```

After:
```
CodeInfo(
1 ─ %1 = Base.sin::typeof(sin)
│        invoke %1(_3::Float64)::Float64
│   %3 = Core.Intrinsics.not_int(true)::Bool
└──      goto #3 if not %3
2 ─      invoke Zygote.notnothing(nothing::Nothing)::Union{}
└──      $(Expr(:unreachable))::Union{}
3 ┄ %7 = invoke Zygote.cos(_3::Float64)::Float64
│   %8 = Base.mul_float(1.0, %7)::Float64
└──      goto #4
4 ─      goto #5
5 ─      goto #6
6 ─      goto #7
7 ─      return %8
) => Float64
```

Which is essentially perfect (there's a bit of junk left over, but LLVM
can take care of that. The only thing that doesn't get removed is the
useless invocation of `sin`, but that's a separate and known issue).
Keno added a commit that referenced this issue Mar 6, 2019
@Keno
WIP: Don't use stacks for simple control flow
aad4a75 
Right now Zygote inserts stacks whenever it needs to use an ssa value
not defined in the first basic block. This is of course unnecessary.
The condition for needing stacks is that the basic block that defines
it is self-reachable (i.e. in a loop). Otherwise, we can simply insert
phi nodes to thread the desired SSA value through to the exit block
(we don't need to do anything in the adjoint, since the reversal of
the CFG ensures dominance). Removing stacks allows for both more
efficient code generation and enables higher order auto-diff (since
we use control flow in Zygote, but can't handle differentiating code
that contains stacks). The headline example is something like the following:

```
function foo(b, x)
    if b
        sin(x)
    else
        cos(x)
    end
end
```

Then looking at `@code_typed derivative(x->foo(true, x), 1.0)`, we get:

Before:
```
CodeInfo(
1 ── %1  = $(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{Int8,1}, svec(Any, Int64), :(:ccall), 2, Array{Int8,1}, 0, 0))::Array{Int8,1}
│    %2  = $(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{Any,1}, svec(Any, Int64), :(:ccall), 2, Array{Any,1}, 0, 0))::Array{Any,1}
│    %3  = $(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{Any,1}, svec(Any, Int64), :(:ccall), 2, Array{Any,1}, 0, 0))::Array{Any,1}
│    %4  = Base.sin::typeof(sin)
│          invoke %4(_3::Float64)::Float64
│    %6  = %new(##334#335{Float64}, x)::##334#335{Float64}
│    %7  = %new(##758#back#336{##334#335{Float64}}, %6)::##758#back#336{##334#335{Float64}}
[snip]
23 ─ %52 = invoke %47(1::Int8)::Tuple{Nothing,Nothing,Any}
│    %53 = Base.getfield(%52, 3, true)::Any
└───       goto #24
24 ─       return %53
) => Any
```

After:
```
CodeInfo(
1 ─ %1 = Base.sin::typeof(sin)
│        invoke %1(_3::Float64)::Float64
│   %3 = Core.Intrinsics.not_int(true)::Bool
└──      goto #3 if not %3
2 ─      invoke Zygote.notnothing(nothing::Nothing)::Union{}
└──      $(Expr(:unreachable))::Union{}
3 ┄ %7 = invoke Zygote.cos(_3::Float64)::Float64
│   %8 = Base.mul_float(1.0, %7)::Float64
└──      goto #4
4 ─      goto #5
5 ─      goto #6
6 ─      goto #7
7 ─      return %8
) => Float64
```

Which is essentially perfect (there's a bit of junk left over, but LLVM
can take care of that. The only thing that doesn't get removed is the
useless invocation of `sin`, but that's a separate and known issue).
bors bot added a commit that referenced this issue Aug 3, 2020
@bors @wkearn
Merge #751
a0f9c8e 
751: Fix FFT type promotions r=CarloLucibello a=wkearn

I ran into an issue with type promotions while using `Conv` layers in Flux when I tried to run model output through FFTs. I tracked it down to the definitions of the adjoints for the various `fft` variants. The 1/N factors needed in the `irfft` adjoints automatically promoted `Float32` arrays to `Float64` arrays, which then caused the error when propagated back through the chain. A minimal example is included below.

This pull request just changes all those multiplications by 1/N into divisions by N, which prevents the promotion. I also added a few tests that assert that the gradients are the right types. The tests with `irfft(x,dims)` throw an error that is not related to the type promotion, so I commented those out.

```julia
x = randn(Float32,16,1,1)
m = Conv((5,),1=>1,relu,pad=SamePad())
loss(x) = sum(abs2,irfft(rfft(m(x)),16))
gradient(()->loss(x),Flux.params(m))

┌ Warning: Slow fallback implementation invoked for ∇conv_data!  You probably don't want this; check your datatypes.
│   yT = AbstractFloat
│   T1 = AbstractFloat
│   T2 = Float32
└ @ NNlib C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\conv.jl:206
ERROR: UndefRefError: access to undefined reference
Stacktrace:
 [1] getindex at .\array.jl:745 [inlined]
 [2] #conv_direct!#149(::Float64, ::Bool, ::typeof(NNlib.conv_direct!), ::Array{AbstractFloat,5}, ::Array{AbstractFloat,5}, ::Array{Float32,5}, ::DenseConvDims{3,(5, 1, 1),1,1,(1, 1, 1),(2, 2, 0, 0, 0, 0),(1, 1, 1),false}) at C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\impl\conv_direct.jl:98
 [3] (::NNlib.var"#kw##conv_direct!")(::NamedTuple{(:alpha, :beta),Tuple{Float64,Bool}}, ::typeof(NNlib.conv_direct!), ::Array{AbstractFloat,5}, ::Array{AbstractFloat,5}, ::Array{Float32,5}, ::DenseConvDims{3,(5, 1, 1),1,1,(1, 1, 1),(2, 2, 0, 0, 0, 0),(1, 1, 1),false}) at .\none:0
 [4] #∇conv_data_direct!#152(::Float64, ::Bool, ::typeof(NNlib.∇conv_data_direct!), ::Array{AbstractFloat,5}, ::Array{AbstractFloat,5}, ::Array{Float32,5}, ::DenseConvDims{3,(5, 1, 1),1,1,(1, 1, 1),(2, 2, 0, 0, 0, 0),(1, 1, 1),false}) at C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\impl\conv_direct.jl:163
 [5] ∇conv_data_direct! at C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\impl\conv_direct.jl:158 [inlined]
 [6] #∇conv_data!#106(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(∇conv_data!), ::Array{AbstractFloat,5}, ::Array{AbstractFloat,5}, ::Array{Float32,5}, ::DenseConvDims{3,(5, 1, 1),1,1,(1, 1, 1),(2, 2, 0, 0, 0, 0),(1, 1, 1),false}) at C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\conv.jl:208
 [7] ∇conv_data!(::Array{AbstractFloat,5}, ::Array{AbstractFloat,5}, ::Array{Float32,5}, ::DenseConvDims{3,(5, 1, 1),1,1,(1, 1, 1),(2, 2, 0, 0, 0, 0),(1, 1, 1),false}) at C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\conv.jl:206
 [8] #∇conv_data!#67(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(∇conv_data!), ::Array{AbstractFloat,3}, ::Array{AbstractFloat,3}, ::Array{Float32,3}, ::DenseConvDims{1,(5,),1,1,(1,),(2, 2),(1,),false}) at C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\conv.jl:148
 [9] ∇conv_data! at C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\conv.jl:148 [inlined]
 [10] #∇conv_data#39 at C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\conv.jl:103 [inlined]
 [11] ∇conv_data at C:\Users\wkearney\.julia\packages\NNlib\sSn9M\src\conv.jl:101 [inlined]
 [12] #1241 at C:\Users\wkearney\.julia\dev\Zygote\src\lib\nnlib.jl:42 [inlined]
 [13] (::Zygote.var"#4101#back#1243"{Zygote.var"#1241#1242"{Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Float32,3},Array{Float32,3},DenseConvDims{1,(5,),1,1,(1,),(2, 2),(1,),false}}})(::Array{AbstractFloat,3}) at C:\Users\wkearney\.julia\packages\ZygoteRules\6nssF\src\adjoint.jl:49
 [14] Conv at C:\Users\wkearney\.julia\packages\Flux\IjMZL\src\layers\conv.jl:147 [inlined]
 [15] (::typeof(∂(λ)))(::Array{Float64,3}) at C:\Users\wkearney\.julia\dev\Zygote\src\compiler\interface2.jl:0
 [16] loss at .\REPL[6]:1 [inlined]
 [17] (::typeof(∂(loss)))(::Float32) at C:\Users\wkearney\.julia\dev\Zygote\src\compiler\interface2.jl:0
 [18] #7 at .\REPL[8]:1 [inlined]
 [19] (::typeof(∂(#7)))(::Float32) at C:\Users\wkearney\.julia\dev\Zygote\src\compiler\interface2.jl:0
 [20] (::Zygote.var"#56#57"{Params,Zygote.Context,typeof(∂(#7))})(::Float32) at C:\Users\wkearney\.julia\dev\Zygote\src\compiler\interface.jl:177
 [21] gradient(::Function, ::Params) at C:\Users\wkearney\.julia\dev\Zygote\src\compiler\interface.jl:54
```

Co-authored-by: William Kearney <William.Kearney.ctr@nrlssc.navy.mil>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@MikeInnes @johnnychen94