similar StackOverflowError

[grahamas]

Not much of an error message, but I'm chasing it down. I'll update this as I make progress. I'm on nightly julia and master AxisIndices

ERROR: StackOverflowError:
Stacktrace:
 [1] similar(#unused#::Type{Array{Float64, N} where N}, shape::Tuple{Base.IdentityUnitRange{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}})
   @ AxisIndices ~/.julia/packages/AxisIndices/ZACgS/src/abstractarray.jl:179

EDIT:

Potentially relevant:

WARNING: Method definition similar(Type{T<:(AbstractArray{T, N} where N where T)}, Tuple{Union{Integer, Base.AbstractUnitRange{T} where T, var"#s36"} where var"#s36"<:(AxisIndices.AbstractAxis{I, Inds} where Inds where I<:Integer), Vararg{Union{Integer, Base.AbstractUnitRange{T} where T, var"#s36"} where var"#s36"<:(AxisIndices.AbstractAxis{I, Inds} where Inds where I<:Integer)}}) where {T<:(AbstractArray{T, N} where N where T)} in module OffsetArrays at /home/graham/.julia/packages/OffsetArrays/rYa26/src/OffsetArrays.jl:261 overwritten in module AxisIndices at /home/graham/.julia/dev/AxisIndices/src/abstractarray.jl:178.
  ** incremental compilation may be fatally broken for this module **

[grahamas]

Resolved

WARNING: Method definition similar(Type{T<:(AbstractArray{T, N} where N where T)}, Tuple{Union{Integer, Base.AbstractUnitRange{T} where T, var"#s36"} where var"#s36"<:(AxisIndices.AbstractAxis{I, Inds} where Inds where I<:Integer), Vararg{Union{Integer, Base.AbstractUnitRange{T} where T, var"#s36"} where var"#s36"<:(AxisIndices.AbstractAxis{I, Inds} where Inds where I<:Integer)}}) where {T<:(AbstractArray{T, N} where N where T)} in module OffsetArrays at /home/graham/.julia/packages/OffsetArrays/rYa26/src/OffsetArrays.jl:261 overwritten in module AxisIndices at /home/graham/.julia/dev/AxisIndices/src/abstractarray.jl:178.
  ** incremental compilation may be fatally broken for this module **

by changing the definition of Base.similar to

function Base.similar(::Type{T}, shape::S) where {T<:AbstractArray, DoA<:DimOrAxes, S<:Tuple{DoA,Vararg{DoA}}}

but the stack overflow remains.

[Tokazama]

What was the original call that caused the stack overflow. I think the problem is IdentityUnitRange. That should typically not be wrapping anything before getting to similar.

[grahamas]

I provoked a better error message by adding @show shape above the offending line (evidently the argument shape itself is the problem... I sort of thought it would've been an infinitely looping call to similar but the call stack doesn't reflect that).

Stacktrace:
  [1] poptask(W::Base.InvasiveLinkedListSynchronized{Task})
    @ Base ./task.jl:755
  [2] wait
    @ ./task.jl:763 [inlined]
  [3] uv_write(s::Base.TTY, p::Ptr{UInt8}, n::UInt64)
    @ Base ./stream.jl:992
  [4] unsafe_write(s::Base.TTY, p::Ptr{UInt8}, n::UInt64)
    @ Base ./stream.jl:1064
  [5] write
    @ ./strings/io.jl:185 [inlined]
  [6] print
    @ ./strings/io.jl:187 [inlined]
  [7] print(::Base.TTY, ::String, ::String, ::Vararg{String})
    @ Base ./strings/io.jl:46
  [8] println(::Base.TTY, ::String, ::Vararg{String})
    @ Base ./strings/io.jl:73
  [9] println(::String, ::String)
    @ Base ./coreio.jl:4
 [10] similar(#unused#::Type{Array{Float64, N} where N}, shape::Tuple{Base.IdentityUnitRange{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}})
    @ AxisIndices ~/.julia/dev/AxisIndices/src/abstractarray.jl:179
 [11] similar(#unused#::Type{Array{Float64, N} where N}, shape::Tuple{Base.IdentityUnitRange{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}})
    @ AxisIndices ~/.julia/dev/AxisIndices/src/abstractarray.jl:180
 [12] similar(#unused#::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Nothing, typeof(*), Tuple{SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}, Int64}, true}, Float64}}, #unused#::Type{Float64}, dims::Tuple{Base.IdentityUnitRange{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}})
    @ Base.Broadcast ./broadcast.jl:197
 [13] similar(bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Nothing, typeof(*), Tuple{SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}, Int64}, true}, Float64}}, #unused#::Type{Float64})
    @ Base.Broadcast ./broadcast.jl:196
 [14] copy
    @ ./broadcast.jl:907 [inlined]
 [15] copy(bc::Base.Broadcast.Broadcasted{AxisIndices.AxisArrayStyle{Base.Broadcast.DefaultArrayStyle{1}}, Tuple{AxisIndices.Axis{Float64, Int64, LinRange{Float64}, AxisIndices.SimpleAxis{Int64, ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}}}, typeof(*), Tuple{AxisIndices.AxisVector{Float64, SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}, Int64}, true}, AxisIndices.Axis{Float64, Int64, LinRange{Float64}, AxisIndices.SimpleAxis{Int64, ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}}}, Float64}})
    @ AxisIndices ~/.julia/dev/AxisIndices/src/arrays.jl:758
 [16] materialize
    @ ./broadcast.jl:882 [inlined]
 [17] (::NeuralModels.NormedDifferenceOfSigmoids{Float64, NeuralModels.DifferenceOfSigmoids{Float64, NeuralModels.DifferenceOfSigmoidsParameter{Float64, NeuralModels.RectifiedSigmoidNonlinearity{Float64}}, Vector{Float64}}})(output::AxisIndices.AxisVector{Float64, SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}, Int64}, true}, AxisIndices.Axis{Float64, Int64, LinRange{Float64}, AxisIndices.SimpleAxis{Int64, ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}}}, ignored_source::AxisIndices.AxisVector{Float64, SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}, Int64}, true}, AxisIndices.Axis{Float64, Int64, LinRange{Float64}, AxisIndices.SimpleAxis{Int64, ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}}}, ignored_t::Float64)
    @ NeuralModels ~/git/NeuralModels/src/nonlinearity.jl:218

That offending line is simply output .* ndos.norm_factor where

typeof(output) = AxisIndices.AxisVector{Float64, SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}, Int64}, true}, AxisIndices.Axis{Float64, Int64, LinRange{Float64}, AxisIndices.SimpleAxis{Int64, ArrayInterface.OptionallyStaticUnitRange{ArrayInterface.StaticInt{1}, Int64}}}}
typeof(ndos.norm_factor) = Float64

(note that previously, that one-element call stack I pasted in the first post was the full error message, so I had no idea where the error was coming from)

[grahamas]

I think the stack overflow must be an infinite recursion into similar even though the call stack doesn't say so -- Base.to_shape sends IdentityUnitRange to itself, so abstractarray.jl:180 will recurse.

shape = (Base.IdentityUnitRange(Static(1):512),)
Base.to_shape(shape) = (Base.IdentityUnitRange(Static(1):512),)

If the IdentityUnitRange shouldn't be there in the first place, that would certainly explain things. I'll see if I can make an MWE.

[Tokazama]

Your correct about to_shape being the issue. I'd welcome any help on it because similar keeps coming back to haunt me.

[grahamas]

I'll take a look at it.

[Tokazama]

I think a laid some pretty good groundwork for resolving issues related to similar in the latest commit, but I still need to put together a lot more tests.