copy(::TrackedArray) doesn't work

[tpapp]

MWE:

using Flux.Tracker
x = ones(5)
tx = TrackedArray(x)
copy(tx)

gives the error

ERROR: MethodError: no method matching Float64(::Flux.Tracker.TrackedReal{Float64})
Closest candidates are:
  Float64(::Real, ::RoundingMode) where T<:AbstractFloat at rounding.jl:185
  Float64(::T<:Number) where T<:Number at boot.jl:722
  Float64(::Int8) at float.jl:60
  ...
Stacktrace:
 [1] convert(::Type{Float64}, ::Flux.Tracker.TrackedReal{Float64}) at ./number.jl:7
 [2] setindex!(::Array{Float64,1}, ::Flux.Tracker.TrackedReal{Float64}, ::Int64) at ./array.jl:769
 [3] setindex! at ./multidimensional.jl:414 [inlined]
 [4] copyto!(::Array{Float64,1}, ::TrackedArray{…,Array{Float64,1}}) at ./multidimensional.jl:822
 [5] copymutable(::TrackedArray{…,Array{Float64,1}}) at ./abstractarray.jl:823
 [6] copy(::TrackedArray{…,Array{Float64,1}}) at ./abstractarray.jl:773
 [7] top-level scope at none:0

on

(v1.1) pkg> status
  [587475ba] Flux v0.6.7+ #master (https://github.com/FluxML/Flux.jl.git)
julia> VERSION
v"1.1.0-DEV.361"

[MikeInnes]

We just need to add a method to catch this. Since tracked arrays are immutable anyway it can just be a no-op.

[tpapp]

What should similar do? Return a Array{Flux.Tracker.TrackedReal{Float64}}?

[MikeInnes]

Similar should always return the equivalent un-tracked object; there's no reason to include tracking for that since there are no gradients anyway.

[tpapp]

I am not sure, since the Interfaces documentation says that similar should

Return a mutable array with the same shape and element type

but in

x = TrackedArray(ones(5))
y = similar(x)
eltype(x) == eltype(y) # => false

which is causing the above code to fail.

[johnnychen94]

@tpapp we could change Base.eltype(x::Type{<:TrackedArray{T}}) where T <: Real = TrackedReal{T} in

Flux.jl/src/tracker/array.jl

Line 34 in 73385b5

Base.eltype(x::Type{<:TrackedArray{T}}) where T <: Real = TrackedReal{T}

to Base.eltype(x::TrackedArray)= eltype(data(x)) to solve this

[tpapp]

@johnnychen94: I am not sure this is the right solution. For a bit of context, consider a function where I want to preallocate an array for results, eg

function unitvec(xs::AbstractVector{T}) where {T <: Real}
    S = typeof(√one(T))         # takes care of T ≡ Int64, etc
    n = length(xs)
    y = Vector{S}(undef, n + 1)
    r = one(S)
    for (i, x) in enumerate(xs)
        z = tanh(x)
        y[i] = z * √r
        r *= 1 - abs2(z)
    end
    y[end] = √r
    y
end

which works fine for all kinds of arrays, except

julia> xs = Flux.param(ones(3))
Tracked 3-element Array{Float64,1}:
 1.0
 1.0
 1.0

julia> unitvec(xs)
S = Float64
ERROR: MethodError: no method matching Float64(::Flux.Tracker.TrackedReal{Float64})
Closest candidates are:
  Float64(::Real, ::RoundingMode) where T<:AbstractFloat at rounding.jl:194
  Float64(::T<:Number) where T<:Number at boot.jl:741
  Float64(::Int8) at float.jl:60
  ...
Stacktrace:
 [1] convert(::Type{Float64}, ::Flux.Tracker.TrackedReal{Float64}) at ./number.jl:7
 [2] setindex!(::Array{Float64,1}, ::Flux.Tracker.TrackedReal{Float64}, ::Int64) at ./array.jl:766
 [3] unitvec(::TrackedArray{…,Array{Float64,1}}) at ./REPL[88]:9
 [4] top-level scope at none:0

I would find it more consistent if the parameter T of a TrackedArray was the same as its eltype, and both were a TrackedReal.

[MikeInnes]

I've tweaked the definition of similar in
bf0b5c5 so that it can be used for this kind of thing.