grad of convert?

[ylxdzsw]

I need to convert type of some variables, but it crashes AutoGrad.

The code I use is:

function loss(w,x,ygold)
    ypred = predict(w,x)
    ynorm = ypred .- log(sum(exp(ypred),1))
    convert(Float32, -sum(ygold .* ynorm) / size(ygold, 2))
end

I tried @primitive like @primitive convert(T,x),dy zerograd() dy or something similar but never get it work. Is there any way to define it properly, or how to hack AutoGrad so that at least convert works without breaking other functionalities?

[ylxdzsw]

finally I got it work:

function zerograd() 0 end
@zerograd zerograd()

import Base.convert
for t in (Float32, Float64)
    @eval @primitive convert(T::Type{$t},x),dy zerograd() T(dy)
end

Wondering is there better solutions.

[denizyuret]

@ylxdzsw Your solution works, here are some suggestions for improvement:

1. Read the comments in src/core.jl (section 6) to see how gradients are actually defined.
2. Your case is closest to 6.4.
3. In your solution, instead of zerograd() you can simply use 0.
4. Instead of T(dy), using convert(T,dy) may be better.

[denizyuret]

I experimented with defining a general convert gradient in AutoGrad. Unfortunately Julia keeps calling convert in all sorts of unexpected places and so far I could not get it to work without breaking AutoGrad. However defining it for specific types like you have done is ok. There is an example for KnetArray / Array conversion in the latest Knet release. It shows an alternative to your solution that does not use the @primitive macro and leaves the gradient wrt first arg undefined.

[ylxdzsw]

Thanks, that solution looks nice.