test_rrule: fail for specifying tangents with implicit structure

[qiyang-ustc]

I met an error when I want to try to use \vdash to specifying tangents.

using LinearAlgebra,ChainRulesTestUtils,ChainRulesCore
import ChainRulesCore.rrule
function myfun(A)
    @assert norm(A+A')<1E-10
    return sum(abs,A)
end

function rrule(::typeof(myfun),A)
    result = myfun(A)
    function pullback_myfun(ȳ)
        return (NO_FIELDS,ȳ*sign.(A))
    end
    return result,pullback_myfun
end

g = rand(4,4) |> x->x-transpose(x)
dg = rand(4,4) |> x->x-transpose(x)
test_rrule(myfun,g⊢dg) # ERROR:  AssertionError: norm(A + A') < 1.0e-10

The error seems to tell me that I did not specify the tangents successfully? Or this is a bug?

[oxinabox]

I am on holidays, so don't have time to look properly.
I think what is happenning is the FiniteDifferences doesn't know that your primal input is constrained to be skew-symmertric.
So it tries to perturb each element of the matrix independently.
which breaks the skew-symmetry.
Which then causes the evalution of the myfun on the perturbed primal to error.

Here perturbing means adding what it thinks is an tiny scalleddown element of the basis for the space.
It thinks that e.g.

0 1 0 0
0 0 0 0
0 0 0 0
0 0 0 0

is in the basis, but it isn't.

 0 1 0 0
-1 0 0 0
 0 0 0 0
 0 0 0 0

is though.

If we had a special matrix type for SkewSymmertic, like we do for Symmetric, UpperTriangualar etc this would just work because it would know the basis.
But because the structure here is implict rather than explicit it does not.

The error seems to tell me that I did not specify the tangents successfully?

The tangent that you pass in is not used for that.
I don't think it can be, but I am no expert on finite differencing. @wesselb might have some ideas.
For test_rrule the tangent it is barely used at all, just used to skip if it is DoesNotExist(), and used to check inplace accumulation, the output_tangent is what is used as that is what gets pulled back.
(In contrast in test_frule it is the other way around, output_tangent is barely used, but the input tangets are used extensively)

---

Workarounds

Right now I see two possibly ways for you to work around this.

Option 1: define or otherwise obtain a explict SkewSymmetric matrix type

This would likely be a think wrapper around a Symmetric matrix.
Defining matrix types in julia is not too hard. Strictly speeking you just need size and getindex
see docs
This might also be useful in other places in your code.

Option 2: write and test a rrule for a function that wraps it

If you define a function:
skewsym_projecting_myfun(A) = myfun(A-A')
and then write a rule for that which calls rrule(myfun,...).
then by testing skewsym_projecting_myfun you will implictly test the rrule for myfun.

---

Long term solution

I am not sure.
More thinking required.

[oxinabox]

Duplicate of #99