Conjugation convention changed from v0.9 to v0.10

[sethaxen]

I noticed when adding FiniteDifferences v0.10 compatibility to ChainRules (JuliaDiff/ChainRules.jl#198) that many of the * tests suddenly failed.

It seems that FiniteDifferences changed its conjugation convention in the change from v0.9.8 to v0.10.0. An example:

Setup:

julia> using FiniteDifferences

julia> x, y, Δz = 1.0 + 2.0im, 3.0 + 4.0im, 5.0 + 6.0im;

On v0.9.8:

julia> j′vp(central_fdm(5, 1), *, Δz, x, y)
(-8.999999999999446 + 38.000000000000284im, -6.999999999999783 + 15.99999999999862im)

julia> conj.(j′vp(central_fdm(5, 1), *, conj(Δz), x, y))
(39.00000000000018 - 1.9999999999994031im, 16.999999999998604 - 4.000000000000034im)

On v0.10.0 (and v0.10.2)

julia> j′vp(central_fdm(5, 1), *, Δz, x, y)
(38.99999999999998 - 2.0000000000012363im, 16.9999999999999 - 3.999999999999728im)

julia> conj.(j′vp(central_fdm(5, 1), *, conj(Δz), x, y))
(-9.000000000000066 + 37.999999999999986im, -7.00000000000015 + 16.00000000000108im)

So it seems j′vp switched conjugate conventions in the jump to v0.10.0. I didn't see this documented anywhere. Was this an intended change?

[nickrobinson251]

I think this was intended but I'm not 100% sure. The reason it was marked as breaking was definitely due to a change in how we treat complex functions. Hopefully @willtebbutt or @simeonschaub can give details :)

[willtebbutt]

I'm going to lay down what I think it is necessary for our rules to satisfy in terms of vectors, their tangents, and cotangents in Euclidean space, and assume that everything that I'm saying generalises to non-Euclidean space and that I can safely talk about cotangents as vectors rather than linear operators. I'm doing this so that we have a better idea regarding what "the right convention" looks like.

Let A : X → Y, and x ∈ X and y ∈ Y such that y = A(x). Let ẋ and ẏ be "tangent" vectors satisfying ẏ = D(A, x)(ẋ) where D(A, x) is the differential of A at x (a.k.a. pushforward / the thing that frules compute). Let x̄ and ȳ be elements of the dual spaces of the spaces in which ẋ and ẏ reside.

Finally, call Dt(A, x) the pullback or adjoint of A at x.

The defining property of D and Dt is

<ȳ, D(A, x)(ẋ)> = <Dt(A, x)(ȳ), ẋ>

where <., .> denotes the usual inner product.

So AFAICT the criterion by which we should just whether what we're doing is correct is whether or not it satisfies the above.

@sethaxen do you agree with this?

Although I suppose that there's also the point that the function name j′vp suggests we compute adjoint(adjoint(ȳ) * J), so we should probably also ask whether we currently satisfy that.

[simeonschaub]

JuliaDiff/ChainRulesCore.jl#165 might also be relevant to this

[sethaxen]

Sorry, it's a busy week, and I haven't had time to work through this carefully.

The defining property of D and Dt is

<ȳ, D(A, x)(ẋ)> = <Dt(A, x)(ȳ), ẋ>

where <., .> denotes the usual inner product.

At first read, I don't follow this. The inner product between a vector and a covector is not I think a well-defined thing. A covector is defined as a map from a vector to a scalar. The inner product between two vectors is equivalent to mapping one vector to a covector via a musical isomorphism and then acting on the other vector to produce a scalar. In index notation:

<v, w> = g_{ij} v^i w^j = v^i w_i,

where v^i is element i of a vector v, v_i is element i of a covector, and g_{ij} is the matrix representation of the metric tensor.

[sethaxen]

Also cc @MasonProtter

[willtebbutt]

At first read, I don't follow this. The inner product between a vector and a covector is not I think a well-defined thing. A covector is defined as a map from a vector to a scalar. The inner product between two vectors is equivalent to mapping one vector to a covector via a musical isomorphism and then acting on the other vector to produce a scalar. In index notation:

I was under the impression that for finite-dimensional vectors in euclidean space (I guess also when the metric tensor is just the identity everywhere?) it's possible to parametrise a covector by a vector and an inner product -- I'm basing this off section 4.5 of Dougal's thesis [1]. My understanding of differential geometry is very limited, so it's very possible that I'm tripping up and not realising it.

[1] - https://dash.harvard.edu/bitstream/handle/1/33493599/MACLAURIN-DISSERTATION-2016.pdf?sequence=4&isAllowed=y

[MasonProtter]

Yes that’s correct. In a euclidean manifold, you can globally use Cartesian coordinates which have a trivial metric (the identity). This is why most people aren’t taught about covectors, they look just like vectors in a Euclidean space.

The only subtlety is that if you allow complex numbers, then the metric has a complex conjugate in it. I.e. the metric in the complex plane is the conjugate, and the metric in the complexification of a N dimensional Euclidean manifold is just the composition of the identity with the complex conjugate.

[oxinabox]

what needs to be done to close this issue?

[sethaxen]

It think all that is needed is to document somewhere that v0.10.0 changed the conjugation convention for j′vp with complex arguments.

With JuliaDiff/ChainRulesCore.jl#167, I am now convinced that this change was correct. It was, however, surprising.

[oxinabox]

I have updated the release notes to reflect that
https://github.com/JuliaDiff/FiniteDifferences.jl/releases/tag/v0.10.0