Gradient of inv and logdet UpperTriangular matrix is not UpperTriangular

[aterenin]

For sparse Gaussian processes, one often wants to parametrize the variational distribution using its Cholesky factors, which form an upper-triangular matrix and are trainable.

However, when taking gradients, Zygote instead computes the gradient with respect to all entries in the UpperTriangular matrix, including ones that are set to zero by virtue of it being UpperTriangular. This is related to #163.

For the moment, a workaround suitable for training models is to simply call UpperTriangular(x) on the upper-triangular matrix x before using it.

[sdewaele]

Perhaps this is more related to #402. As an alternative you could insert the projection function ℙ in your code, see #402 (comment). Then, the adjoint will be an UpperTriangular. You would have to add the definition for the projection of the UpperTriangular:

ℙ(::Type{T},X) where {T<:UpperTriangular} = UpperTriangular(X)