Projection on SPDs is not projecting onto SPDs

[kellertuer]

Hi, nice to see another package doing optimizationon manifolds! I have not yet had the time to check this versus what pymanopt is doing (I think they use tensor flow as a backend, too?) But I just noticed that

https://github.com/master/tensorflow-manopt/blob/93402f6770d5b3c45f232340fddfa92a7126f19a/tensorflow_manopt/manifolds/symmetric_positive.py#L37-L41

This might be wrong. For SPDs, the characteristic property is, that all eigenvalues are positive, so this projection is not projection onto the manifold (of SPDs) but onto the set of positive semidefinite matrices. There is no projection onto the SPDs since that set is open in the set of (symmetric) matrices.

[master]

Hi - thank you for the feedback! tensorflow-manopt differs from pymanopt in two aspects: 1) it implements a different set of optimizers, focusing on (adaptive) SGD methods, and 2) tensorflow-manopt aims for interoperability with the TensorFlow API, rather than just using its autodiff capabilities. See, for example, Keras-compatible layers implementation.

I agree with your remark about SPD projection. It's implemented this way for backward compatibility reasons, so perhaps it should print a warning on calling.

[kellertuer]

Thanks for the explanation, it seems reasonable to do an own project then.

yes, it should bring a warning I think.