Ceres Like API?

[soulslicer]

Hi all,

I have a problem statement, where I am optimizing over parameters T, with several losses L. For speed I actually already implemented the Jacobian computation of dLdT.

In Ceres, I just needed to pass the residuals, current parameters and dLdT and ceres would just go ahead and apply Gauss Newton with Line Search or Trust Region and optimize it for me.

Do you have an example, or a mechanism where I can pass dLdT, residuals and T and you handle the optimization? I want to use this because Ceres is too slow (Takes almost 100ms per iteration) for 200 parameters and 200 Losses. (dLdT Jacobian is a matrix 200x200)

[hjmshi]

Hi,

Since the method that is implemented in this module is an L-BFGS method (not Gauss-Newton), it only relies on computing the gradients of your full objective function, not the residuals. Our implementation was designed to be integrated with PyTorch which allows for easy computation of the full gradient, so it doesn't make sense for us at this point to have a Ceres-like API.

Is your current problem implemented in PyTorch? If so, it is sufficient to simply compute the full gradient using the residual and Jacobian you have already computed (g = J'r), or directly use autograd to get the gradient of the full nonlinear least-squares problem rather than computing gradients of each individual residual.

[soulslicer]

I suppose I can just apply the Gauss Newton math myself. I did not realize this library is Quasi Newton methods.