Efficient factorization of diagonal outer Hessian in convex-over-nonlinear and nonlinear least-squares costs

[sandmaennchen]

• add flags indicating whether outer hessian is diagonal
• if outer hessian is diagonal and ny > 4, its diagonal Cholesky factorization W_chol is computed manually and stored as blasfeo_dvec
• adjust matrix-matrix/matrix-vector multiplications involving W_chol accordingly

[FreyJo]

I went through the code and left some comments.

[FreyJo]

I think that now that we are only storing the diagonal as a vector it should be always more efficient.
Then we can remove this TODO.
@giaf do you agree?

[FreyJo]

I made a small comparison with these settings:

PendulumOcpOptions(
    qp_solver="FULL_CONDENSING_DAQP",
    cost_variant="CONL_LARGE",
    # cost_variant="CONL",
    nlp_solver="SQP_RTI",
    cost_discretization="INTEGRATOR",
    integrator_type="IRK",
    N=50
)

For cost_variant="CONL", with ny=nx+nu=5, there is no speedup:

For cost_variant="CONL", with ny=2*(nx+nu), there is a big speedup:

Thus, I set outer_hess_is_diag = 0 if ny <= 4, in the last commit.

[FreyJo]

I think it will make cost integration for medium+ sized problems much faster.
Great collaboration again 📈