the size of Jacobians when using localparamerization and analytical derivative

[RomaTeng]

Hello,
I am trying Ceres for solving nonlinear least squares.
There is a little problem (but I am not sure):
For the cost function f(x), the residual is n_f dimension, the global size of x is n_xg,
the local size of x is n_xl ( n_xl << n_xg ).
For efficiency, we are trying to use the analytical cost function and provide the Jacobians.
However, in Ceres, we have to provide the n_f * n_xg Jacobians(in cost function) and the n_xg * n_xl
Jacobians (in Local parameterization).

Our current solution is filling the matrix with zeros:
we use [n_f * n_xl 0] as the Jacobians in the costfunction and [ I ; 0 ] as the Jacobians in the localparamerization. From our experiment, this solution works.

If we just want to directly provide the n_f * n_xl Jacobians ( especially for the case n_xg >> n_xl ) , how should we do?

Dear developers, I think you can understand my problem.

Regards,

Teng

[sandwichmaker]

The local parameterization also has to implement a Plus operation which must be compatible with the Jacobian. Therefore, the solution you are using only makes sense if you can ignore the last n_xg - n_xl coordinates, which if you can, then why use that as a parameter size at all?

All that said, the big question is, how much of the time that you are trying to optimize is being spent in the application of the local parameterization that you feel this is necessary? generally speaking this time is quite small.

[RomaTeng]

Thanks for your reply.

For a simple example, we try to use 9 * 1 (3 * 3) elements to denote a rotation matrix and use the exponential mapping ( Exp( ) ) as the oplus operation involving all these 9 elements. We think this way can save time compared to quaternion/axis-angle because the 9 elements can be directly used in the ``Evaluate" without the need to transform the state vector to a rotation matrix. For our understanding, we have to provide a n_f * 9 Jacobian matrix ( for orientation, in Cost function ) and a 9*3 Jacobian in local-parameterization.

However, the size of a real Jacobian matrix for building the Hessian matrix only need the local size of the parameter instead of the global size.

[sandwichmaker]

@RomaTeng I understand that. My question to you is,

1. what do you propose as a solution.
2. have you measured the amount of time being spent constructing the hessian as part of the over all solve? is this a curiosity or do you actually have a performance problem? if so, can you share some numbers?

[RomaTeng]

1. Our current solution is:
   First we calculate the n_f * 3 matrix ``J_real" , and put " J = [ J_real 0_{ nf * 6 } ] " as the Jacobian in the cost function. Then we provide the J_local = [ I_{3 * 3} ; 0_{6 * 3} ] as the Jacobian in the local-parameterization. In this way, J * J_local is equal to J_real, which is used in building the Hessian matrix.
   However, you know, this solution is not elegant. An interesting point is that
   in g2o (another nonlinear least square solver), parameter can be denoted by any struct (not restricted in the double type), and the global size is not needed ( but the local size is needed ).

2. We have not tested in a large scale problem yet. Maybe this issue is not so significant for us now. However, more tricks can be easily used if we only need to provide the local size.
   For example, we can use a number in parameter as a flag to switch between two different oplus operations. For example, we can use a number in parameter as a flag to switch between two different residual equations.

[sandwichmaker]

how does this work with the Plus operation for your local parameterization?

[RomaTeng]

Now our solution works.
I am sure this is very efficient.

[jzubizarreta]

Hi @RomaTeng,

How do you implement the trick of jacobians?
I am trying to do the same without success.

Does the Plus operation need any special treatment?

Thanks in advance,

Jon

[RomaTeng]

@jzubizarreta

In the residual block, I provide
Jacobian = [realJacobian; 0]
In the local parameterization, I provide Jacobian = [ I ; 0]
The Plus operation is defined as the standard rotation plus (exp).
This way works well.

[k-sheridan]

Hi @RomaTeng,

In the LocalParameterization, ceres gives us the ability to change the MultiplyByJacobian function which appears to be the way ceres maps the globally parameterized jacobians into the locally parameterized jacobians.

With the ability to modify this function, we should be able to skip that unnecessary local_matrix = global_matrix * jacobian entirely by simply assigning the already computed local_matrix stored in the global_matrix directly to the actual local_matrix.

I have not tried this yet, but if it works, then only a little time is wasted reassigning variables. This is much better than multiplying by an identity matrix. I will be trying this out in the coming weeks.

[RomaTeng]

Hi @RomaTeng,

In the LocalParameterization, ceres gives us the ability to change the MultiplyByJacobian function which appears to be the way ceres maps the globally parameterized jacobians into the locally parameterized jacobians.

With the ability to modify this function, we should be able to skip that unnecessary local_matrix = global_matrix * jacobian entirely by simply assigning the already computed local_matrix stored in the global_matrix directly to the actual local_matrix.

I have not tried this yet, but if it works, then only a little time is wasted reassigning variables. This is much better than multiplying by an identity matrix. I will be trying this out in the coming weeks.

Thanks. If your method works, could you tell me?
But I know "MultiplyByJacobian is only used in GradientProblem".
@k-sheridan