Hierarchical optimization: avoid extra simulations

[dweindl]

When using hierarchical optimization, there are currently two simulations performed during each objective evaluations. This could be avoided if the gradient is not required and could significantly improve performance.

See here:

pyPESTO/pypesto/hierarchical/calculator.py

Lines 139 to 141 in cfd673d

	# TODO speed gain: if no gradient is required, then simulation can be
	# skipped here, and rdatas can be updated in place
	# (y, sigma, res, llh).

Ideally after AMICI-dev/AMICI#2215

[Doresic]

Will look into and update

[dweindl]

@Doresic : This is finished, right?

[Doresic]

Yes, mostly done.
That's the calculate_directly call of the RelativeCalculator.

Amici is still called twice through call_amici_twice in some special cases. One of those is also if 2 is in sensi_orders.
This is a part that can be further optimized. As far as I know, amici doesn't really calculate 2nd order sensitivities but uses FIM or some BFGS-type of approximation. This is something that could be implemented in the calculate_directly call as well. In that case, one would not have to call amici twice even when requesting 2nd order derivatives.

One special case of calling with 2nd order sensitivities, but can already be used with calculate_directly, is optimization with fides. If one just uses pypesto.optimizer.FidesOptimizer() then it will be used with call_amici_twice as fides always calls for second order sensitivities which are then returned by amici as some approximation.
However, one can instead select pypesto.optimizer.FidesOptimizer(hessian_update = fides.BFGS) which will make fides calculate 2nd order approximations itself, so it will not request them from the objective. Then, the model will not be simulated twice and calcualte_directly will be used.

TLDR: can be closed, but can also be slightly improved.

[dweindl]

Ah, right. Thanks for the update.