Handling Dynamic parameters in CppADCodeGen

[jcarpent]

Current master branch of CppAD allows the use of dynamic parameters.
In other words, it is possible to change at running time some parameters values that are not consider as Independent variables.
See https://github.com/coin-or/CppAD/blob/46be2e2dbcaba47d8a38e8d9110efc48c7245754/cppad/core/new_dynamic.hpp#L103 for further details.

I want to know how to implement this feature in CppADCOdeGen.
@joaoleal can you explain me where I need to edit this new feature?

[jcarpent]

@joaoleal And also if you have some guess on how to proceed?

[joaoleal]

I have not update CodeGen to deal with the changes in CppAD explicitly.
However, it is already possible to do this in CppADCodeGen since you can specify which derivatives you are interested in.
A model can be taped considering all variables/parameters and then the source generation can produce derivatives for custom sparsity patterns (e.g., excluding derivatives for a subset of variables, a.k.a, parameters).
This allows you to redefine values for parameters after the taping/generation process for which no derivatives are computed.
I've used this multiple times.
For instance, see https://github.com/joaoleal/CppADCodeGen/blob/master/test/cppad/cg/model/dynamiclib/dynamic.cpp.

I would say that there is no need to make changes to CppADCodegen for the new feature of CppAD.
It should just work, now you don't have to define custom sparsity patterns.
I have not tested it though.

[joaoleal]

I need to take a better look at the implementation in cppad.
It might not be working the way I initially thought.
Is there a complete example using parameters?

[jcarpent]

Yes, there is a full example here:
https://github.com/coin-or/CppAD/blob/46be2e2dbcaba47d8a38e8d9110efc48c7245754/example/general/new_dynamic.cpp#L30

@bradbell introduces a new concept of Dynamic type as it can be seen in line
https://github.com/coin-or/CppAD/blob/46be2e2dbcaba47d8a38e8d9110efc48c7245754/cppad/local/independent.hpp#L87

[bradbell]

The basic idea is that all of the sparsity patters and derivative computations are relative to the variables and do not include the dynamic parameters.

You can find extensive use of dynamic parameters in the test file
https://github.com/coin-or/CppAD/blob/master/test_more/general/new_dynamic.cpp

I am experimenting, and have seen some improvement, with the effect on speed and memory in the cppad_mixed project:
https://github.com/bradbell/cppad_mixed

It is hard for me to know what the effect would be for the CppADCodeGen project. I would suggest just implementing a small subset of operations and tests to see if it helps.

[joaoleal]

I started adapting CppADCodeGen to support CppAD parameters but it is quite an invasive change.
It will take some time to get it right.
I am currently adding an additional argument for parameters in most model methods (CppADCodeGen models are stateless).
However, this is breaking some functionality which depends on all data coming from the independent variables.
I will push a new branch once I have something that compiles...

[jcarpent]

Sounds really great.
Thanks for these code developments.

[joaoleal]

Started a new branch: feature/parameters
It is still work in progress and I might still make changes to the API but there is a test already working: dynamic_with_params.cpp

[jcarpent]

Ok thanks for reporting this adding.
I will have a look to the code to see the API modifications.

[jcarpent]

@joaoleal Do you have some news for this feature?

[joaoleal]

I still have some issues with the support for atomic functions. At the moment it is not possible to use parameters with adfun wrapped with an atomic functions. Brad has said that CppAD will have a new API that should allow this. I was waiting for this feature. I can also just check if there are parameters and fail for the time being. A quarta, 24/10/2018, 09:35, Justin Carpentier <notifications@github.com> escreveu:

…

@joaoleal <https://github.com/joaoleal> Do you have some news for this feature? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAfTYRqFMUG38uJ1uftb5pr8LZv-dxkUks5uoCZNgaJpZM4WMwpi> .

[jcarpent]

It sounds great. Thanks for this quick feedback.
I'll wait for the next feature from CppAD.

[bradbell]

Thanks for the feedback, that this is important and holding you back. I will try to get to it soon.

[bradbell]

I just realized that this issue is still outstanding. Does the atomic_three interface
https://coin-or.github.io/CppAD/doc/atomic_three.htm
satisfy the requirements for this issue ?

[joaoleal]

It suits my current needs but there is still a test failing related to custom Hessian sparsity patterns. Hopefully I will have some time during the Christmas holidays to solve it.

[mohamdev]

Hi, Is there a way to use dynamic parameters with CppADCodeGen, or is this issue still topical ?

[joaoleal]

I haven't completed the work on the branch yet.
At the moment the way to use dynamic parameters in CppADCodeGen is to consider them as regular variables and to define custom sparsity patterns without them.
The end result is the same but you do not configure dynamic parameters directly in CppAD.

[mohamdev]

Great, thank you very much for your feedback :)

[bradbell]

Using the techniques in
#34 (comment)

You can compute derivatives any way you like in CppAD (using dynamic parameters and such) and then, put those computations in the zero order forward mode for a CppAD function and then generate the source code for that function.

[mohamdev]

Thank you for the example, it's really helpful to me

[bradbell]

One thing that may not be obvious is that in order to use base2ad with atomic functions, the atomic functions need to define the AD<Base> case; see AD<Base> in atomic forward
https://coin-or.github.io/CppAD/doc/atomic_three_forward.htm#AD&lt;Base>
and in atomic reverse
https://coin-or.github.io/CppAD/doc/atomic_three_reverse.htm#AD&lt;Base>

These implementations many in turn call atomic functions to do their operations; e.g., the derivatives of matrix multiply may in tern use matrix multiply; see
https://coin-or.github.io/CppAD/doc/atomic_two_eigen_mat_mul.hpp.htm
I have not yet converted this example to use the atomic_three interface nor the AD<Base> case.

[chrisFlexiv]

By any chance, do you have an update for this feature? Thanks!

[bradbell]

I am working on an example that implements element-wise vector operations as atomic functions. This shows how to implement the AD derivative calculations using atomic functions; see
https://coin-or.github.io/CppAD/doc/atomic_vector.htm

I plan to extend this example to include all the possible atomic function calculations and all the standard math functions.
Once I have this implemented where the base class is double, I plan to convert it to the case where the base class is a template parameter.

@joaoleal I am not sure how one would use such atomic functions in CppADCodeGen (once the base type is a template parameter) ?