New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameter identification using Nelder-Mead - "MoBi toolbox for Matlab" VS "MoBi" #75

Closed
DanielMoj opened this Issue Sep 26, 2017 · 8 comments

Comments

Projects
None yet
5 participants
@DanielMoj

DanielMoj commented Sep 26, 2017

Dear Community,

At the moment i am trying to re-identify parameters within MoBi that I initially (more than 1.5 years ago) identified using the MoBi toolbox for Matlab.
In my past work (toolbox for Matlab) I used Nelder-Mead with a parameter parameter range. Now that I am using the parameter identification from within MoBi i have some issues identifiying the same parameter values as a year ago with the MoBi toolbox for Matlab.

I think that I configured the identification correctly in MoBi (e.g. linear/log scaling) - I used the same simulations and the same observed data.

Now my question is if I can reproduce the old parameter values from within MoBi?
Do you have any experience on that?
Is the Nelder algorithm in MoBi exactly the same as the one used in the MoBi toolbox for Matlab?

I'd very much appreciate your experience on that issue :)

Best
Daniel

@PavelBal

This comment has been minimized.

Show comment
Hide comment
@PavelBal

PavelBal Sep 27, 2017

Member

In my experience, the curret Nelder-Mead (and also the LM) in MoBi behaves completely different from that in the old ToolBox. In MoBi, it ignores the boundary values of the parameters, which is, according to documentation, the correct behavior (unconstrained algo). However, the Nelder-Mead in the old ToolBox very rarely quit the defined range - actually so rare that I thought it was a bug.

Unfortunately, I could not find a way to obtain identical identification results in MoBi and Matlab/R.

Member

PavelBal commented Sep 27, 2017

In my experience, the curret Nelder-Mead (and also the LM) in MoBi behaves completely different from that in the old ToolBox. In MoBi, it ignores the boundary values of the parameters, which is, according to documentation, the correct behavior (unconstrained algo). However, the Nelder-Mead in the old ToolBox very rarely quit the defined range - actually so rare that I thought it was a bug.

Unfortunately, I could not find a way to obtain identical identification results in MoBi and Matlab/R.

@DanielMoj

This comment has been minimized.

Show comment
Hide comment
@DanielMoj

DanielMoj Sep 27, 2017

Hi PavelBal,

thank you very much for your response!
It is reassuring to know that you can confirm my finding!

Did other users have different experiences?

And additionally, is it possible to obtain identical identification results in MoBi and Matlab/R using "MC"?

DanielMoj commented Sep 27, 2017

Hi PavelBal,

thank you very much for your response!
It is reassuring to know that you can confirm my finding!

Did other users have different experiences?

And additionally, is it possible to obtain identical identification results in MoBi and Matlab/R using "MC"?

@Yuri05 Yuri05 added the question label Sep 27, 2017

@msevestre

This comment has been minimized.

Show comment
Hide comment
@msevestre

msevestre Sep 27, 2017

Member

@DanielMoj I had various results trying to compare old and new optimization. In some instances, results were almost identical. In other, the optimal values were widely different.

Now I have a few questions for you:

  • Why do you absolutely want to reproduce the old values?
  • How do old and new error compare?
  • What happens if you start the optimizer in MoBi with your older BEST results. Is the outcome different? Or is MoBi still finding the same solution has before?
Member

msevestre commented Sep 27, 2017

@DanielMoj I had various results trying to compare old and new optimization. In some instances, results were almost identical. In other, the optimal values were widely different.

Now I have a few questions for you:

  • Why do you absolutely want to reproduce the old values?
  • How do old and new error compare?
  • What happens if you start the optimizer in MoBi with your older BEST results. Is the outcome different? Or is MoBi still finding the same solution has before?
@DanielMoj

This comment has been minimized.

Show comment
Hide comment
@DanielMoj

DanielMoj Sep 28, 2017

@msevestre thank you very much for sharing your experience!

Now to the questions :)

  1. I want to reproduce them to be sure that my results (conclusions and so on) of the previously developed models are also valid if I had used the open source version.
  2. I have to check the errors!
  3. Most but not all newly identified values are within a ~15% range from the mobi toolbox PI vlaues, But it seems to be highly dependent on the starting values!

DanielMoj commented Sep 28, 2017

@msevestre thank you very much for sharing your experience!

Now to the questions :)

  1. I want to reproduce them to be sure that my results (conclusions and so on) of the previously developed models are also valid if I had used the open source version.
  2. I have to check the errors!
  3. Most but not all newly identified values are within a ~15% range from the mobi toolbox PI vlaues, But it seems to be highly dependent on the starting values!
@TheiBa

This comment has been minimized.

Show comment
Hide comment
@TheiBa

TheiBa Sep 28, 2017

Member
Member

TheiBa commented Sep 28, 2017

@DanielMoj

This comment has been minimized.

Show comment
Hide comment
@DanielMoj

DanielMoj Oct 2, 2017

Hi Thomas,

thank you for all the good thoughts and impulses! I guess we have to discuss further internally, especially whether the ~15% deviation is indeed an issue for us. As soon as we have made up our minds and decided how to proceed I can share our trail of thoughts on what we decided and how we came to our decision. May help others :)

Best,
Daniel

DanielMoj commented Oct 2, 2017

Hi Thomas,

thank you for all the good thoughts and impulses! I guess we have to discuss further internally, especially whether the ~15% deviation is indeed an issue for us. As soon as we have made up our minds and decided how to proceed I can share our trail of thoughts on what we decided and how we came to our decision. May help others :)

Best,
Daniel

@TheiBa

This comment has been minimized.

Show comment
Hide comment
@TheiBa

TheiBa Oct 4, 2017

Member
Member

TheiBa commented Oct 4, 2017

@msevestre msevestre added the answer label Oct 11, 2017

@msevestre

This comment has been minimized.

Show comment
Hide comment
@msevestre

msevestre Apr 14, 2018

Member

Closing. Please let us know if you have any more questions

Member

msevestre commented Apr 14, 2018

Closing. Please let us know if you have any more questions

@msevestre msevestre closed this Apr 14, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment