Calibration

[bassoste]

Hi again, and thanks for fixing the area-related bug.
I have run the model for some catchments from the CAMELS dataset in order to have an idea of the capability of the model.
In all cases results are not good. I report Nash Sutcliffe Efficiencies to give you an idea:

Although this is ok for educational purposes, it would also be nice to be able to show a case with good fitting.
I guessed the problem might be the calibration method used, which finds an optimal parameter set which corresponds to a local minima. However, this should not be the case, since scipy.optimize.differential_evolution "finds the global minimum of a multivariate function".

Implementation/use of a more advanced calibration method (e.g., the Shuffled Complex Evolution Algorithm) or the integration of available python libraries (e.g., spotpy) might solve the problem (again "might", I did not test them beforehand). But it might also not be necessary.
I am currently trying to fix parameters bounds first, to check if it helps.
Thanks again, Stefano

[bassoste]

I report Nash Sutcliffe Efficiencies (forgotten above) to give you an idea:

basin '11523200'
NSE of the .fit() optimization: -0.1493
NSE of the Monte-Carlo-Simulation: -0.2569

basin '11266500'
NSE of the .fit() optimization: -0.0116
NSE of the Monte-Carlo-Simulation: 0.4024

basin '01047000'
Run 1
NSE of the .fit() optimization: 0.5765
NSE of the Monte-Carlo-Simulation: nan
Run 2
NSE of the .fit() optimization: 0.5745
NSE of the Monte-Carlo-Simulation: 0.4357

basin '11284400'
NSE of the .fit() optimization: nan
NSE of the Monte-Carlo-Simulation: -0.8736

basin '01170100'
Run 1
NSE of the .fit() optimization: nan
NSE of the Monte-Carlo-Simulation: 0.1061 (10000 iterations)
Run 2
NSE of the .fit() optimization: -0.0849
NSE of the Monte-Carlo-Simulation: 0.3046 (100000 iterations)

[kratzert]

Hi @bassoste and sorry for my late reply. I agree that it would be good to have better optimization strategies implemented, I am however not sure when and if I will be able to do it. I created this library some time ago for the pure fun of testing the performance capabilities of the Numba library to some problem of my field. It was never my intent to provide a library that can afterwards be used to get the best model possible out-of-the-box but rather to show how you can speed up numerical models.

I'm also unsure how much time is worth investing into this since a) I don't use this library at all for anything and b) I don't now if there are more people then you using it. Don't get me wrong, but nobody pays me to do it and I am quite busy with doing my PhD and raising a child.

Thus, before doing anything here: Which model did you use for the results above? Are your parameter bounds making sense? Being worth than MCS is not a good sign, though..

[kratzert]

Closed due to missing response.