Nail down stability of IBM with very high orders. #404

pnkraemer · 2021-05-05T09:14:52Z

When playing with the code, I noticed that the changes in #386 were absolutely crucial for stability of high order IBMs.
I thought it would be good to have a test that nails this down. The changes in this PR are:

add an optional mini-damping factor for IBM covariances: this way (using 1e-15), the maximum order for IBM kalman filtering is not 10 anymore, but 15! :D If we do crazy small steps, the max order is "only" 14 now. @nathanaelbosch
write tests for the above point, as well as the no-damping limit of 10.
made mini updates in the car tracking problem zoo that improve its square-root usage in a very minor way.

codecov · 2021-05-05T09:18:25Z

Codecov Report

Merging #404 (cd3dedb) into master (fed9dfc) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #404      +/-   ##
==========================================
- Coverage   83.67%   83.67%   -0.01%     
==========================================
  Files          99       99              
  Lines        4957     4955       -2     
  Branches      670      669       -1     
==========================================
- Hits         4148     4146       -2     
  Misses        598      598              
  Partials      211      211

Impacted Files	Coverage Δ
src/probnum/statespace/discrete_transition.py	`90.12% <ø> (ø)`
...um/problems/zoo/filtsmooth/_filtsmooth_problems.py	`90.90% <100.00%> (+0.09%)`	⬆️
src/probnum/statespace/integrator.py	`99.02% <100.00%> (-0.02%)`	⬇️

schmidtjonathan · 2021-05-05T12:55:46Z

src/probnum/statespace/integrator.py

+        process_noise_cov_cholesky = np.linalg.cholesky(
+            self._proc_noise_cov_mat
+            + self._process_noise_damping * np.eye(len(self._proc_noise_cov_mat))
+        )


Is this nugget necessary for numerical stability?

only for orders > 11 :)

Should we then actually add this argument? I guess it will be needed quite rarely, if not only for the test you added in this PR 😄 Correct me if this is wrong

Can be ignored, I read your answer below

no, not really. I was playing around with it and thought it was cool, but I see your concerns. Shall I remove it?

schmidtjonathan · 2021-05-05T12:59:06Z

src/probnum/statespace/discrete_transition.py

+            state_trans_mat_fun=lambda t: state_trans_mat,
+            shift_vec_fun=lambda t: shift_vec,
+            proc_noise_cov_mat_fun=lambda t: proc_noise_cov_mat,
+            proc_noise_cov_cholesky_fun=lambda t: proc_noise_cov_cholesky,


👍 Thanks! I always get a bit twitchy when more than 2 arguments are passed positionally

schmidtjonathan · 2021-05-05T13:08:25Z

src/probnum/statespace/integrator.py

+        # The Cholesky factor of the process noise covariance matrix of the IBM
+        # always exists, even for non-square root implementations.
+        proc_noise_cov_cholesky = (
+            self.precon(dt)
+            @ self.equivalent_discretisation_preconditioned.proc_noise_cov_cholesky
+        )


Do we get any benefit from computing this Cholesky factor when not using the sqrt implementation?
Otherwise, we might make this conditional.

Hmmm my thinking here was that this cholesky factor is created for the equivalent discretisation (preconditioned), so it was super weird not to translate it here, too. What do you think?

Could we make the computation of the Cholesky factor - both in equivalent_discretisation_preconditioned and in discretise - conditional on self.forward-/backward_implementation?

yes, we could. Thought it only happens once, so the impact on runtime should be almost neglectible. Do you think that it is worth it nevertheless?

schmidtjonathan · 2021-05-05T13:09:59Z

tests/test_filtsmooth/test_gaussfiltsmooth/test_kalman.py

+        step=1e-12,
+        forward_implementation="sqrt",
+        backward_implementation="sqrt",
+        _process_noise_damping=1e-14,


I am surprised that a nugget of 1e-14 does anything at all.

Do you think it necessary to establish an entirely new kwarg, solely for the case that somebody wants to use a 14 times integrated model? 😄

Well, I dont want to stretch the word "necessary" too much here... :) I just thought it was cool to get these high orders! ;)
And it is a "hidden" kwarg that defaults to 0, so I thought it doesnt hurt too much. Though if you find it annoying I am open for talking about it :)

Well maybe I am biased, but I quickly get confused when a method offers me to tweak every part of the inner workings, and especially if the documentation takes half an hour to read (not that this would be the case here!) So I often prefer clarity over control, given that often, the kwargs are only used in very special cases. Do hidden kwargs appear in the docs?

schmidtjonathan · 2021-05-05T13:11:46Z

tests/test_filtsmooth/test_gaussfiltsmooth/test_kalman.py

+    """With damping, the filter achieves really high orders (we test 25, but 50 was
+    possible too, for some reason)."""
+    regression_problem, statespace_components = filtsmooth_zoo.car_tracking(
+        model_ordint=25,


What is happening.

Just for the jokes, the test passes for order=75 as well (no typo!), but I thought 25 was sufficiently extreme :D

All just by removing the matrix inversion, right ?

and by using this mini damping factor that makes the process noise always positive definite.

schmidtjonathan · 2021-05-05T13:17:02Z

Just checking: Your PR description is outdated, right?

the maximum order for IBM kalman filtering is not 10 anymore, but 15!

this should be 25 by now, right? I don't want to be petty here, just check if the 25-order thing is not some kind of special case. 😆

pnkraemer · 2021-05-05T13:18:02Z

25 is only filter. filtsmooth starts complaining at 15

schmidtjonathan · 2021-05-05T13:18:37Z

25 is only filter. filtsmooth starts complaining at 15

I see. Still. That's quite remarkable

pnkraemer · 2021-05-05T13:20:40Z

as I said in one of the comments as well: 75 worked, too, but I thought no one would believe me so I used 25 for the test ;)

schmidtjonathan

Crazy stuff! Thanks for working that out :)

…obnum into high_order_ibm_test

pnkraemer added 4 commits May 5, 2021 11:08

made car tracking problem higher-order and sqrt ready

f0b0b69

wrote tests for very high order IBM filters

9db0fb2

keyword arguments in discrete transition

5dc86e0

fixed square-root detail in IBM and added tiny damping

be383a2

pnkraemer requested a review from schmidtjonathan May 5, 2021 09:17

really high order filtering test

7bd8487

pnkraemer mentioned this pull request May 5, 2021

Mega high order integrated Wiener process priors #405

Closed

pnkraemer added 2 commits May 5, 2021 12:41

simplified equivalent discretistaion code for IBM

8f6613d

Merge branch 'master' into high_order_ibm_test

5c27673

schmidtjonathan reviewed May 5, 2021

View reviewed changes

schmidtjonathan approved these changes May 5, 2021

View reviewed changes

pnkraemer added 2 commits May 5, 2021 15:33

removed damping

4bdf42a

Merge branch 'high_order_ibm_test' of https://github.com/pnkraemer/pr…

cd3dedb

…obnum into high_order_ibm_test

pnkraemer merged commit d050f26 into probabilistic-numerics:master May 5, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nail down stability of IBM with very high orders. #404

Nail down stability of IBM with very high orders. #404

pnkraemer commented May 5, 2021

codecov bot commented May 5, 2021 •

edited

Loading

schmidtjonathan May 5, 2021

pnkraemer May 5, 2021

schmidtjonathan May 5, 2021

schmidtjonathan May 5, 2021

pnkraemer May 5, 2021

schmidtjonathan May 5, 2021

schmidtjonathan May 5, 2021

pnkraemer May 5, 2021

schmidtjonathan May 5, 2021 •

edited

Loading

pnkraemer May 5, 2021

schmidtjonathan May 5, 2021

pnkraemer May 5, 2021

schmidtjonathan May 5, 2021 •

edited

Loading

schmidtjonathan May 5, 2021

pnkraemer May 5, 2021

schmidtjonathan May 5, 2021

pnkraemer May 5, 2021

schmidtjonathan commented May 5, 2021

pnkraemer commented May 5, 2021

schmidtjonathan commented May 5, 2021

pnkraemer commented May 5, 2021

schmidtjonathan left a comment

Nail down stability of IBM with very high orders. #404

Nail down stability of IBM with very high orders. #404

Conversation

pnkraemer commented May 5, 2021

codecov bot commented May 5, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

schmidtjonathan May 5, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

schmidtjonathan May 5, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

schmidtjonathan commented May 5, 2021

pnkraemer commented May 5, 2021

schmidtjonathan commented May 5, 2021

pnkraemer commented May 5, 2021

schmidtjonathan left a comment

Choose a reason for hiding this comment

codecov bot commented May 5, 2021 •

edited

Loading

schmidtjonathan May 5, 2021 •

edited

Loading

schmidtjonathan May 5, 2021 •

edited

Loading