Develop Kalman filter on top of `SimpleArray`

[yungyuc]

Create an initial implementation of the Kalman filter based on Choleskey decomposition.

Quick notes on the alogrithm for Kalman filter:

• Predict

  $$x_{t|t-1} = F_t x_{t-1|t-1} + B_t u_t,$$
  $$P_{t|t-1} = F_t P_{t-1|t-1} F_t^\top + Q_t.$$

• Innovation

  $$y_t = z_t - H_t x_{t|t-1},$$
  $$S_t = H_t P_{t|t-1} H_t^\top + R_t.$$

• Gain (via solve)

  $$K_t = P_{t|t-1} H_t^\top S_t^{-1}.$$

• Update

  $$x_{t|t} = x_{t|t-1} + K_t y_t,$$
  $$P_{t|t} = (I-K_t H_t) P_{t|t-1} (I-K_t H_t)^\top + K_t R_t K_t^\top.$$

To make the calculation precisely, the calculation of gain should be transformed like:

$$ S_t,K_t^\top ;=; \big(P_{t\mid t-1},H_t^\top\big)^\top. $$

Then we would solve the formula with Cholesky decomposition (LLT):

$$ K_t = \text{solve-with-LLT}\bigl(S_t,; (P_{t\mid t-1} H_t^\top)^\top\bigr)^\top. $$

The issue is created based on the comment: #588 (comment)

[yungyuc]

@ThreeMonth03 Please link additional PRs for Kalman filter to this issue.

[ThreeMonth03]

After implementing PR #607, several follow-up issues have been identified and will need to be addressed in the next development cycle:

• Implement symmetrize for 2D SimpleArray
• Implement hermitian for 2D SimpleArray
• Support matrix multiplication between 2D and 1D SimpleArray
• Fix issues related to Bug: F-contigous check in SimpleArray breaks #603 and SimpleArrayComplex128[i] = numpy.complex128 fails with cast error #604
• Implement a high-level API similar to batch_filter
• Add unit tests for prediction — ensuring that, given a reasonable random sequence, the prediction results are stable and correct

[ThreeMonth03]

After implementing PR #607, several follow-up issues have been identified and will need to be addressed in the next development cycle:

• Implement symmetrize for 2D SimpleArray
• Implement hermitian for 2D SimpleArray
• Support matrix multiplication between 2D and 1D SimpleArray
• Fix issues related to Bug: F-contigous check in SimpleArray breaks #603 and SimpleArrayComplex128[i] = numpy.complex128 fails with cast error #604
• Implement a high-level API similar to batch_filter
• Add unit tests for prediction — ensuring that, given a reasonable random sequence, the prediction results are stable and correct

@yungyuc I have a question. Is it a good idea to enhance the operation matmul with supporting multi-dimension? Current matmul operation only support 2d matmul 2d.

[yungyuc]

@yungyuc I have a question. Is it a good idea to enhance the operation matmul with supporting multi-dimension? Current matmul operation only support 2d matmul 2d.

Is implementing multi-dimensional matrix multiplication a good idea for what? I don't see how multi-dimensional matrix multiplication helps your implementation for Kalman filter, but in general, tensor contraction of any rank is useful.