Generator functions, control input, DK smoothing

[GordStephen]

I realized yesterday that my local development has gotten way ahead of master - not a good thing! I'll try to catch this up gradually, starting with small changes. I'll push a few more commits to this PR before requesting it be considered for merging and tagging as 0.1.0 (dropping Julia 0.3 support), namely:

• Convert time-varying system matrices to generator functions
• Add control input / feedthrough to the state equations
• Add standalone Durbin-Koopman smoothing (the existing filtering function and the smoothing method that takes a KalmanFiltered object will be kept)

Once that's done and everyone's happy with it, I'll start a new PR with the next wave of updates - constrained parameter fitting and EM estimation (I don't currently have time-dependent system matrices supported, but I'll get that done). After that I'll start a third PR with alternative implementations of smoothing and parameter estimation using sparse matrices (slower for small systems, but much more efficient for large ones with deterministic state transitions, like seasonal ARMA models).

[GordStephen]

Ok, all done. If we decide to make this 0.1.0, I'll update NEWS before merging.

[GordStephen]

Some basic benchmarking for each commit:

using TimeModels

m, b, s, dt = 5, 2, 2, .1
t = 0:dt:10000
y_true = m*t + b
y_noisy = y_true + s*randn(length(t))
blm = StateSpaceModel([[1 dt; 0 1] zeros(2,28); zeros(28,30)], 1e-6*eye(30), [1. zeros(29)'], s*eye(1), zeros(30), diagm([100; 100; zeros(28)]))

kalman_smooth(y_noisy, blm)
@time kalman_smooth(y_noisy, blm)

Baseline: 20.959700 seconds (17.18 M allocations: 20.914 GB, 6.10% gc time)
Generator functions: 21.151211 seconds (17.18 M allocations: 20.914 GB, 5.92% gc time)
Exogenous inputs: 22.915877 seconds (18.78 M allocations: 21.023 GB, 5.80% gc time)
DK smoothing: 11.030202 seconds (15.99 M allocations: 18.718 GB, 8.10% gc time)

[rob-luke]

Could the control and feed-forward matrices be functions too? I have not seen this done, but I could imagine situations where it could be useful.

[GordStephen]

Sure - I need to put in a bit more thought about how EM estimation of time-independent parameters will best integrate with time-dependent system matrices, but the Kalman filtering / smoothing part should be simple.

[milktrader]

@GordStephen no objections here to your managing the transition to 0.1.0

[GordStephen]

I went all-in and allowed the noise covariance matrices to be time-dependent in addition to the exogenous input matrices.

Performance check: 11.649817 seconds (16.58 M allocations: 18.762 GB, 7.81% gc time)