Join GitHub today
Can keras handle sequence-specific parameters? #12654
I am not sure if this is a feature request or an implementation question. It's the former if what I want to do is not possible, and the latter if it is. Anyway, it is a cross-post with a SO thread, but I'm not getting any bites over there, which makes me suspect that it isn't currently possible.
Consider the following data:
I want to fit a semiparametric model, where a LSTM learns the sine wave, and a linear regression learns the trend for each time series.
Transposing the first term, we've got:
How can I implement this in Keras? Is it possible? I'm running into problems because
If I were doing this by hand, there would be a very obvious form to the gradients. But it's just not clear how to shoe-horn this into something Keras can work with.
Here's a crack at coding it up:
It fails with the error
Generally speaking, it is most common to fit and remove the linear component (can be swapped out for other model types) using your favorite other software package (sklearn, ...) and fit the deep learning model on the residual (y - x*b in your notation).
For this specific model you could construct a graph that does (x->lstm blocks, x-> dense, add dense + lstm block, loss on y).
@briannemsick Thanks for the reply. I could certainly do this, but it'd ignore correlations between the terms being modeled parametrically and the terms being modeled nonparametrically. The reason that you don't want to do that is that the gradients of either component will not be invariant to the values of the other, as the residuals will be continuously updated during the fitting process. I don't follow your second paragraph exactly. Are you proposing a method to execute the partitioned model, or a method to jointly estimate the parametric and nonparametric components?
And, in case it doesn't go without saying, my real use-case isn't the toy model that I put in the example.