New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LSTM for time series prediction #2856

Closed
ashwinnaresh opened this Issue May 31, 2016 · 11 comments

Comments

Projects
None yet
10 participants
@ashwinnaresh

ashwinnaresh commented May 31, 2016

This is more of a conceptual clarification.

I have the following architecture :

model = Sequential()
model.add(LSTM(batch_size,
               batch_input_shape=(batch_size, tsteps, 1)))  
model.compile(loss='mse', optimizer='rmsprop',metrics=["accuracy"])

The LSTM is able to make pretty accurate predictions as of now.

Training data :
issue_tr_le_full

Note that the training data for each of the 3 weeks is almost similar in shape.

LSTM predictions on unseen testing data :
issue_test

As you can see, the LSTM is still able to trace the unexpected peak in the graph which was never seen in the training data.

Any reason as to how the LSTM is able to do the above?

@gamehere007

This comment has been minimized.

Show comment
Hide comment
@gamehere007

gamehere007 Jun 1, 2016

@ashwin123
What is your input and output data for this model

gamehere007 commented Jun 1, 2016

@ashwin123
What is your input and output data for this model

@hijoe320

This comment has been minimized.

Show comment
Hide comment
@hijoe320

hijoe320 Jun 2, 2016

Contributor

@ashwin123 I am sure there is one step lag between the actual time series and the predicted time series, this is the most seen "trap" if you do time series prediction like this, in which the NN will always mimic previous input of time series.

Contributor

hijoe320 commented Jun 2, 2016

@ashwin123 I am sure there is one step lag between the actual time series and the predicted time series, this is the most seen "trap" if you do time series prediction like this, in which the NN will always mimic previous input of time series.

@sjayakum

This comment has been minimized.

Show comment
Hide comment
@sjayakum

sjayakum Jun 3, 2016

@hijoe320 yup you are right, there was a 1 time-step lag between predicted and actual. the NN was mimicking the previous input of the time series.
Thanks!

sjayakum commented Jun 3, 2016

@hijoe320 yup you are right, there was a 1 time-step lag between predicted and actual. the NN was mimicking the previous input of the time series.
Thanks!

@sjayakum

This comment has been minimized.

Show comment
Hide comment
@sjayakum

sjayakum Jun 3, 2016

@hijoe320 Ok, So are we doing any thing wrong to end up with this?

The main problem we have is the "Testing Data" real-time we get only 1 time-step at a time. So they are tested on a batch-size = 1 and time-step = 1.

Anyways to overcome this "trap"? 😁


More information

batch_size = 1
tsteps = 1

We have trained Each Timestep as a sample.

Say lstm requires input of the form => nb_samples x tsteps x attributes

where a,b,c,d... is a time series data
10:00 am -> X = 1 x 1 x a and y = b goes into lstm

10:01 am -> X = 1 x 1 x b and y = c goes into lstm..

10:02 am -> X = 1 x 1 x c and y = d goes into lstm.. And So on..

sjayakum commented Jun 3, 2016

@hijoe320 Ok, So are we doing any thing wrong to end up with this?

The main problem we have is the "Testing Data" real-time we get only 1 time-step at a time. So they are tested on a batch-size = 1 and time-step = 1.

Anyways to overcome this "trap"? 😁


More information

batch_size = 1
tsteps = 1

We have trained Each Timestep as a sample.

Say lstm requires input of the form => nb_samples x tsteps x attributes

where a,b,c,d... is a time series data
10:00 am -> X = 1 x 1 x a and y = b goes into lstm

10:01 am -> X = 1 x 1 x b and y = c goes into lstm..

10:02 am -> X = 1 x 1 x c and y = d goes into lstm.. And So on..

@hijoe320

This comment has been minimized.

Show comment
Hide comment
@hijoe320

hijoe320 Jun 7, 2016

Contributor

@sjayakum you can try:

  1. randomize training samples in each batch, make sure they are not followed one by one
  2. choose or design a better loss function other than MSE
  3. extract some features from the input time series
  4. manually limit the weight of x_{t-1}, x_{t-2}
Contributor

hijoe320 commented Jun 7, 2016

@sjayakum you can try:

  1. randomize training samples in each batch, make sure they are not followed one by one
  2. choose or design a better loss function other than MSE
  3. extract some features from the input time series
  4. manually limit the weight of x_{t-1}, x_{t-2}
@iwinterlovermy

This comment has been minimized.

Show comment
Hide comment
@iwinterlovermy

iwinterlovermy Aug 30, 2016

hi there. May I know how to use LSTM for multiple step ahead forecasting? Assuming I would like to predict 7 days ahead outcome. I've been trying few days but seems to get stuck - how should i prepare the data, and and what to set for batch size and time steps. Thanks in advance!

iwinterlovermy commented Aug 30, 2016

hi there. May I know how to use LSTM for multiple step ahead forecasting? Assuming I would like to predict 7 days ahead outcome. I've been trying few days but seems to get stuck - how should i prepare the data, and and what to set for batch size and time steps. Thanks in advance!

@stale stale bot added the stale label May 23, 2017

@stale stale bot closed this Jun 22, 2017

@safa94

This comment has been minimized.

Show comment
Hide comment
@safa94

safa94 Jul 31, 2017

please did you find something to solve your problem i have the same problem here

safa94 commented Jul 31, 2017

please did you find something to solve your problem i have the same problem here

@SunlnuS

This comment has been minimized.

Show comment
Hide comment
@SunlnuS

SunlnuS Oct 11, 2017

Hello, l'm using LSTM to predict time series and I meet the same problem. It seems that LSTM outputs input as output. Have you solved this problem? Would you like to give me some advice to solve this problem?

SunlnuS commented Oct 11, 2017

Hello, l'm using LSTM to predict time series and I meet the same problem. It seems that LSTM outputs input as output. Have you solved this problem? Would you like to give me some advice to solve this problem?

@yyb1995

This comment has been minimized.

Show comment
Hide comment
@yyb1995

yyb1995 Apr 10, 2018

I have the same problem while using LSTM to predict a complex time series. There is a delay between true value and predicted value and they have a simular shape. It seems that the network copies the last input value to generate an output value. Are there any solutions to this problems? I will be very grateful.

yyb1995 commented Apr 10, 2018

I have the same problem while using LSTM to predict a complex time series. There is a delay between true value and predicted value and they have a simular shape. It seems that the network copies the last input value to generate an output value. Are there any solutions to this problems? I will be very grateful.

@shafferjohn

This comment has been minimized.

Show comment
Hide comment
@shafferjohn

shafferjohn Oct 8, 2018

In my opinion, loss function comes to minimum, so the network chooses to mimic the actual value. You will get same result even without LSTM, I think, even just with one fully connect layer.

shafferjohn commented Oct 8, 2018

In my opinion, loss function comes to minimum, so the network chooses to mimic the actual value. You will get same result even without LSTM, I think, even just with one fully connect layer.

@kartik1225

This comment has been minimized.

Show comment
Hide comment
@kartik1225

kartik1225 Oct 12, 2018

same problem any idea how to overcome this? i have detailed question on stackoverflow

kartik1225 commented Oct 12, 2018

same problem any idea how to overcome this? i have detailed question on stackoverflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment