Can you add a DEMO of LSTM for time series ,Please

[sharpwood]

SiaNet is great ! Thanks for your work.
Can you add a DEMO of LSTM for time series .
thanks!

[deepakkumar1984]

Thanks mate! sure will add to my plan

[sharpwood]

Thanks very much。
I see you mark this issue as a question, is it possible that the existing version of the function can be implemented? But I see that class Recurrent has not been cited
If the existing version can be implemented, can you give a line of key code, thank you.

[deepakkumar1984]

Sorry my bad. I have to implement LSTM, GRU and RNN properly and provide examples. One the plan of development.

[sharpwood]

👍

[sharpwood]

LSTM has been added to the layer folder. Is it available now?

[deepakkumar1984]

I added the LSTM layer yesterday but didn't have time to test with sample data. But you are welcome to test and let me know how it goes.

…

On Wed, 22 Nov 2017 at 2:25 pm, sharpwood ***@***.***> wrote: LSTM has been added to the layer folder. Is it available now? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCQKc2MLgYR3bjzI_RljSWhHS6XSyjlks5s45sagaJpZM4Qj2f-> .

[sharpwood]

I added a LSTM layer, but it was wrong when traning: Objects that can be empty must have a value

[deepakkumar1984]

Ok, Did you get any error. Could you please share the code so that I can debug at my end along with sample training data. Thanks

…

On Wed, Nov 22, 2017 at 4:16 PM, sharpwood ***@***.***> wrote: I added a LSTM layer, but it was wrong when traning: Objects that can be empty must have a value — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCQKQXYBslkCzZ1i3B4ZCRyM8dSsML9ks5s47VTgaJpZM4Qj2f-> .

-- Regards, Deepak

[sharpwood]

I only try add a line :
model.Add(new LSTM(dim:20,cellDim:1));
in HousingRegression BuildModel method

[deepakkumar1984]

Fixed the issue with the LSTM. Can you please check now. Although I am not sure if LSTM will work with MSE loss function. Also with Sparse input like starting with Embedding layer. Please advice?

[deepakkumar1984]

Will try to implement this example: https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/

Hope to complete by tomorrow

[sharpwood]

that is great!!!

[sharpwood]

[sharpwood]

model.Add(new LSTM(20, cellDim: 5));
still in HousingRegression

[deepakkumar1984]

Oh no i checked in the code after adding the test lstm layer

On Wed, 22 Nov 2017 at 8:05 pm, sharpwood ***@***.***> wrote: model.Add(new LSTM(20, cellDim: 5)); still in HousingRegression — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCQKUdOD1RRkeNKEd_eJ9leBe6VWvs5ks5s4-rwgaJpZM4Qj2f-> .

-- Regards, Deepak

[sharpwood]

[deepakkumar1984]

Should work now! Please take the latest

[sharpwood]

That is ok.
When will TimeSeriesPrediction be finished?

[deepakkumar1984]

I added the example code and testing in progress. Having some issue with training. Will try to complete by tomorrow

[deepakkumar1984]

Try out this example code with some of your sample data: https://github.com/deepakkumar1984/SiaNet/blob/master/SieNet.Examples/TimeSeriesPrediction.cs

[sharpwood]

I changed lookback to 3, how to define LSTM shape, as if it was different from Keras

[deepakkumar1984]

The lookback is the shape of the input. The number of lookback will convert the series data to tabular dataset with 3 columns which will be the input shape.

I have updated the time series example, could you please try again?

model.Add(new LSTM(dim: 4, shape: Shape.Create(lookback)));

[sharpwood]

I have got last version, it works ,thanks.
If I have multiple features in csv file, how to define the shape?

[deepakkumar1984]

Ok Its not implemented for multivariate. Let me work on that now and have another example ready for you.

[deepakkumar1984]

And yes it supports multiple LSTM.. model = new Sequential(); model.Add(new LSTM(dim: 4, shape: Shape.Create(lookback), returnSequence: true)); model.Add(new LSTM(dim: 4, shape: Shape.Create(lookback))); model.Add(new Dense(dim: 1)); if you would like to stack the LSTM then need to use returnSequence: true for the previous layer. If returnSequence: false (y default) it will return the last sequence.

[sharpwood]

I think it may be that I don't make it clear. My English is poor.
I mean not a multi-layer LSTM, but a number of features input data.
For example:

[deepakkumar1984]

Do you have any example with keras or another python framework which I can use as reference? Seems like this is a general regression problem and not time series which can be solved by using Dense layer. Maybe I am wrong?

…

On Wed, Nov 29, 2017 at 10:28 AM, sharpwood ***@***.***> wrote: I think it may be that I don't make it clear. My English is poor. I mean not a multi-layer LSTM, but a number of features input data. For example: [image: image] <https://user-images.githubusercontent.com/7440304/33350683-0f5eaa62-d4db-11e7-89df-ed96b5d7857d.png> — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCQKagoGRJsLQxqU6EY6sWoVaBiAqnoks5s7J4fgaJpZM4Qj2f-> .

-- Regards, Deepak

[sharpwood]

https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/

Is this example appropriate?

[sharpwood]

https://machinelearningmastery.com/reshape-input-data-long-short-term-memory-networks-keras/

[deepakkumar1984]

On Wed, 29 Nov 2017 at 8:04 pm, sharpwood ***@***.***> wrote: https://machinelearningmastery.com/reshape-input-data-long-short-term-memory-networks-keras/ — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCQKbwh5eYZXZaSIjf408_HSJK_hPbpks5s7SUXgaJpZM4Qj2f-> .

Thanks mate. This example looks good. I will start the work -- Regards, Deepak

[sharpwood]

Reshape layer has been committed , Would you update the example?

[deepakkumar1984]

Yes, Will do that

…

On Fri, Dec 1, 2017 at 4:32 PM, sharpwood ***@***.***> wrote: Reshape layer has been committed , Would you update the example? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCQKcMmq9gSk_GfQrMr3yr4G3yvtiAIks5s75ZYgaJpZM4Qj2f-> .

-- Regards, Deepak

[bmigette]

Looks interesting :) Will follow up this project / thread.

[sharpwood]

How's the example program going?

[deepakkumar1984]

I will have something by tomorrow. Was working on other fixes on priority.

[deepakkumar1984]

Please use this example to use Reshape layer: https://github.com/deepakkumar1984/SiaNet/blob/master/SieNet.Examples/MiltiVariateTimeSeriesPrediction.cs

[bmigette]

Do you have any documentation that explains what are shapes, and why do we need a reshape layer ?
Nevermind, found explanation here: https://www.tensorflow.org/programmers_guide/tensors :)

[deepakkumar1984]

I need to prepare all that will take some time because i am doing this work in my spare time. Plus some issues are there which need to be fixes on high priority. Contribution are always welcome 😉😉😉

On Thu, 7 Dec 2017 at 8:05 pm, bmigette ***@***.***> wrote: Do you have any documentation that explains what are shapes, and why do we need a reshape layer ? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCQKZSCqPVrZFHi_hgBnSCdFTyVWY5mks5s97FegaJpZM4Qj2f-> .

-- Regards, Deepak

[sharpwood]

Normalize and ConvertTimeSeries method seems to have some problems.

Normalize method generate some NaN values , and The ConvertTimeSeries method does not seem to be able to handle multi - column data.

[deepakkumar1984]

The data frame is bit basic now. Need to enhance with missing values and load datatype other than float. For now you need to sanitize the dataset by yourself and then load it. You can use the framework like Deedle to load the csv and make it ready for training. I may plan to make use of Deedle in the project and way of converting Deedle output to CNTK value.

[deepakkumar1984]

Closing it off as there is no activity in last 180 days