Demo or example code

[jamesmf]

Is there example code anywhere? This is an exciting addition to keras, and I'd love to see it in action. I attempted to adapt your 'untested' code snippet from

keras-team/keras#401

but had trouble going from the TimeDistributedFlatten to the LSTM layers.

model = Sequential()
model.add(TimeDistributedConvolution2D(8, 4, 4, border_mode='same', input_shape=(n_timesteps,1, 28, 28)))
model.add(TimeDistributedMaxPooling2D(pool_size=(2, 2)))
model.add(Activation('relu'))
model.add(TimeDistributedFlatten())
model.add(LSTM(256, return_sequences=False))
model.add(Dense(nb_classes))
model.add(Activation('softmax')

That yields the following dimension mismatch:
Inputs shapes: [(32, 392), (1568, 256)]

My test case is simply processing windows within MNIST images sequentially. My window size is (15,15) resulting in 196 windows.

The shape of X_train is (60000, 196, 1, 15, 15)

If I instead try return_sequences = True in the RNN layer, I get
AssertionError: Incompatible shapes: layer expected input with ndim=2 but previous layer has output_shape (None, 196, 256)

Is there an example anywhere that I could use to troubleshoot?

Thanks

[jamesmf]

Okay I already see at least one thing wrong here.

I was imagining hooking up the last time-step's output from the RNN to a vanilla Dense network. Does that make sense? Or does this implementation depend on the output vector having the same time dimension as the input?

[anayebi]

Yeah if return_sequences=False, then you do want a Dense network, not a TimeDistributedDense.

I think the issue is with your input_shape argument. You mention that X_train is of the shape (60000, 196, 1, 15, 15). If that's the case, then your input_shape to the first TimeDistributedConvolution2D layer should be input_shape=(196,1, 15, 15).

Also, as a sanity check what version of Keras are you using? I can only guarantee support for version 0.3.0 and lower.

[jamesmf]

Yep, I'm using 0.3.0.

Yeah the input shape was the issue - I was focused on the error and not really checking the code.

It seems to be working now. I'll post a link with the working code and then close the issue.

Thanks

[jamesmf]

https://github.com/jamesmf/mnistCRNN

Above is a working TimeDistributedConvolution Example. It takes a set of MNIST images and learns to predict their sum.

[anayebi]

Great thanks! I added the link to your demo in the ReadMe :) 👍