Dropout hidden-to-hidden transition within an RNN

[moisestg]

DropoutWrapper allows to apply dropout to either the cell's inputs, outputs or states. However, I haven't seen an option to do the same thing for the recurrent weights of the cell (for example, 4 out of the 8 different matrices used in the original LSTM formulation). I specifically refer to the hidden-to-hidden transition within an RNN. Take as an example Section 2 from https://arxiv.org/abs/1708.02182

[angerson]

@ebrevdo, can you take a look at this?

[tensorflowbutler]

It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

[tensorflowbutler]

It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

[tensorflowbutler]

Nagging Awaiting TensorFlower: It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

[tensorflowbutler]

Nagging Awaiting TensorFlower: It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

[ebrevdo]

I believe we support this now.

[vivekverma239]

@ebrevdo Can you please comment which Tensorflow function supports this ?

[ebrevdo]

The DropoutWrapper should support intelligent hidden-to-hidden dropout.

…

On Fri, Jun 29, 2018, 11:17 PM vivekverma239 ***@***.***> wrote: @ebrevdo <https://github.com/ebrevdo> Can you please comment which Tensorflow function supports this ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13103 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABtim7BUIEyduO1K6q6D_c2vkZOTuqKbks5uBxfcgaJpZM4PaQ-T> .

[maozezhong]

@ebrevdo Can DropoutWrapper achieve DropConnect?
DropConnect is refered as : http://yann.lecun.com/exdb/publis/pdf/wan-icml-13.pdf