New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request - SRU (Simple Recurrent Unit) Cell #13094

Open
desire2020 opened this Issue Sep 17, 2017 · 10 comments

Comments

Projects
None yet
9 participants
@desire2020

desire2020 commented Sep 17, 2017

ArXiv path: https://arxiv.org/abs/1709.02755

This is a recently proposed, parallelization-friendly RNN cell. The author released his own PyTorch version of the SRU. We are looking forward to an offical tensorflow implementation with Cudnn optimizations.

@tjingrant

This comment has been minimized.

Contributor

tjingrant commented Sep 21, 2017

Hi, I'm interested in taking a crack at this. Will keep you updated.

@buptpriswang

This comment has been minimized.

buptpriswang commented Sep 22, 2017

Hi, it's interesting RNN cell, will keep you updated

@argman

This comment has been minimized.

argman commented Oct 19, 2017

Any progress?

@tjingrant

This comment has been minimized.

Contributor

tjingrant commented Oct 19, 2017

Will submit the initial PR early next week.
Sorry I started late because I have to clear some build errors on master.

@GeorgyZhou

This comment has been minimized.

Contributor

GeorgyZhou commented Oct 23, 2017

I am interested in this question as well. If @tjingrant have some problems, I can also join in.

@tjingrant

This comment has been minimized.

Contributor

tjingrant commented Oct 23, 2017

I apologize for keeping everyone interested waiting and will do my best to release a PR early this week. @GeorgyZhou: thanks for your offer, I will let you know if I need extra hands!

@hadaev8

This comment has been minimized.

hadaev8 commented Jan 21, 2018

Any news?

@stegben

This comment has been minimized.

Contributor

stegben commented Jan 26, 2018

I'm working on a fused version of SRU (process on whole time steps)

@danijar

This comment has been minimized.

Member

danijar commented Feb 13, 2018

Hi @stegben, what's the status of this? It sounds like the fused implementation is really what would make SRU most useful.

@tjingrant

This comment has been minimized.

Contributor

tjingrant commented Feb 14, 2018

To be honest, after implementing the non-fused version, I suspect whether it's worth going deeper. The paper has been rejected on the ground that it is a special case of Quasi-RNN which makes sense. It's probably better for everyone to work on a better version of Quasi-RNN than keeping implementing SRU.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment