Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shyrma casual conv1d #90

Merged
merged 10 commits into from Nov 29, 2019
Merged

Shyrma casual conv1d #90

merged 10 commits into from Nov 29, 2019

Conversation

@shyrma
Copy link

shyrma commented Nov 28, 2019

  • add causal padding mode to conv1d operation
shyrma added 4 commits Nov 27, 2019
Signed-off-by: Yurii <iuriish@yahoo.com>
Signed-off-by: Yurii <iuriish@yahoo.com>
Signed-off-by: Yurii <iuriish@yahoo.com>
@raver119 raver119 removed their assignment Nov 28, 2019
Signed-off-by: raver119 <raver119@gmail.com>
AlexDBlack added 3 commits Nov 29, 2019
Signed-off-by: Alex Black <blacka101@gmail.com>
Signed-off-by: Alex Black <blacka101@gmail.com>
Tweaks
Signed-off-by: Alex Black <blacka101@gmail.com>
@AlexDBlack

This comment has been minimized.

Copy link
Member

AlexDBlack commented Nov 29, 2019

I've pushed up some causal conv1d gradient checks.
Here's output for the first few cases:
https://gist.github.com/AlexDBlack/1bf0c9222f7f16628d1df3e68ac5fd7e
note sz there is sequence length

One thing I did notice is that the input gradients appear to have an off-by-1 issue
So analytic (op) gradient in[0,0,1] is equal to numericalGrad[0,0,0] and so on

shyrma and others added 2 commits Nov 29, 2019
Signed-off-by: Yurii <iuriish@yahoo.com>
Signed-off-by: Alex Black <blacka101@gmail.com>
Copy link
Member

AlexDBlack left a comment

Confirmed passing gradient checks now

@raver119 raver119 merged commit d19eeae into master Nov 29, 2019
1 check failed
1 check failed
continuous-integration/jenkins/pr-head The build of this commit was aborted
Details
@raver119 raver119 deleted the shyrma_casualConv1d branch Nov 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.