New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make the decoder part an autoregressive model #10

Closed

6 tasks done

khanhvu207 opened this issue May 20, 2022 · 0 comments · Fixed by #12

Closed

6 tasks done

Make the decoder part an autoregressive model #10

khanhvu207 opened this issue May 20, 2022 · 0 comments · Fixed by #12

Milestone

Stable supervised...

Owner

khanhvu207 commented May 20, 2022 •

edited

Loading

Give checkpoint folder more meaningful name.
Make the token embedding shared between the BoW encoder and the final decoder.
Add group normalization to the CNN encoder.
Implement autoregressive training with teacher-forcing (e.g. the target sequence is shifted one position to the left).
- BUG: Add attention mask in order to make it truly autoregressive.
bias=False for Conv2d with GroupNorm.

khanhvu207 added this to the Stable supervised learning pipeline milestone

khanhvu207 linked a pull request

that will close this issue

10 make the decoder part an autoregressive model #12

Merged

khanhvu207 closed this as completed in #12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment