Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add AutoregressiveTransformer #225

Merged
merged 2 commits into from
Feb 6, 2023
Merged

Add AutoregressiveTransformer #225

merged 2 commits into from
Feb 6, 2023

Conversation

Warvito
Copy link
Collaborator

@Warvito Warvito commented Feb 5, 2023

Add Basic Autoregressive Transformer

@Warvito
Add AutoregressiveTransformer
5d973cc 
Signed-off-by: Walter Hugo Lopez Pinaya <ianonimato@hotmail.com>
@Warvito Warvito linked an issue Feb 5, 2023 that may be closed by this pull request
Add Transformer network #2
Closed
@Warvito
Copy link
Collaborator Author

Warvito commented Feb 5, 2023

For now, adding Wrapper for x-transformers. Goal is to replace it later

danieltudosiu
danieltudosiu approved these changes Feb 6, 2023
View reviewed changes
Copy link
Contributor

@danieltudosiu danieltudosiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but I left some comments.

If we are using it as a stop-gap then we are good to go.

generative/networks/nets/autoregressive_transformer.py Outdated Show resolved Hide resolved
generative/networks/nets/autoregressive_transformer.py Outdated Show resolved Hide resolved
generative/networks/nets/autoregressive_transformer.py Outdated Show resolved Hide resolved
@Warvito
Add cross-attention and rename model
df52e62 
Signed-off-by: Walter Hugo Lopez Pinaya <ianonimato@hotmail.com>
@Warvito Warvito merged commit 7992e5a into main Feb 6, 2023
@Warvito Warvito deleted the 2-add-transformer-network branch February 6, 2023 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@danieltudosiu danieltudosiu danieltudosiu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add Transformer network
2 participants
@Warvito @danieltudosiu