Skip to content

Add world_language_model transformer and lstm#1005

Open
xuzhao9 wants to merge 35 commits intomainfrom
xz9/add-wlm-trans
Open

Add world_language_model transformer and lstm#1005
xuzhao9 wants to merge 35 commits intomainfrom
xz9/add-wlm-trans

Conversation

@xuzhao9
Copy link
Contributor

@xuzhao9 xuzhao9 commented Jul 1, 2022

Add world_language_model transformer and lstm. They are also used in release testing.

@xuzhao9 xuzhao9 force-pushed the xz9/add-wlm-trans branch from fef120e to 74d18ff Compare July 1, 2022 01:55
@xuzhao9 xuzhao9 changed the title Add world_language_model transformer Add world_language_model transformer and lstm Jul 1, 2022
@xuzhao9
Copy link
Contributor Author

xuzhao9 commented Jul 28, 2022

The PR is blocked by memory leak issue in wlm_lstm_train_cuda: https://app.circleci.com/pipelines/github/pytorch/benchmark/4705/workflows/b9b7e022-0a50-41eb-8088-4e9ca5d4f169/jobs/4836
@robieta I suspect this is because of cyclic references in the LSTM model (since wlm_transformer_train_cuda) doesn't have this problem. But still there might be a bug somewhere?

@xuzhao9 xuzhao9 requested a review from robieta July 28, 2022 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants