Open Source MLM Implementation in Fairseq #635
Conversation
Summary: Pull Request resolved: facebookresearch#635 Adding a task and relevant models, datasets and criteria needed for training Cross-lingual Language Models similar to Masked Language Model used in XLM (Lample and Conneau, 2019 - https://arxiv.org/abs/1901.07291). Reviewed By: liezl200 Differential Revision: D14943776 fbshipit-source-id: 9835d82e9741c2ff9091f24cdbe4bb4be654c5a5
kartikayk
force-pushed
the
export-D14943776
branch
from
e9d0158
to
13194a0
Compare
|
This pull request has been merged in 8776928. |
|
@kartikayk After this PR, i get this error, It seems some files are missing from this? from fairseq.data.masked_lm_dataset import MaskedLMDataset |
@hanyh I'm working on fixing this right now. Will send out an update soon. Sorry for the inconvenience! |
|
@kartikayk Thanks for that implementation :+1 I have one question: could you also provide a kind of example that shows a) to load a trained model and b) that returns embeddings for each subtoken in a given sentence from that model? That would really help me :) |
Summary: Pull Request resolved: facebookresearch/fairseq#635 Adding a task and relevant models, datasets and criteria needed for training Cross-lingual Language Models similar to Masked Language Model used in XLM (Lample and Conneau, 2019 - https://arxiv.org/abs/1901.07291). Reviewed By: liezl200 Differential Revision: D14943776 fbshipit-source-id: 3e416a730303d1dd4f5b92550c78db989be27073
Summary: Pull Request resolved: facebookresearch/fairseq#635 Adding a task and relevant models, datasets and criteria needed for training Cross-lingual Language Models similar to Masked Language Model used in XLM (Lample and Conneau, 2019 - https://arxiv.org/abs/1901.07291). Reviewed By: liezl200 Differential Revision: D14943776 fbshipit-source-id: 3e416a730303d1dd4f5b92550c78db989be27073
Add the missing step to add the arguments to the parser.
Summary: Adding a task and relevant models, datasets and criteria needed for training Cross-lingual Language Models similar to Masked Language Model used in XLM (Lample and Conneau, 2019 - https://arxiv.org/abs/1901.07291).
Differential Revision: D14943776