Skip to content

Finetuning with Megatron-LM#601

Merged
yzhang123 merged 33 commits intomasterfrom
megatron_ref
May 18, 2020
Merged

Finetuning with Megatron-LM#601
yzhang123 merged 33 commits intomasterfrom
megatron_ref

Conversation

@ekmb
Copy link
Copy Markdown
Collaborator

@ekmb ekmb commented Apr 30, 2020

  1. MegatronNM added
  2. Most examples/nlp updated to support Megatron-LM
  3. Docs added

ekmb added 21 commits April 22, 2020 10:35
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
@lgtm-com
Copy link
Copy Markdown

lgtm-com Bot commented May 15, 2020

This pull request introduces 4 alerts when merging 88e2d1a into a22d325 - view on LGTM.com

new alerts:

  • 4 for Unused import

Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
@ekmb ekmb changed the title [WIP] Finetuning with Megatron-LM Finetuning with Megatron-LM May 15, 2020
@ekmb ekmb requested a review from yzhang123 May 15, 2020 14:43
@lgtm-com
Copy link
Copy Markdown

lgtm-com Bot commented May 15, 2020

This pull request introduces 1 alert when merging eb4ccbb into a22d325 - view on LGTM.com

new alerts:

  • 1 for Unused import

ekmb added 4 commits May 15, 2020 08:24
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
@ekmb ekmb requested a review from VahidooX May 15, 2020 19:41
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
@lgtm-com
Copy link
Copy Markdown

lgtm-com Bot commented May 16, 2020

This pull request introduces 2 alerts when merging 005833a into 06b04ca - view on LGTM.com

new alerts:

  • 2 for Unused import

Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
@lgtm-com
Copy link
Copy Markdown

lgtm-com Bot commented May 16, 2020

This pull request introduces 2 alerts when merging acca387 into 06b04ca - view on LGTM.com

new alerts:

  • 2 for Unused import

ekmb added 5 commits May 15, 2020 20:10
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
python question_answering_squad.py \
--train_file PATH_TO_DATA_DIR/squad/v1.1/train-v1.1.json \
--eval_file PATH_TO_DATA_DIR/squad/v1.1/dev-v1.1.json \
--pretrained_model_name megatron-bert-345m-uncased
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whats the difference between megatron-bert-345m-uncased and megatron-bert-uncased? why do we need to have a special case for megatron-bert-345m-uncased

@yzhang123 yzhang123 self-requested a review May 18, 2020 17:59
@yzhang123 yzhang123 merged commit 9d90a95 into master May 18, 2020
@ekmb ekmb deleted the megatron_ref branch May 23, 2020 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants