Add SqueezeBERT #937

renmada · 2021-08-27T15:52:23Z

See README.md

CLAassistant · 2021-08-27T15:52:25Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ ZeyuChen
✅ smallv0221
❌ deeplaying
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

smallv0221

Please change your codes according to the comments in this pr #872
And don't push those Irrelevant files.

smallv0221 · 2021-08-30T03:10:59Z

examples/language_model/squeezebert/README.md

+```
+export CUDA_VISIBLE_DEVICES=0
+python -u ./run_glue.py \
+--model_type bert \


--model_type squeezebert \

renmada · 2021-08-30T07:20:40Z

Irrelevant files

What is Irrelevant files？

smallv0221 · 2021-08-30T07:48:25Z

Irrelevant files

What is Irrelevant files？

These empty files.

renmada · 2021-08-30T16:05:17Z

Please change your codes according to the comments in this pr #872
And don't push those Irrelevant files.

I have changed the code. But I have no idea about where the Irrelevant files came from

smallv0221

Please fix the empty file problem. Pay a attention to your git add.

smallv0221 · 2021-08-31T02:57:52Z

paddlenlp/transformers/squeezebert/modeling.py

+
+
+class SqueezeBertEncoder(nn.Layer):
+    def __init__(self, embedding_size, hidden_size, intermediate_size, num_attention_heads,


The indent here is not correct

Need to fix this.

smallv0221 · 2021-08-31T02:59:43Z

paddlenlp/transformers/squeezebert/tokenizer.py

+    Examples:
+        .. code-block:: python
+            from paddlenlp.transformers import SqueezeBertTokenizer
+            tokenizer = SqueezeBertTokenizer.from_pretrained('SqueezeBert-small-discriminator')


Unknown name SqueezeBert-small-discriminator

smallv0221 · 2021-08-31T03:00:32Z

paddlenlp/transformers/squeezebert/modeling.py

+        return (sequence_output, pooled_output) + encoder_outputs[1:]
+
+
+# class SqueezeBertForPretraining(SqueezeBertPreTrainedModel):


Delete this if not needed

smallv0221 · 2021-08-31T03:01:13Z

paddlenlp/transformers/squeezebert/modeling.py

+@register_base_model
+class SqueezeBertModel(SqueezeBertPreTrainedModel):
+    """
+        Args:


Incorrect indent.

smallv0221 · 2021-08-31T03:02:02Z

paddlenlp/transformers/squeezebert/modeling.py

+            output_attentions=None,
+            output_hidden_states=None,
+    ):
+


Need more doc here.

smallv0221 · 2021-08-31T03:03:04Z

paddlenlp/transformers/squeezebert/modeling.py

+
+
+class SqueezeBertForTokenClassification(SqueezeBertPreTrainedModel):
+


Need more doc for models for specific tasks

Need more doc for models for specific tasks

I have not got your message for two days, is there any wrong?

renmada · 2021-08-31T06:03:25Z

Please review latest submit.

smallv0221

Please delete train_log.txt and dataset file.

smallv0221 · 2021-09-02T05:41:50Z

examples/language_model/squeezebert/models/squeezebert-mnli/tokenizer_config.json

@@ -0,0 +1 @@
+{}


Don't need this file.

smallv0221 · 2021-09-02T05:41:59Z

examples/language_model/squeezebert/models/squeezebert-uncased/tokenizer_config.json

@@ -0,0 +1 @@
+{}


Don't need this file.

I was told to upload all the relevant files. And he will delete all files after testing.

OK, then wait until testing passes.

smallv0221 · 2021-09-25T16:50:59Z

Since your test passed, you can delete all pytorch script, logs and json files

renmada · 2021-10-08T08:41:43Z

Since your test passed, you can delete all pytorch script, logs and json files

How about remove all files under "examples/language_model/squeeze_bert" ?

smallv0221 · 2021-10-11T09:30:49Z

Since your test passed, you can delete all pytorch script, logs and json files

How about remove all files under "examples/language_model/squeeze_bert" ?

Yes. You can remove all examples.

renmada · 2021-10-14T02:48:47Z

Since your test passed, you can delete all pytorch script, logs and json files

How about remove all files under "examples/language_model/squeeze_bert" ?

Yes. You can remove all examples.
Done.

smallv0221

LGTM

ZeyuChen · 2021-10-30T15:09:04Z

@yingyibiao 这个PR有效吗？

smallv0221

LGTM

smallv0221 · 2021-11-01T08:21:17Z

Please sign our Contributor License Agreement for this pr to be merged

Add squeezebert

2a506c2

smallv0221 reviewed Aug 30, 2021

View reviewed changes

smallv0221 requested changes Aug 31, 2021

View reviewed changes

Add doc and remove empty files

3110b5e

smallv0221 requested changes Sep 2, 2021

View reviewed changes

deeplaying added 3 commits September 6, 2021 23:19

更新pytorch glue结果

adabb16

fix bug

21405c4

增加cpu速度测试

c1fbede

ZeyuChen assigned smallv0221 Sep 13, 2021

ZeyuChen added this to In progress in PaddleNLP v2.2 via automation Oct 10, 2021

delete example

aa5c280

PaddleNLP v2.2 automation moved this from In progress to Reviewer approved Oct 21, 2021

smallv0221 previously approved these changes Oct 21, 2021

View reviewed changes

chenxiaozeng closed this Nov 1, 2021

PaddleNLP v2.2 automation moved this from Reviewer approved to Done Nov 1, 2021

chenxiaozeng reopened this Nov 1, 2021

PaddleNLP v2.2 automation moved this from Done to In progress Nov 1, 2021

Merge branch 'develop' into squeezebert

423232b

smallv0221 dismissed their stale review via 423232b November 1, 2021 08:16

PaddleNLP v2.2 automation moved this from In progress to Review in progress Nov 1, 2021

smallv0221 approved these changes Nov 1, 2021

View reviewed changes

PaddleNLP v2.2 automation moved this from Review in progress to Reviewer approved Nov 1, 2021

ZeyuChen changed the title ~~Add squeezebert~~ Add SqueezeBERT Nov 2, 2021

ZeyuChen added the enhancement New feature or request label Nov 2, 2021

Merge branch 'develop' into squeezebert

4ce4eaf

ZeyuChen merged commit a77af2e into PaddlePaddle:develop Nov 2, 2021

PaddleNLP v2.2 automation moved this from Reviewer approved to Done Nov 2, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add SqueezeBERT #937

Add SqueezeBERT #937

renmada commented Aug 27, 2021

CLAassistant commented Aug 27, 2021 •

edited

Loading

smallv0221 left a comment •

edited

Loading

smallv0221 Aug 30, 2021

renmada commented Aug 30, 2021

smallv0221 commented Aug 30, 2021

renmada commented Aug 30, 2021

smallv0221 left a comment

smallv0221 Aug 31, 2021

smallv0221 Sep 2, 2021

smallv0221 Sep 2, 2021

smallv0221 Aug 31, 2021

smallv0221 Aug 31, 2021

smallv0221 Aug 31, 2021

smallv0221 Aug 31, 2021

smallv0221 Aug 31, 2021

renmada Sep 2, 2021

renmada commented Aug 31, 2021

smallv0221 left a comment

smallv0221 Sep 2, 2021

smallv0221 Sep 2, 2021

renmada Sep 2, 2021

smallv0221 Sep 2, 2021

smallv0221 commented Sep 25, 2021

renmada commented Oct 8, 2021

smallv0221 commented Oct 11, 2021

renmada commented Oct 14, 2021

smallv0221 left a comment

ZeyuChen commented Oct 30, 2021

smallv0221 left a comment

smallv0221 commented Nov 1, 2021



		class SqueezeBertEncoder(nn.Layer):
		def __init__(self, embedding_size, hidden_size, intermediate_size, num_attention_heads,

		return (sequence_output, pooled_output) + encoder_outputs[1:]


		# class SqueezeBertForPretraining(SqueezeBertPreTrainedModel):



		class SqueezeBertForTokenClassification(SqueezeBertPreTrainedModel):

Add SqueezeBERT #937

Add SqueezeBERT #937

Conversation

renmada commented Aug 27, 2021

CLAassistant commented Aug 27, 2021 • edited Loading

smallv0221 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

renmada commented Aug 30, 2021

smallv0221 commented Aug 30, 2021

renmada commented Aug 30, 2021

smallv0221 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

renmada commented Aug 31, 2021

smallv0221 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smallv0221 commented Sep 25, 2021

renmada commented Oct 8, 2021

smallv0221 commented Oct 11, 2021

renmada commented Oct 14, 2021

smallv0221 left a comment

Choose a reason for hiding this comment

ZeyuChen commented Oct 30, 2021

smallv0221 left a comment

Choose a reason for hiding this comment

smallv0221 commented Nov 1, 2021

CLAassistant commented Aug 27, 2021 •

edited

Loading

smallv0221 left a comment •

edited

Loading