Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(transformers): Add DeBERTa model implementation to transformers … #5414

Open
wants to merge 61 commits into
base: develop
Choose a base branch
from

Conversation

Zhiyuan-Fan
Copy link
Contributor

…library

PR types

PR changes

Description

@paddle-bot
Copy link

paddle-bot bot commented Mar 25, 2023

Thanks for your contribution!

@codecov
Copy link

codecov bot commented Mar 25, 2023

Codecov Report

Attention: Patch coverage is 65.82880% with 503 lines in your changes are missing coverage. Please review.

Project coverage is 63.20%. Comparing base (f54e80d) to head (b6f945a).
Report is 1268 commits behind head on develop.

❗ Current head b6f945a differs from pull request most recent head 2741703. Consider uploading reports for the commit 2741703 to get more accurate results

Files Patch % Lines
paddlenlp/transformers/deberta_v2/modeling.py 66.50% 205 Missing ⚠️
paddlenlp/transformers/deberta/modeling.py 78.44% 125 Missing ⚠️
paddlenlp/transformers/deberta/tokenizer.py 22.53% 110 Missing ⚠️
paddlenlp/transformers/deberta/converter.py 0.00% 31 Missing ⚠️
paddlenlp/transformers/deberta_v2/converter.py 0.00% 31 Missing ⚠️
paddlenlp/transformers/deberta_v2/configuration.py 97.36% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #5414      +/-   ##
===========================================
+ Coverage    54.61%   63.20%   +8.59%     
===========================================
  Files          480      512      +32     
  Lines        67855    72473    +4618     
===========================================
+ Hits         37056    45807    +8751     
+ Misses       30799    26666    -4133     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@linjieccc linjieccc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

devertav2 -> deverta_v2

需要补充下modeling和Tokenizer的实现,并且支持AutoModel及AutoTokenizer的from_pretrained调用

@@ -0,0 +1,431 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

缺少文档,需要补充执行方式及具体的实验结果

dev_ds_matched = dev_ds_matched.map(trans_func, lazy=True)
dev_ds_mismatched = dev_ds_mismatched.map(trans_func, lazy=True)
dev_batch_sampler_matched = paddle.io.BatchSampler(
dev_ds_matched, batch_size=self.args.per_device_eval_batch_size * 2, shuffle=False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的batch_size为什么是self.args.per_device_eval_batch_size * 2?

@@ -0,0 +1,13 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

补充deberta tokenizer单测及deberta_v2的相关单测

@Zhiyuan-Fan Zhiyuan-Fan marked this pull request as draft June 4, 2023 00:47
@Zhiyuan-Fan Zhiyuan-Fan marked this pull request as ready for review June 4, 2023 00:48


PRETRAINED_VOCAB_FILES_MAP = {
"vocab_file": {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个key应该是sentencepiece_model_file, 不是vocab_file

@github-actions
Copy link

github-actions bot commented Aug 5, 2023

This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动,被标记为stale。

@github-actions
Copy link

This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动,被标记为stale。

@github-actions github-actions bot added the stale label Oct 29, 2023
@github-actions github-actions bot removed the stale label Mar 9, 2024
@w5688414 w5688414 mentioned this pull request Apr 3, 2024
Copy link

github-actions bot commented May 8, 2024

This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动,被标记为stale。

@github-actions github-actions bot added the stale label May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants