下表汇总介绍了目前PaddleNLP支持的RoBERTa模型对应预训练权重。 关于模型的具体细节可以参考对应链接。
| Pretrained Weight | Language | Details of the model |
|---|---|---|
hfl/roberta-wwm-ext |
Chinese | 12-layer, 768-hidden, 12-heads, 102M parameters. Trained on English Text using Whole-Word-Masking with extended data. |
hfl/roberta-wwm-ext-large |
Chinese | 24-layer, 1024-hidden, 16-heads, 325M parameters. Trained on English Text using Whole-Word-Masking with extended data. |
hfl/rbt3 |
Chinese | 3-layer, 768-hidden, 12-heads, 38M parameters. |
hfl/rbtl3 |
Chinese | 3-layer, 1024-hidden, 16-heads, 61M parameters. |
hfl/rbt4 |
Chinese | 4-layer, 768-hidden, 12-heads, 47M parameters. |
hfl/rbt6 |
Chinese | 6-layer, 768-hidden, 12-heads, 60M parameters. |
deepset/roberta-base-squad2 |
English | 12-layer, 768-hidden, 12-heads, 124M parameters. Trained on English text. Please refer to: deepset/roberta-base-squad2 |
uer/roberta-base-chinese-extractive-qa |
Chinese | 12-layer, 768-hidden, 12-heads, 101M parameters. Trained on Chinese text. Please refer to: uer/roberta-base-chinese-extractive-qa |
uer/roberta-base-finetuned-chinanews-chinese |
Chinese | 12-layer, 768-hidden, 12-heads, 102M parameters. Trained on Chinese text. Please refer to: uer/roberta-base-finetuned-chinanews-chinese |
uer/roberta-base-finetuned-cluener2020-chinese |
Chinese | 12-layer, 768-hidden, 12-heads, 101M parameters. Trained on Chinese text. Please refer to: uer/roberta-base-finetuned-cluener2020-chinese |
roberta-base |
English | Please refer to: roberta-base |
cardiffnlp/twitter-roberta-base-sentiment |
English | Please refer to: cardiffnlp/twitter-roberta-base-sentiment |
roberta-large |
English | Please refer to: roberta-large |
distilroberta-base |
English | Please refer to: distilroberta-base |
cross-encoder/nli-distilroberta-base |
English | Please refer to: cross-encoder/nli-distilroberta-base |
siebert/sentiment-roberta-large-english |
English | Please refer to: siebert/sentiment-roberta-large-english |
j-hartmann/emotion-english-distilroberta-base |
English | Please refer to: j-hartmann/emotion-english-distilroberta-base |
roberta-base-openai-detector |
English | Please refer to: roberta-base-openai-detector |
huggingface/CodeBERTa-small-v1 |
English | Please refer to: huggingface/CodeBERTa-small-v1 |
mrm8488/distilroberta-finetuned-financial-news-sentiment-analysis |
English | Please refer to: mrm8488/distilroberta-finetuned-financial-news-sentiment-analysis |
cardiffnlp/twitter-roberta-base-emotion |
English | Please refer to: cardiffnlp/twitter-roberta-base-emotion |
seyonec/PubChem10M_SMILES_BPE_396_250 |
English | Please refer to: seyonec/PubChem10M_SMILES_BPE_396_250 |
textattack/roberta-base-SST-2 |
English | Please refer to: textattack/roberta-base-SST-2 |
sshleifer/tiny-distilroberta-base |
English | Please refer to: sshleifer/tiny-distilroberta-base |
thatdramebaazguy/roberta-base-squad |
English | Please refer to: thatdramebaazguy/roberta-base-squad |
ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli |
English | Please refer to: ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli |
ufal/robeczech-base |
Czech | Please refer to: ufal/robeczech-base |
seyonec/PubChem10M_SMILES_BPE_450k |
English | Please refer to: seyonec/PubChem10M_SMILES_BPE_450k |
cardiffnlp/twitter-roberta-base |
English | Please refer to: cardiffnlp/twitter-roberta-base |
seyonec/PubChem10M_SMILES_BPE_50k |
English | Please refer to: seyonec/PubChem10M_SMILES_BPE_50k |
microsoft/codebert-base-mlm |
English | Please refer to: microsoft/codebert-base-mlm |
textattack/roberta-base-MNLI |
English | Please refer to: textattack/roberta-base-MNLI |
cardiffnlp/twitter-roberta-base-offensive |
English | Please refer to: cardiffnlp/twitter-roberta-base-offensive |
cross-encoder/stsb-roberta-large |
English | Please refer to: cross-encoder/stsb-roberta-large |
seyonec/ChemBERTa_zinc250k_v2_40k |
English | Please refer to: seyonec/ChemBERTa_zinc250k_v2_40k |
uklfr/gottbert-base |
German | Please refer to: uklfr/gottbert-base |
seyonec/ChemBERTa-zinc-base-v1 |
English | Please refer to: seyonec/ChemBERTa-zinc-base-v1 |
roberta-large-openai-detector |
English | Please refer to: roberta-large-openai-detector |
cross-encoder/quora-roberta-base |
English | Please refer to: cross-encoder/quora-roberta-base |
cross-encoder/stsb-roberta-base |
English | Please refer to: cross-encoder/stsb-roberta-base |
microsoft/graphcodebert-base |
English | Please refer to: microsoft/graphcodebert-base |
cardiffnlp/twitter-roberta-base-hate |
English | Please refer to: cardiffnlp/twitter-roberta-base-hate |
chkla/roberta-argument |
English | Please refer to: chkla/roberta-argument |
Salesforce/grappa_large_jnt |
English | Please refer to: Salesforce/grappa_large_jnt |
vinai/bertweet-large |
English | Please refer to: vinai/bertweet-large |
allenai/biomed_roberta_base |
English | Please refer to: allenai/biomed_roberta_base |
facebook/muppet-roberta-base |
English | Please refer to: facebook/muppet-roberta-base |
Rakib/roberta-base-on-cuad |
English | Please refer to: Rakib/roberta-base-on-cuad |
cross-encoder/stsb-distilroberta-base |
English | Please refer to: cross-encoder/stsb-distilroberta-base |
nyu-mll/roberta-base-1B-1 |
English | Please refer to: nyu-mll/roberta-base-1B-1 |
nyu-mll/roberta-med-small-1M-1 |
English | Please refer to: nyu-mll/roberta-med-small-1M-1 |
SkolkovoInstitute/roberta_toxicity_classifier |
English | Please refer to: SkolkovoInstitute/roberta_toxicity_classifier |
facebook/muppet-roberta-large |
English | Please refer to: facebook/muppet-roberta-large |
lassl/roberta-ko-small |
Korean | Please refer to: lassl/roberta-ko-small |
huggingface/CodeBERTa-language-id |
English | Please refer to: huggingface/CodeBERTa-language-id |
textattack/roberta-base-imdb |
English | Please refer to: textattack/roberta-base-imdb |
macedonizer/mk-roberta-base |
Macedonian | Please refer to: macedonizer/mk-roberta-base |
cross-encoder/nli-MiniLM2-L6-H768 |
English | Please refer to: cross-encoder/nli-MiniLM2-L6-H768 |
textattack/roberta-base-QNLI |
English | Please refer to: textattack/roberta-base-QNLI |
deepset/roberta-base-squad2-covid |
English | Please refer to: deepset/roberta-base-squad2-covid |
textattack/roberta-base-MRPC |
English | Please refer to: textattack/roberta-base-MRPC |
bhadresh-savani/roberta-base-emotion |
English | Please refer to: bhadresh-savani/roberta-base-emotion |
aychang/roberta-base-imdb |
English | Please refer to: aychang/roberta-base-imdb |
cross-encoder/quora-distilroberta-base |
English | Please refer to: cross-encoder/quora-distilroberta-base |
csarron/roberta-base-squad-v1 |
English | Please refer to: csarron/roberta-base-squad-v1 |
seyonec/ChemBERTA_PubChem1M_shard00_155k |
English | Please refer to: seyonec/ChemBERTA_PubChem1M_shard00_155k |
mental/mental-roberta-base |
English | Please refer to: mental/mental-roberta-base |
textattack/roberta-base-CoLA |
English | Please refer to: textattack/roberta-base-CoLA |
navteca/quora-roberta-base |
English | Please refer to: navteca/quora-roberta-base |
cardiffnlp/twitter-roberta-base-emoji |
English | Please refer to: cardiffnlp/twitter-roberta-base-emoji |
benjamin/roberta-base-wechsel-german |
Multilingual | Please refer to: benjamin/roberta-base-wechsel-german |
textattack/roberta-base-ag-news |
English | Please refer to: textattack/roberta-base-ag-news |
johngiorgi/declutr-base |
English | Please refer to: johngiorgi/declutr-base |
salesken/query_wellformedness_score |
English | Please refer to: salesken/query_wellformedness_score |
blinoff/roberta-base-russian-v0 |
Russian | Please refer to: blinoff/roberta-base-russian-v0 |
allenai/reviews_roberta_base |
English | Please refer to: allenai/reviews_roberta_base |
ruiqi-zhong/roberta-base-meta-tuning-test |
English | Please refer to: ruiqi-zhong/roberta-base-meta-tuning-test |
mrm8488/distilroberta-finetuned-tweets-hate-speech |
English | Please refer to: mrm8488/distilroberta-finetuned-tweets-hate-speech |
cointegrated/roberta-large-cola-krishna2020 |
English | Please refer to: cointegrated/roberta-large-cola-krishna2020 |
deepset/roberta-base-squad2-distilled |
English | Please refer to: deepset/roberta-base-squad2-distilled |
tli8hf/unqover-roberta-base-squad |
English | Please refer to: tli8hf/unqover-roberta-base-squad |
cross-encoder/nli-roberta-base |
English | Please refer to: cross-encoder/nli-roberta-base |
nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large |
English | Please refer to: nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large |
seyonec/BPE_SELFIES_PubChem_shard00_160k |
English | Please refer to: seyonec/BPE_SELFIES_PubChem_shard00_160k |
CLTL/MedRoBERTa.nl |
Dutch | Please refer to: CLTL/MedRoBERTa.nl |
HooshvareLab/roberta-fa-zwnj-base |
Persian | Please refer to: HooshvareLab/roberta-fa-zwnj-base |
nyu-mll/roberta-base-100M-1 |
English | Please refer to: nyu-mll/roberta-base-100M-1 |
deepset/tinyroberta-squad2 |
English | Please refer to: deepset/tinyroberta-squad2 |
youscan/ukr-roberta-base |
Ukrainian | Please refer to: youscan/ukr-roberta-base |
navteca/roberta-base-squad2 |
English | Please refer to: navteca/roberta-base-squad2 |
bertin-project/bertin-roberta-base-spanish |
Spanish | Please refer to: bertin-project/bertin-roberta-base-spanish |
shiyue/roberta-large-tac08 |
English | Please refer to: shiyue/roberta-large-tac08 |
softcatala/julibert |
Catalan | Please refer to: softcatala/julibert |
elozano/tweet_sentiment_eval |
English | Please refer to: elozano/tweet_sentiment_eval |
cahya/roberta-base-indonesian-1.5G |
Indonesian | Please refer to: cahya/roberta-base-indonesian-1.5G |
elozano/tweet_emotion_eval |
English | Please refer to: elozano/tweet_emotion_eval |
navteca/roberta-large-squad2 |
English | Please refer to: navteca/roberta-large-squad2 |
elozano/tweet_offensive_eval |
English | Please refer to: elozano/tweet_offensive_eval |
ynie/roberta-large_conv_contradiction_detector_v0 |
English | Please refer to: ynie/roberta-large_conv_contradiction_detector_v0 |