Skip to content

Commit

Permalink
update few-shot README.md (PaddlePaddle#455)
Browse files Browse the repository at this point in the history
* "add mlm params to dygraph ernie1.0"

* finish p-tuning v1.0

* mend

* delete unused coment

* add label_normalized

* P-tuning: support Chid task of FewCLUE

* 1. decouple evaluate and train

* 1.add FewCLUE datasets(9/9)
2.implement p-tuning strategy by transform_function
3.unify train_script beteween `chid` task and other 8 tasks of FewCLUE

* add README.md

* update FewCLUE data

* add predict.py for FewCLUE

* update README

* update README.md
  • Loading branch information
tianxin committed May 27, 2021
1 parent 605f96c commit ca75dc8
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 19 deletions.
4 changes: 1 addition & 3 deletions examples/few_shot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Few-Shot Learning 旨在研究如何从少量有监督的训练样本中学习

随着大规模预训练模型的不断涌现,FSL 结合预训练模型的先验知识和强大的泛化能力在下游任务效果上取得了显著提升,为大规模预训练模型结合 FSL 的工业落地应用带来了无限可能性。

我们旨在为 FSL 领域的研究者提供简单易用、全面、前沿的 FSL 策略库,便于研究者基于 FSL 策略库将注意力集中在算法创新上。我们会持续开源 FSL 领域的前沿学术工作,并在中文小样本学习测评基准 [FewCLUE](https://github.com/CLUEbenchmark/FewCLUE) 上给出常见预训练模型的 Benchmark
我们旨在为 FSL 领域的研究者提供简单易用、全面、前沿的 FSL 策略库,便于研究者基于 FSL 策略库将注意力集中在算法创新上。我们会持续开源 FSL 领域的前沿学术工作,并在中文小样本学习测评基准 [FewCLUE](https://github.com/CLUEbenchmark/FewCLUE) 上进行评测

## Benchmark
| 算法 | 预训练模型 | Score | eprstmt | bustm | ocnli | csldcp | tnews | wsc | iflytek | csl | chid |
Expand All @@ -15,7 +15,5 @@ Few-Shot Learning 旨在研究如何从少量有监督的训练样本中学习
- PET(Todo)
- EFL(Todo)

## 如何贡献

## References
[1]X. Liu et al., “GPT Understands, Too,” arXiv:2103.10385 [cs], Mar. 2021, Accessed: Mar. 22, 2021. [Online]. Available: http://arxiv.org/abs/2103.10385
27 changes: 11 additions & 16 deletions examples/few_shot/p-tuning/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,17 @@ python -u -m paddle.distributed.launch --gpus "0" \
模型每训练 1 个 epoch, 会在验证集和测试集上进行评估。

### 模型预测
Todo
通过如下命令,指定 GPU 0 卡,使用 1 个 P-embedding 在 `FewCLUE``iflytek` 数据集上进行预测
```
python -u -m paddle.distributed.launch --gpus "0" predict.py \
--task_name "iflytek" \
--device gpu \
--init_from_ckpt "${model_params_file}" \
--p_embedding_num 1 \
--output_dir "./output" \
--batch_size 32 \
--max_seq_length 512
```

## 基于自定义数据进行 P-tuning 实验

Expand Down Expand Up @@ -97,20 +107,5 @@ python -u -m paddle.distributed.launch --gpus "0" \
--max_seq_length 512
```

### 模型预测

通过如下命令,指定 GPU 0 卡,使用 1 个 P-embedding 在 `FewCLUE``iflytek` 数据集上进行预测

```
python -u -m paddle.distributed.launch --gpus "0" predict.py \
--task_name "iflytek" \
--device gpu \
--init_from_ckpt "${model_params_file}" \
--p_embedding_num 1 \
--output_dir "./output" \
--batch_size 32 \
--max_seq_length 512
```

## References
[1]X. Liu et al., “GPT Understands, Too,” arXiv:2103.10385 [cs], Mar. 2021, Accessed: Mar. 22, 2021. [Online]. Available: http://arxiv.org/abs/2103.10385

0 comments on commit ca75dc8

Please sign in to comment.