Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full-Dialogue DiaASQ zh #15

Open
Tracked by #14
Nana2929 opened this issue Jun 7, 2023 · 0 comments
Open
Tracked by #14

Full-Dialogue DiaASQ zh #15

Nana2929 opened this issue Jun 7, 2023 · 0 comments

Comments

@Nana2929
Copy link
Owner

Nana2929 commented Jun 7, 2023

  1. 每個例子都很長,需要良好的分隔符號或提示:不在這邊加上「範例」或引導詞彙的話,ChatGPT 會以為範例是需要一起 inference 的例子。
    full_dialogue_dataset.py
    image

  2. ChatGPT results

INFO:__main__:Found 22 files to be concatenated ...
INFO:dataset.diaasq.full_dialogue_dataset:Legal pool (k-examples pool) size: 409
INFO:__main__:Sanity check passed!
INFO:__main__:Writing inference file to output/diaasq/gpt-full-dialog-zh/FullDiaAsqDataset_gpt_eval.csv ...
INFO:__main__:Starting evaluation on valid (100 examples)...
{'aspect_f1': 0.4953959483845313,
 'iden_f1': 0.12240553480962135,
 'opinion_f1': 0.21926105385779768,
 'pair_ao_f1': 0.14426229503281143,
 'pair_ta_f1': 0.2990881458468997,
 'pair_to_f1': 0.11902231663541006,
 'quad_micro_f1': 0.084703537569097,
 'target_f1': 0.5363457759829979} 
  1. Paper statistics
image
@Nana2929 Nana2929 mentioned this issue Jun 7, 2023
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant