New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

instruction tuningについて #13

Open

keisks opened this issue Apr 27, 2023 · 4 comments

Collaborator

keisks commented Apr 27, 2023 •

edited

Loading

事前学習後にinstruction tuning を行う。（RLHFよりも効果が大きいという話を以前どこかで聞いた記憶がある。）

日本語で行う場合のデータをどうするか。また、evaluation用のデータ（タスク）とinstruction tuning用のデータ（タスク）は分ける必要がありそう。

Collaborator Author

keisks commented May 7, 2023 •

edited

Loading

https://huggingface.co/datasets/kunishou/databricks-dolly-15k-ja
https://huggingface.co/datasets/kunishou/databricks-dolly-69k-ja-en-translation（翻訳タスク）
これがオープンソースコミュニティの力?

Collaborator Author

keisks commented May 7, 2023

RLFHについては
https://huggingface.co/datasets/Anthropic/hh-rlhf
を日本語化する必要がある。

Collaborator Author

keisks commented Jun 29, 2023

https://ai.googleblog.com/2023/03/presto-multilingual-dataset-for-parsing.html?m=1

Collaborator Author

keisks commented Jun 29, 2023

https://huggingface.co/datasets/MBZUAI/LaMini-instruction

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment