Any plans to train a instruction-following version?

[matthewdm0816]

Alpaca released their dataset on instruction tuning which is used on many other LLMs, any plan to finetune the RWKV on similar data?
Here are a source of Chinse alpaca:https://github.com/LC1332/Chinese-alpaca-lora

[innerop]

Maybe using this?

https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html

[BlinkDL]

Update ChatRWKV v2 to latest version and you can try an RWKV 7B Alpaca test model:

https://huggingface.co/BlinkDL/rwkv-4-pile-7b/blob/main/RWKV-4-Pile-7B-Instruct-test4-20230326.pth

It's recommended to use +i for "Alpaca Instruct". Examples:

+i Explain the following metaphor: "Life is like cats". 
+i write a python function to read data from an excel file.

[winglian]

Update ChatRWKV v2 to latest version and you can try an RWKV 7B Alpaca test model:

https://huggingface.co/BlinkDL/rwkv-4-pile-7b/blob/main/RWKV-4-Pile-7B-Instruct-test4-20230326.pth

It's recommended to use +i for "Alpaca Instruct". Examples:

+i Explain the following metaphor: "Life is like cats". 
+i write a python function to read data from an excel file.

@BlinkDL how are instruct datasets prepared? I assume use https://github.com/BlinkDL/RWKV-v2-RNN-Pile/blob/main/RWKV-v3/prepare_data.py. Are all the instructions concatenated into the same array, or is into a multi-dimensional array? tia!