This is the implementation for Chinese Web fiction Genrative Pre-Trained model:GPT-WF.
This repo is tested on Python 3.6+, PyTorch 1.5.0+. To use the program the following prerequisites need to be installed.
- pytorch>=1.5.0
- nltk>=3.4
- numpy>=1.15.4
- pandas>=0.24.0
- sentencepiece>=0.1.8
- tensorflow>=1.12.0
- apex
Please download the pre-trained models from the following links and put it in folder ~/Megatron-LM/checkpoints/generic/
- Basic model: https://share.weiyun.com/r6Kdrm7T password：xqsuuv
- Stylized models https://share.weiyun.com/ELBwoIMU password：7gyavd
When you have downloaded the pre-trained models, you should run the following scripts to generate web fiction.
cd Megatron-LM
bash scripts/generate_text.sh
For business reason, we don't present CWFC corpus. But we will present the corpus for fine-tuning stylized models soon.