Llama2RNN.c：一个用C语言实现的终身 RNN 模型

这是一个使用 Llama2 权重设计的循环神经网络（RNN）模型，旨在无限期运行（终身）。

如何训练

参考README_llama2.c.md处理好数据

python3 tinystories.py download
python3 tinystories.py train_vocab --vocab_size=4096
python3 tinystories.py pretokenize --vocab_size=4096

python3 train.py config/train_tinystories_token4096_memorynorm.py

python3 tokenizer.py --tokenizer-model ./data/tok4096.model
export.py out_path/model_q80.bin --version 2 --mem --checkpoint out_path/ckpt.pt

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 519 Commits
.github/workflows		.github/workflows
assets		assets
config		config
doc		doc
.gitignore		.gitignore
LICENSE		LICENSE
Makefile		Makefile
README.en.md		README.en.md
README.md		README.md
README_llama2.c.md		README_llama2.c.md
build_msvc.bat		build_msvc.bat
configurator.py		configurator.py
data_generator.py		data_generator.py
dataset.py		dataset.py
datatask.py		datatask.py
eval.sh		eval.sh
export.py		export.py
jiiov.py		jiiov.py
jiiov_chat.py		jiiov_chat.py
jiiov_rephrase.py		jiiov_rephrase.py
model.py		model.py
requirements.txt		requirements.txt
run.c		run.c
runq.c		runq.c
runqm.c		runqm.c
sample.py		sample.py
test.c		test.c
test_all.py		test_all.py
tinystories.py		tinystories.py
tokenizer.bin		tokenizer.bin
tokenizer.model		tokenizer.model
tokenizer.py		tokenizer.py
train.py		train.py
ultrachat.py		ultrachat.py
wiki.py		wiki.py
wiki_zh.py		wiki_zh.py
wikipedia_en.py		wikipedia_en.py
win.c		win.c
win.h		win.h
zhihu.py		zhihu.py