I reproduce the results of Block-ap PPL, wikitext2 7.76 c4 9.50 [Llama2-7b-w2g64], logs below.
I use the auther released weight [1] to test, the result is 7.65, 9.36. Are there something to notice for training the model,
or just the randomness to cause the PPL shift?
[1]https://huggingface.co/ChenMnZ/Llama-2-7b-BlockAP-w2g64/tree/main
I reproduce the results of Block-ap PPL, wikitext2 7.76 c4 9.50 [Llama2-7b-w2g64], logs below.
I use the auther released weight [1] to test, the result is 7.65, 9.36. Are there something to notice for training the model,
or just the randomness to cause the PPL shift?
[1]https://huggingface.co/ChenMnZ/Llama-2-7b-BlockAP-w2g64/tree/main