ModuleNotFoundError: No module named 'llama'

[yufan412]

Hello, I don't know what functions are implemented in your llama file. Could you please send me a copy

[chawins]

Oh my apologies. I probably did not update the README. You can pip install this from https://github.com/meta-llama/llama3/tree/main?tab=readme-ov-file#quick-start.

As a matter of fact, this Llama-3 tokenizer is not necessary anymore. I originally thought that there was a bug with HuggingFace Llama-3 tokenizer, but I was told that there is an easy fix for it (ref: huggingface/transformers#31187). I will try to remove this custom Llama-3 tokenizer in the future and stick to the HuggingFace tokenizer for simplicity.