Integrate evaluation methods into tests

[LianxinGao]

No description provided.

[mikecovlee]

You can use this light-weight evaluator: https://github.com/scukdde-llm/llm-evaluators

[LianxinGao]

@mikecovlee is the right way to test? python eval.py --base_model /data/llama-7b-hf --no_lora=true

it throw error:

Traceback (most recent call last):
  File "/data/glx/code/llm-evaluators/eval.py", line 36, in <module>
    tokenizer = LlamaTokenizer.from_pretrained(BASE_MODEL)
  File "/opt/conda/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1854, in from_pretrained
    return cls._from_pretrained(
  File "/opt/conda/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2017, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 156, in __init__
    self.sp_model = self.get_spm_processor()
  File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 164, in get_spm_processor
    model_pb2 = import_protobuf()
  File "/opt/conda/lib/python3.10/site-packages/transformers/convert_slow_tokenizer.py", line 40, in import_protobuf
    return sentencepiece_model_pb2
UnboundLocalError: local variable 'sentencepiece_model_pb2' referenced before assignment

[mikecovlee]

Try pip install sentencepiece ?

[LianxinGao]

Try pip install sentencepiece ?

@mikecovlee already installed。。。
Can you run the code successfully? python eval.py --base_model=YOUR_MODEL --no_lora=true

[mikecovlee]

Just try google it.

pip install google protobuf