You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello!
To run Falcon for perplexity calculation use code in https://github.com/Vahe1994/SpQR/blob/main/main_for_rw.py , it is similar to main.py with some additional code to allow run falcon.
As for LM_eval - you can use lm_eval_main.py and provide falcon model in the args: --model hf-causal --model_args pretrained=tiiuae/falcon-40b.
Feel free to report any problem that you encounter.
P.s. We are working on combining scripts for Falcon and LLama and deduplicating code, see PR #7 .We will add more explicit instruction how to run Falcon in ReadMe.
Hi Authors,
Thank you for sharing the evaluation code for llama models. Could you please release the code for the evaluation of Falcon models?
Best regards,
Abdelrahman.
The text was updated successfully, but these errors were encountered: