-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix llm as judge warnings #173
Conversation
Now loads openai lazily. the dep is no linger required when using lighteval |
You could add a warning/error when loading metrics at the task level, to check there the deps if needed. |
you mean like I did in task.py to check that the token is set ? |
And for both of these, I mean using the same checks as for all the other optional deps :) (using new functions defined in utils) |
@NathanHB I added the import testing I was thinking about - I'll let you review and if it works for you you can merge |
* commit * fixes * fix style * fixes * make style * Fix import error detection for open ai package (llm as a judge metric) --------- Co-authored-by: Nathan Habib <nathan.habib@huggingface.com> Co-authored-by: Clémentine Fourrier <22726840+clefourrier@users.noreply.github.com>
No description provided.