New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix call _get_keys for custom_evaluator #9763
fix call _get_keys for custom_evaluator #9763
Conversation
In the function _load_run_evaluators _get_keys is was not called in only custom_evaluator are used
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 2 Ignored Deployments
|
[isinstance(e, EvaluatorType) for e in config.evaluators] | ||
if config.evaluators or config.custom_evaluators or any( | ||
[isinstance(e, EvaluatorType) for e in config.evaluators] or any( | ||
[isinstance(e, EvaluatorType) for e in config.custom_evaluators]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should only be StringRunEvaluatorChain
for this second check
Hi, @hinthornw I finally set up the dev env properly. All tests passed. Let me know. |
Hi, @hinthornw @baskaryan Cloud you merge this PR? Or any other suggestions ? |
Thank you! |
In the function _load_run_evaluators the function _get_keys was not called if only custom_evaluators parameter is used