You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
scorers.text (deterministic, no LLM): exact_match, contains_all, contains_any, levenshtein_similarity, embedding_similarity
(provider-neutral -- pass an embed_fn, this package makes no
embedding calls itself), valid_json, json_diff, numeric_diff.
scorers.rubric (LLM-graded, provider-neutral): RubricTemplate and LLMRubricEvaluator (pass a complete_fn; this package never calls a
model itself). Ships FACTUALITY, CLOSED_QA, SUMMARY_QUALITY, BATTLE (pairwise A/B), and MODERATION templates.
default_registry() returns an EvaluatorRegistry pre-populated with
every text/trajectory scorer under a stable name, ready to pass to evaluate_suite(suite, samples, registry=default_registry()).
All of the above are re-exported from the package root.