-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Closed
Description
Describe the bug
Ragas documentation here states the ragas_score
can be retrieved directly from result of evaluate
. However, this throws a KeyError
Is there an intended workaround or was this removed with the intention that we would calculate harmonic mean using output?
Ragas version:
Python version: 3.10.13, 3.12.1
Code to Reproduce
# OPENAI API key set here...
from datasets import load_dataset
from ragas.metrics import (
answer_relevancy,
faithfulness,
context_recall,
context_precision,
context_relevancy
)
from ragas.metrics.critique import harmfulness
from ragas import evaluate
fiqa_eval = load_dataset("explodinggradients/fiqa", "ragas_eval")
result = evaluate(
fiqa_eval["baseline"].select(range(3)),
metrics=[
answer_relevancy,
faithfulness,
context_recall,
context_precision,
context_relevancy,
harmfulness
]
)
print(result['ragas_score'])
Expected behavior
The result object would have a key/value pair for ragas_score
as described in documentation.
Metadata
Metadata
Assignees
Labels
No labels