Skip to content

ragas_score removed from Result class in November 2023 but still in documentation #415

@mwertheim-zco

Description

@mwertheim-zco

Describe the bug
Ragas documentation here states the ragas_score can be retrieved directly from result of evaluate. However, this throws a KeyError
Is there an intended workaround or was this removed with the intention that we would calculate harmonic mean using output?

Ragas version:
Python version: 3.10.13, 3.12.1

Code to Reproduce

# OPENAI API key set here...

from datasets import load_dataset
from ragas.metrics import (
    answer_relevancy,
    faithfulness,
    context_recall,
    context_precision,
    context_relevancy
)
from ragas.metrics.critique import harmfulness
from ragas import evaluate

fiqa_eval = load_dataset("explodinggradients/fiqa", "ragas_eval")


result = evaluate(
    fiqa_eval["baseline"].select(range(3)),
    metrics=[
        answer_relevancy,
        faithfulness,
        context_recall,
        context_precision,
        context_relevancy,
        harmfulness
    ]
)

print(result['ragas_score'])

Expected behavior
The result object would have a key/value pair for ragas_score as described in documentation.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions