Skip to content

Conversation

jjmachan
Copy link
Member

@jjmachan jjmachan commented Jan 23, 2024

usage:

from ragas.testset.generator import TestsetGenerator

# generator with openai models
generator = TestsetGenerator.with_openai()
generator

# specify distributions
from ragas.testset.evolutions import simple, reasoning, multi_context

distributions = {
    simple: 0.5,
    multi_context: 0.4,
    reasoning: 0.1
}
distributions

# generate testset
testset = generator.generate_with_llamaindex_docs(documents, 100, distributions)
testset.to_pandas()

100 rows in <4mins

part of #380

@jjmachan jjmachan requested a review from shahules786 January 23, 2024 02:52
"answer": "" if data.answer is None else data.answer,
"question_type": question_type,
"episode_done": True,
"evolution_elimination": data.evolution_elimination,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

evolution_elimination is an intermediate filtering step, so I wouldn't expose it in the final output.

@shahules786 shahules786 merged commit 8969173 into explodinggradients:main Jan 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants