Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: MVP for the new TestsetGenerator - SimpleEvolution #464

Merged
merged 10 commits into from Jan 18, 2024

Conversation

jjmachan
Copy link
Member

perform an evolution

from ragas.testset.evolutions import SimpleEvolution, NodeFilter, QuestionFilter, logger

node_filter = NodeFilter(gpt4)
ques_filter = QuestionFilter(gpt4)
se = SimpleEvolution(node_filter, ques_filter)

await se.aevolve(llm, docstore)

run evolutions with executor

from ragas.executor import Executor

exec = Executor(is_async=False)
qs = []
for i in range(10):
    se = SimpleEvolution(node_filter, ques_filter)
    exec.submit(se.evolve, llm, docstore, name=f"SimpleEvolution-{i}")

try:
    qs = exec.results()
except ValueError:
    se = SimpleEvolution(node_filter, ques

generates 300 samples in <6min, should be scalable enough

related to #380

@jjmachan jjmachan merged commit 49592ff into explodinggradients:main Jan 18, 2024
15 checks passed
@jjmachan jjmachan deleted the testsetgen/mvp branch January 18, 2024 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants