New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Evaluation Dataset Generation #530

Open

penguine-ip opened this issue Feb 26, 2024 · 1 comment

Assignees

Labels

enhancement help wanted

Contributor

penguine-ip commented Feb 26, 2024 •

edited

Evaluation dataset generation is coming to deepeval by end of this week. For this feature, we're looking at the following:

Allow users to generate test cases based on their knowledge base
Allow users to choose how to chunk their knowledge base
Allow users to specify how many test cases to generate
Allow users to complicate test cases to make them more realistic (https://arxiv.org/pdf/2304.12244.pdf, https://mlabonne.github.io/blog/notes/Large%20Language%20Models/phi1.htm)

Feedback, suggestions, and contributions for any of the points above are welcomed 😊

penguine-ip added enhancement help wanted labels

penguine-ip self-assigned this

piseabhijeet commented Mar 4, 2024

link not found:
https://mlabonne.github.io/blog/notes/Large%20Language%20Models/phi1.htm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment