Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RAG Evaluation #8

Closed
2 tasks done
RobinQu opened this issue Apr 3, 2024 · 3 comments
Closed
2 tasks done

RAG Evaluation #8

RobinQu opened this issue Apr 3, 2024 · 3 comments
Milestone
v0.1.5

Comments

@RobinQu
Copy link
Owner

RobinQu commented Apr 3, 2024 •
edited
Loading

A notebook for experiment, including:

  • comparision with langchain as baseline.
  • ablation test for different retrivers, spliter paramters.
@RobinQu
Copy link
Owner Author

RobinQu commented Apr 3, 2024

intial notebook is located at https://github.com/RobinQu/instinct.cpp/blob/sprint/v0.1.1/modules/instinct-examples/doc-agent/test/rag_evaluation/rag_evaluation.ipynb

@RobinQu RobinQu added this to the v0.1.2 milestone Apr 3, 2024
@RobinQu RobinQu modified the milestones: v0.1.2, v0.1.4 May 21, 2024
@RobinQu RobinQu modified the milestones: v0.1.4, 0.1.5 Jun 16, 2024
@RobinQu
Copy link
Owner Author

RobinQu commented Jun 16, 2024

Inconsistency problem of RecursiveCharacterSplitter is resolved. Evaulation should be fair now.

@RobinQu RobinQu mentioned this issue Jun 16, 2024
Closed
4 tasks
@RobinQu
Copy link
Owner Author

RobinQu commented Jul 1, 2024

Close this issue. The evaulation script is stable since last version.

@RobinQu RobinQu closed this as completed Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
First GA Release of instinct.cpp
Status: Done
Milestone
v0.1.5
Development

No branches or pull requests
1 participant
@RobinQu