Skip to content

feat/executor: make ragas faster and more robust to run #390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 38 commits into from
Jan 4, 2024
Merged

feat/executor: make ragas faster and more robust to run #390

merged 38 commits into from
Jan 4, 2024

Conversation

jjmachan
Copy link
Member

@jjmachan jjmachan commented Dec 17, 2023 •
edited
Loading

  1. async vs threadpool
  2. executor APIs and how to use them
  3. BaseRagasLLM - how generate_text and agenerate_text work - what is the diff
  4. change in each metric - this is the most important to get review from @ikka because there might be small bugs in calculations I missed - the code might be working now but calculation errors can be very bad

@jjmachan
changed name RagasLLM -> BaseRagasLLM
4b06818
@jjmachan
added langchain_core llm
f30cf57
@jjmachan
changed faithfulness to single metric
1fcea66
@jjmachan
fixing llm for executor
84d9cb2
@jjmachan
added callbacks
8a41af7
@jjmachan
ported a couple of metrics
0c61475
@jjmachan
executor tested
8cf84fc
@jjmachan
executor object created
21c1456
@jjmachan
error handling for both
a779aea
@jjmachan
fixed up BaseRagasLLM and BaseRagasEmbeddings
44558d9
@jjmachan
as_complete functionality
b4e080d
@jjmachan
fix BaseRagasLLM
db71c5d
@jjmachan
basic callbacks configured
7161674
@jjmachan
fixed results
f8b98f5
@jjmachan
merged with main
8d654c4
@jjmachan
remove DS_store
8163d45
@jjmachan
fix prompts
9395c29
@jjmachan
moved tests
9dcc4fc
@jjmachan
Merge branch 'fix/prompt' into feat/executor
30422c0
@jjmachan
fixed some metrics
14aa522
@jjmachan
answer_correctness ported
777db4d
@jjmachan
formating
baebebc
@jjmachan
critique ported
4318d1b
@jjmachan
contex_recall ported
306ebe1
@jjmachan
context_relevancy ported
6ff94a2
@jjmachan
added benchmark
3691653
@jjmachan
Merge branch 'main' into feat/executor
8ccce15
@jjmachan jjmachan marked this pull request as ready for review January 3, 2024 06:55
@jjmachan jjmachan requested a review from shahules786 January 4, 2024 14:59
@jjmachan jjmachan merged commit bad3a8e into explodinggradients:main Jan 4, 2024
@jjmachan jjmachan deleted the feat/executor branch January 4, 2024 15:30
This was referenced Jan 8, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shahules786 shahules786 Awaiting requested review from shahules786

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jjmachan