Skip to content

Conversation

shahules786
Copy link
Member

@shahules786 shahules786 commented Dec 25, 2023

Automatic prompt adaption for different languages.

  • Add support for all metrics
  • Testing and experimentation
  • Add docs (with explanation)

To be merged only after #401

Usage

from ragas.metrics import faithfulness
faithfulness.adapt("hindi")
faithfulness.save()

shahules786 and others added 30 commits October 19, 2023 17:22
@shahules786 shahules786 marked this pull request as ready for review December 31, 2023 13:09
@shahules786 shahules786 requested a review from jjmachan January 1, 2024 03:06
@shahules786 shahules786 added the enhancement New feature or request label Jan 1, 2024
Copy link
Member

@jjmachan jjmachan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • logging module instead of print
  • that part of code in prompt.py that I didn't quite understand, it could be why cache is slow.

Also try to put in a bit more effort into variable names after the llm.generate step. more thoughtful var names can make a big difference in improving readability of the code esp after llm.generate because normally we have no idea what the output of that function is.

Comment on lines +157 to +159
assert len(grouped_results) == len(
self.examples
), "examples and adapted examples must be of equal length"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this raise an exception or is it okay

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leaving it as assert itself - with the assumption that there is no try - except method we can do to catch this.
the con is that the whole operation fails in this case. I think we should raise an Exception and catch it and throw a warning

Copy link
Member

@jjmachan jjmachan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix these things

@shahules786 shahules786 mentioned this pull request Jan 1, 2024
@shahules786 shahules786 merged commit 54ac551 into explodinggradients:main Jan 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants