Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added prompt class #388

Merged
merged 8 commits into from Dec 19, 2023

Conversation

tinomaxthayil
Copy link
Contributor

  • Added class RagasPrompt for creating prompt objects
    - RagasPrompt.format - Will take the input parameters and returns the ChatPromptTemplate
    - RagasPrompt.validate - Will check if the prompt object has the field in correct formay

  • Added RagasPrompt to init

  • Added test_prompt to test prompt object

Copy link
Member

@jjmachan jjmachan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks really good but just 2 things

  1. write some tests - you can write some simple tests and reflect the usecases that prompts will be used for (based on different types of prompts we have) and that would be very valuable
  2. no need for a module named /prompt just a file is enough - we are not going to subclass this right. Prompt will only be a container for the data

src/ragas/prompts/__init__.py Outdated Show resolved Hide resolved
src/ragas/prompts/base.py Outdated Show resolved Hide resolved
src/ragas/prompts/base.py Outdated Show resolved Hide resolved
src/ragas/prompts/base.py Outdated Show resolved Hide resolved
src/ragas/prompts/base.py Outdated Show resolved Hide resolved
Copy link
Member

@shahules786 shahules786 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add missing dependency langchain-core

@shahules786
Copy link
Member

It would be good to have a function that returns examples as str. For example:

statement_prompt = RagasPrompt(
    instruction = "Create one or more statements from each sentence in the given answer.",
    examples = [
    {"question": "Who was  Albert Einstein and what is he best known for?",
    "answer": "He was a German-born theoretical physicist, widely acknowledged to be one of the greatest and most influential physicists of all time. He was best known for developing the theory of relativity, he also made important contributions to the development of the theory of quantum mechanics.",
    "statements in json":
    """{
        "statements": [
            "Albert Einstein was born in Germany.",
            "Albert Einstein was best known for his theory of relativity."
        ]
    }"""}],
    input_keys=["question","answer"],
    output_key="statements in json",
    output_type="JSON")

statement_prompt.get_example_str(0)

Output

question: Who was  Albert Einstein and what is he best known for?
answer: He was a German-born theoretical physicist, widely acknowledged to be one of the greatest and most influential physicists of all time. He was best known for developing the theory of relativity, he also made important contributions to the development of the theory of quantum mechanics.
statements in json: {{
        "statements": [
            "Albert Einstein was born in Germany.",
            "Albert Einstein was best known for his theory of relativity."
        ]
    }}

Copy link
Member

@jjmachan jjmachan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just some minor changes, feel free to merge it after fixing it
Note: do make the CI passes before - that is important

requirements/dev.txt Outdated Show resolved Hide resolved
src/ragas/__init__.py Outdated Show resolved Hide resolved
Copy link
Member

@shahules786 shahules786 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shahules786 shahules786 merged commit 9850f09 into explodinggradients:main Dec 19, 2023
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants