Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[14/N][RAG Demo fixes] [dev] run 5 trials over 4 queries #1204

Merged
merged 4 commits into from
Feb 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 25 additions & 6 deletions cookbooks/RAG-with-Model-Graded-Eval-v2/rag.aiconfig.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,16 +76,19 @@ metadata:
remember_chat_context: false
name: Rag Demo With Model-graded Eval
prompts:
- input: "Answer the following question using the provided context. Question: {{query}}\
\ \nContext: {{context}}"
- input: "Answer the following question using the provided context. \n\nQuestion:\
\ {{query}} \nContext: {{context}}"
metadata:
model: gpt-3.5-turbo
model:
name: gpt-3.5-turbo-16k-0613
settings: {}
parameters: {}
remember_chat_context: false
name: generate
outputs: []
- input: "Given the following question, and answer, does the answer satisfactorily\
\ answer the question? \n\nQuestion: {{query}}\nAnswer: {{generate.output}}"
\ answer the question? \n\nGive a relevance verdict (YES or NO) with an explanation.\n\
\nQuestion: {{query}}\nAnswer: {{generate.output}}\n\nVerdict:\nExplanation:"
metadata:
model: gpt-4
parameters: {}
Expand All @@ -103,15 +106,31 @@ prompts:
name: evaluate_faithfulness
outputs: []
- input: 'Given the following answer, is the answer self-consistent and easy to understand?
Answer: {{generate.output}}'
Answer: {{generate.output}}


Give a relevance verdict (YES or NO) with an explanation.


Verdict:

Explanation:'
metadata:
model: gpt-4
parameters: {}
remember_chat_context: false
name: evaluate_coherence
outputs: []
- input: 'Given the following answer, is the answer self-consistent and easy to understand?
Answer: {{generate.output}}'
Answer: {{generate.output}}


Give a succinctness verdict (YES or NO) with an explanation.


Verdict:

Explanation:'
metadata:
model: gpt-4
parameters: {}
Expand Down
Loading