Skip to content

Commit

Permalink
fix(evals): incorrect wording in hallucinations (#3085)
Browse files Browse the repository at this point in the history
* fix(evals): incorrect wording in hallucinations

* re-run hallucinations eval

* fix
  • Loading branch information
mikeldking committed May 3, 2024
1 parent 672cbed commit 7aa0292
Show file tree
Hide file tree
Showing 2 changed files with 119 additions and 119 deletions.
12 changes: 6 additions & 6 deletions packages/phoenix-evals/src/phoenix/evals/default_templates.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,10 @@
HALLUCINATION_PROMPT_RAILS_MAP = OrderedDict({True: "hallucinated", False: "factual"})
HALLUCINATION_PROMPT_BASE_TEMPLATE = """
In this task, you will be presented with a query, a reference text and an answer. The answer is
generated to the question based on the reference text. The answer may contain false information, you
generated to the question based on the reference text. The answer may contain false information. You
must use the reference text to determine if the answer to the question contains false information,
if the answer is a hallucination of facts. Your objective is to determine whether the reference text
contains factual information and is not a hallucination. A 'hallucination' in this context refers to
if the answer is a hallucination of facts. Your objective is to determine whether the answer text
contains factual information and is not a hallucination. A 'hallucination' refers to
an answer that is not based on the reference text or assumes information that is not available in
the reference text. Your response should be a single word: either "factual" or "hallucinated", and
it should not include any other text or characters. "hallucinated" indicates that the answer
Expand All @@ -76,10 +76,10 @@
"""
HALLUCINATION_PROMPT_TEMPLATE_WITH_EXPLANATION = """
In this task, you will be presented with a query, a reference text and an answer. The answer is
generated to the question based on the reference text. The answer may contain false information, you
generated to the question based on the reference text. The answer may contain false information. You
must use the reference text to determine if the answer to the question contains false information,
if the answer is a hallucination of facts. Your objective is to determine whether the reference text
contains factual information and is not a hallucination. A 'hallucination' in this context refers to
if the answer is a hallucination of facts. Your objective is to determine whether the answer text
contains factual information and is not a hallucination. A 'hallucination' refers to
an answer that is not based on the reference text or assumes information that is not available in
the reference text.
Expand Down

0 comments on commit 7aa0292

Please sign in to comment.