-
Notifications
You must be signed in to change notification settings - Fork 0
/
prompt_template_premise.py
164 lines (107 loc) · 7.8 KB
/
prompt_template_premise.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
# Note on PROMPT_TEMPLATE_4_SHOT_COT - the 4 shot is chosen from PROMPT_TEMPLATE_4SHOT, it will allow us to compare the peformance between
# PROMPT_TEMPLATE_4SHOT and PROMPT_TEMPLATE_4_SHOT_COT.
PROMPT_TEMPLATE_ZEROSHOT_WITH_INSTRUCTION=f"""In this task, you will be given a text and your goal is to classify the text as either "premise" or "non-premise" based on the definitions below. The texts are from the Decisions and Judgements categories of the European Court of Human Rights (ECHR).
"premise": In the context of argumentation in case law, a "premise" refers to the statements, facts, or assertions that provide the basis/reason for a non-premise. They are the reasons given to support the final decision of the Commission or Court. They form the building blocks of the argumentative structure leading to the non-premise.
"non-premise": In the context of argumentation in case law, a "non-premise" is the final decision or judgment made by the Commission or Court. It is often supported by one or more premises. The non-premise is the result of the argumentative process and is the central point that the argument is trying to establish.
Text to classify:[TEST_TEXT]
Classification:"""
# This prompt can be used for any 2shot example that are
# similar_sentence, disimilar_sentence, random_sentence
PROMPT_TEMPLATE_2SHOT_WITH_INSTRUCTION_SEMANTICALLY_CHOSEN="""In this task, you will be given a text and your goal is to classify the text as either "premise" or "non-premise" based on the definitions below. The texts are from the Decisions and Judgements categories of the European Court of Human Rights (ECHR).
"premise": In the context of argumentation in case law, a "premise" refers to the statements, facts, or assertions that provide the basis/reason for a non-premise. They are the reasons given to support the final decision of the Commission or Court. They form the building blocks of the argumentative structure leading to the non-premise.
"non-premise": In the context of argumentation in case law, a "non-premise" is the final decision or judgment made by the Commission or Court. It is often supported by one or more premises. The non-premise is the result of the argumentative process and is the central point that the argument is trying to establish.
Below are examples of texts that are correctly classified as "premise"/"non-premise".
Example:[EXAMPLE_1_TEXT]
Classification:[EXAMPLE_1_LABEL]
Example:[EXAMPLE_2_TEXT]
Classification:[EXAMPLE_2_LABEL]
Text to classify:[TEST_TEXT]
Classification:
"""
# This prompt can be used for any 4shot example that are
# similar_sentence, disimilar_sentence, random_sentence
PROMPT_TEMPLATE_4SHOT_WITH_INSTRUCTION_SEMANTICALLY_CHOSEN="""In this task, you will be given a text and your goal is to classify the text as either "premise" or "non-premise" based on the definitions below. The texts are from the Decisions and Judgements categories of the European Court of Human Rights (ECHR).
"premise": In the context of argumentation in case law, a "premise" refers to the statements, facts, or assertions that provide the basis/reason for a non-premise. They are the reasons given to support the final decision of the Commission or Court. They form the building blocks of the argumentative structure leading to the non-premise.
"non-premise": In the context of argumentation in case law, a "non-premise" is the final decision or judgment made by the Commission or Court. It is often supported by one or more premises. The non-premise is the result of the argumentative process and is the central point that the argument is trying to establish.
Below are examples of texts that are correctly classified as "premise"/"non-premise".
Example:[EXAMPLE_1_TEXT]
Classification:[EXAMPLE_1_LABEL]
Example:[EXAMPLE_2_TEXT]
Classification:[EXAMPLE_2_LABEL]
Example:[EXAMPLE_3_TEXT]
Classification:[EXAMPLE_3_LABEL]
Example:[EXAMPLE_4_TEXT]
Classification:[EXAMPLE_4_LABEL]
Text to classify:[TEST_TEXT]
Classification:
"""
'''
Below I have removed this sentence for answering the reviwer 1 question
" The texts are from the Decisions and Judgements categories of the European Court of Human Rights (ECHR)."
Originally it was like this
"In this task, you will be given a text and your goal is to classify the text as either "conclusion" or "non-conclusion" based on the definitions below. The texts are from the Decisions and Judgements categories of the European Court of Human Rights (ECHR)."
14.Oct.2023
'''
# This prompt can be used for any 8shot example that are
# similar_sentence, disimilar_sentence, random_sentence
PROMPT_TEMPLATE_8SHOT_WITH_INSTRUCTION_SEMANTICALLY_CHOSEN="""In this task, you will be given a text and your goal is to classify the text as either "premise" or "non-premise" based on the definitions below.
"premise": In the context of argumentation in case law, a "premise" refers to the statements, facts, or assertions that provide the basis/reason for a non-premise. They are the reasons given to support the final decision of the Commission or Court. They form the building blocks of the argumentative structure leading to the non-premise.
"non-premise": In the context of argumentation in case law, a "non-premise" is the final decision or judgment made by the Commission or Court. It is often supported by one or more premises. The non-premise is the result of the argumentative process and is the central point that the argument is trying to establish.
Below are examples of texts that are correctly classified as "premise"/"non-premise".
Example:[EXAMPLE_1_TEXT]
Classification:[EXAMPLE_1_LABEL]
Example:[EXAMPLE_2_TEXT]
Classification:[EXAMPLE_2_LABEL]
Example:[EXAMPLE_3_TEXT]
Classification:[EXAMPLE_3_LABEL]
Example:[EXAMPLE_4_TEXT]
Classification:[EXAMPLE_4_LABEL]
Example:[EXAMPLE_5_TEXT]
Classification:[EXAMPLE_5_LABEL]
Example:[EXAMPLE_6_TEXT]
Classification:[EXAMPLE_6_LABEL]
Example:[EXAMPLE_7_TEXT]
Classification:[EXAMPLE_7_LABEL]
Example:[EXAMPLE_8_TEXT]
Classification:[EXAMPLE_8_LABEL]
Text to classify:[TEST_TEXT]
Classification:
"""
def create_prompt_premise(test_text,
*args,
zeroshot_with_instruction=False,
twoshot_with_instruction=False,
fourshot_with_instruction=False,
eightshot_with_instruction=False
):
list_length = len(args)
if zeroshot_with_instruction:
return PROMPT_TEMPLATE_ZEROSHOT_WITH_INSTRUCTION.replace("[TEST_TEXT]", test_text)
elif list_length == 2 and twoshot_with_instruction:
prompt = PROMPT_TEMPLATE_2SHOT_WITH_INSTRUCTION_SEMANTICALLY_CHOSEN
for i in range(1, list_length + 1):
text, label, _ = args[i - 1]
prompt = prompt.replace(f'[EXAMPLE_{i}_TEXT]', text).replace(f'[EXAMPLE_{i}_LABEL]', label)
prompt = prompt.replace('[TEST_TEXT]', test_text)
return prompt
elif list_length == 4 and fourshot_with_instruction:
prompt = PROMPT_TEMPLATE_4SHOT_WITH_INSTRUCTION_SEMANTICALLY_CHOSEN
for i in range(1, list_length + 1):
text, label, _ = args[i - 1]
prompt = prompt.replace(f'[EXAMPLE_{i}_TEXT]', text).replace(f'[EXAMPLE_{i}_LABEL]', label)
prompt = prompt.replace('[TEST_TEXT]', test_text)
return prompt
elif list_length == 8 and eightshot_with_instruction:
prompt = PROMPT_TEMPLATE_8SHOT_WITH_INSTRUCTION_SEMANTICALLY_CHOSEN
for i in range(1, list_length + 1):
text, label, _ = args[i - 1]
prompt = prompt.replace(f'[EXAMPLE_{i}_TEXT]', text).replace(f'[EXAMPLE_{i}_LABEL]', label)
prompt = prompt.replace('[TEST_TEXT]', test_text)
return prompt
else:
raise ValueError("""Please provide a valid flag:
'zeroshot_with_instruction',
'twoshot_with_instruction'
'fourshot_with_instruction',
'eightshot_with_instruction'
""")