-
Notifications
You must be signed in to change notification settings - Fork 4
/
log.txt
22 lines (22 loc) · 1.52 KB
/
log.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Reponse length
+---+--------------------+-------+------+
| | model | mean | std |
+---+--------------------+-------+------+
| 0 | gpt-3.5-turbo-1106 | 384.5 | 7.5 |
| 1 | gpt-4-0125-preview | 730.0 | 60.0 |
+---+--------------------+-------+------+
Total comparisons: 8. Failed to response: 0
Total comparisons: 8. Non exact matches: 8
Extracted 8 answers from the responses (extraction success rate 100.00%)
(Win + Draw / 2) w. gpt-3.5-turbo-1106:
+---+--------------------+--------------+-----+-----------+------------------------+-------------------+
| | model | Overall-lang | EN | uml_class | uml_class-faithfulness | uml_class-general |
+---+--------------------+--------------+-----+-----------+------------------------+-------------------+
| 0 | gpt-4-0125-preview | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
+---+--------------------+--------------+-----+-----------+------------------------+-------------------+
(Win + Both Good) w. gpt-3.5-turbo-1106:
+---+--------------------+--------------+-----+-----------+------------------------+-------------------+
| | model | Overall-lang | EN | uml_class | uml_class-faithfulness | uml_class-general |
+---+--------------------+--------------+-----+-----------+------------------------+-------------------+
| 0 | gpt-4-0125-preview | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
+---+--------------------+--------------+-----+-----------+------------------------+-------------------+