Got errors when using gpt-3.5-turbo-1106

[wenlinyao]

Hi,

python -m alpha_codium.solve_problem --dataset_name valid_and_test_processed --split_name test --problem_number 0

The above script works well when I use gpt-3.5-turbo-0613. But when I use 'gpt-3.5-turbo-1106', it always shows the following error. Can you have a try on 'gpt-3.5-turbo-1106' to see if you have the same error? Thanks!

2024-01-25 21:11:46.480 | INFO | alpha_codium.gen.coding_competitor:solve_problem:118 - problem['name']: 1575_A. Another Sorting Problem
2024-01-25 21:11:46.484 | INFO | alpha_codium.gen.coding_competitor:run:60 - Running code contests competitor, model gpt-3.5-turbo-1106
2024-01-25 21:11:46.485 | INFO | alpha_codium.gen.stages.run_self_reflect:run_self_reflect:18 - --reflection stage--
2024-01-25 21:11:46.491 | INFO | alpha_codium.llm.ai_handler:chat_completion:86 - -----------------
2024-01-25 21:11:46.491 | INFO | alpha_codium.llm.ai_handler:chat_completion:87 - Running inference ...
2024-01-25 21:11:56.045 | INFO | alpha_codium.llm.ai_handler:chat_completion:133 - done
2024-01-25 21:11:56.045 | INFO | alpha_codium.llm.ai_handler:chat_completion:134 - -----------------
ERROR:root:'run_self_reflect' stage, counter_retry 0, Error: while scanning for the next token
found character '`' that cannot start any token
in "", line 1, column 1:
```yaml
^

[mrT23]

Thanks @wenlinyao
i added a small post-process, and now everything works fine:
b55f41d

There are small differences in variants of GPT we need to accommodate, especially when using them more like "completion" models than chat models.
sometimes they add
"```yaml" in the beginning of the answer. sometimes they don't ...

[wenlinyao]

Thanks!