Automatic Correction of Errors in the Generated Program

[lmanhes]

First of all, congratulations on this implementation, the code is very clear!

In case of a program error during evaluation, could we loop back to modify the program using the errors as feedback?

[codelion]

It would be interesting to let the LLM fix the error and regenerate the code. I haven't tried it yet in some of the examples I modified the system prompt after seeing the LLM make some obvious errors like using an API that didn't exist.

We can consider adding it but it is actually quite easy to use something like optillm - https://github.com/codelion/optillm It already provides an OpenAI Compatible API so we can just use prepend the model name with executecode or any of the approaches implemented in optillm and use them during search in openevolve.