Question about parallel evaluation

[passing2961]

I have one question. When I run Qwen3-8B with OpenEvolve on the Erdős problem, the performance varies depending on the parallel evaluation configuration. For example, when I set the parallel evaluation to 1, it shows 0.3810, but when I set it to 16, it shows 0.495 (which is worse).
In the AlgoTune task, your team already demonstrated that parallel evaluation performs much better than sequential evaluation. In my case, however, I am trying to understand why the opposite trend appears. Do you have any comments on this?

[codelion]

When parallel_evaluations is too high, OpenEvolve can't build on top of successful ideas fast enough. New candidates get generated before recent improvements make it into the database, so exploration stays shallow.

The AlgoTune blog showed serial (1) was catastrophically bad vs parallel (16), but that doesn't mean more parallelism is always better. For most examples we run with parallel_evaluations: 4 and num_islands: 4-5 as the recommended default. That balance gives enough throughput without starving the feedback loop.

For the Erdős problem with Qwen3-8B, may be the mathematical refinements need that tighter feedback loop. Try parallel_evaluations: 4 as a middle ground between your two data points.

[passing2961]

@codelion Thanks for your kind response. By the way, under the parallel_evaluations: 1 setting on the Erdős problem, Qwen3-8B achieves 0.381082 while GPT-5.3-Codex achieves 0.381394. Intuitively, I would have expected GPT-5.3-Codex to perform better. Have you also observed cases where Qwen3-8B performs surprisingly well on the Erdős problem or other tasks?

[codelion]

On individual tasks there is variance in model responses. May have to test on a number of examples to say if a model is really better than the other.

[passing2961]

So, you mean that I have to test the models on other math tasks such as first_ineq. Am I correctly understanding?