Limiting fitness evaluations

[HenrikMettler]

This PR draft is a suggestion for a solution to #224
It works by counting the calls of the objective function using a decorator (as suggested eg: https://stackoverflow.com/questions/21716940/is-there-a-way-to-track-the-number-of-times-a-function-is-called). I am not sure if this is a reasonable approach, mainly two things bother me:

• the subtraction if fitness is not reevalutated (necessary since we don't want to count those)
• if a caching decorator is used for the inner objective, the counts of the objective function calls might not correspond to full function evaluations anymore

[jakobj]

thanks for working on this! didn't have time yet to check it in detail, but as i already mentioned we could think about introducing a general "objective" decorator, that would allow us to do such things like counting the number of objective evaluations in a simpler fashion. this would also remove the need to manually add the if individual.fitness is None statement to every objective. it would look something like:

@cgp.objective
def objective(individual):
    ....
    return individual

what do you think @mschmidt87?

[mschmidt87]

Totally agree, a general objective wrapper sounds awesome. So, what you're saying is that within this PR, we would use @HenrikMettler 's code and

• add the if individual.fitness is None check to it
• move the decorator from the example to utils.py
  , right?

[mschmidt87]

Little activity on this PR, @HenrikMettler are you working on this PR?

[HenrikMettler]

I moved the wrapper into utils.py. Since I am not sure this is the way you were thinking @jakobj I didn't adapt all the examples. Also many tests in test_hl_api.py and test_utils.py fail under the current, because their objectives have no attribute calls

[jakobj]

thanks for the changes 👌 i've left some inline comments that should be addressed.

but before getting into that, we need to make sure this works in parallel contexts. would this implementation work there? how are the evaluations counted across different processes?

[HenrikMettler]

Should I introduce the objective decorator for all examples and tests within this PR, or in a separate PR?

[jakobj]

Should I introduce the objective decorator for all examples and tests within this PR, or in a separate PR?

let's first figure out a suitable implementation that works with more than one process. we might end up with a different solution than the decorator after all.

[HenrikMettler]

As suggested by @jakobj the counter for objective function calls is now moved to the evolutionary algorithm. Removing it from the objective decorator enables using it with multiple processes running in parallel. I left everything for the objective decorator within this PR, even though it doesn't really belong to this topic anymore

[jakobj]

thanks for (re)implementing this so quickly! 🚀

i've added a few inline comments. also please add a unit test.

[jakobj]

thanks for the update! 👌

some more inline comments, then it's ready to merge!

[jakobj]

excellent, thanks a lot! 👍

as always, please reorganize the history before merging.