issue 19 : A note on reproducibility

[geoffreyp]

add the function set_seed()

[sjvrijn]

Looks good to me, do you agree @chkoar?

[sjvrijn]

You may want to include the use of your new set_seed() function in the tests:

• test_combinatorial_algorithm.py#L16
• test_genetic_operator.py#L16
• test_numerical_algorithm.py#L16
• test_parent_selector.py#L19
• test_solution.py#L16
• test_tool.py#L15

[chkoar]

Looks good to me, do you agree @chkoar?

Yes. But I have the impression that this approach is not safe in threaded environments.
A small naive example:

import random
import numpy as np
import time
from moead_framework.problem.numerical import Zdt1
from concurrent.futures import ThreadPoolExecutor

random.seed(0)
problem = Zdt1(2)
expected = [
    [0.8444218515250481, 0.7579544029403025],
    [0.420571580830845, 0.25891675029296335],
]

solutions = []
solutions.append(problem.generate_random_solution().solution)
solutions.append(problem.generate_random_solution().solution)

assert np.array_equal(solutions, expected)

def worker(*args):
    time.sleep(np.random.randint(1,3))
    return problem.generate_random_solution().solution


for i in range(10):
    random.seed(0)
    with ThreadPoolExecutor() as executor:
        solutions = executor.map(worker, range(2))
        solutions = [x for x in solutions]

    assert np.array_equal(solutions, expected)

[geoffreyp]

You may want to include the use of your new set_seed() function in the tests:

It is available in 2c322fa

Yes. But I have the impression that this approach is not safe in threaded environments.

From the python documentation :

By re-using a seed value, the same sequence should be reproducible from run to run as long as multiple threads are not running.

The solution available is to instantiate a new instance of Random for each Thread and use this instance in problems and algorithms but it requires a lot of refactoring in the framework to replace each call of "import random" (and it is only for the Random module, the problem is maybe the same for np.random)

local_random = random.Random(n)
for i in xrange(100):
    a.append(local_random.randint(0, 1000))

I propose to add a note to prevent that this approach is not safe in threaded environments in python.

[sjvrijn]

A note in the docstring set_seed and wherever else relevant in the rest of the documentation would sufficiently address this issue for me.

[geoffreyp]

A note in the docstring set_seed and wherever else relevant in the rest of the documentation would sufficiently address this issue for me.

Available in 2b313c1

[sjvrijn]

This would be sufficient for me

[chkoar]

Well, having this I believe is sufficient.