[joss-reviews#2974] Many component combinations not possible?

[sjvrijn]

Related: openjournals/joss-reviews#2974

As a quick experiment, I tried to run all possible combinations of components, but many fail due to code incompatibliities.

Looking at the documentation, it is not immediately clear if some components cannot work together with others. For example, for moead_framework.core.sps_strategy.sps_dra.SpsDra, the description is 'The strategy used in MOEA/D-DRA', but does that mean it exclusively works with that algorithm?

Can you explain to what extent it is the goal of your framework to enable all such combinations?

Code used to test combinations:

from moead_framework.aggregation import Tchebycheff, WeightedSum
aggregations = [Tchebycheff, WeightedSum]

from moead_framework.core.genetic_operator.combinatorial import BinaryMutation, Crossover, CrossoverAndMutation
genetic_operators = [CrossoverAndMutation, BinaryMutation, Crossover]

from moead_framework.core.parent_selector import TwoRandomParentSelector, TwoRandomAndCurrentParentSelector, OneRandomAndCurrentParentSelector
parent_selectors = [TwoRandomParentSelector, TwoRandomAndCurrentParentSelector, OneRandomAndCurrentParentSelector]

from moead_framework.core.selector import DeltaSelector, ClosestNeighborsSelector
selectors = [ClosestNeighborsSelector, DeltaSelector]

from moead_framework.core.sps_strategy import SpsAllSubproblems, SpsDra, SpsRandomAndBoundaries
sps_strategies = [SpsAllSubproblems, SpsDra, SpsRandomAndBoundaries]

from moead_framework.algorithm.combinatorial import Moead, MoeadDeltaNr, MoeadSPSRandom, MoeadDRA
comb_algorithms = [Moead] #, MoeadDRA, MoeadDeltaNr, MoeadSPSRandom]

from moead_framework.problem.combinatorial import Rmnk, Mubqp, KnapsackProblem
rmnk = Rmnk(instance_file="moead_framework/test/data/instances/rmnk_0_2_100_1_0.dat")
comb_problems = [rmnk]

number_of_weight = 10
number_of_weight_neighborhood = 2
number_of_evaluations = 100
weight_file = "moead_framework/test/data/weights/SOBOL-" + str(rmnk.number_of_objective) + "objs-" + str(number_of_weight) + "wei.ws"

from itertools import product
all_combinations = list(product(
    comb_algorithms,
    comb_problems,
    aggregations,
    genetic_operators,
    selectors,
    parent_selectors,
    sps_strategies
))

num_success = 0
for i, (algorithm, problem, aggregation, genetic_operator, selector, parent_selector, sps_strategy) in enumerate(all_combinations):
    try:
        moead = algorithm(
            problem=problem,
            max_evaluation=number_of_evaluations,
            number_of_weight_neighborhood=number_of_weight_neighborhood,
            weight_file=weight_file,
            aggregation_function=aggregation,
            genetic_operator=genetic_operator,
            parent_selector=parent_selector,
            sps_strategy=sps_strategy,
            mating_pool_selector=selector,
        )
        moead.run()
    except Exception as e:
        print(f'{i:03} / {len(all_combinations)} failed: {e}')
        continue

    # print(f'{i} / {len(all_combinations)} succeeded')
    num_success += 1

print(f'\n\n{num_success} / {len(all_combinations)} succeeded')

[geoffreyp]

All components are not compatible with each algorithm because they have some requirements.
For example :

• moead_framework.core.sps_strategy.sps_dra.SpsDra needs the attribute pi
• SpsRandomAndBoundaries needs the attribute number_of_subproblem
• The multi-point Crossover needs 2 solutions as parents so we have to use a mating selector that give us 2 parents
• ...

I added more information in the documentation and I improved errors to say how the component ... is not compatible with the current algorithm. PR #66

[sjvrijn]

Thanks for elaborating that not all components are expected to work together. Here are my suggestions to address this issue:

• Fix numerical and combinatorial Moead
  Lines 68 in combinatorial and 65 in numerical Moead still need to initialize the selector with parent_selector(algorithm=self)

  The if-else block may even be simplified to the example below for the combinatorial, and equivalently for the numerical case:

if parent_selector is None:
    parent_selector = TwoRandomParentSelector
self.parent_selector = parent_selector(algorithm=self)

• Documentation:
  In the global documentation, I would add a clear generic note stating something like

  Note: Not all available components are compatible with each other.
  Refer to each component's documentation for more details on required attributes.

  at the Tutorials / Implement own algorithm and Library Reference / Other components pages

• Exceptions:
  I'll post my explicit suggestions in the PR you already opened

[geoffreyp]

Thank you for your suggestions !
The PR is updated with the commits 233c694 and de0285d