[Bug] Assertion Error with Multi-Objective

[MazurekP]

Description

I wanted to add a multi-objective optimization to to already working code and got
assert len(configs) == len(config_instance_seed_budget_keys)
AssertionError

Steps/Code to Reproduce

https://gist.github.com/MazurekP/409b450d19965763e57b85af4ddcd732

Expected Results

normal optimization process

Actual Results

Traceback (most recent call last):
File "/home/pmazurek/NA61-SHINE-filtration-ML-model/SMAC_approach/SMAC_with_manual_instances/back_to_basics/Multi-objective/SMAC_Forest_multi_objective_single_dataset.py", line 220, in
incumbent = smac.optimize()
File "/home/pmazurek/miniconda3/envs/SHINE/lib/python3.9/site-packages/smac/facade/abstract_facade.py", line 289, in optimize
incumbents = self._optimizer.optimize()
File "/home/pmazurek/miniconda3/envs/SHINE/lib/python3.9/site-packages/smac/main/smbo.py", line 279, in optimize
trial_info = self.ask()
File "/home/pmazurek/miniconda3/envs/SHINE/lib/python3.9/site-packages/smac/main/smbo.py", line 151, in ask
trial_info = next(self._trial_generator)
File "/home/pmazurek/miniconda3/envs/SHINE/lib/python3.9/site-packages/smac/intensifier/successive_halving.py", line 391, in iter
successful_configs = self._get_best_configs(configs, bracket, stage, isb_keys)
File "/home/pmazurek/miniconda3/envs/SHINE/lib/python3.9/site-packages/smac/intensifier/successive_halving.py", line 545, in _get_best_configs
selected_configs = sort_by_crowding_distance(rh, configs, all_keys)[:n_configs]
File "/home/pmazurek/miniconda3/envs/SHINE/lib/python3.9/site-packages/smac/utils/pareto_front.py", line 108, in sort_by_crowding_distance
F = _get_costs(runhistory, configs, config_instance_seed_budget_keys)
File "/home/pmazurek/miniconda3/envs/SHINE/lib/python3.9/site-packages/smac/utils/pareto_front.py", line 31, in _get_costs
assert len(configs) == len(config_instance_seed_budget_keys)
AssertionError

Versions

AttributeError: module 'smac' has no attribute '__version__' :|

here is pip show result:
Name: smac
Version: 2.0.0

[dengdifan]

Hi,
thanks for the report, this is indeed a bug.
The problem is due to the combination of multi-fidelity and multi-objective optimization. This part is currently not well designed and we need to look deeper into that and fix this soonish. Before that, maybe you could use HPOFacade instead of MFFacade as a temporary solution.

[MazurekP]

Thank You very much the workaround works and even isn't much worse than multi fidelity facade