information displayed in the optimization process

[vanquanTRAN]

Thank you for your code sharing and your extraordinary development. I try to modify your code to show each iteration versus best score, that help us to show a graph n_iter vs best score.
print('Iteration {}: Best Cost = {}'.format(best_iter, best_score=)).
But i cant not succeed,
Could you help me to handle this issue ? Thank you for your help.

[SimonBlanke]

Hello @vanquanTRAN,

could you describe your problem in more detail?

Do you want to print this information in each iteration?
Do you really need the information at runtime? You could just get the search data after the run finished via hyper.results(model).

The more information you give about your problem and what you want to do, the better I can help you.

[vanquanTRAN]

Hello Simon, thank you for your enthusiasm, in fact, your code can help us to verify the best score over each iteration.
But perhaps due to my mistake or the optimization algorithms are coded in different ways because should the best score such as R2

of iteration (n+1) th be greater or equal (at least) than that of n th, for example in the attached fig for PSO ? Could you explain?
Iteration 0: Best Cost = 0.7615630337611912
Iteration 1: Best Cost = 0.761734744272929
Iteration 2: Best Cost = 0.7665962344898513
Iteration 3: Best Cost = 0.7665962344898513
Iteration 4: Best Cost = 0.7665962344898513
Iteration 5: Best Cost = 0.7665962344898513
Iteration 6: Best Cost = 0.7665962344898513
Iteration 7: Best Cost = 0.7665962344898513
Iteration 8: Best Cost =0.7665962344898513
Iteration 9: Best Cost = 0.7665962344898513
Iteration 10: Best Cost = 0.7667057693160213
Iteration 11: Best Cost = 0.7670653200716335

[SimonBlanke]

Hello @vanquanTRAN,

thank you for going into so much detail! I am currently working on a new feature that should solve this problem.
I will give you an update (probably tomorrow) on how you can output this data during the optimization run.

[SimonBlanke]

Hello @vanquanTRAN,

I looked into a solution for your problem. I had to do some small additions to the optimization backend, so you should update it via: pip install gradient_free_optimizers==0.3.2

After the update you can run the example code below:

from sklearn.datasets import load_boston
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.model_selection import cross_val_score

from hyperactive import Hyperactive


data = load_boston()
X, y = data.data, data.target


def model(para):
    gbr = GradientBoostingRegressor(
        n_estimators=para["n_estimators"],
        max_depth=para["max_depth"],
        min_samples_split=para["min_samples_split"],
    )
    scores = cross_val_score(gbr, X, y, cv=3)

    print(
        "Iteration:", para.optimizer.nth_iter, " Best score", para.optimizer.best_score
    )

    return scores.mean()


search_space = {
    "n_estimators": list(range(10, 150, 5)),
    "max_depth": list(range(2, 12)),
    "min_samples_split": list(range(2, 22)),
}


hyper = Hyperactive()
hyper.add_search(model, search_space, n_iter=20)
hyper.run()

Please let me know if this solution works for you.

[vanquanTRAN]

Thank you Simon, your solution is well done for me now, your work will be cited in my paper if it is published
Thank you

[SimonBlanke]

Very good @vanquanTRAN! I would appreciate a citation in your paper.

I will leave this issue open for now, because I have an additional new feature, that will be released within the next week.

[SimonBlanke]

Since v3.2.0 a streamlit-based dashboard for the visualization of the search data (automatically) collected during the optimization run has been added: Example
This should give enough flexibility to display information during the optimization progress.