'str object has no attribute reproduce'

[Alistair-Turcan]

Describe the bug
When running the optimize step, I get a strange error which keeps reproducing after the 520th iteration. Sometimes it can work fine when I lower the starting points/iterations. I pasted it below.

Any help would be appreciated. Thanks for making a great package! Seems to work much faster than other optimizers.

To Reproduce

Unfortunately the code loads in a lot of data and has much preprocessing and I don't think it would be straightforward to reproduce, hopefully this is enough info.

from bayes_opt import BayesianOptimization

def optimization_wrapper_bayes():

    bins = 2
    bn=2**7
    precomputed_values = precompute_values_pairTrait(betXY, pi1, sig1, w8s, bn, bins)

    # Define the objective function for Bayesian Optimization
    def objective(h2X, h2Y, tX, tY, axy, ayx, iXY):
        theta = [h2X, h2Y, tX, tY, axy, ayx, iXY]
        return pairTrait_twoStep_likelihood(theta, betXY, pi1, sig1, w8s, m0, precomputed_values,
                                                 pi_U=piU, pi_X=piX, pi_Y=piY,
                                                 i_X=iX, i_Y=iY, nX=nX, nY=nY,
                                                 model="comp", bn = bn, bins=bins, M=1e7)

    # Getting bounds
    pbounds = {
        'h2X': (0.05, 0.5), #(par_df['sp_h2X'].min(), par_df['sp_h2X'].max()),
        'h2Y': (0.05, 0.5), #(par_df['sp_h2Y'].min(), par_df['sp_h2Y'].max()),
        'tX': (0.001, 0.1), #(par_df['sp_tX'].min(), par_df['sp_tX'].max()),
        'tY': (-0.1, 0.1), #(par_df['sp_tY'].min(), par_df['sp_tY'].max()),
        'axy': (-0.1, 0.3), #(par_df['sp_axy'].min(), par_df['sp_axy'].max()),
        'ayx': (-0.1, 0.1), #(par_df['sp_ayx'].min(), par_df['sp_ayx'].max()),
        'iXY': (-0.1, 0.1), #(par_df['sp_iXY'].min(), par_df['sp_iXY'].max()),
    }

    optimizer = BayesianOptimization(
        f=objective,
        pbounds=pbounds,
        random_state=1,
        verbose=2,
    )
    #optimizer.set_gp_params(alpha=1e-3, n_restarts_optimizer=5)
    
    optimizer.maximize(
        init_points=500,
        n_iter=50
    )
    
    return optimizer.max

**Error**
---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
~/.local/lib/python3.7/site-packages/bayes_opt/bayesian_optimization.py in maximize(self, init_points, n_iter, acquisition_function, acq, kappa, kappa_decay, kappa_decay_delay, xi, **gp_params)
    304             try:
--> 305                 x_probe = next(self._queue)
    306             except StopIteration:

~/.local/lib/python3.7/site-packages/bayes_opt/bayesian_optimization.py in __next__(self)
     26         if self.empty:
---> 27             raise StopIteration("Queue is empty, no more objects to retrieve.")
     28         obj = self._queue[0]

StopIteration: Queue is empty, no more objects to retrieve.

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-91-f7dc02174309> in <module>
      3 
      4 # Call the function with the first element of par_list
----> 5 result = optimization_wrapper_bayes()
      6 
      7 # Measure end time

<ipython-input-90-60a130b91fb1> in optimization_wrapper_bayes()
     36     optimizer.maximize(
     37         init_points=500,
---> 38         n_iter=50
     39     )
     40 

~/.local/lib/python3.7/site-packages/bayes_opt/bayesian_optimization.py in maximize(self, init_points, n_iter, acquisition_function, acq, kappa, kappa_decay, kappa_decay_delay, xi, **gp_params)
    306             except StopIteration:
    307                 util.update_params()
--> 308                 x_probe = self.suggest(util)
    309                 iteration += 1
    310             self.probe(x_probe, lazy=False)

~/.local/lib/python3.7/site-packages/bayes_opt/bayesian_optimization.py in suggest(self, utility_function)
    218         with warnings.catch_warnings():
    219             warnings.simplefilter("ignore")
--> 220             self._gp.fit(self._space.params, self._space.target)
    221             if self.is_constrained:
    222                 self.constraint.fit(self._space.params,

/opt/anaconda3-cluster/lib/python3.7/site-packages/sklearn/gaussian_process/_gpr.py in fit(self, X, y)
    247                     optima.append(
    248                         self._constrained_optimization(obj_func, theta_initial,
--> 249                                                        bounds))
    250             # Select result from run with minimal (negative) log-marginal
    251             # likelihood

/opt/anaconda3-cluster/lib/python3.7/site-packages/sklearn/gaussian_process/_gpr.py in _constrained_optimization(self, obj_func, initial_theta, bounds)
    502                 obj_func, initial_theta, method="L-BFGS-B", jac=True,
    503                 bounds=bounds)
--> 504             _check_optimize_result("lbfgs", opt_res)
    505             theta_opt, func_min = opt_res.x, opt_res.fun
    506         elif callable(self.optimizer):

/opt/anaconda3-cluster/lib/python3.7/site-packages/sklearn/utils/optimize.py in _check_optimize_result(solver, result, max_iter, extra_warning_msg)
    241                 "    [https://scikit-learn.org/stable/modules/"](https://scikit-learn.org/stable/modules/%22)
    242                 "preprocessing.html"
--> 243             ).format(solver, result.status, result.message.decode("latin1"))
    244             if extra_warning_msg is not None:
    245                 warning_msg += "\n" + extra_warning_msg

AttributeError: 'str' object has no attribute 'decode'

[till-m]

Hey @Alistair-Turcan,

I think this specific, strange error is caused by a scipy/sklearn version mismatch. See the line in newer sklearn. However, I expect you would generally run into problems here and the reason why you only start seeing it so late in the process, is because the GP fitting stops converging -- I would assume because there's too many points. Do you really need that many init_points?

[Alistair-Turcan]

Hi, @till-m , thanks for the quick reply!

I tried downgrading scipy and sklearn, and upgrading this package, same issue. Performance seemed to increase when I increased the starting points, but maybe it doesn't need to be so high. Do you have any intuition of how many starting points I can specify before this error occurs?

Thanks again!

[till-m]

Hi @Alistair-Turcan,

your code is not enough to reproduce the issue, but I adapted it. For me this runs without a problem -- could you check if this is also true for you?
Environment:

scikit-learn==1.3.1
scipy==1.10.0
bayesian-optimization==1.4.3

from bayes_opt import BayesianOptimization
import numpy as np

def optimization_wrapper_bayes():
    # Define the objective function for Bayesian Optimization
    def objective(h2X, h2Y, tX, tY, axy, ayx, iXY):
        return np.random.rand()

    # Getting bounds
    pbounds = {
        'h2X': (0.05, 0.5), #(par_df['sp_h2X'].min(), par_df['sp_h2X'].max()),
        'h2Y': (0.05, 0.5), #(par_df['sp_h2Y'].min(), par_df['sp_h2Y'].max()),
        'tX': (0.001, 0.1), #(par_df['sp_tX'].min(), par_df['sp_tX'].max()),
        'tY': (-0.1, 0.1), #(par_df['sp_tY'].min(), par_df['sp_tY'].max()),
        'axy': (-0.1, 0.3), #(par_df['sp_axy'].min(), par_df['sp_axy'].max()),
        'ayx': (-0.1, 0.1), #(par_df['sp_ayx'].min(), par_df['sp_ayx'].max()),
        'iXY': (-0.1, 0.1), #(par_df['sp_iXY'].min(), par_df['sp_iXY'].max()),
    }

    optimizer = BayesianOptimization(
        f=objective,
        pbounds=pbounds,
        random_state=1,
        verbose=2,
    )
    #optimizer.set_gp_params(alpha=1e-3, n_restarts_optimizer=5)
    
    optimizer.maximize(
        init_points=500,
        n_iter=50
    )
    
    return optimizer.max

print(optimization_wrapper_bayes())

[Alistair-Turcan]

I no longer seem to have the issue. Thanks so much! Works with any amount of starting points/iterations I specify. scipy's minimize function actually had the same issue when I specified enough iterations, so I guess it was just some package thing.

[till-m]

Happy to hear, let us know if you run into any other issues :)