BUG: scipy.optimize._differentialevolution incorrectly converted np.inf values to self.bounds when using scipy.optimized.Bounds

[FloWolfStatworx]

Describe your issue.

Hello scipy team,

thank you for providing such an amazing package. While working with your differential evolution algorithm, I experienced a conversion problem with the optimization variable bounds when working with np.inf values together with scipy.optimize.Bounds. In line 680 of the file scipy.optimized._differentialevolution we have

self.limits = np.array(new_bounds_to_old(bounds.lb,
                                         bounds.ub,
                                         len(bounds.lb)),
                       dtype=float).T

which results in nan-values in self.limits, caused by line 400-401 in scipy.optimize._bounds

lb = [float(x) if x > -np.inf else None for x in lb]
ub = [float(x) if x < np.inf else None for x in ub]

One fix would probably be to substitute the lines above by

lb = [float(x) if x > -np.inf else -np.inf for x in lb]
ub = [float(x) if x < np.inf else np.inf for x in ub]

Thank you very much in advance :)

Reproducing Code Example

import numpy as np
from scipy.optimize import Bounds, differential_evolution

bounds = Bounds(lb=np.array([0, -np.inf]), ub=np.array([np.inf, 0]))

func = lambda x,y: x**2 + y **2

res = differential_evolution(
    func = func,
    bounds = bounds
)

Error message

Traceback (most recent call last):
  File "secret/scipy_bug.py", line 8, in <module>
    res = differential_evolution(
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "secret/.venv/lib/python3.11/site-packages/scipy/optimize/_differentialevolution.py", line 377, in differential_evolution
    with DifferentialEvolutionSolver(func, bounds, args=args,
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "secret/.venv/lib/python3.11/site-packages/scipy/optimize/_differentialevolution.py", line 689, in __init__
    raise ValueError('bounds should be a sequence containing '
ValueError: bounds should be a sequence containing real valued (min, max) pairs for each value in x

SciPy/NumPy/Python version and system information

1.9.3 1.24.2 sys.version_info(major=3, minor=11, micro=2, releaselevel='final', serial=0)
openblas64__info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None)]
    runtime_library_dirs = ['/usr/local/lib']
blas_ilp64_opt_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None)]
    runtime_library_dirs = ['/usr/local/lib']
openblas64__lapack_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)]
    runtime_library_dirs = ['/usr/local/lib']
lapack_ilp64_opt_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)]
    runtime_library_dirs = ['/usr/local/lib']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2
    not found = AVX512F,AVX512CD,AVX512_KNL,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL

[mdhaber]

Thanks for reporting. That might work, although there are some optimizers that work with None instead of an inf in their bounds. That's probably why the line was written the way it is, and DE only started using the code later. This should be easy to fix in any case, but that's something to keep in mind.

[andyfaff]

As @mdhaber says None is used by several other minimisers in optimize to signify no bounds. One example is L-BFGS-B.

If the stacktrace your interpreter provided was more expansive the context would be revealed:

    701 if (np.size(self.limits, 0) != 2 or not
    702         np.all(np.isfinite(self.limits))):
--> 703     raise ValueError('bounds should be a sequence containing '
    704                      'real valued (min, max) pairs for each value'
    705                      ' in x')

The way differential_evolution works is by generating random numbers between the lower and upper bounds (remember no gradients here). So we actually want the lower and upper bounds to be finite.

Perhaps adding the word finite to the error message would be a good change: ... containing finite real valued ... .

It's been suggested before that if np.inf is observed in a bounds then replace it by np.finfo(float).max (similar argument for lower limit), but that's not really a good solution. Firstly, it'll cause float arithmetic to overflow in a lot of cases. Secondly, because guesses are generated within those bounds via random number generation, if the objective function is in the least bit noisy there is no chance that a minimum would be found.

I don't think changes to the limits processing makes sense here. Are there other changes you can suggest that would improve usability here @FloWolfStatworx?

[mdhaber]

I don't think changes to the limits processing makes sense here

Oops. Yeah, clearly DE can't have infinite bounds. (I wasn't thinking about DE in particular very carefully this morning.) I agree that all that is needed is the addition of the word "finite". I'll submit a PR.