Issue when calling gp.predictive_gradients()

[blaahhrrgg]

I am running Python v2.7.12 using Anaconda 64bit on Windows 7. I installed GPy using 'pip install GPy' to get version 1.5.6 which I have been using for a while on small problems without any issues. More, recently I have wanted to make use of the predictive_gradients() function but I have run into some issues.

Firstly, when I import GPy I get the following warning: "warning in stationary: failed to import cython module: falling back to numpy" which I have not thought too much of in the past since everything else seemed to work okay. Secondly, when I try and run the following code,

import numpy as np
import GPy as gp

k = gp.kern.RBF(input_dim=1, variance=1., lengthscale=1.)

X = np.random.uniform(-3.,3., (20,1))
Y = np.sin(X) + np.random.randn(20,1)*0.05

m = gp.models.GPRegression(X, Y, kernel=k)

grad = m.predictive_gradients(X)

I get the following error,

NameError                                 Traceback (most recent call last)
<ipython-input-10-964a0d158030> in <module>()
----> 1 m.predictive_gradients(X)

C:\Program Files\Anaconda2\lib\site-packages\GPy\core\gp.pyc in predictive_gradi
ents(self, Xnew, kern)
    335
    336         for i in range(self.output_dim):
--> 337             mean_jac[:,:,i] = kern.gradients_X(self.posterior.woodbury_v
ector[:,i:i+1].T, Xnew, self._predictive_variable)
    338
    339         # gradients wrt the diagonal part k_{xx}

C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_operation
s.pyc in wrap(self, dL_dK, X, X2)
    116     def wrap(self, dL_dK, X, X2=None):
    117         with _Slice_wrap(self, X, X2) as s:
--> 118             ret = s.handle_return_array(f(self, dL_dK, s.X, s.X2))
    119         return ret
    120     return wrap

C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.pyc in grad
ients_X(self, dL_dK, X, X2)
    234         """
    235         if config.getboolean('cython', 'working'):
--> 236             return self._gradients_X_cython(dL_dK, X, X2)
    237         else:
    238             return self._gradients_X_pure(dL_dK, X, X2)

C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.pyc in _gra
dients_X_cython(self, dL_dK, X, X2)
    321         X, X2 = np.ascontiguousarray(X), np.ascontiguousarray(X2)
    322         grad = np.zeros(X.shape)
--> 323         stationary_cython.grad_X(X.shape[0], X.shape[1], X2.shape[0], X,
 X2, tmp, grad)
    324         return grad/self.lengthscale**2
    325

NameError: global name 'stationary_cython' is not defined

If I use a linear kernel I do not get the above error.

I wondered if this was related to the documented issues on using cython code with anaconda on Windows 64bit due to bundling with mingw64 (see: ContinuumIO/anaconda-issues#175) but I am now using TDM-GCC and the issue persists.

Has this been documented before? Is there anything I have missed that I can try to get this to work?

Many thanks for your help.

[mzwiessele]

Hmm this seems odd, indeed. If the cython module is missing, it should never get into that part of the if statement. As you can see above in the same file, it should write into the config, that cython is not available. as a sanity check could you try putting

[cython]
working = False

in your %userprofile%\.config\GPy\user.cfg file and rerun the script?

I am more bamboozled about the fact, that cython is not working for you. We build the cython modules for windows and upload them to pypi, so it should install on windows without problems. Have you installed the cython module using anaconda?

[blaahhrrgg]

Hi Max,

I actually re-installed Anaconda this morning to sanity check a few things, I don't recall explicitly installing cython so I'm not sure whether it was already there or pip took care of it. Nevertheless, I currently have cython v0.25.1 installed according to conda.

With the above flag in the user.cfg file I get the same errors,

warning in stationary: failed to import cython module: falling back to numpy
Traceback (most recent call last):
  File "H:\Projects\test\test\test.py", line 11, in <module>
    grad = m.predictive_gradients(X)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\core\gp.py", line 337,
in predictive_gradients
    mean_jac[:,:,i] = kern.gradients_X(self.posterior.woodbury_vector[:,i:i+1].T
, Xnew, self._predictive_variable)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_o
perations.py", line 118, in wrap
    ret = s.handle_return_array(f(self, dL_dK, s.X, s.X2))
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.py"
, line 236, in gradients_X
    return self._gradients_X_cython(dL_dK, X, X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.py"
, line 323, in _gradients_X_cython
    stationary_cython.grad_X(X.shape[0], X.shape[1], X2.shape[0], X, X2, tmp, gr
ad)
NameError: global name 'stationary_cython' is not defined

[mzwiessele]

Does the following report False?

from GPy.util.config import config
print(config.getboolean('cython', 'working'))

[blaahhrrgg]

It reports True.

[mzwiessele]

can you import

from GPy.kern.src import stationary_cython

?

[blaahhrrgg]

No, I get the following error:

ImportError: DLL load failed: The application has failed to start because its si
de-by-side configuration is incorrect. Please see the application event log or u
se the command-line sxstrace.exe tool for more detail.

[mzwiessele]

So the config reports True, although you have given cython working as false? This is getting more odd by the minute. Are you able to import cython?

On another note, the gradient computation for the ARD parameters makes use of cython, too. What happens if you use an ARD kernel:

k = gp.kern.RBF(input_dim=1, variance=1., lengthscale=1., ARD=True)

and run a normal GP as you described above, without the predictive gradients?

[blaahhrrgg]

Hmmm, I can import cython with no errors. To double check I ran the cython hello world example here. When I first ran this tutorial there was an "Unable to find vcvarsall.bat" error which I have now resolved.

Without the predictive gradients, i.e., running the following code:

import numpy as np
import GPy as gp

k = gp.kern.RBF(input_dim=1, variance=1., lengthscale=1.)

X = np.random.uniform(-3.,3., (20,1))
Y = np.sin(X) + np.random.randn(20,1)*0.05

m = gp.models.GPRegression(X, Y, kernel=k)
print(m.predict(X))

I get the following output:

warning in stationary: failed to import cython module: falling back to numpy
(array([[-0.59422249],
       [ 0.37418019],
       [-0.55712291],
       [-0.23910571],
       [ 0.1361988 ]]), array([[ 1.34195044],
       [ 1.48095964],
       [ 1.35368817],
       [ 1.46978674],
       [ 1.49680015]]))

If I usethe RBF kernel with ARD=True, then the code fails again with a similar output to before:

warning in stationary: failed to import cython module: falling back to numpy
Traceback (most recent call last):
  File "test.py", line 9, in <module>
    m = gp.models.GPRegression(X, Y, kernel=k)
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\parameterized.py", l
ine 54, in __call__
    self.initialize_parameter()
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\parameter_core.
py", line 331, in initialize_parameter
    self.trigger_update()
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\updateable.py",
 line 79, in trigger_update
    self._trigger_params_changed(trigger_parent)
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\parameter_core.
py", line 128, in _trigger_params_changed
    self.notify_observers(None, None if trigger_parent else -np.inf)
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\observable.py",
 line 91, in notify_observers
    [callble(self, which=which) for _, _, callble in self.observers]
  File "C:\Program Files\Anaconda2\lib\site-packages\paramz\core\parameter_core.
py", line 498, in _parameters_changed_notification
    self.parameters_changed()
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\core\gp.py", line 195,
in parameters_changed
    self.kern.update_gradients_full(self.grad_dict['dL_dK'], self.X)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_o
perations.py", line 102, in wrap
    ret = f(self, dL_dK, s.X, s.X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\rbf.py", line
107, in update_gradients_full
    super(RBF,self).update_gradients_full(dL_dK, X, X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_o
perations.py", line 102, in wrap
    ret = f(self, dL_dK, s.X, s.X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.py"
, line 193, in update_gradients_full
    self.lengthscale.gradient = self._lengthscale_grads_cython(tmp, X, X2)
  File "C:\Program Files\Anaconda2\lib\site-packages\GPy\kern\src\stationary.py"
, line 228, in _lengthscale_grads_cython
    stationary_cython.lengthscale_grads(N, M, Q, tmp, X, X2, grads)
NameError: global name 'stationary_cython' is not defined

[mzwiessele]

So you said you are using your own compiler. Can you download the source and run the setup code from the source directory with your compiler directly? That should compile the resources directly and make everything work.

You might have to look up the compilation options for the setup script (i only know the two options --compiler=msvc or mingw, not sure about tdm).

I think we have brought it down to the compilation, though. It might be windows 7, which is the most likely to me, but the compiled version seems to not work for you :(

[blaahhrrgg]

Hmmm, I've tried to install GPy from source. I uninstalled GPy using pip, then downloaded the source and ran 'python setup.py install' from the source directory with distutils.cfg pointing to msvc (I have that installed as well) and still ended up with the same errors when running the above scripts. Am I missing anything here?

Thanks for your help so far.

[mzwiessele]

It seems the compilation is not working. You should be able to still use the numpy code by providing cython working = false in the config. Unfortunately I do not have Windows and cannot go deeper, I am afraid :/

On 22 Nov 2016, at 17:22, JM notifications@github.com wrote:

Hmmm, I've tried to install GPy from source. I uninstalled GPy using pip, then downloaded the source and ran 'python setup.py install' from the source directory with distutils.cfg pointing to msvc (I have that installed as well) and still ended up with the same errors when running the above scripts. Am I missing anything here?

Thanks for your help so far.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[blaahhrrgg]

I provided cython working = False but that didn't help. This morning I uninstalled 64bit Anaconda and installed 32bit Anaconda and everything now works fine. It must be a 64bit compiler issue as I mentioned in the first post. I am not sure whether I am being naive or whether there is a bug with respect to the cython working = False as that seems to not have the desired effect on my machine. Either way, I now have a work around and at least this issue Anaconda 64bit on windows is documented here in case anyone else comes across it. Thanks for your time and help with this.

[mzwiessele]

On the contrary, thanks for reporting and using GPy!

On 23 Nov 2016, at 09:54, JM notifications@github.com wrote:

I provided cython working = False but that didn't help. This morning I uninstalled 64bit Anaconda and installed 32bit Anaconda and everything now works fine. It must be a 64bit compiler issue as I mentioned in the first post. I am not sure whether I am being naive or whether there is a bug with respect to the cython working = False as that seems to not have the desired effect on my machine. Either way, I now have a work around and at least this issue Anaconda 64bit on windows is documented here in case anyone else comes across it. Thanks for your time and help with this.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[fedorzh]

I'm still getting the same error. Also, I don't want to change everything to use 32-bit version.

@blaahhrrgg The reason why cython is still True is because during import process,
config.set('cython', 'working', 'True') is done in linalg.py and other files.

I removed all these lines, and was able to run your commands without errors

[AliBaheri]

Hi!
I am running a python 2.7 code (containing GPy and GPyOpt) from Matlab on Anaconda 64bit on Windows 10 and I am facing with the following error:

warning in stationary: failed to import cython module: falling back to numpy Error using stationary>_gradients_X_cython (line 323) Python Error: NameError: global name 'stationary_cython' is not defined

When I run the code in python I do not have any problem, but the problem comes when I call the script from MATLAB (I have run the code from MATLAB a few months ago without any problem.)

I have to mention, recently for some reason, I have downgraded the numpy to Numpy=1.11.0.
The reason was Matlab has a bug with the latest version of numpy.

Also I am facing with the following window:
An application has made an attempt to load the C runtime library ...

Could you please help me to resolve the issue?

Python Error: NameError: global name 'stationary_cython' is not defined

Error in stationary>gradients_X (line 236)
            return self._gradients_X_cython(dL_dK, X, X2)

Error in kernel_slice_operations>wrap (line 118)
            ret = s.handle_return_array(f(self, dL_dK, s.X, s.X2))

Error in prod>gradients_X (line 80)
            target += self.parts[0].gradients_X(dL_dK*self.parts[1].K(X, X2), X, X2)

Error in kernel_slice_operations>wrap (line 118)
            ret = s.handle_return_array(f(self, dL_dK, s.X, s.X2))

Error in gp>predictive_gradients (line 337)
            mean_jac[:,:,i] = kern.gradients_X(self.posterior.woodbury_vector[:,i:i+1].T, Xnew, self._predictive_variable)

Error in gpmodel>predict_withGradients (line 113)
        dmdx, dvdx = self.model.predictive_gradients(X)

Error in EI>_compute_acq_withGradients (line 47)
        m, s, dmdx, dsdx = self.model.predict_withGradients(x)

Error in base>acquisition_function_withGradients (line 46)
        f_acqu,df_acqu = self._compute_acq_withGradients(x)

Error in LP>d_acquisition_function (line 128)
            _, grad_acq_x = self.acq.acquisition_function_withGradients(x)

Error in LP>acquisition_function_withGradients (line 139)
        aqu_x_grad = self.d_acquisition_function(x)

Error in acquisition_optimizer>fp_dfp (line 165)
            fp_xx , dfp_xx = f_df(xx)

Error in optimizer>_f_df (line 60)
                return f(x), f_df(x)[1][0]

Error in optimize>__call__ (line 63)
        fg = self.fun(x, *args)

Error in optimize>function_wrapper (line 289)
        return function(*(wrapper_args + args))

Error in lbfgsb>func_and_grad (line 278)
            f = fun(x, *args)

Error in lbfgsb>_minimize_lbfgsb (line 330)
                f, g = func_and_grad(x)

Error in lbfgsb>fmin_l_bfgs_b (line 193)
                           **opts)

Error in optimizer>optimize (line 64)
            res = scipy.optimize.fmin_l_bfgs_b(_f_df, x0=x0, bounds=self.space.get_bounds(), maxiter=self.maxiter)

Error in acquisition_optimizer>optimize (line 177)
                x_min, f_min = self.optimizer.optimize(x0, f =fp, df=None, f_df=fp_dfp)

Error in base>optimize (line 59)
            out = self.optimizer.optimize(f=self.acquisition_function, f_df=self.acquisition_function_withGradients)[0]

Error in batch_local_penalization>compute_batch (line 34)
        X_batch = self.acquisition.optimize()

Error in bo>_compute_next_evaluations (line 186)
        return self.evaluator.compute_batch()

Error in bo>run_optimization (line 108)
            self.suggested_sample = self._compute_next_evaluations()

Error in bayesian_optimization>run_optimization (line 458)
        super(BayesianOptimization, self).run_optimization(max_iter = max_iter, max_time = max_time,  eps = eps,
        verbosity=verbosity, save_models_parameters = save_models_parameters, report_file = report_file, evaluations_file=
        evaluations_file, models_file=models_file)

Error in bayesian_optimization>__init__ (line 244)
        self.run_optimization(max_iter=0,verbosity=self.verbosity)

Error in BatchBO>BAYESOPT2 (line 37)
                                                 acquisition_weight = 2)

[debipe20]

I am facing the following issue. I am using python 2.7 and anaconda 32bit.

NameErrorTraceback (most recent call last)
in ()
----> 1 myProblem = GPyOpt.methods.BayesianOptimization(myf,bounds)

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\methods\bayesian_optimization.pyc in init(self, f, domain, constrains, cost_withGradients, model_type, X, Y, initial_design_numdata, initial_design_type, acquisition_type, normalize_Y, exact_feval, acquisition_optimizer_type, model_update_interval, evaluator_type, batch_size, num_cores, verbosity, verbosity_model, bounds, maximize, **kwargs)
242
243 # --- Initialize everything
--> 244 self.run_optimization(max_iter=0,verbosity=self.verbosity)
245
246 def _model_chooser(self):

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\methods\bayesian_optimization.pyc in run_optimization(self, max_iter, max_time, eps, verbosity, save_models_parameters, report_file, evaluations_file, models_file, **kwargs)
456 self.acquisition_optimizer.optimizer ='CMA'
457 print('WARNING: "acqu_optimize_method" will be deprecated in the next version!')
--> 458 super(BayesianOptimization, self).run_optimization(max_iter = max_iter, max_time = max_time, eps = eps, verbosity=verbosity, save_models_parameters = save_models_parameters, report_file = report_file, evaluations_file= evaluations_file, models_file=models_file)
459
460 def _sign(self,f):

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\core\bo.pyc in run_optimization(self, max_iter, max_time, eps, verbosity, save_models_parameters, report_file, evaluations_file, models_file)
106
107 # --- Update and optimize acquisition and compute the exploration level in the next evaluation
--> 108 self.suggested_sample = self._compute_next_evaluations()
109
110 if not ((self.num_acquisitions < self.max_iter) and (self._distance_last_evaluations() > self.eps)):

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\core\bo.pyc in _compute_next_evaluations(self)
184 Computes the location of the new evaluation (optimizes the acquisition in the standard case).
185 """
--> 186 return self.evaluator.compute_batch()
187
188 def _update_model(self):

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\core\evaluators\sequential.pyc in compute_batch(self)
19 Selects the new location to evaluate the objective.
20 """
---> 21 return self.acquisition.optimize()
22
23

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\acquisitions\base.pyc in optimize(self)
57 out = self.optimizer.optimize(f=self.acquisition_function)[0]
58 else:
---> 59 out = self.optimizer.optimize(f=self.acquisition_function, f_df=self.acquisition_function_withGradients)[0]
60 return out
61

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\optimization\acquisition_optimizer.pyc in optimize(self, f, df, f_df)
175 if self.search:
176 if self.f_df == None: fp_dfp = None # -- In case no gradients are available
--> 177 x_min, f_min = self.optimizer.optimize(x0, f =fp, df=None, f_df=fp_dfp)
178 return self._expand_vector(x_min), f_min
179 else:

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\optimization\optimizer.pyc in optimize(self, x0, f, df, f_df)
62 res = scipy.optimize.fmin_l_bfgs_b(f, x0=x0, bounds=self.space.get_bounds(),approx_grad=True, maxiter=self.maxiter)
63 else:
---> 64 res = scipy.optimize.fmin_l_bfgs_b(_f_df, x0=x0, bounds=self.space.get_bounds(), maxiter=self.maxiter)
65 return np.atleast_2d(res[0]),np.atleast_2d(res[1])
66

C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\lbfgsb.pyc in fmin_l_bfgs_b(func, x0, fprime, args, approx_grad, bounds, m, factr, pgtol, epsilon, iprint, maxfun, maxiter, disp, callback, maxls)
191
192 res = _minimize_lbfgsb(fun, x0, args=args, jac=jac, bounds=bounds,
--> 193 **opts)
194 d = {'grad': res['jac'],
195 'task': res['message'],

C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\lbfgsb.pyc in _minimize_lbfgsb(fun, x0, args, jac, bounds, disp, maxcor, ftol, gtol, eps, maxfun, maxiter, iprint, callback, maxls, **unknown_options)
326 # until the completion of the current minimization iteration.
327 # Overwrite f and g:
--> 328 f, g = func_and_grad(x)
329 elif task_str.startswith(b'NEW_X'):
330 # new iteration

C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\lbfgsb.pyc in func_and_grad(x)
276 else:
277 def func_and_grad(x):
--> 278 f = fun(x, *args)
279 g = jac(x, *args)
280 return f, g

C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\optimize.pyc in function_wrapper(*wrapper_args)
290 def function_wrapper(wrapper_args):
291 ncalls[0] += 1
--> 292 return function((wrapper_args + args))
293
294 return ncalls, function_wrapper

C:\ProgramData\Anaconda2\lib\site-packages\scipy\optimize\optimize.pyc in call(self, x, *args)
61 def call(self, x, *args):
62 self.x = numpy.asarray(x).copy()
---> 63 fg = self.fun(x, *args)
64 self.jac = fg[1]
65 return fg[0]

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\optimization\optimizer.pyc in _f_df(x)
58 if f_df is not None:
59 def _f_df(x):
---> 60 return f(x), f_df(x)[1][0]
61 if f_df is None and df is None:
62 res = scipy.optimize.fmin_l_bfgs_b(f, x0=x0, bounds=self.space.get_bounds(),approx_grad=True, maxiter=self.maxiter)

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\optimization\acquisition_optimizer.pyc in fp_dfp(x)
163 xx = self._expand_vector(x)
164
--> 165 fp_xx , dfp_xx = f_df(xx)
166 dfp_xx = dfp_xx[:,np.array(self.free_dims)]
167 return fp_xx, dfp_xx

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\acquisitions\base.pyc in acquisition_function_withGradients(self, x)
44 Takes an acquisition and it gradient and weights it so the domain and cost are taken into account.
45 """
---> 46 f_acqu,df_acqu = self._compute_acq_withGradients(x)
47 cost_x, cost_grad_x = self.cost_withGradients(x)
48 f_acq_cost = f_acqu/cost_x

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\acquisitions\EI.pyc in _compute_acq_withGradients(self, x)
45 """
46 fmin = self.model.get_fmin()
---> 47 m, s, dmdx, dsdx = self.model.predict_withGradients(x)
48 phi, Phi, _ = get_quantiles(self.jitter, fmin, m, s)
49 f_acqu = (fmin - m + self.jitter) * Phi + s * phi

C:\ProgramData\Anaconda2\lib\site-packages\GPyOpt\models\gpmodel.pyc in predict_withGradients(self, X)
111 m, v = self.model.predict(X)
112 v = np.clip(v, 1e-10, np.inf)
--> 113 dmdx, dvdx = self.model.predictive_gradients(X)
114 dmdx = dmdx[:,:,0]
115 dsdx = dvdx / (2*np.sqrt(v))

C:\ProgramData\Anaconda2\lib\site-packages\GPy\core\gp.pyc in predictive_gradients(self, Xnew, kern)
335
336 for i in range(self.output_dim):
--> 337 mean_jac[:,:,i] = kern.gradients_X(self.posterior.woodbury_vector[:,i:i+1].T, Xnew, self.predictive_variable)
338
339 # gradients wrt the diagonal part k{xx}

C:\ProgramData\Anaconda2\lib\site-packages\GPy\kern\src\kernel_slice_operations.pyc in wrap(self, dL_dK, X, X2)
116 def wrap(self, dL_dK, X, X2=None):
117 with _Slice_wrap(self, X, X2) as s:
--> 118 ret = s.handle_return_array(f(self, dL_dK, s.X, s.X2))
119 return ret
120 return wrap

C:\ProgramData\Anaconda2\lib\site-packages\GPy\kern\src\stationary.pyc in gradients_X(self, dL_dK, X, X2)
234 """
235 if config.getboolean('cython', 'working'):
--> 236 return self._gradients_X_cython(dL_dK, X, X2)
237 else:
238 return self._gradients_X_pure(dL_dK, X, X2)

C:\ProgramData\Anaconda2\lib\site-packages\GPy\kern\src\stationary.pyc in _gradients_X_cython(self, dL_dK, X, X2)
321 X, X2 = np.ascontiguousarray(X), np.ascontiguousarray(X2)
322 grad = np.zeros(X.shape)
--> 323 stationary_cython.grad_X(X.shape[0], X.shape[1], X2.shape[0], X, X2, tmp, grad)
324 return grad/self.lengthscale**2
325

NameError: global name 'stationary_cython' is not defined

[tfwnicholson]

I was having a similar issue, which seems to be caused by not having gcc and gmp installed in Conda.

I was able to use:
conda install -y gcc gmp

To install the necessary libraries, and then I ran:
python setup --verbose build_ext
making sure that the cython modules were being rebuilt.

[yutaoki]

Maybe you need to install paramz first, and then GPy, and then finally GPyOpt.

I had a similar issue but now I don't get the error message:
"warning in stationary: failed to import cython module: falling back to numpy"
after this procedure.

[isaacmasher]

I was still getting the "unable to load cython" error now (Nov 2019), with GPy installed on python 2.7 from pip. I can run everything fine, but runtime seems excessively slow. I suspect that this is due to using the numpy rather than cython version of things.

I solved this by installing the Microsoft Visual C++ redistributable 2008 SP1 (example here). My guess is that the cythonized files in the GPy package were compiled with this version of MSVC. This is kind of an old version, and newer Windows machines (e.g. fresh install of Windows 10) do not come with all of the redistributables for older MSVC++. The "side-by-side configuration error" is caused by not having the correct version of the MSVC++ redistributables on your system (or windows being unable to find them).

To the maintainers: I would suggest including this (i.e. the required MSVC++ redistributable version) in the error message when the cython libraries fail to load due to a side-by-side configuration error.