Windows+Py3.8: A dynamic link library (DLL) initialization routine failed.

[inducer]

@cgohlke reported in #213, moved here:

ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed

> py -3.8 -c"import pycuda._driver"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

It seems that this error is not due to CUDA, boost, or pybind11 since the simple _pvt_struct extension and pyopencl-2019.1 also fail:

> py -3.8 -c"import pycuda._pvt_struct"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

> py -3.8 -c"import pyopencl"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "X:\Python38\lib\site-packages\pyopencl\__init__.py", line 39, in <module>
    import pyopencl._cl as _cl
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

[inducer]

Thanks for the report! Unfortunately, I'm completely at a loss for what's going on here, and I can't reproduce this for lack of a Windows machine. Any clues?

[cgohlke]

I have started to look into this issue, but have no clue yet. Do pycuda and pyopencl work with Python 3.8.0b2 on Linux?

[inducer]

I installed 3.8b2 from Debian, but none of pip, virtualenv, -m venv are working for me at the moment, so I'm not able to test easily. I've set a reminder to revisit this in a week.

[inducer]

Just retried this endeavor on Linux (amd64) and Debian's Python 3.8.0~b2-5, and I had no issue. I had to compile numpy==1.18.0.dev0+a162f64 by hand after manually installing Cython, but that was about all the difficulty I encountered.

========================================================================================================= test session starts =========================================================================================================
platform linux -- Python 3.8.0b2, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: /home/andreas, inifile: pytest.ini
collected 210 items                                                                                                                                                                                                                   

test_algorithm.py ......................................................                                                                                                                                                        [ 25%]
test_array.py ........................................................................                                                                                                                                          [ 60%]
test_clmath.py ......................sssss.                                                                                                                                                                                     [ 73%]
test_clrandom.py .....................                                                                                                                                                                                          [ 83%]
test_enqueue_copy.py ..                                                                                                                                                                                                         [ 84%]
test_wrapper.py .......ss.................s......                                                                                                                                                                               [100%]

========================================================================================================== warnings summary ===========================================================================================================
/home/andreas/tmp/xxy/lib/python3.8/site-packages/_pytest/mark/structures.py:328
  /home/andreas/tmp/xxy/lib/python3.8/site-packages/_pytest/mark/structures.py:328: PytestUnknownMarkWarning: Unknown pytest.mark.bitonic - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    warnings.warn(

src/pyopencl/test/test_algorithm.py::test_elwise_kernel[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
  /home/andreas/tmp/xxy/lib/python3.8/site-packages/pytools-2019.1.1-py3.8.egg/pytools/py_codegen.py:26: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

src/pyopencl/test/test_clmath.py::test_exp[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
src/pyopencl/test/test_clmath.py::test_log[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
src/pyopencl/test/test_clmath.py::test_sqrt[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
src/pyopencl/test/test_clmath.py::test_sin[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
src/pyopencl/test/test_clmath.py::test_cos[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
src/pyopencl/test/test_clmath.py::test_tan[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
src/pyopencl/test/test_clmath.py::test_sinh[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
src/pyopencl/test/test_clmath.py::test_cosh[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
src/pyopencl/test/test_clmath.py::test_tanh[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
  /home/andreas/tmp/xxy/lib/python3.8/site-packages/pyopencl-2019.1-py3.8-linux-x86_64.egg/pyopencl/array.py:2092: ComplexWarning: Casting complex values to real discards the imaginary part
    size = int(ceil((stop-start)/step))

src/pyopencl/test/test_wrapper.py::test_compile_link[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
  /home/andreas/src/pyopencl/test/test_wrapper.py:1084: UserWarning: Pre-build attribute access defeats compiler caching.
    vsink_prg = cl.Program(ctx, """//CL//

src/pyopencl/test/test_wrapper.py::test_compile_link[<context factory for <pyopencl.Device 'pthread-Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz' on 'Portable Computing Language'>]
  /home/andreas/src/pyopencl/test/test_wrapper.py:1089: UserWarning: Pre-build attribute access defeats compiler caching.
    main_prg = cl.Program(ctx, """//CL//

-- Docs: https://docs.pytest.org/en/latest/warnings.html
======================================================================================== 202 passed, 8 skipped, 13 warnings in 570.13 seconds =========================================================================================

[cgohlke]

I retried with Python 3.8.0b3 on Windows without luck. Not sure how to tackle this. Maybe rebuilding in debug mode will reveal something...

[cgohlke]

This seems to be related to how numpy is initialized in numpy_init.hpp, which is used by pycuda and pyopencl.

Things work when I take out the code from numpy_init.hpp and move import_array into the PyMODINIT_FUNC function. I don't know why this would fail only on Python 3.8 for Windows.

For pyopencl, this works for me:

diff --git a/src/numpy_init.hpp b/src/numpy_init.hpp
index 2cf7fe0..527bbdb 100644
--- a/src/numpy_init.hpp
+++ b/src/numpy_init.hpp
@@ -32,6 +32,7 @@
 #include <numpy/arrayobject.h>
 #include <stdexcept>

+/*

 namespace
 {
@@ -55,7 +56,7 @@ namespace
   } _array_importer;
 }

-
+*/


 #endif
diff --git a/src/wrap_cl.cpp b/src/wrap_cl.cpp
index 2ea3d59..6ec9095 100644
--- a/src/wrap_cl.cpp
+++ b/src/wrap_cl.cpp
@@ -41,6 +41,11 @@ extern void pyopencl_expose_mempool(py::module &m);

 PYBIND11_MODULE(_cl, m)
 {
+  if (_import_array() < 0) {
+    PyErr_SetString(PyExc_ImportError, "numpy.core.multiarray failed to import");
+    return;
+  }
+
   pyopencl_expose_constants(m);
   pyopencl_expose_part_1(m);
   pyopencl_expose_part_2(m);

[inducer]

Thanks for doing the detective work on this! Looking at the documentation, it is actually kind of surprising that the previous scheme worked at all. Could you check if these recent changes work for you? (I've also made analogous changes to PyOpenCL.) If so, I would roll a new release.

[cgohlke]

Could you check if these recent changes work for you?

PyOpenCl and PyCUDA are now working for me on Python 3.8 for Windows.. Thank you!

[inducer]

Likewise thanks for for your help. I've just released new versions of PyCUDA and PyOpenCL.