PyArray_API is *not* a unique symbol. (Trac #1928) #2521
Description
Original ticket http://projects.scipy.org/numpy/ticket/1928 on 2011-08-10 by trac user phil@..., assigned to @pearu.
numpy makes the API to its array type available through the symbol PyArray_API. But it also allows users to make it available through some other symbol by defining PY_ARRAY_UNIQUE_SYMBOL to be whatever other symbol is desired. (Why anyone would want to use a different symbol name never became clear to me, but some do.)
__multiarray_api.h has some C preprocessor magic that defines the symbol one way if PY_ARRAY_UNIQUE_SYMBOL is defined, and another way if PY_ARRAY_UNIQUE_SYMBOL is not defined. It all works well, except...
...In the numpy header fortranobject.h, PY_ARRAY_UNIQUE_SYMBOL is defined as PyArray_API. This makes it look as if a different symbol is being used, but the "different" symbol being used is the same as the "real" symbol for the API. This results in anyone including fortranobject.h (scipy does) having a definition of PyArray_API that does not play well at all with the "real" global definition of PyArray_API.
I was sometimes getting a crash when invoking "import scipy.signal" I tracked it down to:
- signal imports special (directly or indirectly)
- import of special results in call of C function initspecfun()
- initspecfun() calls _import_array
- _import_array executes this line of code:
PyArray_API = (void **)PyCObject_AsVoidPtr(c_api);
It was this statement that was causing trouble. My debugger was telling me that the value being returned from PyCObject_AsVoidPtr was the same as the address of PyArray_API. In other words, after the above statement, &PyArray_API == PyArray_API is true. I'm pretty sure that's not good. It resulted in the first attempt to access the contents of PyArray_API (a call to PyArray_GetNDArrayCVersion() a few lines later) crashing.
The crash only happens if I do the "import scipy.signal" after I import my own C-python module that uses numpy arrays. I suppose it could be a bug in my code, but I can't see how. My source file just has a "#include <numarray/libnumarray.h>", without define any optional symbols (no PY_ARRAY_UNIQUE_SYMBOL, no NO_IMPORT or NO_IMPORT_ARRAY). It does not anywhere (directly) reference PyArray_API. The only thing the init routine does with numpy is call _import_array. Of course, my module has routines that do stuff with numpy arrays, but I don't call any of those between importing my module and importing scipy.signal to get the crash. If I am doing something wrong, then consider this to be a bug report on the docs, which have not made it clear what I am doing wrong. (I've looked. A bit.)
I tried to fix the problem by removing the #define of PY_ARRAY_UNIQUE_SYMBOL in fortranobject.h. That did indeed prevent the crash when I import my module followed by scipy.signal. However, if I import scipy.signal without first importing my module, the attempt to import scipy.signal raises in ImportError with complaint of undefined symbol PyArray_API.
I then tried to fix it by changing the define of PY_ARRAY_UNIQUE_SYMBOL in fortranobject.h to something (anything) other than PyArray_API. That worked. I am now able to import scipy.signal whether or not I first import my module. Is that the "right" fix? You decide.
Python 2.5.4
numpy 1.5.1rc2
scipy 0.8.0