cython doesn't correctly generate C extension with name different from py file

[pitrou]

In my setup.py I have the following code:

    from Cython.Build import cythonize

    extensions = [
        Extension("distributed._cy_schedulerstate",
                  ["distributed/schedulerstate.py"])
    ]
    ext_modules = cythonize(extensions)

    setup(..., ext_modules=ext_modules)

The idea is that i can choose at runtime between the pure Python version (distributed.schedulerstate) and the Cython-accelerated version (distributed._cy_schedulerstate). Unfortunately the C extension fails loading:

>>> from distributed import _cy_schedulerstate
Traceback (most recent call last):
  File "<ipython-input-3-6878069d7b91>", line 1, in <module>
    from distributed import _cy_schedulerstate
ImportError: dynamic module does not define module export function (PyInit__cy_schedulerstate)

And indeed the C code generated by Cython doesn't use the extension name defined in the Extension constructor:

#if PY_MAJOR_VERSION < 3
PyMODINIT_FUNC initschedulerstate(void); /*proto*/
PyMODINIT_FUNC initschedulerstate(void)
#else
PyMODINIT_FUNC PyInit_schedulerstate(void); /*proto*/
PyMODINIT_FUNC PyInit_schedulerstate(void)

[stefanos82]

The only thing that came in mind as soon as I read _cy_schedulerstate is private variables, even though it should work with two underscores instead with one.

Could it somehow got confused during parsing?

[scoder]

Might seem like an obvious bug but it isn't that easy. cythonize() has several features that make its interface a little ... special. You can use file patterns, for example, as in Extension("**/*.pyx"). That makes it less obvious to decide (in a user understandable way) when the module name should be overridden. I hope I found a safe way to implement this.