Cythonize/Cython compilation fails to import/recognize .pxd from external modules that only have __init__.so

[ghost]

When trying to use a .pxd from an external module and making sure it can be found on sys.path, Cythonize/Cython compilation will still fail to pick it up if that module only has __init__.cpython-36m-x86_64-linux-gnu.so but no __init__.py (which it obviously couldn't have, given the __init__ part is already that C extension).

As soon as I do touch __init__.py in that folder it's all picked up fine, but of course that's not really a meaningful thing to do and renders the module broken, which I think I shouldn't have to do just so I can import .pxd files from it, should I?

So this looks a bit like a Cython bug to me, tested with Cython 0.29.6

[scoder]

PR welcome that extends the package check in Utils.is_package_dir() and adds a test for it.

[HSR05]

@scoder I would like to work on it. Can you guide me a little. I will be more than thankful. 👍

[scoder]

Sure. Read the code in Cython.Utils.is_package_dir(), it checks for package files in the directories. As for a test, see tests/run/different_package_names.srctree. That's a text file containing multiple files, which the test runner extracts into a test directory. See https://github.com/cython/cython/wiki/HackerGuide#tests

[ghost]

For what it's worth the workaround for this bug in a setup.py can become pretty involved and regularly likes to break some of my things, so it'd be quite a maintenance burden relief if Cython could handle this case properly out of the box (I just added this comment because I lost another hour of debugging to making some importlib quirk not break things, once more adding more code to work around this 😢 )