Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug in absolute import handling in python 2.7 #2638

Closed
ibell opened this issue Oct 1, 2018 · 4 comments
Closed

Bug in absolute import handling in python 2.7 #2638

ibell opened this issue Oct 1, 2018 · 4 comments

Comments

@ibell
Copy link
Contributor

ibell commented Oct 1, 2018

I think this is the same issue: https://grokbase.com/t/gg/cython-users/1417hszq2d/pxd-import-problem-when-modules-added-to-package

I have code in a .pyx file in my package PDSim (https://github.com/ibell/pdsim) that reads:

from PDSim.flow cimport flow_models

which yields the error (though somehow not terminal):

PDSim/scroll/_scroll.pyx: cannot find cimported module 'PDSim.flow'

in which the folder flow in my package has the contents:

>> ls -l
-rw-rw-r-- 1 ian ian  2216 Dec 19  2017 fanno.pyx
-rw-rw-r-- 1 ian ian  1933 Dec 19  2017 flow_models.pxd
-rw-rw-r-- 1 ian ian 24767 Sep 30 18:32 flow_models.pyx
-rw-rw-r-- 1 ian ian  2407 Dec 19  2017 flow.pxd
-rw-rw-r-- 1 ian ian 13527 Dec 19  2017 flow.pyx
-rw-rw-r-- 1 ian ian     0 Dec 19  2017 __init__.py

Why does it not find this module?

@ibell
Copy link
Contributor Author

ibell commented Oct 1, 2018

I can "fix" the issue with the replacement of

from PDSim.flow cimport flow_models

by

cimport PDSim.flow.flow_models as flow_models

which is pretty verbose, and smells like a bug. The parallel python import is

from PDSim.flow import flow_models

which works fine.

@scoder
Copy link
Contributor

scoder commented Oct 14, 2018

This case might need special-casing in the path search. I guess it tries to look for a cimportable PDSim.flow package, which would succeed if there was a PDSim/flow/__init__.pxd file, but ignores the __init__.py file as something non-cimportable. Instead, it should look for it, and continue the name list search only for name.pxd files.

@bweigel
Copy link
Contributor

bweigel commented Oct 27, 2018

From what we could figure out the problem stems from the cimported_files-method in Dependencies.py, when pxd_file is None:

    def cimported_files(self, filename):
        ...
        for module in self.cimports(filename):
            ...
            pxd_file = self.find_pxd(module, filename)
            if pxd_file is not None:
                pxd_list.append(pxd_file)
            elif not self.quiet:
                print("%s: cannot find cimported module '%s'" % (filename, module))

This seems to be ultimately due to search_include_directories-function in Utils.py, which doesn't cover all the cases of package/module-discovery, which are:

  1. pkg/a.pxd where b is definied in the a.pxd
  2. pkg/a/__init__.pxd where b is definied in the __init__.pxd
  3. pkg/a/b.pxd and pkg/a/__init__.[pxd|py|pyx]

Cases 1 and 2 seem to be processed just fine, while support of case 3 is missing.

@bweigel
Copy link
Contributor

bweigel commented Oct 27, 2018

The self.cimports(filename) call returns an "incomplete" list of possible dependencies to include.
i.e. in case of from pkg.a cimport b it would only return pkg.a, which would miss the aforementioned case 3 (package structure of pkg/a/b.pxd together with pkg/a/__init__.[pxd|py|pyx])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants