Speed up relative re-imports into absolute imports

[scoder]

Cython has to know the qualified module name at compile time, so the absolute package path of a relative import is actually known at compile time. We cannot change the way relative first imports work in general, because they are actually looked up relative to the current package, but we can still speed up re-imports (which are actually not that uncommon). Cython can construct the qualified name of a relative module import at compile time and look that up in sys.modules (as done in the ImportDottedModule helper function for absolute imports).

Requires C / C-API knowledge.

[xorb0ss]

Mind if I take/attempt this as a "good first issue"?

[scoder]

Mind if I take/attempt this as a "good first issue"?

Sure, if you know C? The import code is mostly written using C-API code. Look at the FromImportStatNode in Nodes.py and the ImportNode in ExprNodes.py that generate the relevant C code. I suggest writing a test case (see the HackerGuide) with repeated relative imports and looking at the generated C code to see what we currently do and what can be done.

[xorb0ss]

Sure, if you know C?

Yup.

@scoder Does this have any relation to #2854 and #2859 in terms of steps taken or good ways to approach it? I'm also having a look at tests/run/absolute_import.srctree.

[scoder]

Yes, related. The __Pyx__ImportDottedModule_Lookup helper function does the right module lookup, once you have the fully qualified module name, so that can be reused (move it to a new /////… section in ImportExport.c and use //@requires to depend on it). I'm also ok with doing this only for Py3, if Py2 proves any problematic.

The qualified module name needs to be constructed for relative imports in FromImportStatNode (I think), then passed as a Python string constant (e.g. created with code.intern_identifier()) into the helper, if that returns NULL, it means that the module needs to be imported normally. Otherwise, just use what we got.