Allow for built-in modules to be submodules

[kesmit13]

Bug report

If you statically link a namespaced extension (such as numpy.core._multiarray_umath) and add it to the inittab, it will not get loaded properly. In this case, the path to the extension is passed to the BultinImporter.find_spec method which causes it to return None. The path is a pointer to the namespace of the extension.

One possible fix would be to check for a . in the name of the extension being loaded. If it contains that, consider it to be a fully-qualified import path and check to see if that fully-qualified name is in the builtins list before moving on to other checks.

Your environment

• CPython 3.10 (although I believe it's the same in newer versions)
• Linux x86 and Wasm

Linked PRs

• GH-102700: allow built-in modules to be submodules #103162
• [3.11] GH-102700: allow built-in modules to be submodules (GH-103162) #103322

[FFY00]

In GH-102768 @brettcannon wrote:

As discovered while trying to get NumPy statically compiled into CPython for WASI, the built-in importer automatically rejects any built-in module name with a .. But honestly I don't know if there's a need to be so strict since there technically isn't a reason why a built-in module couldn't be a submodule.

[FFY00]

@kesmit13 thanks for opening the issue.

numpy and numpy.core are normal packages, with only numpy.core._multiarray_umath being a built-in module, right?

the built-in importer automatically rejects any built-in module name with a .

The root issue here is that path is set, which happens because the parent module is a package.

cpython/Lib/importlib/_bootstrap.py

Lines 1262 to 1269 in 405739f

	try:
	path = parent_module.__path__
	except AttributeError:
	msg = f'{_ERR_MSG_PREFIX} {name!r}; {parent!r} is not a package'
	raise ModuleNotFoundError(msg, name=name) from None
	parent_spec = parent_module.__spec__
	child = name.rpartition('.')[2]
	spec = _find_spec(name, path)

I think the question we want to ask here is if we want to support having a built-in module be a submodule of a normal package? If so, we should simply remove the path check from BultinImporter.find_spec.

[kesmit13]

Your comments about numpy.core._multiarray_umath are correct. The issue that I'm running into is trying to create a statically linked Python binary that includes numpy and pandas. In order to do this, I have to use PyImport_AppendInittab with the C extensions from those packages.

[FFY00]

Thanks, I think it makes sense to drop this limitation. I don't think there's anything else preventing built-in modules from being submodules other than the path check, so removing that check should be all that's needed.