How to support the essential layout?

[jaraco]

Please don't hate me for this request 😬 . At this stage, I'm only looking for advice.

I've embarked on a project to re-envision how (Python) projects are developed. This system relies on a new source layout. Instead of the "src" layout or the "flat" layout, this system uses the essential layout. This approach puts the code for any project in the root of a repo, such that its installed location is relative to the project name. For example, the Python package tempora can be found as the root of this repo and the package coherent.build can be found as the root of this repo. The projects' path isn't repeated in the repo itself.

I've worked out the details to build and install packages from this layout. I also have a path forward to support editable installs for this layout. To close the loop on the MVP, however, I also want to be able to support pytest and in particular doctests in the modules. Unfortunately and unsurprisingly, when running pytest --doctest-modules on one of these projects, if there are any relative imports in the modules under test, they will fail, because pytest doesn't know how to resolve ./__init__.py to the name tempora or ./__init__.py to coherent.build.

I'd like to get support for this layout in pytest, either through a plugin or possibly with native support. I've laid out a few approaches to consider and I'd like to start by considering what it would take to add native support, and I'd appreciate your expert advice. Note that I'm not asking for acceptance or integration of this support, but only advice to help uncover complications that may arise. I'm hoping to limit the scope of this support to the importlib import-mode.

I remember reading elsewhere that pytest has a special responsibility to resolve Python packages from file paths, so that it can resolve ./src/towncrier/__init__.py to towncrier and ./zc/interface/__init__.py to zc.interface.

I've looked at DoctestModule and I see it has no special logic for resolving self.obj (the module), so the logic for discovery must be in the parent _pytest.python.Module, so I'll continue exploring there.

In the meantime, I'll start with my questions.

• Are there any hidden dangers with my envisioned approach (allowing ./*.py to resolve to a given package root)?
• Are there existing abstractions to handle the more common "src-" and "flat-" layouts that could be extended with a third layout?
• Is restricting this behavior to importlib import mode a viable approach? Recommended?

[jaraco]

I'm seeing now that a lot of the logic for these concerns is handled in import_path and several of the concerns are entangled (import mode, root path concerns, namespace package support). And it seems that the concerns about the layout are not modeled explicitly in the code.

I'll want to spent some more time getting to know this code better before embarking on a solution.

[RonnyPfannschmidt]

The concept of import roots could be a help

I hope to turn it feasible in the sprint

[jaraco]

For the short term, I've created this hacky monkey patch to unblock the Coherent System.