Does Pants support adding prefix namespaces?

[Asmhoa]

Hello, my team has a python apps monorepo that looks like:

umbrella/
├─ libs/
│  ├─ lib1/
│  │  ├─ setup.py
│  ├─ lib2/
│  │  ├─ setup.py
│  ├─ lib3/
│  │  ├─ setup.py
├─ apps/
│  ├─ app1/
│  │  ├─ src/
│  │  │  ├─ __init__.py
│  │  ├─ setup.py
│  ├─ app2/
│  │  ├─ setup.py
pants.toml

For each of these libs and apps, we want to include them in a namespace. For example, umbrella/apps/app1/setup.py contains

from setuptools import setup
setup(
    package_dir={"umbrella.apps.app1": "src"}
)

which allows us to pip install apps/app1 independently and then import umbrella.apps.app1.

Is this kind of namespace prefixing supported with source roots in Pants?

[kaos]

Hi, with your source tree, it seems a source root of / would do what you want, and that is part of the default source roots. Does it not work as expected?
https://www.pantsbuild.org/docs/source-roots

Edit: sorry, I missed the src/ sub-folder under each app.. so no, that won't work with pants out-of-the-box.
To make it work, you'd need to move the code around a bit, so that you have your desired end-result under your source root(s). That is, going from a source root, for umbrella.apps.app1 it would be a path of <source-root>/umbrella/apps/app1/__init__.py.

[Asmhoa]

Thanks @kaos. I know this is the standard way to make a python namespace, but that nesting can get excessive really fast 😅 . The setup.py workaround works when installing the package (i.e. python setup.py build or pip install) but not in editable mode (i.e pip install -e .). So I was hoping I could use Pants to just trigger a build whenever a file gets changed.

Please let me know if you have any other ideas other than changing the tree, thank you!

[kaos]

OK, yea I have no ideas that doesn't involve changing the tree, but I do have a "compromise" idea:

project/
  apps/
    app1/
      src/
        umbrella/
          app1/
            *.py
    app2/
      src/
        umbrella/
          app2/

That is, you have the same level of separation in your apps and libs as now, but your sources would be dug deeper in each src folder. So with source roots of src/ that ought to work.

[benjyw]

Pants can't work with your layout because there are no "correct" source roots in it, and the code is unusable from source (since imports will fail), and you have to pip install it locally to use it. Pants exists precisely so you don't have to do that sort of thing... Why is this desirable for your use case?