Add topic on namespace packages #290
Conversation
There was a problem hiding this comment.
Thanks for this!
The overall structure looks good to me, just two major comments (and a few minor ones inline):
- since the import system is covered in the language reference these days, I think we should adopt "native namespace packages" as the preferred term over "PEP-420-style namespace packages"
- at the import system level, the compatibility between native namespace packages and pkgutil-style namespace packages should be solid, and I think we should commit to also maintaining that compatibility across the PyPA tooling so folks can mix and match them freely (leaving pkg_resources-style namespace packages as the only "all-or-nothing" case)
source/namespace_packages.rst
Outdated
| :Last Reviewed: 2017-05-04 | ||
|
|
||
| Namespace packages allow you to split a single Python package across multiple | ||
| distributions. For instance, if you have the following package structure: |
There was a problem hiding this comment.
While distributions is formally correct, it never really caught on in general use, so I suspect this opening sentence might prove confusing. Perhaps:
Namespace packages allow you to share a single Python import namespace between multiple projects or :term:`distribution packages <Distribution Package>`.
There was a problem hiding this comment.
Good point. I updated it to reference terms an disambiguate distributions. Let me know what you think. I suspect no matter what we do this sentence will be difficult to grok, but the example below is hopefully illustrative enough to convey understanding.
source/namespace_packages.rst
Outdated
| packages (such as a large corpus of client libraries for multiple products from | ||
| a single company). However, namespace packages come with several caveats and | ||
| are not appropriate in all cases. A simple alternative is to use a prefix on | ||
| all of your distributions such as ``import mynamespace_subpackage_a``. |
There was a problem hiding this comment.
Perhaps import mynamespace_subpackage_a as subpackage_a to get an alternative closer to the from imports above?
There was a problem hiding this comment.
Perhaps, but most style guides discourage aliasing. I've added it as an or. :)
source/namespace_packages.rst
Outdated
|
|
||
| The three methods of creating namespace packages are largely not | ||
| cross-compatible. It's inadvisable to use different methods in distributions of | ||
| the same namespace package. |
There was a problem hiding this comment.
I'd suggest "distributions that contribute to the same namespace package", since most namespaces either have no particular owning distribution (e.g. backports.*), or else one owning project and multiple contributing projects.
source/namespace_packages.rst
Outdated
|
|
||
| There are currently three different approaches to creating namespace packages: | ||
|
|
||
| #. Use `PEP 420-style namespace packages`_. This is recommended if packages in |
There was a problem hiding this comment.
I'd suggest referring to these as "native namespace packages" and linking to https://docs.python.org/3/reference/import.html#namespace-packages, rather than codifying the PEP number here - if the native namespace behaviour changes in the future, the language reference will be updated, but PEP 420 won't.
However, the "PEP 420 namespace package" term would still be worth mentioning as an alternative name for native namespace packages.
| subpackage_a/ | ||
| # Sub-packages have __init__.py. | ||
| __init__.py | ||
| module.py |
There was a problem hiding this comment.
The current examples may give the impression that namespace packages only support subpackages, and not submodules - it may simplify things to instead only show submodules, and then state somewhere that any form of submodule (including full subpackages) is supported.
There was a problem hiding this comment.
Done in the very first section and examples and explicitly called out that sub-packages and modules can participate in namespace packages.
source/namespace_packages.rst
Outdated
|
|
||
| It is extremely important that every distribution that uses the namespace | ||
| package omits the ``__init__.py``. If any distribution does not, it will cause | ||
| the namespace logic to fail and the other sub-packages will not be importable. |
There was a problem hiding this comment.
It's probably worth stating that the __init__.py files for pkgutil-style explicit namespace packages are also OK here, since they replicate the logic otherwise used by native namespace packages.
source/namespace_packages.rst
Outdated
| you need compatibility with packages already using this method or if your | ||
| package needs to be zip-safe. | ||
|
|
||
| The three methods of creating namespace packages are largely not |
There was a problem hiding this comment.
I'd adjust this to state that we do expect native namespace packages and pkgutil-style namespace packages to be cross-compatible. If mixing them in any given Python version fails, then that's a bug in either pkgutil.extend_path() or the import machinery.
(And we should have stdlib tests in place to ensure that cross-compatibility if we don't already)
| __path__ = __import__('pkgutil').extend_path(__path__, __name__) | ||
|
|
||
| **Every** distribution that uses the namespace package must include an | ||
| identical ``__init__.py``. If any distribution does not, it will cause the |
There was a problem hiding this comment.
As above, I think we should soften the recommendation to also allow "native namespace packages" here.
That way namespaces can more easily support a mix of pure Python 3 projects using native namespace packages, and hybrid Python 2/3 ones using pkgutil-style namespace packages, rather than believing that everyone has to use pkgutil-style namespace as long as anyone is.
There was a problem hiding this comment.
Done via a tip in the native section.
SGTM, I'll incorporate that.
For sure, the problem is that according to my tests there are a few issues with that. Notably, if you install the first package with setuptools and the second with pip it'll fail. I'm hesitate to advertise compatibility until that's resolved. @jaraco Any ideas what's going on in that case? |
|
@ncoghlan Thanks for the review, it's ready for another round. :) |
|
Very nice, thank you! In reviewing this, it made me realise that now that namespace packages are documented, it might be feasible to also cover the two main options to writing plugin systems for Python applications: #291 |
|
@ncoghlan my pleasure! One small note - the examples for namespace packages currently live on my personal github, it might make more sense for those to live under the pypa org. I'm happy to populate that repo if someone can create it. |
Resolves #265.