DeprecationWarning: pkg_resources is deprecated as an API

[adamjstewart]

Bug description

setuptools 67.3, released yesterday, officially deprecates pkg_resources.declare_namespace: https://setuptools.pypa.io/en/latest/history.html#v67-3-0

How to reproduce the bug

With setuptools 67.3+ installed:

$ python -Werror -c 'from pytorch_lightning import LightningDataModule'

Error messages and logs

Traceback (most recent call last):
  File "<string>", line 1, in <module>
    from pytorch_lightning import LightningDataModule
/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/pytorch_lightning/__init__.py:34: in <module>
    from lightning_fabric.utilities.seed import seed_everything  # noqa: E402
/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/lightning_fabric/__init__.py:29: in <module>
    __import__("pkg_resources").declare_namespace(__name__)
/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/pkg_resources/__init__.py:2297: in declare_namespace
    warnings.warn(msg, DeprecationWarning, stacklevel=2)
DeprecationWarning: Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages

Environment

Current environment

#- Lightning Component: LightningModule
#- PyTorch Lightning Version: 1.9.1
#- Lightning App Version: N/A
#- PyTorch Version: 1.13.1
#- Python version: 3.8.16
#- OS: Ubuntu 22.04.1
#- CUDA/cuDNN version: N/A
#- GPU models and configuration: N/A
#- How you installed Lightning: pip
#- Running environment of LightningApp: cloud

More info

No response

cc @tchaton

[awaelchli]

@Borda Do you know what's the alternative here?

[adamjstewart]

I'm not super familiar with namespace packages, but based on the link in the warning message, it seems like you're supposed to set the namespace_packages field in setup.py and let setuptools handle namespacing instead of doing it manually in __init__.

[adamjstewart]

Note that the the warning message changed slightly in setuptools 67.4:

DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('lightning_fabric')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`.

[adamjstewart]

Any updates on this?

[awaelchli]

@Borda Should we do what @adamjstewart said #16756 (comment)?

[Borda]

I am not much familiar with namespace, let me check it

[adamjstewart]

Update: setuptools 67.5 officially deprecates all of pkg_resources: https://setuptools.pypa.io/en/latest/history.html#v67-5-0

[tonybaloney]

See
https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#finding-namespace-packages for instructions on what to do.

TLDR if you require namespace packages (searching where there isn't nessecarily a __init__.py in every folder), use

from setuptools import find_namespace_packages

setup(
    # ...
    packages=find_namespace_packages(),
    # ...
)

Otherwise:

from setuptools import find_packages

setup(
    # ...
    packages=find_packages(),
    # ...
)