[BUG]: egg_info should be executed as part of the build

[kloczek]

setuptools version

57.4.0

Python version

3.8.11

OS

Linux/x86_64

Additional environment information

N/A

Description

Many modules which have sphinx documentation which is possible to generate using setuptools<>sphinx integration needs fails on build_sphinx only because module metadata are not available.
Those metadata is possiblr to generate using egg_info command nad some of the modules dring tge build calls that setuptools comman but it is only because include_package_data=true or or gecause they are installing some data files.
More details with my try to prepare patch (which is wrong) are in #2756

Expected behavior

builld should check are module metadata available and if not egg_info should be executed.
Without that it is necessary to call manually egg_info beffore for example build_sphinx.

How to Reproduce

N/A

Output

N/A

Code of Conduct

• I agree to follow the PSF Code of Conduct

[jaraco]

Some commands, such as bdist_egg, do invoke egg_info prior to running. The build command, however, is part of distutils and doesn't know about egg_info.

In my experience, running build_sphinx as a setuptools command is of little value over simply invoking sphinx after installing the package (which would make the metadata present).

What is the use-case / advantage of using the setuptools command?

[kloczek]

In my experience, running build_sphinx as a setuptools command is of little value over simply invoking sphinx after installing the package (which would make the metadata present).

Completly wrong impression :)
With sphinx-doc/sphinx#9520 I've manage to build doculemtation as man pages for almost half of may rpm packages with python modules.

[tkloczko@barrel SPECS]$ grep -l %py3_build_sphinx_man python-*| wc -l; ls -l python-* | wc -l
298
618

result ..

[tkloczko@barrel SPECS]$ man python-<tab><tab>
Display all 279 possibilities? (y or n)
python-amqp                               python-gitdb                              python-paramiko                           python-six
python-anyiodoc                           python-gitpython                          python-parso                              python-smartypants
python-anytree                            python-graphviz                           python-parver                             python-smmap.tex
python-argcomplete                        python-greenlet                           python-paste                              python-sniffio
python-argon2-cffi                        python-h11                                python-paste-deploy                       python-sos
python-arrow                              python-h2                                 python-path                               python-sphinx-argparse
python-asgi                               python-hacking                            python-pathspec                           python-sphinx-click
python-aspectlib                          python-hpack                              python-pep517                             python-sphinxcontrib-asyncio
python-astor                              python-html5lib                           python-pillow                             python-sphinxcontrib-autoprogram
python-astroid                            python-httpcore                           python-platformdirs                       python-sphinxcontrib-bibtex
python-async_generator                    python-httplib2                           python-pluggy                             python-sphinxcontrib-httpdomain
python-atomicwrites                       python-hypercorn                          python-polib                              python-sphinxcontrib-programoutput
python-attrs                              python-hyperframe                         python-portend                            python-sphinxcontrib-trio
python-augeas                             python-hyperlink                          python-prb                                python-sphinxcopybutton
python-autodocsumm                        python-hypothesis                         python-priority                           python-sphinxext-opengraph
python-automat                            python-ifaddr                             python-productmd-compose                  python-sphinx-inline-tabs
python-babel                              python-importlib-metadata                 python-productmd-composeinfo              python-sphinx-readable-heme
python-backcall                           python-inflect                            python-productmd-discinfo                 python-sphinx-removed-in
python-backports.entry-points-selectable  python-ipykernel                          python-productmd-images                   python-sphinx_rtd_theme
python-beautifulsoup                      python-ipythonparallel                    python-productmd-rpms                     python-sphinx-tabs
python-benchmark                          python-itsdangerous                       python-productmd-terminology              python-sphinx-typlog-theme
python-betamax                            python-jaraco-classes                     python-productmd-treeinfo                 python-sphobjinv
python-billiard                           python-jaraco-collections                 python-prompt_toolkit                     python-sqlalchemy
python-black                              python-jaraco-envs                        python-psutil                             python-sqlparse
python-bleach                             python-jaraco-functools                   python-ptyprocess                         python-stdlib-list
python-blinker                            python-jaraco.itertools                   python-purl                               python-stem
python-boto3                              python-jaraco-packaging                   python-py                                 python-sure
python-botocore                           python-jaraco-path                        python-pyasn1                             python-sybil
python-bottle                             python-jaraco-text                        python-pybtex                             python-systemd
python-breathe                            python-jedi                               python-pybtex-docutils                    python-tempora
python-build                              python-Jinja                              python-pycodestyle                        python-terminado
python-cachetools                         python-jinja2_pluralize                   python-pydocstyle                         python-testpath
python-cbor2                              python-jmespath                           python-pyfakefs                           python-test_server
python-cffi                               python-jsonschema                         python-pyftpdlib                          python-testtools
python-chai                               python-jupyter_client                     python-pygal                              python-tidy
python-characteristic                     python-jupyter_core                       python-pygments                           python-tinycss2
python-chardet                            python-jupytext                           python-pyhamcrest                         python-toolz
python-charset-normalizer                 python-kiwi                               python-pyiso8601                          python-tornado
python-cheroot                            python-kombu                              python-pylama                             python-tox
python-click                              python-lark                               python-pylint                             python-traitlets
python-click-log                          python-latexcodec                         python-pymeeus                            python-transaction
python-commonmark-py                      python-lazy-object-proxy                  python-pynacl                             python-trio
python-contextlib2                        python-libevdev                           python-pyopenssl                          python-trustme
python-convertdate                        python-linkify-it-py                      python-pyrad                              python-twisted
python-coveragepy                         python-lockfile                           python-pyrsistent                         python-uritemplate
python-coveralls                          python-lxml                               python-pyscss                             python-urllib3
python-cppy                               python-lz4                                python-pytest                             python-validators
python-cssselect2                         python-m2r                                python-pytest-cov                         python-vine
python-cython                             python-mako                               python-pytest-regressions                 python-virtualenv
python-dateutil                           python-markupsafe                         python-pytest-runner                      python-waitress
python-decorator                          python-mdit-py-plugins                    python-pytest-trio                        python-wcwidth
python-dictdiffer                         python-metaextract                        python-pytest-xprocess                    python-webcolors
python-dnspython                          python-mistune                            python-python-sphinx-contribspelling      python-webencodings
python-dulwich                            python-mock                               python-pyudev                             python-webob
python-elasticsearch-py                   python-more-itertools                     python-pyxattr                            python-websocket-client
python-elementpath                        python-mpi4py                             python-pyxdg                              python-websockets
python-entrypoints                        python-msgpack                            python-requests                           python-webtest
python-evdev                              python-multidict                          python-requests-mock                      python-werkzeug
python-eventlet                           python-multipledispatch                   python-requests_toolbelt                  python-wheel
python-execnet                            python-mypy                               python-rfc3986                            python-Whoosh
python-factory-boy                        python-myst-parser                        python-rich                               python-wrapt
python-faker                              python-nbclient                           python-rsa                                python-WSGIProxy2
python-fields                             python-nbformat                           python-rst.linker                         python-wsproto
python-flake8                             python-netaddr                            python-scons                              python-xmlschema
python-flask                              python-nose2                              python-scripttest                         python-yamlloader
python-flask-sqlalchemy                   python-notebook                           python-selenium                           python-yarl
python-flit                               python-numpydoc                           python-semantic-version                   python-zeroconf
python-fonttools                          python-olefile                            python-service-identity                   python-zipp
python-gcovr                              python-openstackdocstheme                 python-setuptools                         python-zope-event
python-gidocgen                           python-outcome                            python-simpleline

What is the use-case / advantage of using the setuptools command?

With setuptools<>sphinx integration it is possible to not care where cppy.py file is and is possible to build dociulemntation using standarised command sequence
In my case I'm using rpm macri whicjh looks like below:

[tkloczko@barrel SPECS]$ rpm -E %py3_build_sphinx_man
\
        PBR_VERSION=%{version} \
        SETUPTOOLS_SCM_PRETEND_VERSION=%{version} \
        /usr/bin/python3 setup.py build_sphinx -b man --build-dir build/sphinx

Only missing in that approach is that many copy.py files are accessing to module metadata so this is why it is necessry to take care of executilg egg_info before build_sphinx.

[kloczek]

I'm moving all build to pep517 (python3 -sBm build -w) type builds and I see that using that procedure egg_info metadata are are always produced in build tree.
As I'm no longer interested old style of the build this ticket could be IMO closed.

Only thing is that instead generate egg_info metadata IMO should be generated dist_info.
I'm leaving you decision what to do with this ticket.

[jaraco]

In #1988, we are exploring the deprecation of many of the setup.py invocations. Based on your experience and mine and that of RTD, I’d like to consider deprecating and removing the build_sphinx command. I’ll open a separate issue for that.

[kloczek]

Just one stat from all my rpm packages spec files used to build python modules

[tkloczko@ss-desktop SPECS]$ ls -1 python-*|wc -l; grep %py3_build_sphinx_man python-*|wc -l
743
382

As you see in more than half of all my packages I'm using setuptools<>sphinx integration which is IMO EXTREAMY useful because it allows generate documentation without cate where conf.py file and documentation source is located in source tree.
IMO it would be really bad if such generic procedure would be removed without provide some replacement.

I understand intention to remove some legacy stuff however I would not complain if that setuptools tool would be somehow replaced by something which will be functionally as useful as current setuptools<>sphinx integration allowing use some hooks for other pep517 backends like flit or poetry.

IMO it is kind of need to extend current pep517 spec to handle build and install documentation depends on chosen format.
I've been trying to explain and those needs with flit developers

pypa/flit#429
pypa/flit#491

Generally speaking such documentation interface would need two thing:

• on build stage should be always like it is now with setuptools generated package metadata because those metadata may be necessary to generate documentation (look on 3.5.1: cannot add module metadata in project tree flit#491)
• probably in project.toml file would be good to add possibility to specify where is base directory with documentation source form (like it is now with setup.cfg and default base directory should be sill 'docs/`)
• project.toml should as well contain some details about documentation rendering tooling like sphinx or mkdoc (some projects stated using it recently and however I don't like it because it allows render only html output and want to have produced man page(s))

Feel free to keep me on the loop on any discussion about documentation build and install subject :)

[jaraco]

@brettcannon recently posted this tweet seeking something similar.