0.21.1: sphinx fails

[kloczek]

Looks like something is wrong with validators sphinx documentation

+ /usr/bin/sphinx-build -n -T -b man docs build/sphinx/man
Running Sphinx v6.1.3
making output directory... done
myst v1.0.0: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=set(), disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_
domains=None, fence_as_directive=set(), number_code_blocks=[], title_to_header=False, heading_anchors=0, heading_slug_func=None, html_meta={}, footnote_transition=True, words_per_minute=200
, substitutions={}, linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2
jax_process|mathjax_process|math|output_area', enable_checkboxes=False, suppress_warnings=[], highlight_code_blocks=True)
building [mo]: targets for 0 po files that are out of date
writing output...
building [man]: all manpages
updating environment: [new config] 0 added, 0 changed, 0 removed
reading sources...

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 284, in build_main
    app.build(args.force_all, args.filenames)
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 347, in build
    self.builder.build_update()
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 308, in build_update
    self.build(['__all__'], to_build)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    updated_docnames = set(self.read())
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 437, in read
    raise SphinxError('root file %s not found' %
sphinx.errors.SphinxError: root file /home/tkloczko/rpmbuild/BUILD/validators-0.21.1/docs/index.md not found

Sphinx error:
root file /home/tkloczko/rpmbuild/BUILD/validators-0.21.1/docs/index.md not found

That is odd a bit because actually that file exist

[tkloczko@pers-jacek SPECS]$ ls -l /home/tkloczko/rpmbuild/BUILD/validators-0.21.1/docs/index.md
-rw-r--r-- 1 tkloczko tkloczko 1481 Apr 13 01:33 /home/tkloczko/rpmbuild/BUILD/validators-0.21.1/docs/index.md

[yozachar]

Documentation is dynamically generated, all you need to do is run gen_docs.py.

validators/docs/gen_docs.py

Lines 79 to 111 in 506d22b

	def _gen_rst_docs(source: Path, refs_path: Path, only_web: bool = False, only_man: bool = False):
	"""Generate reStructuredText docs."""
	# external
	from pypandoc import convert_file # type: ignore
	with open(source / "docs/index.rst", "wt") as idx_f:
	idx_f.write(
	convert_file(source_file=source / "docs/index.md", format="md", to="rst")
	+ "\n\n.. toctree::"
	+ "\n :hidden:"
	+ "\n :maxdepth: 2"
	+ "\n :caption: Reference:"
	+ "\n :glob:\n"
	+ "\n reference/*\n"
	)
	# generate RST reference documentation
	_generate_reference(source / "validators/__init__.py", refs_path, "rst")
	rc = 0
	if not only_man:
	# build sphinx web pages as subprocess
	web_build = run(("sphinx-build", "docs", "docs/_build/web"), capture_output=True)
	print(web_build.stderr.decode() + web_build.stdout.decode())
	rc = web_build.returncode
	if not only_web:
	# build sphinx man pages as subprocess
	man_build = run(
	("sphinx-build", "-b", "man", "docs", "docs/_build/man"), capture_output=True
	)
	print(man_build.stderr.decode() + man_build.stdout.decode())
	copy(source / "docs/_build/man/validators.1", source / "docs/validators.1")
	print(f"Man page copied to: {source / 'docs/validators.1'}")
	rc = man_build.returncode if rc == 0 else rc
	return rc

Some changes might be required, here:

validators/docs/gen_docs.py

Lines 146 to 156 in 506d22b

	rc = generate_documentation(
	project_root,
	only_md=True,
	only_rst_web=False,
	only_rst_man=False,
	# # NOTE: use
	# discard_refs=len(argv) <= 1 or argv[1] != "--keep",
	# # instead of
	discard_refs=True,
	# # for debugging
	)

I suspect you're seeing the error, because of sphinx configuration conf.py and some intermediate files, generated by gen_docs.py, required for the build.

validators/docs/conf.py

Line 28 in 506d22b

exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "*.md"]

[kloczek]

Issue has been resolved? 🤔

[yozachar]

Developers (at least I) intend to generate documentation using that script, you could do the same. You haven't given me a strong reason to using sphinix-build directly.

---

gen_docs.py uses pypandoc to convert the dynamically generated markdown files to reStructuredText. Only then sphinx-build would have something to work on.

From #258 I suspect you're automating something. You can use (or modify or even create an equivalent bash script for) the following function to suit your needs.

validators/docs/gen_docs.py

Lines 79 to 111 in 506d22b

	def _gen_rst_docs(source: Path, refs_path: Path, only_web: bool = False, only_man: bool = False):
	"""Generate reStructuredText docs."""
	# external
	from pypandoc import convert_file # type: ignore
	with open(source / "docs/index.rst", "wt") as idx_f:
	idx_f.write(
	convert_file(source_file=source / "docs/index.md", format="md", to="rst")
	+ "\n\n.. toctree::"
	+ "\n :hidden:"
	+ "\n :maxdepth: 2"
	+ "\n :caption: Reference:"
	+ "\n :glob:\n"
	+ "\n reference/*\n"
	)
	# generate RST reference documentation
	_generate_reference(source / "validators/__init__.py", refs_path, "rst")
	rc = 0
	if not only_man:
	# build sphinx web pages as subprocess
	web_build = run(("sphinx-build", "docs", "docs/_build/web"), capture_output=True)
	print(web_build.stderr.decode() + web_build.stdout.decode())
	rc = web_build.returncode
	if not only_web:
	# build sphinx man pages as subprocess
	man_build = run(
	("sphinx-build", "-b", "man", "docs", "docs/_build/man"), capture_output=True
	)
	print(man_build.stderr.decode() + man_build.stdout.decode())
	copy(source / "docs/_build/man/validators.1", source / "docs/validators.1")
	print(f"Man page copied to: {source / 'docs/validators.1'}")
	rc = man_build.returncode if rc == 0 else rc
	return rc

[kloczek]

Moment .. so why calling that script is not added to the conf.py and or why it is not part of that file? 🤔

[yozachar]

How, can you please explain?

[kloczek]

conf.py it is just python script and it can contain any code as well.

[yozachar]

But, it's supposed to be a configuration file for Sphinx.

Does it have some sort of hooks to run other scripts as well? Any reference would be helpful.

[kloczek]

It is only impession that it is config files.
Please have a look on conf.py files in other projects.
You can ffjnd many examples of preproessing files which are used by sphinx to render document in one of the requested formats.
Really .. it is possible to integrate that external script straight into conf.py.