Use antsibull sphinx extension #73170

felixfontein · 2021-01-09T22:11:24Z

SUMMARY

The Pygments lexer and part of the CSS is contained in the antsibull sphinx extension (https://github.com/ansible-community/antsibull/tree/main/sphinx_antsibull_ext), so there's no need to duplicate these parts in the docsite sources.

ISSUE TYPE

Docs Pull Request

COMPONENT NAME

docsite

ansibot · 2021-01-09T22:30:14Z

The test ansible-test sanity --test docs-build [explain] failed with the error:

Command "/usr/bin/python3.6 /root/ansible/test/sanity/code-smell/docs-build.py" returned exit status 1.
>>> Standard Error
Command 'make base_singlehtmldocs' failed with status code: 2
--> Standard Output
../../hacking/build-ansible.py collection-meta --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/ ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
../../hacking/build-ansible.py document-config --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ ../../lib/ansible/config/base.yml
mkdir -p rst/cli
../../hacking/build-ansible.py generate-man --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
../../hacking/build-ansible.py document-keywords --template-dir=../templates --output-dir=rst/reference_appendices/ ../../lib/ansible/keyword_desc.yml
../../hacking/build-ansible.py docs-build base -o rst ;\

../bin/testing_formatter.sh
CPUS=2 make -f Makefile.sphinx singlehtml
make[1]: Entering directory '/root/ansible/docs/docsite'
sphinx-build -M singlehtml "rst" "_build" -j 2 -n -w rst_warnings 
Running Sphinx v1.7.9
Makefile.sphinx:24: recipe for target 'singlehtml' failed
make[1]: Leaving directory '/root/ansible/docs/docsite'
Makefile:66: recipe for target 'base_singlehtmldocs' failed
--> Standard Error
ERROR:antsibull:func=write_rst:mod=antsibull.write_docs:nonfatal_errors=['Unable to normalize config: return due to: 1 validation error for PluginReturnSchema\nreturn -> _raw -> type\n  string does not match regex "^(bool|complex|dict|float|int|list|str)$" (type=value_error.str.regex; pattern=^(bool|complex|dict|float|int|list|str)$)']:plugin_name=ansible.builtin.config:plugin_type=lookup|ansible.builtin.config did not return correct RETURN or EXAMPLES.
ERROR:antsibull:func=write_rst:mod=antsibull.write_docs:nonfatal_errors=['Unable to normalize random_choice: return due to: 1 validation error for PluginReturnSchema\nreturn -> _raw -> type\n  string does not match regex "^(bool|complex|dict|float|int|list|str)$" (type=value_error.str.regex; pattern=^(bool|complex|dict|float|int|list|str)$)']:plugin_name=ansible.builtin.random_choice:plugin_type=lookup|ansible.builtin.random_choice did not return correct RETURN or EXAMPLES.
../bin/testing_formatter.sh: 38: ../bin/testing_formatter.sh: cannot open ../docsite/rst/dev_guide/testing/sanity/index.rst: No such file

Extension error:
Could not import extension pygments_lexer (exception: No module named 'pygments_lexer')
make[1]: *** [singlehtml] Error 2
make: *** [base_singlehtmldocs] Error 2

click here for bot help

ansibot · 2021-01-09T22:46:00Z

The test ansible-test sanity --test docs-build [explain] failed with the error:

Command "/usr/bin/python3.6 /root/ansible/test/sanity/code-smell/docs-build.py" returned exit status 1.
>>> Standard Error
Command 'make base_singlehtmldocs' failed with status code: 2
--> Standard Output
../../hacking/build-ansible.py collection-meta --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/ ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
../../hacking/build-ansible.py document-config --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ ../../lib/ansible/config/base.yml
mkdir -p rst/cli
../../hacking/build-ansible.py generate-man --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
../../hacking/build-ansible.py document-keywords --template-dir=../templates --output-dir=rst/reference_appendices/ ../../lib/ansible/keyword_desc.yml
../../hacking/build-ansible.py docs-build base -o rst ;\

../bin/testing_formatter.sh
CPUS=2 make -f Makefile.sphinx singlehtml
make[1]: Entering directory '/root/ansible/docs/docsite'
sphinx-build -M singlehtml "rst" "_build" -j 2 -n -w rst_warnings 
Running Sphinx v1.7.9
Makefile.sphinx:24: recipe for target 'singlehtml' failed
make[1]: Leaving directory '/root/ansible/docs/docsite'
Makefile:66: recipe for target 'base_singlehtmldocs' failed
--> Standard Error
ERROR:antsibull:func=write_rst:mod=antsibull.write_docs:nonfatal_errors=['Unable to normalize config: return due to: 1 validation error for PluginReturnSchema\nreturn -> _raw -> type\n  string does not match regex "^(bool|complex|dict|float|int|list|str)$" (type=value_error.str.regex; pattern=^(bool|complex|dict|float|int|list|str)$)']:plugin_name=ansible.builtin.config:plugin_type=lookup|ansible.builtin.config did not return correct RETURN or EXAMPLES.
ERROR:antsibull:func=write_rst:mod=antsibull.write_docs:nonfatal_errors=['Unable to normalize random_choice: return due to: 1 validation error for PluginReturnSchema\nreturn -> _raw -> type\n  string does not match regex "^(bool|complex|dict|float|int|list|str)$" (type=value_error.str.regex; pattern=^(bool|complex|dict|float|int|list|str)$)']:plugin_name=ansible.builtin.random_choice:plugin_type=lookup|ansible.builtin.random_choice did not return correct RETURN or EXAMPLES.
../bin/testing_formatter.sh: 38: ../bin/testing_formatter.sh: cannot open ../docsite/rst/dev_guide/testing/sanity/index.rst: No such file

Exception occurred:
  File "/usr/local/lib/python3.6/dist-packages/sphinx_antsibull_ext/assets.py", line 45, in setup_assets
    app.add_css_file(file)
AttributeError: 'Sphinx' object has no attribute 'add_css_file'
The full traceback has been saved in /tmp/sphinx-err-yzrz69z_.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make[1]: *** [singlehtml] Error 2
make: *** [base_singlehtmldocs] Error 2

click here for bot help

See ansible/ansible#73170 (comment) and sphinx-doc/sphinx@3afc72f

felixfontein · 2021-01-10T08:58:32Z

This is strange. Sphinx 2.1.2 (https://github.com/ansible/ansible/blob/devel/docs/docsite/requirements.txt#L5) already has add_css_file (https://github.com/sphinx-doc/sphinx/blob/v2.1.2/sphinx/application.py#L960). Maybe an even older version of Sphinx is used here? It must have been before version 1.8.0, since add_css_file was added then (sphinx-doc/sphinx@3afc72f).

This should get fixed by ansible-community/antsibull#233, but maybe the very old Sphinx version used for docsite testing also needs to be updated :)

See ansible/ansible#73170 (comment) and sphinx-doc/sphinx@3afc72f

abadger · 2021-01-11T18:15:08Z

CC @samccann about the older Sphinx requirement

* Add compatibility hack for old Sphinx versions. See ansible/ansible#73170 (comment) and sphinx-doc/sphinx@3afc72f

webknjaz · 2021-01-11T18:54:09Z

@abadger FYI there's a PR allowing better sphinx: #73176.

felixfontein · 2021-01-11T19:38:18Z

In any case, whatever comes first (#73176 merged or new antsibull release), this issue should be fixed soon :)

felixfontein · 2021-01-15T19:26:19Z

Restarting tests (since rstcheck / sphinx were updated).

ansibot · 2021-01-15T19:38:29Z

The test ansible-test sanity --test docs-build [explain] failed with the error:

Command "/usr/bin/python3.6 /root/ansible/test/sanity/code-smell/docs-build.py" returned exit status 1.
>>> Standard Error
Command 'make base_singlehtmldocs' failed with status code: 2
--> Standard Output
../../hacking/build-ansible.py collection-meta --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/ ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
../../hacking/build-ansible.py document-config --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ ../../lib/ansible/config/base.yml
mkdir -p rst/cli
../../hacking/build-ansible.py generate-man --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
../../hacking/build-ansible.py document-keywords --template-dir=../templates --output-dir=rst/reference_appendices/ ../../lib/ansible/keyword_desc.yml
../../hacking/build-ansible.py docs-build base -o rst ;\

../bin/testing_formatter.sh
CPUS=2 make -f Makefile.sphinx singlehtml
make[1]: Entering directory '/root/ansible/docs/docsite'
sphinx-build -M singlehtml "rst" "_build" -j 2 -n -w rst_warnings 
Running Sphinx v1.7.9
Makefile.sphinx:24: recipe for target 'singlehtml' failed
make[1]: Leaving directory '/root/ansible/docs/docsite'
Makefile:66: recipe for target 'base_singlehtmldocs' failed
--> Standard Error
ERROR:antsibull:func=write_rst:mod=antsibull.write_docs:nonfatal_errors=['Unable to normalize config: return due to: 1 validation error for PluginReturnSchema\nreturn -> _raw -> type\n  string does not match regex "^(bool|complex|dict|float|int|list|str)$" (type=value_error.str.regex; pattern=^(bool|complex|dict|float|int|list|str)$)']:plugin_name=ansible.builtin.config:plugin_type=lookup|ansible.builtin.config did not return correct RETURN or EXAMPLES.
ERROR:antsibull:func=write_rst:mod=antsibull.write_docs:nonfatal_errors=['Unable to normalize random_choice: return due to: 1 validation error for PluginReturnSchema\nreturn -> _raw -> type\n  string does not match regex "^(bool|complex|dict|float|int|list|str)$" (type=value_error.str.regex; pattern=^(bool|complex|dict|float|int|list|str)$)']:plugin_name=ansible.builtin.random_choice:plugin_type=lookup|ansible.builtin.random_choice did not return correct RETURN or EXAMPLES.
../bin/testing_formatter.sh: 38: ../bin/testing_formatter.sh: cannot open ../docsite/rst/dev_guide/testing/sanity/index.rst: No such file

Exception occurred:
  File "/usr/local/lib/python3.6/dist-packages/sphinx_antsibull_ext/assets.py", line 45, in setup_assets
    app.add_css_file(file)
AttributeError: 'Sphinx' object has no attribute 'add_css_file'
The full traceback has been saved in /tmp/sphinx-err-7t4im9y9.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make[1]: *** [singlehtml] Error 2
make: *** [base_singlehtmldocs] Error 2

click here for bot help

ansibot · 2021-01-15T20:10:10Z

The test ansible-test sanity --test docs-build [explain] failed with the error:

Command "/usr/bin/python3.6 /root/ansible/test/sanity/code-smell/docs-build.py" returned exit status 1.
>>> Standard Error
Command 'make base_singlehtmldocs' failed with status code: 2
--> Standard Output
../../hacking/build-ansible.py collection-meta --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/ ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
../../hacking/build-ansible.py document-config --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ ../../lib/ansible/config/base.yml
mkdir -p rst/cli
../../hacking/build-ansible.py generate-man --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
../../hacking/build-ansible.py document-keywords --template-dir=../templates --output-dir=rst/reference_appendices/ ../../lib/ansible/keyword_desc.yml
../../hacking/build-ansible.py docs-build base -o rst ;\

../bin/testing_formatter.sh
CPUS=2 make -f Makefile.sphinx singlehtml
make[1]: Entering directory '/root/ansible/docs/docsite'
sphinx-build -M singlehtml "rst" "_build" -j 2 -n -w rst_warnings 
Running Sphinx v1.7.9
Makefile.sphinx:24: recipe for target 'singlehtml' failed
make[1]: Leaving directory '/root/ansible/docs/docsite'
Makefile:66: recipe for target 'base_singlehtmldocs' failed
--> Standard Error
ERROR:antsibull:func=write_rst:mod=antsibull.write_docs:nonfatal_errors=['Unable to normalize config: return due to: 1 validation error for PluginReturnSchema\nreturn -> _raw -> type\n  string does not match regex "^(bool|complex|dict|float|int|list|str)$" (type=value_error.str.regex; pattern=^(bool|complex|dict|float|int|list|str)$)']:plugin_name=ansible.builtin.config:plugin_type=lookup|ansible.builtin.config did not return correct RETURN or EXAMPLES.
ERROR:antsibull:func=write_rst:mod=antsibull.write_docs:nonfatal_errors=['Unable to normalize random_choice: return due to: 1 validation error for PluginReturnSchema\nreturn -> _raw -> type\n  string does not match regex "^(bool|complex|dict|float|int|list|str)$" (type=value_error.str.regex; pattern=^(bool|complex|dict|float|int|list|str)$)']:plugin_name=ansible.builtin.random_choice:plugin_type=lookup|ansible.builtin.random_choice did not return correct RETURN or EXAMPLES.
../bin/testing_formatter.sh: 38: ../bin/testing_formatter.sh: cannot open ../docsite/rst/dev_guide/testing/sanity/index.rst: No such file

Exception occurred:
  File "/usr/local/lib/python3.6/dist-packages/sphinx_antsibull_ext/assets.py", line 45, in setup_assets
    app.add_css_file(file)
AttributeError: 'Sphinx' object has no attribute 'add_css_file'
The full traceback has been saved in /tmp/sphinx-err-008tzu2v.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make[1]: *** [singlehtml] Error 2
make: *** [base_singlehtmldocs] Error 2

click here for bot help

felixfontein · 2021-01-15T20:23:06Z

@samccann for some reason, the sanity test is still using Sphinx v1.7.9.

felixfontein · 2021-01-28T19:40:09Z

/rebuild

felixfontein · 2021-01-28T19:44:30Z

@mattclay about the requirement constraings change: is this change OK? I could also change it to >=1.8.0,<=2.1.2. (Alternatively this can also be fixed by creating a new version of the default test docker container which has a newer version of Sphinx, and by not modifying the constraints.)

docs/docsite/sphinx_conf/2.10_conf.py

docs/docsite/_static/core.css

outdated

felixfontein · 2021-06-19T12:11:38Z

Marking as WIP since two more things should better happen first:

~~new antsibull release with the reduced extension;~~
upgrade to the latest ansible Sphinx theme.

I think it's best to continue with this once #74956 has been merged.

webknjaz · 2021-06-23T11:01:34Z

@felixfontein should this be unWIPed now?

felixfontein · 2021-06-23T13:21:03Z

@webknjaz yep it should. Done, thanks!

felixfontein · 2021-06-23T19:31:52Z

@abadger @webknjaz @mattclay @samccann @acozine and everyone else who was involved, thanks for reviewing, testing, ..., and merging!

* Use antsibull sphinx extension. * Require antsibull 0.34.0. (cherry picked from commit 2c0f050)

felixfontein added a commit to felixfontein/antsibull that referenced this pull request Jan 10, 2021

Add compatibility hack for old Sphinx versions.

36c39a1

See ansible/ansible#73170 (comment) and sphinx-doc/sphinx@3afc72f

felixfontein mentioned this pull request Jan 10, 2021

Add compatibility hack for old Sphinx versions ansible-community/antsibull#233

Merged

felixfontein added a commit to felixfontein/antsibull that referenced this pull request Jan 11, 2021

Add compatibility hack for old Sphinx versions.

b1ade76

See ansible/ansible#73170 (comment) and sphinx-doc/sphinx@3afc72f

abadger pushed a commit to ansible-community/antsibull that referenced this pull request Jan 11, 2021

Add compatibility hack for old Sphinx versions (#233)

21c4fa1

* Add compatibility hack for old Sphinx versions. See ansible/ansible#73170 (comment) and sphinx-doc/sphinx@3afc72f

sivel removed the needs_triage Needs a first human triage before being processed. label Jan 12, 2021

felixfontein closed this Jan 15, 2021

felixfontein reopened this Jan 15, 2021

felixfontein force-pushed the use-antsibull-extension branch from 718c2f0 to 0229826 Compare January 15, 2021 19:59

ansibot added the needs_ci This PR requires CI testing to be performed. Please close and re-open this PR to trigger CI. label Jan 15, 2021

ansibot removed the needs_ci This PR requires CI testing to be performed. Please close and re-open this PR to trigger CI. label Jan 15, 2021

ansibot added the test This PR relates to tests. label Jan 18, 2021

ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Jan 27, 2021

felixfontein changed the title ~~[WIP] Use antsibull sphinx extension~~ Use antsibull sphinx extension Jan 28, 2021

webknjaz reviewed Jun 19, 2021

View reviewed changes

docs/docsite/sphinx_conf/2.10_conf.py Outdated Show resolved Hide resolved

webknjaz reviewed Jun 19, 2021

View reviewed changes

docs/docsite/_static/core.css Outdated Show resolved Hide resolved

ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Jun 19, 2021

felixfontein mentioned this pull request Jun 19, 2021

Remove CSS that lives in the antsibull Sphinx extension ansible-community/sphinx_ansible_theme#34

Merged

ansibot added core_review In order to be merged, this PR must follow the core review workflow. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Jun 19, 2021

felixfontein force-pushed the use-antsibull-extension branch from a757096 to fb35de6 Compare June 19, 2021 12:07

felixfontein changed the title ~~Use antsibull sphinx extension~~ [WIP] Use antsibull sphinx extension Jun 19, 2021

felixfontein mentioned this pull request Jun 20, 2021

Bump sphinx_ansible_theme to v0.7.0+ #75059

Merged

felixfontein added 2 commits June 22, 2021 21:22

Use antsibull sphinx extension.

3418f9d

Require antsibull 0.34.0.

aa58de3

felixfontein force-pushed the use-antsibull-extension branch from fb35de6 to aa58de3 Compare June 22, 2021 19:22

webknjaz approved these changes Jun 23, 2021

View reviewed changes

felixfontein changed the title ~~[WIP] Use antsibull sphinx extension~~ Use antsibull sphinx extension Jun 23, 2021

ansibot added core_review In order to be merged, this PR must follow the core review workflow. and removed WIP This issue/PR is a work in progress. Nevertheless it was shared for getting input from peers. labels Jun 23, 2021

acozine merged commit 2c0f050 into ansible:devel Jun 23, 2021

felixfontein deleted the use-antsibull-extension branch June 23, 2021 19:30

felixfontein added a commit to felixfontein/ansible that referenced this pull request Jun 23, 2021

Use antsibull sphinx extension (ansible#73170)

6913618

* Use antsibull sphinx extension. * Require antsibull 0.34.0. (cherry picked from commit 2c0f050)

felixfontein mentioned this pull request Jun 23, 2021

[2.11] Use antsibull sphinx extension #75100

Merged

acozine pushed a commit that referenced this pull request Jun 29, 2021

Use antsibull sphinx extension (#73170) (#75100)

49950fa

* Use antsibull sphinx extension. * Require antsibull 0.34.0. (cherry picked from commit 2c0f050)

ansible locked and limited conversation to collaborators Jul 21, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use antsibull sphinx extension #73170

Use antsibull sphinx extension #73170

felixfontein commented Jan 9, 2021

ansibot commented Jan 9, 2021

ansibot commented Jan 9, 2021

felixfontein commented Jan 10, 2021

abadger commented Jan 11, 2021

webknjaz commented Jan 11, 2021

felixfontein commented Jan 11, 2021

felixfontein commented Jan 15, 2021

ansibot commented Jan 15, 2021

ansibot commented Jan 15, 2021

felixfontein commented Jan 15, 2021

felixfontein commented Jan 28, 2021

felixfontein commented Jan 28, 2021

felixfontein commented Jun 19, 2021 •

edited

webknjaz commented Jun 23, 2021

felixfontein commented Jun 23, 2021

felixfontein commented Jun 23, 2021

Use antsibull sphinx extension #73170

Use antsibull sphinx extension #73170

Conversation

felixfontein commented Jan 9, 2021

SUMMARY

ISSUE TYPE

COMPONENT NAME

ansibot commented Jan 9, 2021

ansibot commented Jan 9, 2021

felixfontein commented Jan 10, 2021

abadger commented Jan 11, 2021

webknjaz commented Jan 11, 2021

felixfontein commented Jan 11, 2021

felixfontein commented Jan 15, 2021

ansibot commented Jan 15, 2021

ansibot commented Jan 15, 2021

felixfontein commented Jan 15, 2021

felixfontein commented Jan 28, 2021

felixfontein commented Jan 28, 2021

felixfontein commented Jun 19, 2021 • edited

webknjaz commented Jun 23, 2021

felixfontein commented Jun 23, 2021

felixfontein commented Jun 23, 2021

felixfontein commented Jun 19, 2021 •

edited