Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix documentation build #50913

Merged
merged 10 commits into from Dec 19, 2018

Conversation

Projects
None yet
3 participants
@bdrung
Copy link
Contributor

commented Dec 19, 2018

This pull requests repairs building the documentation with Python 3.7 on Debian unstable (first six commits). Then it addresses pylint issues in doc/conf.py followed by some refactoring there to improve code quality. See the individual commits for details.

cbosdo and others added some commits Jun 18, 2018

Fix sphynx error about tornado.version_info
This is fix errors like the following when building docs:

WARNING: autodoc: failed to import module 'salt.states.saltmod'; the following exception was raised:
Traceback (most recent call last):
  File "/public/src/salt/env/lib/python3.6/site-packages/sphinx/ext/autodoc/importer.py", line 140, in import_module
    __import__(modname)
  File "/public/src/salt/salt/states/saltmod.py", line 36, in <module>
    import salt.output
  File "/public/src/salt/salt/output/__init__.py", line 19, in <module>
    import salt.loader
  File "/public/src/salt/salt/loader.py", line 23, in <module>
    import salt.config
  File "/public/src/salt/salt/config/__init__.py", line 27, in <module>
    import salt.utils.network
  File "/public/src/salt/salt/utils/network.py", line 35, in <module>
    import salt.utils.zeromq
  File "/public/src/salt/salt/utils/zeromq.py", line 39, in <module>
    if tornado.version_info < (5,):
TypeError: '<' not supported between instances of 'Mock' and 'tuple'

(cherry picked from commit 8199700)
Do not mock json when building the documentation
Building the documentation on Debian unstable with Python 3.7 fails:

```
debian-unstable$ HTML_THEME=saltstack make -C doc html
make: Entering directory 'doc'
No need to update translations. Skipping...
sphinx-build -b html -d _build/doctrees    . _build/html
Running Sphinx v1.7.9
loading translations [en]... done

Exception occurred:
  File "/usr/lib/python3/dist-packages/sphinx/util/jsonimpl.py", line 22, in <module>
    class SphinxJSONEncoder(json.JSONEncoder):
TypeError: __mro_entries__ must return a tuple
The full traceback has been saved in /tmp/sphinx-err-wzl9_n0k.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make: *** [Makefile:72: html] Error 2
make: Leaving directory 'doc'
debian-unstable$ cat /tmp/sphinx-err-wzl9_n0k.log

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sphinx/cmdline.py", line 303, in main
    args.warningiserror, args.tags, args.verbosity, args.jobs)
  File "/usr/lib/python3/dist-packages/sphinx/application.py", line 187, in __init__
    self.setup_extension(extension)
  File "/usr/lib/python3/dist-packages/sphinx/application.py", line 411, in setup_extension
    self.registry.load_extension(self, extname)
  File "/usr/lib/python3/dist-packages/sphinx/registry.py", line 315, in load_extension
    mod = __import__(extname, None, None, ['setup'])
  File "/usr/lib/python3/dist-packages/sphinx/builders/applehelp.py", line 20, in <module>
    from sphinx.builders.html import StandaloneHTMLBuilder
  File "/usr/lib/python3/dist-packages/sphinx/builders/html.py", line 43, in <module>
    from sphinx.util import jsonimpl, logging, status_iterator
  File "/usr/lib/python3/dist-packages/sphinx/util/jsonimpl.py", line 22, in <module>
    class SphinxJSONEncoder(json.JSONEncoder):
TypeError: __mro_entries__ must return a tuple
```

The json module is a standard module. I is always present. So do not mock it.

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
doc: Do not mock non-existing __mro_entries__ attribute
Building the documentation with the Python 3.7 version of sphinx fails:

```
$ make -C doc html SPHINXBUILD="python3.7 /usr/bin/sphinx-build"
[...]
WARNING: autodoc: failed to import module 'salt.states.pkg'; the following exception was raised:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sphinx/ext/autodoc/importer.py", line 152, in import_module
    __import__(modname)
  File "salt/states/pkg.py", line 84, in <module>
    import salt.utils.pkg
  File "salt/utils/pkg/__init__.py", line 13, in <module>
    import salt.utils.data
  File "salt/utils/data.py", line 23, in <module>
    import salt.utils.yaml
  File "salt/utils/yaml.py", line 9, in <module>
    from salt.utils.yamldumper import *
  File "salt/utils/yamldumper.py", line 34, in <module>
    class IndentMixin(Dumper):
TypeError: __mro_entries__ must return a tuple
```

Instead of returning a Mock object for the `__mro_entries__` attribute, raise
an AttributeError instead.

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

@bdrung bdrung force-pushed the bdrung:fix-doc-build branch from 7aa1a26 to 104169f Dec 19, 2018

bdrung added some commits Dec 19, 2018

doc: Fix iterating over the Mock object in Python 3
Building the documentation for salt.modules.snapper fails with Python 3:

```
$ make -C doc html
[...]
WARNING: [autosummary] failed to import 'salt.modules.snapper': no module named salt.modules.snapper
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 109 source files that are out of date
updating environment: [config changed] 1608 added, 17 changed, 0 removed
reading sources... [100%] topics/yaml/index
doc/ref/modules/all/index.rst:20: WARNING: failed to import snapper
doc/ref/modules/all/index.rst:20: WARNING: toctree references unknown document 'ref/modules/all/snapper'
WARNING: autodoc: failed to import module 'salt.modules.snapper'; the following exception was raised:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sphinx/ext/autodoc/importer.py", line 152, in import_module
    __import__(modname)
  File "salt/modules/snapper.py", line 72, in <module>
    if SNAPPER_DBUS_OBJECT in bus.list_activatable_names():
TypeError: argument of type 'Mock' is not iterable
```

Fix the Mock object to be iterable (as it was intended to).

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
doc: Mark preamble variable as raw string
Importing `doc/conf.py` fails with Python 3.7:

```
>>> from conf import Mock
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "doc/conf.py", line 454
    ''',
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 5-6: truncated \uXXXX escape
```

Therefore mark the string as raw due to the backslashes.

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
doc: Replace \xa0 (non-breaking space) by spaces
Creating the documentation produces a warning:

```
doc/topics/targeting/index.rst:90: WARNING: Could not lex literal_block as "yaml". Highlighting skipped.
```

Therefore replace the non-breaking spaces \xa0 by normal spaces.

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
doc: Fix reference to salt executors
sphinx shows a warning:

doc/topics/development/modules/index.rst:133: WARNING: undefined label:
all-salt_executors (if the link has no caption the label must precede a section
header)

The executors reference is called all-salt.executors (with a dot instead of an
underscore).

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
doc: Address pylint issues in conf.py
pylint finds issues in doc/conf.py:

```
pylint3 conf.py
************* Module conf
conf.py:226:0: C0330: Wrong hanging indentation (remove 4 spaces).
        os.pardir,  # salt itself (for autodoc)
    |   ^ (bad-continuation)
conf.py:227:0: C0330: Wrong hanging indentation (remove 4 spaces).
        '_ext',  # custom Sphinx extensions
    |   ^ (bad-continuation)
conf.py:374:0: C0301: Line too long (106/100) (line-too-long)
conf.py:442:0: C0330: Wrong hanging indentation (add 2 spaces).
  ('contents', 'Salt.tex', 'Salt Documentation', 'SaltStack, Inc.', 'manual'),
  ^ | (bad-continuation)
conf.py:485:0: C0301: Line too long (107/100) (line-too-long)
conf.py:27:0: W0613: Unused argument 'args' (unused-argument)
conf.py:27:0: W0613: Unused argument 'kwargs' (unused-argument)
conf.py:58:4: R0201: Method could be a function (no-self-use)
conf.py:198:8: R1705: Unnecessary "else" after "return" (no-else-return)
conf.py:197:0: W0613: Unused argument 'iargs' (unused-argument)
conf.py:197:0: W0613: Unused argument 'ikwargs' (unused-argument)
conf.py:187:0: W0613: Unused argument 'oargs' (unused-argument)
conf.py:187:0: W0613: Unused argument 'okwargs' (unused-argument)
conf.py:533:0: R0913: Too many arguments (6/5) (too-many-arguments)
conf.py:533:25: W0613: Unused argument 'app' (unused-argument)
conf.py:533:30: W0613: Unused argument 'what' (unused-argument)
conf.py:533:47: W0613: Unused argument 'skip' (unused-argument)
conf.py:533:53: W0613: Unused argument 'options' (unused-argument)
conf.py:542:7: W0621: Redefining name 'path' from outer scope (line 230) (redefined-outer-name)
conf.py:6:0: W0611: Unused import functools (unused-import)
conf.py:310:4: W0611: Unused import sphinxcontrib.spelling (unused-import)
```

Address those or silince pylint for wanted behavior.

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
doc: Indroduce MOCK_MODULES_MAPPING
Some mocks needs some attributes defined (instead of having them returning a
mock object). Introduce a MOCK_MODULES_MAPPING mapping that defines these
attributes. This gets rid of a special handling for psutil.

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
doc: Move mock_decorator_with_params up
Refactor conf.py and move mock_decorator_with_params for the following commit.

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

@bdrung bdrung force-pushed the bdrung:fix-doc-build branch from 104169f to ecdb452 Dec 19, 2018

@garethgreenaway garethgreenaway requested a review from saltstack/team-core Dec 19, 2018

@terminalmage terminalmage merged commit 59df6b4 into saltstack:2018.3 Dec 19, 2018

10 checks passed

WIP Ready for review
Details
continuous-integration/jenkins/pr-merge This commit looks good
Details
jenkins/pr/docs The docs job has passed
Details
jenkins/pr/lint Python lint test has passed
Details
jenkins/pr/py2-centos-7 The py2-centos-7 job has passed
Details
jenkins/pr/py2-ubuntu-1604 The py2-ubuntu-1604 job has passed
Details
jenkins/pr/py2-windows-2016 The py2-windows-2016 job has passed
Details
jenkins/pr/py3-centos-7 The py3-centos-7 job has passed
Details
jenkins/pr/py3-ubuntu-1604 The py3-ubuntu-1604 job has passed
Details
jenkins/pr/py3-windows-2016 The py3-windows-2016 job has passed
Details

@bdrung bdrung deleted the bdrung:fix-doc-build branch Dec 19, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.