Warnings in plot directive aren't treated as such by Sphinx

[astrofrog]

Bug report

Bug summary

sphinx-build has a -W option to turn warnings into errors. However, this doesn't seem to have any effect on warnings emitted inside plot directives.

Code for reproduction

To reproduce, create an empty directory then add a conf.py file with:

import matplotlib.sphinxext.plot_directive
extensions = [matplotlib.sphinxext.plot_directive.__name__]

and an index.rst file with:

.. plot::

    import warnings
    warnings.warn("A warning occurred")

Then run:

sphinx-build -b html -d _build/doctrees   . _build/html -W

The -W flag should turn warnings into errors, but this is not the case.

Actual outcome

Running Sphinx v2.3.0+
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: [new config] 1 added, 0 changed, 0 removed
<string>:2: UserWarning: A warning occurred                                                                                                                                                                                                                                          

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index                                                                                                                                                                                                                                                       
generating indices...  genindexdone
writing additional pages...  searchdone
copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded.

Expected outcome

The build should fail with an error.

Matplotlib version

• Operating system: Ubuntu 19.10
• Matplotlib version: 3.2.0rc1 (from pip)
• Python version: 3.7.3

[anntzer]

See sphinx-doc/sphinx#9142: I guess we can implement this on our side, but it may make more sense as a feature request on sphinx's side?

[github-actions]

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

[github-actions]

Good first issue - notes for new contributors

This issue is suited to new contributors because it does not require understanding of the Matplotlib internals. To get started, please see our contributing guide.

We do not assign issues. Check the Development section in the sidebar for linked pull requests (PRs). If there are none, feel free to start working on it. If there is an open PR, please collaborate on the work by reviewing it rather than duplicating it in a competing PR.

If something is unclear, please reach out on any of our communication channels.

[tacaswell]

This is a good first issue because it has no API design choices, but hard because it will require understanding how the sphinx extension system works and possibly working upstream with sphinx to add functionality there to support converting warnings to errors.