Regression with 10db540 and custom directive incorrectly warning about circular inclusion

[ianw]

Describe the bug

Hello,

Via bisection I have found that 10db540 causes a regression with zuul-sphinx, incorrectly warning about circular inclusion.

This plugin looks as README RST files in subdirectories and builds overview pages. It seems somehow this has started causing docutils to show a circular reference warning.

There has been some discussion of this in https://sourceforge.net/p/docutils/bugs/459/ but it still remains unclear what is happening.

Running the build with Sphinx 5.3 produces the output

Warning, treated as error:
/home/iwienand/tmp/sphinx/zuul-jobs/doc/source/container-roles.rst:3:circular inclusion in "include" directive: roles/build-docker-image/common.rst < roles/build-docker-image/common.rst < roles/build-container-image/common.rst < roles/build-container-image/README.rst

These are not recursively included, what is actually happening is that the same file (common.rst) is included from multiple README files, which themselves are included (serially) in a parent document. e.g.

container-roles.rst
 .. include:: build-container-image/README.rst
   .. include:: build-container-image/common.rst
 .. include:: build-docker-image/README.rst
   .. include:: build-container-image/common.rst

It is the second inclusion of common.rst here that triggers this. Why this change affects this, I frankly have no idea!

This is some sort of weird interaction with the docutils 0.17 releases. If I downgrade to docutils 0.16, the problem does not occur, and if I force upgrade to 0.18 it also does not occur. However, we are stuck on 0.17 because sphinx_rtd_theme is currently capped there.

zuul-sphinx is available at the above, but it all boils down to just the single file https://opendev.org/zuul/zuul-sphinx/src/branch/master/zuul_sphinx/zuul.py. The file that is causing problems is https://opendev.org/zuul/zuul-jobs/raw/branch/master/doc/source/container-roles.rst

How to Reproduce

git clone https://opendev.org/zuul/zuul-jobs
cd zuul-jobs
# This project has pinned itself to 5.2.3 to avoid this issue.  Remove the pin
sed -i 's/^sphinx<.*//' ./doc/requirements.txt
tox -e docs

To replicate with the commit in question reverted

git clone https://opendev.org/zuul/zuul-jobs
git clone https://github.com/sphinx-doc/sphinx.git
pushd sphinx
git checkout v5.3.0
git revert --no-edit 10db540e9824a3747
popd
pushd zuul-jobs
# remove the pin of sphinx to 5.2.3
sed -i 's/^sphinx<.*//' ./doc/requirements.txt
# recreate tox environment
tox -re docs
# overwrite with the checkout version of sphinx
.tox/docs/bin/pip install -e ../sphinx
# rerun tox
tox -e docs

Environment Information

text Platform: linux; (Linux-5.19.6-200.fc36.x86_64-x86_64-with-glibc2.35) Python version: 3.10.6 (main, Aug 2 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)]) Python implementation: CPython Sphinx version: 5.3.0 Docutils version: 0.17.1 Jinja2 version: 3.1.2

Sphinx extensions

No response

Additional context

No response