Share doctest namespaces across a page, and honour directive options

[tony]

Summary

• Add namespace scoping to the doctest finder. Blocks that name a group (.. doctest:: intro, ```{doctest} intro) share one namespace, so a narrative page can build state across the prose that explains it (Doctest blocks in one document do not share a namespace, unlike pytest text files #83). Blocks that name no group keep their own namespace unless a project opts into a page-wide one.
• Fix four directive options the finder parsed and then discarded (Doctest directive options are parsed and then discarded #84): an inline # doctest: flag inside a directive, :options:, :skipif:, and :pyversion: — which raised InvalidVersion and aborted collection of the page.
• Fix node IDs that embedded the machine's absolute path, so a checked-in --deselect line resolves on another checkout and JUnit XML stops carrying a home directory (pytest node IDs embed an absolute path #85).
• Fix collection order. DocTest.__lt__ compares names, and names carry the block index as text, so any page past nine blocks ran page.md[10] before page.md[1].
• Fix a crash on a doctest block nested in a .. note::, a list item, or a block quote — docutils leaves such a node's line unset, and reading it as a number took the whole file down.
• Report a block that is skipped end to end. It collects as its own item and says which page and line it came from, instead of disappearing into the namespace it shared.

The default for a page that names no group is unchanged, and every existing example in libtmux, libvcs, tmuxp and cihai collects under the same node IDs and the same counts.

Changes by area

Namespaces

• src/doctest_docutils.py: group a page's blocks by namespace and merge each into one doctest.DocTest, spliced by file position so reported lines and the %03d gutter stay where each block alone would put them.
• src/pytest_doctest_docutils.py: --doctest-docutils-namespace-scope and the matching doctest_docutils_namespace_scope ini option, block (default) or document, resolved once so a misspelling stops the session rather than erroring per file.
• The scope reaches every entry point: DocutilsDocTestFinder(namespace_scope=…), testdocutils(…), and python -m doctest_docutils --namespace-scope.
• A block may name several groups, comma separated, and * stands for every group the page declares — how a page writes one .. testsetup:: for all of them.
• .. testsetup:: and .. testcleanup:: run before and after the rest of their group whatever order the page writes them in. Both render as comments, so authors move them out of a reader's way.

Directive options

• node["test"] is read, so an inline # doctest: flag survives a directive that trims it out of the rendered code — and <BLANKLINE> inside a directive now compares against the marker rather than a real blank line.
• node["options"] seeds each example's flags; the example's own inline flag still wins. A true :skipif: is the exception: it is a gate, and an inline -SKIP cannot reopen it.
• node["skipif"] is evaluated, and a true condition marks the block SKIP rather than dropping it, so it reports like :options: +SKIP.
• :pyversion: passes its arguments in the order the local signature declares, and a malformed specifier stays a reporter warning.

Reporting

• Test names come from the page's base name; the full path stays as DocTest.filename so failures still resolve.
• A namespace is laid out in page order but run in phase order, so a .. testcleanup:: written above its group still runs last while every example still reports the line it actually sits on. Anchoring the merged text on the run order instead reported examples against whichever block came first in it, and could point past the end of the file.
• A wholly skipped item is marked at collection, so its fixtures do not set up for a test that runs nothing, and its reason names the page and line instead of pytest's own file.
• Logging is lazy %s with doctest_source_file and doctest_block_type, replacing a pprint dump of the test list, source string, globals and seen-map on every document.

Dependencies

• pyproject.toml: pytest-xdist in the dev and testing groups, so the test that pins distributed behaviour runs rather than skipping.

Design decisions

Merge a namespace into one test, rather than sharing a live globals dict. Sphinx reassigns test.globs and passes clear_globs=False; that shape breaks under process-parallel distribution, which libtmux's CI already runs over its docs/. A state-building page passes serially and fails with NameError under pytest -n 2 when items share a dict. One test per namespace has nothing to share across items.

Keep the per-block default. Merging also merges fixture lifetime: one item means one function-scoped fixture setup for the whole namespace. Switching the default to page-wide took libtmux from green to four failures and libvcs to two, all of them pages whose blocks each expected a fresh server, session or example_git_repo. Sharing is therefore something a page or a project asks for.

:options: are defaults; :skipif: is a gate. An example's own # doctest: -SKIP overrides the directive's :options:, which is what Sphinx does. It cannot override a true :skipif:, because Sphinx drops a gated block before its source is ever read — and an example that could reopen one would run on exactly the interpreter or platform the condition named.

A block skipped end to end leaves its namespace. It binds nothing the other blocks could read, so nothing is lost by collecting it separately — and it regains the SKIPPED line, the count, the JUnit entry and the node ID that merging had cost it. A block only partly skipped still has work to do and stays put.

Breaking changes

Node IDs no longer carry an absolute path. page.md::/home/you/docs/page.md[0] becomes page.md::page.md[0]. Any checked-in --deselect or -k written against the old form needs updating — though the old form only ever matched on the machine that produced it.

Two blocks naming one group collect as one item. They previously collected as two items sharing a single node ID, page.rst::shared, which no selector could tell apart.

A gated block now appears in the run. :skipif: True used to remove the block during collection; it now collects and reports SKIPPED, so a page carrying one gains an item.

CHANGES is deliberately untouched — this project writes changelog entries at release time, not from a feature branch.

It also ships #89's opt-in answer to that trade: doctest_docutils_namespace_items = per-block keeps a node id for every block of a shared page, handing the blocks one globals mapping rather than merging them into a single test. It is off by default, and a run that never sets it is byte-identical to before the setting existed.

A live mapping is a Python object, so it neither crosses a worker process nor survives an item being run twice. Where the run named no scheduler, one is filled in that keeps each page whole while everything else still spreads; a scheduler the run asked for by name that would split a page is refused, naming the page. A block that runs a second time — under --reruns, or anything else that repeats an item — is refused rather than trusted, because it would otherwise run against the globals it already changed and report an expectation that only came true on the retry as a pass.

Verification

The lexicographic sort is gone — expect no matches:

$ rg -c 'tests\.sort\(\)' src/

No line number is read as a string — expect no matches:

$ rg -c 'int\(source_lines' src/

No f-strings in log calls — expect no matches:

$ rg -c 'logger\.(debug|info|warning|error)\(f"' src/

The untrimmed source is what gets parsed — expect one match:

$ rg -n 'node\.get\("test"\)' src/

Test plan

• uv run pytest --reruns 0 — full suite green
• uv run ruff check . and uv run ruff format . — clean
• uv run mypy src tests — clean (this project configures mypy; ty is not a declared dependency)
• just build-docs — builds with no new warning and no broken cross-reference
• test_document_scope_survives_xdist — a page-wide namespace passes under pytest -n 2
• test_a_failing_block_still_fails_beside_a_skipped_one — a gate never turns a broken page green, under pytest and under python -m doctest_docutils
• test_lifting_a_gated_block_moves_no_reported_line — a skipped block's removal does not shift the lines its neighbours report
• test_merging_moves_no_reported_line and test_merged_examples_keep_their_gutter — failure locations match the unmerged page
• test_a_group_survives_an_include — a group split across .. include:: collects and runs
• Downstream: libtmux, libvcs, tmuxp and cihai run their docs/ and README.md with this branch on PYTHONPATH — zero failures, and --collect-only node IDs byte-identical to the base

[tony]

Code review

Found 1 issue:

1. Merged tests anchor their line numbers on phase order rather than document order, so a page that writes .. testsetup:: or .. testcleanup:: away from its group reports wrong failure lines — sometimes past the end of the file. _merge_blocks takes origin from blocks[0], but _find passes blocks as testsetup, test, testcleanup, so blocks[0] is not the block that appears first on the page. This is the exact layout the phase-ordering change exists to support, and which the how-to now tells authors to use ("you can move them out of a reader's way"). test_a_group_runs_setup_first_and_cleanup_last only asserts the run passes, so it does not catch this.

gp-libs/src/doctest_docutils.py

Lines 579 to 581 in c2fc2e3

	"""
	origin = blocks[0].lineno or 0
	lines: list[str] = []

The call site that supplies phase order:

gp-libs/src/doctest_docutils.py

Lines 1135 to 1148 in c2fc2e3

	for namespace, phases in namespaces.items():
	in_phase_order = [
	*phases["testsetup"],
	*phases["test"],
	*phases["testcleanup"],
	]
	kept, lifted = _split_skipped_blocks(in_phase_order)
	anchored.append(
	(
	# Every block anchors its namespace, lifted or not, so
	# lifting the first one cannot let another namespace
	# declared below it collect first.
	min(held.position for held in in_phase_order),
	# Ties with a block this namespace lifted break toward the

Reproduced on a 15-line page with .. testcleanup:: demo at line 4, .. doctest:: demo at line 8 (prompt on line 10) and .. testsetup:: demo at line 13: the failure reports File "page.rst", line 15, in demo for the prompt on line 10, and a third example computes line 17.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[codecov]

Codecov Report

❌ Patch coverage is 93.52679% with 87 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.38%. Comparing base (baf73b4) to head (8a25122).

Files with missing lines	Patch %	Lines
src/doctest_docutils.py	75.78%	62 Missing ⚠️
src/pytest_doctest_docutils.py	80.76%	25 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master      #87       +/-   ##
===========================================
+ Coverage   76.68%   89.38%   +12.69%     
===========================================
  Files          15       15               
  Lines        1025     2326     +1301     
===========================================
+ Hits          786     2079     +1293     
- Misses        239      247        +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[tony]

Code review

Found 1 issue:

1. _merge_blocks still documents its blocks parameter as shifting example.lineno in place, and cites that as the reason no block may be merged twice while keep holds it. c860165 rewrote the loop to shift a copy.copy(example) carrying its own dict(example.options) precisely so a block can be merged twice — into a second group it names, or on its own once lifted. That commit updated the Returns section and the inline comment at the loop but left this paragraph, which now states the causality backwards: keep exclusion is no longer what prevents double-shifting, the copy is. test_merging_reads_its_blocks_rather_than_consuming_them pins the new behavior, so the code is correct and only the docstring disagrees.

gp-libs/src/doctest_docutils.py

Lines 597 to 602 in e51fad6

	Blocks of one namespace, each parsed on its own, in the order they run.
	Each kept block's ``example.lineno`` is shifted **in place**, so no
	block may be merged twice while `keep` holds it — which is why
	:meth:`DocutilsDocTestFinder._find` leaves a lifted block out of `keep`
	before merging that block on its own.
	name : str

[tony]

Code review

Found 1 issue:

1. _worker_count diverges from the pytest-xdist expansion it reproduces, and the divergence can silently disable the per-block guard. Upstream builds a list (xspeclist.extend([spec] * num)), so a negative multiplier contributes zero specs; this sums the integer instead, so it contributes a negative number. --tx -1*popen --tx 2*popen counts 1 here against xdist's 2, workers < 2 short-circuits, and the run proceeds on two workers with no page-level scheduler and no refusal. The failure direction is permissive: a namespace splits across processes silently, which is the case pytest_xdist_node_collection_finished exists to stop. Clamping each spec's contribution at zero matches parse_tx_spec_config.

gp-libs/src/pytest_doctest_docutils.py

Lines 338 to 345 in 55115db

	total = 0
	for spec in specs:
	count, star, _ = spec.partition("*")
	try:
	total += int(count) if star else 1
	except ValueError:
	total += 1
	return total

[tony]

Measured this branch against libtmux by pinning it through [tool.uv.sources], then read the same problem in Sybil, Sphinx, stdlib doctest, pytest, xdoctest and pytest-asyncio to see how each one handles it. Three findings below: a sharp edge in per-block, the five-line fix another project already ships, and one place this branch is ahead of all of them.

Headline: namespace_items=per-block does what it claims. On libtmux's full docs/ tree, --doctest-docutils-namespace-scope=document --doctest-docutils-namespace-items=per-block gives 190 items, 0 failures — every node id preserved, sharing genuinely on. The default merged gives 31 items and 3 failures on the same tree. Splitting "who shares" from "does sharing cost the ids" into two settings is the right call, and it is the thing Sphinx does not offer.

A shared name outlives the fixture that made it

Under per-block the namespace persists across items but fixtures do not, so an object a block derives from a fixture and stashes under its own name keeps answering after its fixture has been finalized. Self-contained repro, no libtmux needed:

conftest.py

import pytest


class Resource:
    def __init__(self, n):
        self.n = n
        self.alive = True


COUNT = [0]


@pytest.fixture
def resource():
    COUNT[0] += 1
    r = Resource(COUNT[0])
    yield r
    r.alive = False


@pytest.fixture(autouse=True)
def _seed(doctest_namespace, resource):
    doctest_namespace["resource"] = resource

page.md

```
>>> saved = resource
>>> saved.n, saved.alive
(1, True)
```

```
>>> saved.n, saved.alive
(1, False)
>>> resource.n, resource.alive
(2, True)
>>> saved is resource
False
```

$ pytest page.md -q -p no:randomly --doctest-docutils-namespace-scope=document --doctest-docutils-namespace-items=per-block
2 passed

That page passes, which is the point — it asserts the surprise rather than tripping over it. The fixture's own name is rebound fresh each block (resource.n goes 1 to 2), but saved still resolves and reports alive as False. Under merged the same page fails, because one item means one fixture and the object stays alive.

In libtmux this is not hypothetical. A window carried across blocks answers w.window_name correctly from a cached field while w.server.is_alive() is False — a plausible wrong answer rather than an error. Nothing in the output indicates the object is dead. Every one of libtmux's 14 multi-fence how-to pages carries objects across blocks exactly this way, so this is the shape of failure that would greet them.

Sybil fixes this in five lines, and they are portable

Sybil aliases module scope to document scope, so pytest's own module-scoped fixture machinery gives a fixture exactly the shared-state lifetime with no custom teardown:

def getparent(self, cls):
    if cls is Module:
        return self.parent
    if cls is Session:
        return self.session

src/sybil/integration/pytest.py#L66-L70 — pytest's get_scope_node asks node.getparent(Module) for module scope; returning the file collector means @pytest.fixture(scope="module") lives exactly as long as the shared namespace.

Then every resolved fixture is copied back into the namespace on every item, so a fixture name can never go stale — only an object a block derived itself:

def setup(self) -> None:
    self._request._fillfixtures()
    for name, fixture in self.funcargs.items():
        self.example.namespace[name] = fixture

src/sybil/integration/pytest.py#L72-L75

The surrounding design is worth reading as a whole, since it is the same shape this branch arrived at independently. One dict owned above the items at src/sybil/document.py#L39, handed by identity to every example at src/sybil/document.py#L93-L103, N items yielded from one parse at src/sybil/integration/pytest.py#L104-L113, and stdlib's copy defeated by subclassing at src/sybil/evaluators/doctest.py#L14-L24 with clear_globs=False at src/sybil/evaluators/doctest.py#L138.

Sphinx solves only the copy half, in three lines and with a comment naming the obstacle: sphinx/ext/doctest.py#L609-L613. The obstacle itself is Lib/doctest.py#L559.

Where per-block is already ahead of Sybil

A live shared dict cannot cross a process, and this branch is the only implementation surveyed that handles it — the xdist_group marks, the LoadScope scheduler, and the UsageError on --dist load / worksteal. Sybil ships no guard at all: a shared page silently breaks under --dist load and needs --dist loadfile to work, which is documented nowhere. That is a real advantage and worth keeping visible in the docs, because it is the failure mode a downstream project hits only in CI.

What would make this maximally useful downstream

Ranked by how much each would change what a project like libtmux can do.

1. Close the fixture-lifetime gap. This is the one blocker between per-block and a project retiring hand-rolled machinery. Two shapes, both proven: Sybil's getparent alias plus refill-on-every-item, or pytest-asyncio's contract, where the plugin declares the scope and the project opts in per unit (loop_scope=) rather than the plugin widening anything unilaterally. The pytest-asyncio shape is the better fit here, because a project must consent — widening server/session to module scope is a real change to a project's test semantics, not a flag flip. Whichever shape, the failure mode to design against is silence: a stale object reading fine is worse than a NameError.

2. Collect fences that carry no >>>. Today the collection predicate requires a >>>, which excludes the entire class of pages whose visible code a reader is meant to copy verbatim — a prompt breaks paste, and inline expected output puts an assertion in the reader's paste path. libtmux has 16 such pages and hand-built a page collector for them precisely because this branch cannot see them. myst_fence_as_directive makes this reachable without changing the pages: register a directive whose option_spec gp-libs owns, and a plain fence can carry group and scope options while still rendering as a plain fence. This is the single change with the largest downstream surface.

3. Make debugging a shared page cheap. Sharing turns an independent failure into a dependent one, and the diagnostics should say so. Under merged, blocks after the first failure do not run and are invisible; under per-block, they run and fail confusingly against a half-built namespace. Both are improved by naming the dependency in the report — which block first failed, and which later ones were skipped or are downstream of it. A --doctest-docutils-continue-on-failure mirroring pytest's own flag would also let a page report all its real failures in one run rather than one per iteration.

4. State the independence limit in the docs. No surveyed project makes a shared block runnable alone, and none can: running block 2 by node id will always NameError once block 1 is what defines its names. per-block preserves the id for -k, --lf, --deselect and JUnit reporting, which is genuinely useful, but it does not make the block independently runnable. Saying so plainly prevents a class of confused bug reports, since the id looks like a promise of independence.

5. Keep merged the default. The measurement in the PR description already justifies it, and libtmux reproduces it: flipping libtmux's tree to document + merged breaks three pages by three unrelated mechanisms — a monotonic id counter drifting once one server serves a page, a name collision between two blocks that each create the same-named object, and an object killed by an earlier block. All three are fixture-lifetime consequences of the collapse, and all three vanish under per-block. That is a good argument for documenting per-block as the recommended setting whenever sharing is turned on at all.

Sources reviewed

Every claim above was read at the ref listed, not recalled.

Project	Ref	What was read
Sybil	10.0.1	Document namespace ownership, SybilFile.collect, SybilItem.getparent / setup / request_fixtures, doctest evaluator, parsers, clear-namespace, invisible-code-block
Sphinx	v9.1.0	sphinx/ext/doctest.py — TestGroup, test_group, group directive semantics, testsetup / testcleanup
CPython	3479e45	Lib/doctest.py — DocTest.__init__ glob copying, DocTestRunner.run clear_globs, DocTestParser, subtest reporting
pytest	f306da7	_pytest/doctest.py — DoctestTextfile.collect, DoctestItem, doctest_namespace scope, --doctest-continue-on-failure
xdoctest	v1.3.2	parsing granularity, --xdoctest-style freeform vs google, per-part diagnostics
pytest-asyncio	v1.4.0	loop_scope / asyncio_mode, scoped-fixture construction, scope-mismatch surfacing
MyST-Parser	v5.1.0	fence representation in the doctree, attrs_block, myst_fence_as_directive
docutils	trunk	Directive.option_spec plumbing, reporting thresholds

Downstream verification was libtmux with this branch pinned via [tool.uv.sources]: full suite green at the default block scope, and the docs/ tree measured at every combination of the two new settings.

[tony]

Correction to my previous comment. Recommendation 1 there — "close the fixture-lifetime gap", citing Sybil's getparent alias as the fix — was asking for something this branch already ships. Withdrawing it. I also re-measured against ff0c68c and have a parallelism result that was not in the earlier comment.

Withdrawn: there is no fixture-lifetime gap

DocTestDocutilsFile subclasses pytest.Module, and pytest.Module is the node pytest resolves module scope against. A page therefore already is the scope, and @pytest.fixture(scope="module") already gets exactly the shared-namespace lifetime. That is the same outcome Sybil reaches through SybilItem.getparent, which returns the file collector when pytest asks for Module — this branch gets it structurally instead of through a shim, which is the better of the two.

cbb796d pins it and ff0c68c documents it, and both predate my earlier comment. I had the answer available and misread it.

The staleness I reproduced was real but self-inflicted: my conftest.py declared the fixture function-scoped, which correctly sets up once per item. Re-running the identical page with the one-word change:

@pytest.fixture(scope="module")
def resource():
    ...

>>> saved.n, saved.alive
(1, True)
>>> saved is resource
True

The object survives every block of the page. The tell was in my own earlier numbers and I walked past it — merged kept the object alive because one item meant one function-scoped setup, which is the same fact as "widen the scope and it survives".

So the correct guidance for a downstream project carrying an object across blocks is not "wait for a fix", it is "declare that fixture scope="module"", which is what fixtures.md now says. libtmux's server and session are function-scoped, which is right for its per-block pages and would need to change only for pages that deliberately share.

New measurement: per-block under -n auto

7a78afa is doing real work. libtmux's docs/ tree, 190 blocks over 16 pages, --doctest-docutils-namespace-scope=document:

Layout	-n auto	serial
merged	4 failed, 27 passed	3 failed, 28 passed
per-block	1 failed, 189 passed	190 passed

The single -n auto failure under per-block is not sharing-related, and the control proves it: running the same page at the default block scope with no sharing whatsoever produces the identical failure, same block index, same line. It is a pre-existing timing flake in that page — a time.sleep(0.3) followed by a poll with a 2s budget, where a cold shell in a freshly created pane can exceed the budget under load. A branch of libtmux that replaces those polls with a wait-for handshake passes the same page 5/5 under -n auto.

Net: sharing and full parallelism now coexist. That is the result I could not get before 7a78afa, and it is worth calling out in the PR description, because "shared namespace" and "runs under -n auto" reading as mutually exclusive is the assumption most people will arrive with.

Full libtmux suite on ff0c68c with the branch pinned through [tool.uv.sources]: 1552 passed, 1 skipped.

What is left, re-ranked

With the fixture question closed, the list shortens and reorders.

1. Collect fences that carry no >>>. Unchanged from my earlier comment and now the only item with a large downstream surface. The collection predicate requires a >>>, which excludes every page whose visible code a reader is meant to copy verbatim — a prompt breaks paste, and inline expected output puts an assertion in the reader's paste path. libtmux has 16 such pages and hand-built a page collector for them solely because this branch cannot see them. myst_fence_as_directive reaches this without changing how the pages read: register a directive whose option_spec this plugin owns, and a plain fence can carry group and scope options while still rendering as a plain fence. Worth noting the ceiling honestly — a project would still keep whatever it uses for per-page environment isolation and for assertions it does not want in the reader's copy path, so this shrinks such a harness rather than retiring it.

2. Make a shared failure cheap to debug. Sharing converts an independent failure into a dependent one, and the report should say which. Under merged, blocks after the first failure never run and are invisible in the output; under per-block they do run, and fail against a half-built namespace, which reads as several unrelated bugs. Naming the relationship — which block failed first, and which later failures are downstream of it — turns a confusing cascade into one line. A continue-on-failure mirroring pytest's own flag would also let a page surface all its genuine failures in one run instead of one per iteration.

3. Say plainly that an id is not independence. No surveyed project makes a shared block runnable alone, and none can: selecting block 2 by id will always fail once block 1 is what defines its names. per-block preserves the id for -k, --lf, --deselect and JUnit reporting, which is genuinely valuable, but the id looks like a promise of independent runnability and is not one. The per-block costs section already covers the mechanics; stating this one as a flat sentence would prevent a category of confused reports.

4. Recommend per-block wherever sharing is on. Keeping merged as the default is well justified by the measurement in the PR description. But every failure mode libtmux hit under document + merged — a monotonic id counter drifting once one server serves a whole page, two blocks colliding on a same-named object, an object killed by an earlier block — is a consequence of the collapse, and all of them vanish under per-block. merged is the safe default for someone who has not opted in; per-block is the better setting for someone who has.

Sources reviewed

Everything below was read at the ref listed rather than recalled. gp-libs is cited by symbol and commit subject rather than line anchor, since this branch rebased between my two comments and line anchors on a moving head rot.

Project	Ref	What was read
gp-libs	ff0c68c	DocTestDocutilsFile, the per-block finder arm, the items and scope options, tests/regressions/test_autouse_fixtures.py, the fixtures and per-block-costs docs, and the eight commits added since 55115db
Sybil	10.0.1	Document namespace ownership, SybilFile.collect, SybilItem.getparent / setup / request_fixtures, the doctest evaluator, parsers, clear-namespace, invisible-code-block
Sphinx	v9.1.0	sphinx/ext/doctest.py — TestGroup, test_group, group directive semantics, testsetup / testcleanup
CPython	3479e45	Lib/doctest.py — DocTest.__init__ glob copying, DocTestRunner.run clear_globs, DocTestParser, subtest reporting
pytest	f306da7	_pytest/doctest.py — DoctestTextfile.collect, DoctestItem, doctest_namespace scope, --doctest-continue-on-failure
xdoctest	v1.3.2	parsing granularity, --xdoctest-style freeform vs google, per-part diagnostics
pytest-asyncio	v1.4.0	loop_scope / asyncio_mode, scoped-fixture construction, scope-mismatch surfacing
MyST-Parser	v5.1.0	fence representation in the doctree, attrs_block, myst_fence_as_directive
docutils	trunk	Directive.option_spec plumbing, reporting thresholds

Downstream verification throughout was libtmux with this branch pinned via [tool.uv.sources], measured at every combination of the two settings, serial and under -n auto, with a no-sharing control run for every failure before attributing it.

Relevant anchors for the corrected claim: src/sybil/integration/pytest.py#L66-L70 and src/sybil/integration/pytest.py#L72-L75 for the shim this branch does not need, src/sybil/document.py#L39 and src/sybil/document.py#L93-L103 for one dict owned above the items, sphinx/ext/doctest.py#L609-L613 for the three-line copy defeat, and Lib/doctest.py#L559 for the obstacle both are working around.

[tony]

What libtmux would need from this plugin to test its documentation the way it wants it tested. Three asks, ranked, plus what libtmux would change on its own side. One of them turns out to be much cheaper than I thought when I raised it, because Sphinx already defines the semantics.

Context for the ranking: libtmux has three documentation surfaces, and this branch already serves two of them completely.

Surface	What it needs	Status
src/**.py docstrings, 206 tests	unaffected by namespace scope at either setting	served
docs/topics/*.md, 16 pages / 190 blocks	a fresh namespace and a fresh tmux server per block	served, at the default block scope
docs/howto/*.md, 16 pages	six things, below	blocked on the first

Ask 1, blocking: collect a block that carries no >>>

Nothing else matters until this exists, because the pages that need everything else are invisible to the plugin today. Measured on this branch, four ways of writing a runnable Python block with no prompt:

Form	Collected
```python	0
```{doctest}	0
```{testcode}	0
```{code-block} python	0

Control, the same file with >>> added and nothing else changed: 1 collected. So the prompt is the sole determinant, and no directive form escapes it.

The reason libtmux's how-to pages carry no prompt is not stylistic. A reader is meant to select the block and paste it into a file. A >>> prefix breaks that, and an expected-output line puts an assertion into the text the reader just pasted. The pages are the product; the checking has to happen somewhere the reader never sees.

Ask 2, strongly wanted: an assertion channel outside the visible block

This is the one that got cheaper. I originally described it as Sybil's invisible-code-block shape, which is a real answer — see the (invisible-)?code(-block)? pattern its MyST parser accepts at src/sybil/parsers/myst/codeblock.py — but it means inventing syntax.

Sphinx already has this, and has had it for years. TestcodeDirective accepts a hide flag at sphinx/ext/doctest.py#L173-L177, and the directive turns a hidden block into a comment node rather than a literal block at sphinx/ext/doctest.py#L92-L93:

nodetype: type[TextElement] = nodes.literal_block
if self.name in {'testsetup', 'testcleanup'} or 'hide' in self.options:
    nodetype = nodes.comment

So {testcode} runs and does not check output, and {testcode} :hide: runs and does not render. That is asks 1 and 2 together, in one directive, with semantics Sphinx defined and downstream Sphinx tooling already renders correctly.

Which makes the concrete proposal narrower than "accept fences without prompts": honour {testcode}, and honour its :hide: flag. A page then reads as plain prose with a visible {testcode} a reader can copy, and a hidden {testcode} :hide: doing the asserting. No new syntax, no change to how the pages read, and the fallback for projects that would rather not use a directive at all is myst_fence_as_directive, a MyST setting that maps a bare language fence onto a directive name — see the field at myst_parser/config/main.py#L265-L273.

Ask 3, probably out of scope: re-run a block against a fresh namespace

libtmux's how-to harness can execute a block a second time in a different world, which is how a page showing if server.is_alive(): gets both branches checked from one visible block. I am not asking for this. It is unusual, no surveyed project offers it, and libtmux can keep it locally. Recording it only so the gap is known rather than discovered later.

What libtmux would change on its own side

None of this is a request for the plugin to accommodate libtmux's fixtures — that part already works, and the parts I previously thought were missing are not.

Add a docs/howto/conftest.py declaring server and session at scope="module". A page is a pytest.Module here, so module scope is page scope, and a how-to page's blocks then share one tmux server, which is what those pages need. Leave the root conftest.py function-scoped so docs/topics/*.md keeps a fresh server per block, which is what those pages need. I verified the two coexist in one run: two directories, two different scopes for a same-named fixture, four blocks, all passing.

Keep the per-page TMUX_TMPDIR and HOME isolation in libtmux's own harness. That is a tmux concern, not a doctest concern, and I would not want it in a doctest plugin.

For CI: -n auto with no --dist is correct and needs no flag, since the scheduler is filled in. Verified end to end — --dist load and --dist worksteal with per-block both exit 4, --dist loadfile exits 0, and the error message names all three remedies.

Net

Of the six things libtmux's how-to pages need, three are already available on this branch: a shared namespace per page, a fixture lifetime that matches it, and the ability to differ from the per-block pages next door. One is an absolute blocker, one is strongly wanted, and one is mine to keep.

The two that matter collapse into a single change if {testcode} and :hide: are the chosen spelling — which would also mean libtmux's how-to pages become ordinary Sphinx documents rather than a format only its own collector understands.

And nothing at all is needed for the 206 docstring tests or the 190 topics blocks. Those are served today, at the default settings, with no configuration.