Skip to content

Commit 26914fd

Browse files
committed
Wire subint_forkserver as first-class backend
Promote `_subint_forkserver` from primitives-only into a registered spawn backend: `'subint_forkserver'` is now a `SpawnMethodKey` literal, dispatched via `_methods` to the new `subint_forkserver_proc()` target, feature-gated under the existing `subint`-family py3.14+ case, and selectable via `--spawn-backend=subint_forkserver`. Deats, - new `subint_forkserver_proc()` spawn target in `_subint_forkserver`: - mirrors `trio_proc()`'s supervision model — real OS subprocess so `Portal.cancel_actor()` + `soft_kill()` on graceful teardown, `os.kill(SIGKILL)` on hard-reap (no `_interpreters.destroy()` race to fuss over bc the child lives in its own process) - only real diff from `trio_proc` is the spawn mechanism: fork from a main-interp worker thread via `fork_from_worker_thread()` (off-loaded to trio's thread pool) instead of `trio.lowlevel.open_process()` - child-side `_child_target` closure runs `tractor._child._actor_child_main()` with `spawn_method='trio'` — the child is a regular trio actor, "subint_forkserver" names how the parent spawned, not what the child runs - new `_ForkedProc` class — thin `trio.Process`-compatible shim around a raw OS pid: `.poll()` via `waitpid(WNOHANG)`, async `.wait()` off-loaded to a trio cache thread, `.kill()` via `SIGKILL`, `.returncode` cached for repeat calls. `.stdin`/`.stdout`/`.stderr` are `None` (fork-w/o-exec inherits parent FDs; we don't marshal them) which matches `soft_kill()`'s `is not None` guards Also, new backend-tier test `test_subint_forkserver_spawn_basic` drives the registered backend end-to-end via `open_root_actor` + `open_nursery` + `run_in_actor` w/ a trivial portal-RPC round-trip. Uses a `forkserver_spawn_method` fixture to flip `_spawn_method`/`_ctx` for the test's duration + restore on teardown (so other session-level tests don't observe the global flip). Test module docstring reworked to describe the three tiers now covered: (1) primitive-level, (2) parent-trio-driven primitives, (3) full registered backend. Status: still-open work (tracked on `tractor#379`) doc'd inline in the module docstring — no cancel/hard-kill stress coverage yet, child-side subint-hosted root runtime still future (gated on `msgspec#563`), thread-hygiene audit pending the same unblock. (this patch was generated in some part by [`claude-code`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-code
1 parent cf2e71d commit 26914fd

3 files changed

Lines changed: 449 additions & 22 deletions

File tree

tests/spawn/test_subint_forkserver.py

Lines changed: 126 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,17 @@
11
'''
22
Integration exercises for the `tractor.spawn._subint_forkserver`
3-
primitives (`fork_from_worker_thread()` + `run_subint_in_worker_thread()`)
4-
driven from inside a real `trio.run()` in the parent process —
5-
the runtime shape tractor will need when we move toward wiring
6-
up a `subint_forkserver` spawn backend proper.
3+
submodule at three tiers:
4+
5+
1. the low-level primitives
6+
(`fork_from_worker_thread()` +
7+
`run_subint_in_worker_thread()`) driven from inside a real
8+
`trio.run()` in the parent process,
9+
10+
2. the full `subint_forkserver_proc` spawn backend wired
11+
through tractor's normal actor-nursery + portal-RPC
12+
machinery — i.e. `open_root_actor` + `open_nursery` +
13+
`run_in_actor` against a subactor spawned via fork from a
14+
main-interp worker thread.
715
816
Background
917
----------
@@ -16,17 +24,20 @@
1624
host its own `trio.run()` inside a fresh subint.
1725
1826
Those smoke-test scenarios are standalone — no trio runtime
19-
in the *parent*. These tests exercise the same primitives
20-
from inside `trio.run()` in the parent, proving out the
21-
piece actually needed for a working spawn backend.
27+
in the *parent*. Tiers (1)+(2) here cover the primitives
28+
driven from inside `trio.run()` in the parent, and tier (3)
29+
(the `*_spawn_basic` test) drives the registered
30+
`subint_forkserver` spawn backend end-to-end against the
31+
tractor runtime.
2232
2333
Gating
2434
------
2535
- py3.14+ (via `concurrent.interpreters` presence)
26-
- no backend restriction (these tests don't use
27-
`--spawn-backend` — they drive the forkserver primitives
28-
directly rather than going through tractor's spawn-method
29-
registry).
36+
- no `--spawn-backend` restriction — the backend-level test
37+
flips `tractor.spawn._spawn._spawn_method` programmatically
38+
(via `try_set_start_method('subint_forkserver')`) and
39+
restores it on teardown, so these tests are independent of
40+
the session-level CLI backend choice.
3041
3142
'''
3243
from __future__ import annotations
@@ -36,6 +47,7 @@
3647
import pytest
3748
import trio
3849

50+
import tractor
3951
from tractor.devx import dump_on_hang
4052

4153

@@ -50,6 +62,8 @@
5062
run_subint_in_worker_thread,
5163
wait_child,
5264
)
65+
from tractor.spawn import _spawn as _spawn_mod # noqa: E402
66+
from tractor.spawn._spawn import try_set_start_method # noqa: E402
5367

5468

5569
# ----------------------------------------------------------------
@@ -212,3 +226,104 @@ def test_fork_and_run_trio_in_child() -> None:
212226
),
213227
)
214228
assert isinstance(pid, int) and pid > 0
229+
230+
231+
# ----------------------------------------------------------------
232+
# tier-3 backend test: drive the registered `subint_forkserver`
233+
# spawn backend end-to-end through tractor's actor-nursery +
234+
# portal-RPC machinery.
235+
# ----------------------------------------------------------------
236+
237+
238+
async def _trivial_rpc() -> str:
239+
'''
240+
Minimal subactor-side RPC body: just return a sentinel
241+
string the parent can assert on.
242+
243+
'''
244+
return 'hello from subint-forkserver child'
245+
246+
247+
async def _happy_path_forkserver(
248+
reg_addr: tuple[str, int | str],
249+
deadline: float,
250+
) -> None:
251+
'''
252+
Parent-side harness: stand up a root actor, open an actor
253+
nursery, spawn one subactor via the currently-selected
254+
spawn backend (which this test will have flipped to
255+
`subint_forkserver`), run a trivial RPC through its
256+
portal, assert the round-trip result.
257+
258+
'''
259+
with trio.fail_after(deadline):
260+
async with (
261+
tractor.open_root_actor(
262+
registry_addrs=[reg_addr],
263+
),
264+
tractor.open_nursery() as an,
265+
):
266+
portal: tractor.Portal = await an.run_in_actor(
267+
_trivial_rpc,
268+
name='subint-forkserver-child',
269+
)
270+
result: str = await portal.wait_for_result()
271+
assert result == 'hello from subint-forkserver child'
272+
273+
274+
@pytest.fixture
275+
def forkserver_spawn_method():
276+
'''
277+
Flip `tractor.spawn._spawn._spawn_method` to
278+
`'subint_forkserver'` for the duration of a test, then
279+
restore whatever was in place before (usually the
280+
session-level CLI choice, typically `'trio'`).
281+
282+
Without this, other tests in the same session would
283+
observe the global flip and start spawning via fork —
284+
which is almost certainly NOT what their assertions were
285+
written against.
286+
287+
'''
288+
prev_method: str = _spawn_mod._spawn_method
289+
prev_ctx = _spawn_mod._ctx
290+
try_set_start_method('subint_forkserver')
291+
try:
292+
yield
293+
finally:
294+
_spawn_mod._spawn_method = prev_method
295+
_spawn_mod._ctx = prev_ctx
296+
297+
298+
@pytest.mark.timeout(60, method='thread')
299+
def test_subint_forkserver_spawn_basic(
300+
reg_addr: tuple[str, int | str],
301+
forkserver_spawn_method,
302+
) -> None:
303+
'''
304+
Happy-path: spawn ONE subactor via the
305+
`subint_forkserver` backend (parent-side fork from a
306+
main-interp worker thread), do a trivial portal-RPC
307+
round-trip, tear the nursery down cleanly.
308+
309+
If this passes, the "forkserver + tractor runtime" arch
310+
is proven end-to-end: the registered
311+
`subint_forkserver_proc` spawn target successfully
312+
forks a child, the child runs `_actor_child_main()` +
313+
completes IPC handshake + serves an RPC, and the parent
314+
reaps via `_ForkedProc.wait()` without regressing any of
315+
the normal nursery teardown invariants.
316+
317+
'''
318+
deadline: float = 20.0
319+
with dump_on_hang(
320+
seconds=deadline,
321+
path='/tmp/subint_forkserver_spawn_basic.dump',
322+
):
323+
trio.run(
324+
partial(
325+
_happy_path_forkserver,
326+
reg_addr,
327+
deadline,
328+
),
329+
)

tractor/spawn/_spawn.py

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,13 @@
7272
# `ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md`
7373
# + issue #379 for the full analysis.
7474
'subint_fork',
75+
# EXPERIMENTAL — the `subint_fork` workaround. `os.fork()`
76+
# from a non-trio worker thread (never entered a subint)
77+
# is CPython-legal and works cleanly; forked child runs
78+
# `tractor._child._actor_child_main()` against a trio
79+
# runtime, exactly like `trio_proc` but via fork instead
80+
# of subproc-exec. See `tractor.spawn._subint_forkserver`.
81+
'subint_forkserver',
7582
]
7683
_spawn_method: SpawnMethodKey = 'trio'
7784

@@ -124,13 +131,14 @@ def try_set_start_method(
124131
case 'trio':
125132
_ctx = None
126133

127-
case 'subint' | 'subint_fork':
128-
# Both subint backends need no `mp.context`; both
129-
# feature-gate on the py3.14 public
134+
case 'subint' | 'subint_fork' | 'subint_forkserver':
135+
# All subint-family backends need no `mp.context`;
136+
# all three feature-gate on the py3.14 public
130137
# `concurrent.interpreters` wrapper (PEP 734). See
131138
# `tractor.spawn._subint` for the detailed
132-
# reasoning and the distinction between the two
133-
# (`subint_fork` is WIP/experimental).
139+
# reasoning. `subint_fork` is blocked at the
140+
# CPython level (raises `NotImplementedError`);
141+
# `subint_forkserver` is the working workaround.
134142
from ._subint import _has_subints
135143
if not _has_subints:
136144
raise RuntimeError(
@@ -469,6 +477,7 @@ async def new_proc(
469477
from ._mp import mp_proc
470478
from ._subint import subint_proc
471479
from ._subint_fork import subint_fork_proc
480+
from ._subint_forkserver import subint_forkserver_proc
472481

473482

474483
# proc spawning backend target map
@@ -483,4 +492,8 @@ async def new_proc(
483492
# clean `NotImplementedError` with pointer to the analysis,
484493
# rather than an "invalid backend" error.
485494
'subint_fork': subint_fork_proc,
495+
# WIP — fork-from-non-trio-worker-thread, works on py3.14+
496+
# (validated via `ai/conc-anal/subint_fork_from_main_thread_smoketest.py`).
497+
# See `tractor.spawn._subint_forkserver`.
498+
'subint_forkserver': subint_forkserver_proc,
486499
}

0 commit comments

Comments
 (0)