Auditing a new main_thread_forkserver spawning backend#453
Open
Auditing a new main_thread_forkserver spawning backend#453
main_thread_forkserver spawning backend#453Conversation
Brings `tractor`'s `pyproject.toml` (+ `uv.lock`) up to the
final state on the `subint_forkserver_backend` dev branch:
- `requires-python = ">=3.13, <3.15"` — stay on 3.13 floor
for working `stackscope` task-tree introspection;
3.14 still allowed for opt-in subint-family backends.
- New `[dependency-groups]`:
* `pytest-timeout>=2.3` — opt-in per-test cap (NOT used
as a global cap; see the long NOTE in
`[tool.pytest.ini_options]` for why pytest-timeout
breaks trio under fork-based backends).
* `psutil>=7.0.0` — used by `tractor._testing._reap` for
the `tractor-reap` zombie-actor + leaked-shm cleanup
utility (xplatform `Process.memory_maps`,
`Process.open_files`).
* `subints` group gated to `>=3.14` (`msgspec>=0.21.0`
for future PEP-684 isolated-subint work).
* `eventfd` + `sync_pause` gated to `>=3.13, <3.14`.
- `[tool.pytest.ini_options]` block now carries the
capture-mode + global-timeout tradeoffs as inline NOTEs.
Provenance: this is the squashed equivalent of ~7 commits
on `subint_forkserver_backend` between `9cf3d588` (initial
work) and `3c366cac` (drop global pytest-timeout cap). See
that branch for the full incremental history including the
`subint`-research dead-ends.
(this patch was generated in some part by [\`claude-code\`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Pulls the full conc-anal trail captured during the `subint_forkserver_backend` dev branch into `main`: - `subint_*_issue.md` (5 docs) — hang catalog + diagnosis for the explored `subint`-family backends. Several docs are referenced from the `_main_thread_forkserver.py` module docstring (CPython post-fork refusal, PEP 684 thread constraints, etc.) and need to be present for those links to resolve. - `subint_fork_blocked_by_cpython_post_fork_issue.md` — the canonical writeup of the CPython-level refusal that gates `os.fork()` from a sub-interpreter. Drives the design rationale for the main-thread-worker forkserver pattern. - `subint_fork_from_main_thread_smoketest.py` — standalone script empirically validating that `fork()` from a regular `threading.Thread` against the main interp works cleanly. Foundation for the variant-1 backend. - `fork_thread_semantics_execution_vs_memory.md` — reconciles trio's "leaked thread stacks" framing with the docstring's "every other thread is gone" framing as the memory-side vs execution-side of the same POSIX `fork()` reality. Provenance: squashed equivalent of ~10 commits on `subint_forkserver_backend` (`0f48ed2e`, `de4f470b`, `3ab99d55`, `4b5176e2`, `cf2e71d8`, `e3f4f5a3`, `c99d475d`, `f3cea714`, `4a325458`, `532a9834`). (this patch was generated in some part by [\`claude-code\`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-code
Captured Claude prompt I/O alongside the original `subint_forkserver_backend` dev work per the NLNet generative-AI-attribution policy (see `.claude/skills/prompt-io/`). Each pair of `.md`/`.raw.md` files records the prompt + response that drove a corresponding code commit on the source branch. Imported as a single squash so the historical provenance is preserved in `main`-bound history but without polluting the per-feature commit timeline. Provenance: squashed from ~4 commits on `subint_forkserver_backend` (`a65fded4`, `c041518b`, + inline addenda). (this patch was generated in some part by [\`claude-code\`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-code
Tooling configuration that came with the dev branch: - `.claude/skills/run-tests/SKILL.md` — extended with fork-spawn-aware test running guidance (zombie-actor cleanup, capture-mode tradeoffs, SIGINT-first cleanup ladder, lastfailed cache inspection). - `.claude/skills/conc-anal/SKILL.md` — minor refinements to the concurrency-analysis skill referenced by the new `ai/conc-anal/` docs. - `.claude/settings.local.json` — additional perms used by the new skills (`Skill(run-tests)`, `Skill(open-wkt)`, etc.) + `Read/Write(.claude/**)` for commit-msg authoring. - `.github/workflows/ci.yml` — pickup new dep groups + py-floor change. - `.gitignore` — add `notes/`, `snippets/`, `.claude/wkts/` patterns. Provenance: squashed from ~6 commits on `subint_forkserver_backend` (`9cf3d588`, `d3d6f646`, `ba86d482`, `b1a0753a`, `4106ba73`, `70d58c4b`, `d093c319`, `76d12060`). (this patch was generated in some part by [\`claude-code\`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-code
Standalone reaper utility used to scrub state left
behind by hard-killed actor sessions during
fork-spawn-backend hang triage:
- `scripts/tractor-reap` — CLI with `--zombies`
(reap orphaned `tractor._child` processes via
`psutil`) and `--shm` (sweep leaked
`multiprocessing.SharedMemory` segments backed by
dead pids).
- `tractor/_testing/_reap.py` — backing impl using
`psutil.Process.{memory_maps, open_files,
children}`. Layered so individual fixtures
(`reap_subactors_per_test` later) can call the
same primitives.
- `tractor/_testing/addr.py` — fix
`get_rando_addr()` cross-process collisions so
parallel pytest sessions don't fight over the same
random `reg_addr` ports / UDS sock paths.
Provenance: squashed from ~5 commits on
`subint_forkserver_backend` (`eae478f3`, `6d76b604`,
`4f12d69b`, `7c5dd4d0`).
(this patch was generated in some part by [\`claude-code\`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Plugin-level test infrastructure shared by both the
existing backends and the new fork-spawn one:
- `tractor/_testing/pytest.py`:
* Add `skipon_spawn_backend(<name>, reason=...)`
pytest marker. Lets per-test-file modules opt
out of specific backends with an explicit
reason string. Used heavily by the
cross-cutting test suite updates (Bucket H).
* `--enable-stackscope` CLI flag — installs the
`stackscope` SIGUSR1 handler at runtime gate
time WITHOUT pulling in the full `--tpdb`
debug-mode side effects. Pairs with
`tractor.devx._stackscope`'s file-tee +
run-sync-soon dispatch (Bucket E).
* `reap_subactors_per_test` fixture (opt-in) that
sweeps any `tractor._child` orphans whose
parent pid matches the current pytest process,
via the `tractor._testing._reap` helper.
* Backend-aware `--spawn-backend` option
parametrization.
- `tests/conftest.py`,
`tests/devx/conftest.py` — wire the new fixture
+ filter `start_method` allowlist for
pexpect-based debugger tests to include
`main_thread_forkserver`.
Provenance: squashed from ~6 commits on
`subint_forkserver_backend` (`3b26b59d`, `b376eb03`,
`f8178df0`, `5418f2dc`, `4c133ab5`, `d6e70e9d`).
(this patch was generated in some part by [\`claude-code\`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
The headline change: a new in-process forkserver that runs `os.fork()` from a regular `threading.Thread` attached to the main interpreter, sidestepping trio's classic fork-hostility (signal-wakeup-fd, epoll, threadpool caches) by isolating the syscall in a worker that has provably never entered trio. Spawn-method key: `'main_thread_forkserver'`. Reserves a sibling `'subint_forkserver'` key for the future variant-2 (subint-isolated child runtime, gated on jcrist/msgspec#1026 + PEP 684). Module layout (`tractor/spawn/`): - `_main_thread_forkserver.py` (NEW, 1024 LOC) — variant-1 IMPL. Includes the in-process forkserver worker thread, `_ForkedProc` `trio.Process`-shaped shim, post-fork fd scrubber (`_close_inherited_fds`), pidfd-based cancellable wait, plus the substantial module docstring covering POSIX `fork()` semantics ("execution-side gone" vs trio's "memory-side leaked" framing) + design rationale. - `_subint_forkserver.py` (NEW, 310 LOC) — variant-2 PLACEHOLDER. Today aliases dispatch to variant-1's proc spawner; reserved for the subint-hosted-trio child runtime. - `_subint.py` (NEW, 474 LOC) — variant-3 standalone subint backend. Wedged on cross-trio-instance hang upstream; carried as documented dead-code scaffold. - `_subint_fork.py` (NEW, 153 LOC) — variant-2 fork research. CPython-blocked (`_PyInterpreterState_DeleteExceptMain`); carried as a `NotImplementedError` stub pointing at the analysis doc. - `_spawn.py` — registers the four new spawn keys in `SpawnMethodKey` `Literal`, wires dispatch in `try_set_start_method` + `_methods` table. Runtime support: - `tractor/_child.py` — `_actor_child_main()` learns the `spawn_method` arg + post-fork `_state` reset. - `tractor/_root.py` — `open_root_actor` accepts `spawn_method=` + carries it through to actor init. - `tractor/runtime/_runtime.py` — relax the `enable_stack_on_sig` gate to honor `TRACTOR_ENABLE_STACKSCOPE` env-var (set by the pytest `--enable-stackscope` flag, Bucket D). - `tractor/runtime/_state.py` — refactor `_runtime_vars` into pure get/set API for clean post-fork reset; surface `spawn_method` to RPC log/repr paths. Devx: - `tractor/devx/_stackscope.py` — file-tee (`/tmp/tractor-stackscope-<pid>.log`) + `/dev/tty` writes that bypass pytest's stdio capture; `run_sync_soon`-based dispatch onto the trio loop so `stackscope.extract` recurses nursery children when called from a signal handler; suppress benign import-time `RuntimeWarning`s from `stackscope._glue`. - `tractor/devx/_debug_hangs.py` (NEW, 227 LOC) — hang-triage primitives (`dump_on_hang` etc.). IPC tweaks for fork-survival: - `tractor/ipc/_shm.py` — `SharedMemory` wedge fix under `main_thread_forkserver` (re-init posix primitive map post-fork). - `tractor/ipc/_mp_bs.py`, `tractor/ipc/_linux.py` — minor adjustments for fork-child IPC bringup. Tests (`tests/spawn/`): - `test_main_thread_forkserver.py` (652 LOC) — backend smoke + xfail draft for the orphaned-actor SIGINT cleanup behavior (xfail until issue #449 lands). - `test_subint_cancellation.py` (245 LOC) — variant-3 subint backend cancellation/teardown audit. Provenance: this is the squashed equivalent of ~30+ commits on `subint_forkserver_backend` between `82332fbc` (lift fork prims into a dedicated mod) and `2d4995e0` (route stackscope SIGUSR1 onto trio loop). See that branch for the full incremental history including the stepwise hang diagnoses, per-fix conc-anal annotations, and the variant-1/2 naming split (Phase D in the dev branch's history). Linked issues: - #379 — subint umbrella tracking issue - #449 — orphaned-subactor SIGINT cleanup hang (xfailed in the test suite) - #451 — Mode-A cancel-cascade hang - #452 — discovery-client `CLOSE_WAIT` fd leak (this patch was generated in some part by [\`claude-code\`][claude-code-gh]) [claude-code-gh]: https://github.com/anthropics/claude-code
Cross-cutting test-suite updates so the existing tests
work under the new `main_thread_forkserver` backend
alongside `trio` / `mp_*`:
- `skipon_spawn_backend(<name>, reason=...)` marks
added to tests that genuinely don't apply or are
known-broken under specific backends:
* `tests/test_pubsub.py` — `subint*`-family hangs
* `tests/test_shm.py` — `subint*` SharedMemory wedge
* `tests/test_inter_peer_cancellation.py` —
`subint*` cancellation leak
* `tests/discovery/test_registrar.py` —
`test_trynamic_trio` peer-discovery double-run
* `tests/test_cancellation.py` — known-failing
cases that need separate triage
- `reg_addr` standardization — replace ad-hoc port
hard-codings with the `reg_addr` fixture so
parallel pytest sessions don't collide
(`test_advanced_streaming`, `test_pubsub`,
`test_context_stream_semantics`,
`test_inter_peer_cancellation`,
`test_infected_asyncio`).
- Backend-aware timeouts — `maybe_expect_raises` in
`test_pldrx_limiting` and `test_dynamic_pub_sub`'s
`fail_after` cap honor `main_thread_forkserver`'s
per-spawn cost (30s vs 12s).
- `tests/devx/test_debugger.py` — extends the
`start_method` allowlist to enable pexpect-based
debugger tests under the fork backend (the forked
child runs its own `trio.run` exactly like the
`trio` backend so pexpect's process-boundary
expectations hold).
- `tests/test_ringbuf.py`, `tests/devx/test_tooling.py`,
`tests/devx/test_pause_from_non_trio.py` — minor
adjustments for fork-survival.
Provenance: squashed from ~15 commits on
`subint_forkserver_backend` (`b350aa09`, `2ca0f41e`,
`e3f4f5a3`, `1af21210`, `9b05f659`, `66f1941f`,
`44bdb169`, `aa3e2309`, `c99d475d`, `985ea76d`,
`f3cea714`, `4b2a0886`, `060f7d24`, `383b0fdd`,
`530160fa`, `65fcfbf2`, plus inline polish).
(this patch was generated in some part by [\`claude-code\`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
There was a problem hiding this comment.
Pull request overview
Mainlines the new main_thread_forkserver spawn backend (and related hardening) to enable fork-based spawning that is compatible with subinterpreter work, while adding diagnostics and test-harness safeguards for known hang classes.
Changes:
- Adds/threads new spawn backends (
main_thread_forkserver, stubs forsubint_fork/subint_forkserver) through the spawn registry and runtime. - Hardens teardown/cancellation paths (esp. fork-related deadlocks) and adds hang-diagnostic tooling + pytest harness improvements.
- Adjusts tests/fixtures and introduces
tractor-reapfor orphan subactor and leaked shm cleanup.
Reviewed changes
Copilot reviewed 59 out of 62 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
tractor/spawn/_subint_forkserver.py |
Adds reserved/stub backend and helper for running subints on worker threads. |
tractor/spawn/_subint_fork.py |
Adds subint_fork stub backend documenting CPython limitation. |
tractor/spawn/_spawn.py |
Registers new spawn method keys, gating, and dispatch table entries. |
tractor/runtime/_state.py |
Adds runtime-vars defaults/reset and a setter API used for fork-child state reset. |
tractor/runtime/_runtime.py |
Adjusts spawn-handshake handling; improves stackscope enablement; fixes fork-teardown deadlock and bounds peer shutdown wait. |
tractor/ipc/_shm.py |
Adjusts shm unlink behavior and makes teardown more robust to races. |
tractor/ipc/_mp_bs.py |
Disables multiprocessing.resource_tracker and forces SharedMemory(track=False) for fork-safety. |
tractor/ipc/_linux.py |
Refines cffi import error reporting for eventfd support. |
tractor/devx/_stackscope.py |
Improves SIGUSR1 stackscope dump reliability (tee to file/tty) and schedules dump on trio loop when possible. |
tractor/devx/_debug_hangs.py |
Adds reusable hang diagnostics (faulthandler-to-file + resource delta tracking). |
tractor/devx/__init__.py |
Re-exports new hang-diagnostic helpers. |
tractor/_testing/pytest.py |
Adds --enable-stackscope, skipon_spawn_backend marker, and orphan subactor reaping fixtures; improves typing. |
tractor/_testing/addr.py |
Reduces cross-process test port collisions by salting randomness with PID. |
tractor/_root.py |
Adds env-var overrides for loglevel/spawn method; expands debug-backend compatibility list. |
tractor/_child.py |
Refactors child entry into _actor_child_main for shared usage by multiple backends. |
tests/test_spawning.py |
Extends capture-related skip to fork-based backend. |
tests/test_shm.py |
Skips shm tests on subint backend (known hang class) and documents fork workaround expectations. |
tests/test_ringbuf.py |
Skips ringbuf tests by default; adds cffi import guard. |
tests/test_pubsub.py |
Adds skipon_spawn_backend('subint') marker for known hangs. |
tests/test_inter_peer_cancellation.py |
Adds skipon_spawn_backend('subint') marker for known hangs. |
tests/test_infected_asyncio.py |
Adds per-test reaping fixture usage and routes registries explicitly to reduce cascade failures. |
tests/test_context_stream_semantics.py |
Threads reg_addr into nurseries to isolate runs and reduce interference. |
tests/test_cancellation.py |
Adds backend skips/typing tweaks; adds timeouts in some cases and routes reg_addr into nurseries. |
tests/test_advanced_streaming.py |
Reworks cancellation expectations and avoids pytest-timeout for fork backends (uses trio timeouts). |
tests/spawn/test_subint_cancellation.py |
Adds subint cancellation audit tests with hang dumps. |
tests/msg/test_pldrx_limiting.py |
Makes timeouts backend-aware to reduce fork-backend flakiness. |
tests/discovery/test_registrar.py |
Adds hang dump and backend skip notes for known subint hang class. |
tests/devx/test_tooling.py |
Skips greenback-dependent tooling tests when greenback isn’t installed. |
tests/devx/test_pause_from_non_trio.py |
Skips greenback-dependent pause tests when greenback isn’t installed. |
tests/devx/test_debugger.py |
Adjusts timeout usage for debugger test. |
tests/devx/conftest.py |
Uses env vars to select spawn backend/loglevel for example-script runs; expands supported spawners. |
tests/conftest.py |
Adds typing to fixtures touched by harness changes. |
scripts/tractor-reap |
Introduces CLI for orphan subactor reap and optional /dev/shm sweep. |
pyproject.toml |
Updates Python support range; reorganizes dependency groups; sets pytest defaults (notably --capture=sys). |
.github/workflows/ci.yml |
Adjusts pytest invocation formatting and capture mode override. |
.gitignore |
Expands ignore rules for local/AI tooling artifacts. |
ai/prompt-io/claude/20260422T200723Z_797f57c_prompt_io.raw.md |
Adds provenance log (raw). |
ai/prompt-io/claude/20260422T200723Z_797f57c_prompt_io.md |
Adds provenance log (summary). |
ai/prompt-io/claude/20260420T192739Z_5e8cd8b2_prompt_io.raw.md |
Adds provenance log (raw). |
ai/prompt-io/claude/20260420T192739Z_5e8cd8b2_prompt_io.md |
Adds provenance log (summary). |
ai/prompt-io/claude/20260418T042526Z_26fb820_prompt_io.raw.md |
Adds provenance log (raw). |
ai/prompt-io/claude/20260418T042526Z_26fb820_prompt_io.md |
Adds provenance log (summary). |
ai/prompt-io/claude/20260417T124437Z_5cd6df5_prompt_io.raw.md |
Adds provenance log (raw). |
ai/prompt-io/claude/20260417T124437Z_5cd6df5_prompt_io.md |
Adds provenance log (summary). |
ai/conc-anal/subint_forkserver_thread_constraints_on_pep684_issue.md |
Adds analysis/tracking doc for future constraints cleanup. |
ai/conc-anal/subint_forkserver_mp_shared_memory_issue.md |
Adds analysis/post-mortem doc for shm/resource_tracker fork issues. |
ai/conc-anal/subint_fork_from_main_thread_smoketest.py |
Adds CPython-level smoke test for the main-thread worker fork approach. |
ai/conc-anal/subint_fork_blocked_by_cpython_post_fork_issue.md |
Adds analysis doc for CPython post-fork refusal from subinterpreters. |
ai/conc-anal/subint_cancel_delivery_hang_issue.md |
Adds analysis doc for Ctrl-C-able parent-side hang after subint teardown. |
ai/conc-anal/fork_thread_semantics_execution_vs_memory.md |
Adds explanatory doc on fork semantics in multithreaded programs. |
.claude/skills/conc-anal/SKILL.md |
Adds internal guidance on teardown waits and capture-pipe hang pattern. |
.claude/settings.local.json |
Updates local tooling permissions/config. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+13
to
+16
| 'XXX SUBINT HANGING TEST XXX\n' | ||
| 'See oustanding issue(s)\n' | ||
| # TODO, put issue link! | ||
| ) |
Comment on lines
+26
to
+30
| reason=( | ||
| 'XXX SUBINT HANGING TEST XXX\n' | ||
| 'See oustanding issue(s)\n' | ||
| # TODO, put issue link! | ||
| ) |
Comment on lines
+269
to
+272
| infect_asyncio: bool = False, | ||
| task_status: TaskStatus[Portal] = trio.TASK_STATUS_IGNORED, | ||
| proc_kwargs: dict[str, any] = {}, | ||
|
|
Comment on lines
+21
to
+26
| > **Status:** placeholder. Today | ||
| > `--spawn-backend=subint_forkserver` aliases to | ||
| > `main_thread_forkserver_proc` (variant 1, see | ||
| > `tractor.spawn._main_thread_forkserver`). A follow-up commit | ||
| > in this PR series flips the alias to a `NotImplementedError` | ||
| > stub reserving the `'subint_forkserver'` key for the literal |
Comment on lines
+79
to
+82
| infect_asyncio: bool = False, | ||
| task_status: TaskStatus[Portal] = trio.TASK_STATUS_IGNORED, | ||
| proc_kwargs: dict[str, any] = {}, | ||
|
|
| # is honored inside `tractor._root.open_root_actor()`, | ||
| # so no per-script edits are required. | ||
| 'main_thread_forkserver', | ||
| 'subint_forkserver', |
Comment on lines
+156
to
+157
| --capture=fd | ||
| # ^XXX^ can't work with --spawn-method=main_thread_forkserver |
Comment on lines
+32
to
+35
| 'XXX SUBINT GIL-CONTENTION HANGING TEST XXX\n' | ||
| 'See oustanding issue(s)\n' | ||
| # TODO, put issue link! | ||
| ) |
Comment on lines
+24
to
+31
| try: | ||
| import cffi | ||
| except ImportError as ie: | ||
| if sys.version_info < (3, 14): | ||
| ie.add_note( | ||
| f'The `cffi` pkg has no 3.14 support yet.\n' | ||
| ) | ||
|
|
Comment on lines
+288
to
+355
| # `TRACTOR_LOGLEVEL` env-var wins over any caller-passed | ||
| # `loglevel` so devs/test-runs can crank (or silence) | ||
| # console verbosity without touching application code. | ||
| env_ll_report: str = '' | ||
| if env_ll := os.environ.get('TRACTOR_LOGLEVEL'): | ||
| loglevel = env_ll | ||
| env_ll_report: str = ( | ||
| f'Detected env-var setting,\n' | ||
| f'TRACTOR_LOGLEVEL={env_ll!r}\n' | ||
| f'\n' | ||
| f'Setting console loglevel per,\n' | ||
| f'loglevel={loglevel!r}\n' | ||
| ) | ||
| if ( | ||
| loglevel | ||
| and | ||
| loglevel.upper() != env_ll.upper() | ||
| ): | ||
| env_ll_report += ( | ||
| f'\n' | ||
| f'NOTE env-var OVERRIDES caller-passed,\n' | ||
| f'loglevel={loglevel!r}\n' | ||
| ) | ||
|
|
||
| loglevel: str = ( | ||
| loglevel | ||
| or | ||
| log._default_loglevel | ||
| ) | ||
| loglevel: str = loglevel.upper() | ||
|
|
||
| assert loglevel | ||
| _log = log.get_console_log( | ||
| level=loglevel, | ||
| name='tractor', | ||
| logger=logger, | ||
| ) | ||
| assert _log | ||
| if env_ll_report: | ||
| _log.info(env_ll_report) | ||
|
|
||
| # `TRACTOR_SPAWN_METHOD` env-var wins over any caller-passed | ||
| # `start_method` so devs/test-runs can swap the actor spawn | ||
| # backend without touching application code (e.g. driving | ||
| # the `examples/debugging/<script>.py` suite under each | ||
| # backend from `tests/devx/conftest.py::spawn`). | ||
| if env_sm := os.environ.get('TRACTOR_SPAWN_METHOD'): | ||
| start_method: str = env_sm | ||
| env_sm_report: str = ( | ||
| f'Detected env-var setting,\n' | ||
| f'TRACTOR_SPAWN_METHOD={env_sm!r}\n' | ||
| f'\n' | ||
| f'Setting spawn backend as,\n' | ||
| f'start_method={env_sm!r}\n' | ||
| ) | ||
| if ( | ||
| start_method | ||
| and | ||
| start_method != env_sm | ||
| ): | ||
| _log.warning( | ||
| env_sm_report | ||
| + | ||
| f'NOTE env-var OVERRIDES caller-passed,\n' | ||
| f'`start_method={start_method!r}`\n' | ||
| ) | ||
| else: | ||
| _log.info(env_sm_report) |
Owner
Author
|
Heh, looking through these commits, i actually think they're a bit too chunky 😂 🫣 .. so i might replace this with something else FYI. either way gonna use this one to get CI green.. |
main_thread_forkserver spawning backendmain_thread_forkserver spawning backend
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-worked/squashed history from #447 but focused on promoting and mainlining the (nearly CI audited as working)
tractor.spawn._main_thread_forkserverbackend tinkered from the histories (commit-chronologically) of,os.fork()spawning backend #448Surprisingly (at least to me) this patch set was mostly vibe coded with much human intuition/guidance to get us to what seems to be a much superior approach to the stdlib's
multiprocessing"forkserver" spawn method,which has been very problematic for us within rigorous "SC safe" constraints and more generally for nested subactor trees per,
Differences from #447 in this patch
despite the diff being identical this instead,
focuses/emphasizes the 3.13+ and-already-working
.spawn._main_thread_forkerserbackend despite bringing in the code first prototyped in thespawn._subint,spawn._subint_forkandspawn._subint_forkserverbackends each first presented in the draft PRs above.minimized set of larger commits reworked by
claudeto make the history by commit more immediately obvious, high-level-grokable for human onlookers.will ensure we get a clean CI run for all of,
--spawn-backend=main_thread_forkserver+--tpt-proto=[tcp|uds]detail out various new
pytestharness features developed while prototyping this backend and debugging various complex testing edge cases including some very difficult to decipher test-suite hangs on the new backend.Still TODO before this lands,