Skip to content

v3.0

@RhysU RhysU tagged this 20 Jun 05:03
- [API] The Jobserver is now a thin handle over a separate slot-owning
  Resources object. Submission controls (env, preexec, sleep) are no longer
  per-call keyword arguments on submit()/map(); instead revise_env(),
  replace_preexec(), and replace_sleep() each return a sibling handle that
  shares the same slots and selector. Teardown is reference-counted across
  context-manager entries on the shared Resources.

- [API] submit() and map() lost their env=, preexec_fn=, and sleep_fn=
  parameters. Code that passed these per call must now derive a configured
  handle first (e.g. js.revise_env({...}).replace_preexec(fn)) and submit
  through it. The shared-slots contract spans every sibling handle.

- [API] modify_env was renamed to revise_env and now validates eagerly:
  keys must be str (values str-or-None to unset), keys cannot be empty,
  contain '=', or contain NUL, and values cannot contain NUL. The diff is
  stored canonically as a read-only MappingProxyType so a shared sibling
  handle cannot mutate it.

- [API] replace_preexec() now forwards positional and keyword arguments to
  the replacement callable: replace_preexec(replacement, /, *args, **kwargs).
  This removes the need for functools.partial or lambda wrappers when a
  pre-exec hook needs configuration (#428).

- [API] The SubmissionDied exception was renamed to LostResult and its
  meaning was reframed around the result pipe: it now reports that a worker
  was killed, exited, or exec'd away before sending a result, rather than
  "a child process died unexpectedly." Update imports and except clauses
  accordingly.

- [API] Future.when_done()'s fn parameter is now positional-only so a
  callback may itself be forwarded a keyword argument named 'fn' without a
  name collision. The method still returns a per-Future registration seqno.

- [API] The package now exposes a __version__ string sourced from
  importlib.metadata, falling back to "0.0.0+unknown" in an uninstalled
  source tree. A test asserts it agrees with the installed distribution
  metadata.

- Timeouts are validated up front: negative timeouts and non-finite values
  (NaN, Inf) now raise ValueError instead of being silently accepted. This
  closes a class of hangs and undefined waits across wait(), result(),
  submit(), and map().

- The internal queue layer was redesigned from a single MinimalQueue into an
  AbstractQueue hierarchy: AbstractPicklingQueue with SPSCQueue and MPMCQueue
  variants for objects, plus a FixedBytesQueue for the single-byte slot
  tokens. Slot counts are now bounded by the atomic pipe_buf() write size.

- KeyboardInterrupt and other BaseExceptions raised inside a worker no longer
  dump multiprocessing's _bootstrap traceback to the parent's stderr; the
  noise is suppressed via the preexec path while the failure still surfaces
  as LostResult (#420).

- Result deserialization in the parent was hardened: Future.wait() now
  catches BaseException from unpickling, the worker catches all pickle
  failures in its result-send fallback, and pickle errors from
  process.start() are wrapped in a clearer message. PICKLE_DUMP_ERRORS is
  centralized in _compat so the worker and executor classify failures
  identically.

- A flaky Connection-leak test on Python 3.12 (a race between the child's
  resource-sharer teardown and the parent's fd detach, surfacing as either
  FileNotFoundError or EOFError) was replaced with deterministic per-branch
  tests (#410).

- __setstate__ on both Resources and Jobserver gained explicit tuple
  shape/length checks that survive python -O, and the worker self-heals if a
  KeyboardInterrupt escapes from __setstate__. Lazy per-process selector
  rebuild and a refcount lock keep nested fan-out and sibling handles safe
  across forks.

- Packaging and docs were modernized: a PyPI-flavored README is generated and
  shipped as the project readme, the license is declared as the PEP 639 SPDX
  expression "MPL-2.0" (requiring setuptools>=77), email addresses were
  removed from copyright headers, and executor/process-pool/worker-pool
  keywords were added.

- The test and example suites were overhauled: numerous ResourceWarning
  leaks were fixed, design tests were reorganized and ~18 unit tests were
  distilled from torture-test coverage, executor forward-progress under slot
  saturation is now covered, and the examples were renumbered ex01-ex12 with
  new lifecycle and cancellation samples plus a stabilized benchmark harness.
Assets 2
Loading