Skip to content

v1.0.0

Choose a tag to compare

@takeokunn takeokunn released this 25 Jul 16:59
· 75 commits to main since this release
v1.0.0
9f305d6

First stable release. The exported API is unchanged from 0.2.0 and is now
covered by semantic versioning; what this release adds is evidence about how
it behaves on the platforms it claims to support, gathered by actually
running the suite on Linux rather than inferring from macOS. That turned up
two real defects, both invisible on macOS, and one broken CI check.

The first of the two ### Known Limitations recorded under 0.1.0 -- the
drain-timeout bound -- is resolved. The second, a group of timing-sensitive
process-group tests, is not: it is narrowed, re-diagnosed, and still skipped
on Linux.

Correctness

  • drain-timeout-seconds is now honoured on Linux. A copier thread
    drained a child's stdout/stderr with a blocking read(2), and
    %drain-copiers unstuck one that overran its deadline by force-closing
    the stream under it. Closing a descriptor wakes a thread already parked
    in read(2) on macOS/BSD, but that is a BSD courtesy rather than
    anything POSIX promises, and Linux does not do it: the reader stayed
    parked until whoever else held the pipe's write end let go. After
    sh -c "sleep 5 & exit 0" that is a backgrounded descendant which
    outlives the leader by design, so the wait was effectively unbounded.
    The observed symptom on Linux was worse than 0.1.0's note described --
    not merely run overrunning its documented bound, but run signalling
    process-io-error :cleanup ("Copier thread did not terminate after its
    stream was closed") once the force-close failed to land.

    The read loop now waits with poll(2) on a bounded timeout and checks a
    stop flag between turns (%await-fd-readable), so the decision to give up
    is taken by the reader itself instead of being inflicted on it through the
    descriptor. The bound then holds by construction on any POSIX platform
    rather than by accident on some. The flag is checked before readiness,
    which bounds the loop even against a child that never stops producing:
    one poll interval plus one bounded read(2) per turn. Polling costs no
    wakeup rate the library was not already paying, since communicate's own
    deadline loop already runs at +default-poll-interval+.

    %drain-copiers' escalation gained the cooperative stop as its first
    rung -- ask, then force-close, then signal -- ordered by what each costs
    when it fires, in the same continuation-passing shape as
    escalate-unless-gone's SIGTERM -> SIGKILL -> give-up. Force-closing is
    now a fallback for a copier parked where a flag cannot reach it (a
    read-sequence on a non-fd stream) rather than the primary mechanism.
    %communicate-base's cleanup path was reordered to match: it retires the
    copiers before their streams, so close-process-streams no longer closes
    a descriptor under a live reader on every pass -- a descriptor the OS is
    free to reissue the moment it is closed.

  • :on-timeout and :on-cancel are validated at the entry points that
    own them.
    Every entry point resolves these by comparing against
    :error and treating anything else as :return, which made an
    unrecognised value indistinguishable from a deliberate :return instead
    of an error. run, run-command and run-pipeline compounded it: each
    hands communicate a hardcoded :on-timeout :return and decides for
    itself whether to signal, so %validate-communication-options' guard
    never saw what the caller wrote. A misspelt :errror therefore read as
    :return and silently swallowed the very timeout or cancellation the
    caller had asked to have signalled. %validate-outcome-policy now guards
    each policy where it is accepted -- run (:on-timeout, :on-cancel),
    run-command (both), run-pipeline (both), and communicate
    (:on-cancel, which had no guard either).

Testing

  • The drain-timeout regression test ("run returns boundedly when an exited
    leader leaves a pipe-holding descendant") is no longer skipped on Linux --
    the fix above makes it platform-independent, and it passes on CI.

  • The other seven Linux skips are re-diagnosed rather than removed. They had
    been filed under the same "process-group/communicate timing is not
    guaranteed identical on Linux" heading as the drain bug, which conflated
    two unrelated things. They do not share its cause: each asserts that a
    process group is gone within a 0.1s grace period, which a contended
    shared CI runner cannot reliably deliver. (They pass on an uncontended
    aarch64 Linux container and fail on GitHub's x86_64 runners -- contention,
    not architecture.) Their skip reasons now say that instead. They are not
    given more headroom because a timing assertion loose enough to survive
    arbitrary contention no longer asserts the timing; the honest fix is to
    make the assertion event-driven rather than deadline-driven, which is
    deferred.

  • The coverage ratchet had been failing every Linux CI build, on a
    comparison it should never have made. The floor tracked the figure from a
    complete run (macOS, nothing skipped) but was enforced against the reduced
    Linux run, so CI reported the skipped tests as a coverage "regression" --
    86.9% against an 87.0% floor -- with every test passing. The floors are now
    enforced only when the whole suite ran (+suite-complete-p+); otherwise
    coverage is reported with an explicit note that it is not comparable.

  • A guard-clause test that names a program which does not exist proves
    nothing.
    t/run-timeout-test.lisp's "run rejects invalid timeout
    controls before spawning" spawned /bin/true, which macOS does not ship
    (true lives in /usr/bin there). All ten of its assertions passed on a
    process-launch-error for the missing file -- identically, and just as
    green, whether or not the guard under test existed. That is what hid the
    :on-timeout gap above: the assertion only had to mean something once
    the suite was first run on Linux, where /bin/true does exist. A
    %true-program fixture now resolves true through the ambient PATH,
    matching the existing %spawn-sleeping fixture, and is used by the 17
    call sites that actually spawn. The remaining literal /bin/true
    occurrences are in make-command data tables that never spawn, where the
    string is inert.

  • Added a regression test asserting that all four entry points reject an
    unrecognised outcome policy rather than reading it as :return.

  • Coverage ratchet advanced to 87.4% expression / 81.5% branch (from
    87.0/79.5), measured on a complete run.

Documentation

  • Documented the timeout and cleanup deadlines. grace-period,
    poll-interval, timeout-signal, kill-signal and
    drain-timeout-seconds previously appeared only inside run's signature
    in the options table -- five knobs with no stated meaning or default,
    covering the escalation behaviour that is the library's whole reason to
    exist. The execution guide now gives them a table of their own, explains
    why signals go to the process group rather than the child, and explains
    why draining has a deadline separate from the child's (a descendant that
    outlives the leader inherits the same pipe and can hold it open
    indefinitely).

  • Added a "Running the suite on both platforms" section to the development
    guide, covering the container invocation for checking Linux behaviour
    locally, and the two habits this release's bugs argue for: never name a
    program a guard-clause test does not intend to execute, and prefer fixing
    a platform difference to skipping the test that catches it.

Build/environment

  • flake.nix hardcoded "0.2.0" in four separate places (the docs
    derivation, the cl-process-kit package, and the cl-process-kit-pty
    package), plus two more in cl-process-kit.asd (cl-process-kit and
    cl-process-kit/test) -- six places a release has to remember to bump in
    lockstep, with no build-time check that they agree. nerima-lisp/cl-boundary-kit
    v0.6.0 already fixed the identical problem in its own flake.nix (found
    while auditing this project's own environment setup for the same class
    of drift); ported the same technique here: a version let binding
    parses the :version form out of cl-process-kit.asd line-by-line
    (Nix's builtins.match is whole-string-anchored and . doesn't span
    newlines, so a single multi-line regex doesn't work) and every Nix
    package now inherits it. A release now only ever edits the .asd.
  • cl-process-kit/pty and cl-process-kit/pty-test were missing the
    :version/:author/:maintainer/:license/:homepage/:bug-tracker/
    :source-control metadata the other two systems in the same file
    already carry -- added it for consistency.

Production readiness

  • Added SECURITY.md, SUPPORT.md, and CONTRIBUTING.md (GitHub's
    standard community-health filenames, which its UI surfaces automatically
    regardless of README content) -- this repository had none, unlike sibling
    nerima-lisp projects (cl-weave has all three). Scoped and sized for
    this project specifically rather than copied verbatim: SECURITY.md
    names the concrete classes of report that actually apply here (shell/
    argument injection via run-shell/make-command, process-group
    isolation failures, spawn-native's privilege/credential handling), not
    a generic template; CONTRIBUTING.md documents the coverage ratchet and
    nix flake check as the authoritative (not just convenient) verification
    step, matching how this project is actually developed and verified
    throughout this changelog.

CPS

  • src/copier.lisp's %drain-copiers had a two-step "join within the
    remaining time; if that times out, force-close the stream and give it
    one more brief join" escalation inlined as nested whens. Extracted
    %join-copier-unless-timed-out (copier timeout on-timeout), which calls
    the on-timeout continuation only if the join actually times out --
    the same shape as communicate.lisp's escalate-unless-gone, applied
    one level down at the copier-thread join instead of the process-group
    signal escalation. %drain-copiers's two-step escalation is now two
    nested calls to it instead of two nested whens, matching this
    codebase's established call-with-X/continuation-parameter idiom.

Correctness

  • src/copier.lisp's %join-copier accepted an &optional timeout and, if
    omitted, joined a copier thread with no bound at all -- an unbounded wait
    on a thread that can be blocked reading a live (possibly-hung) child
    process's stdout/stderr. Audited every SB-THREAD:JOIN-THREAD/
    PROCESS-WAIT/SB-EXT:PROCESS-WAIT call in src/ for this same class of
    risk: the other four unbounded PROCESS-WAIT calls (communicate.lisp,
    native-spawn.lisp, process-handle.lisp, spawn.lisp) are each
    provably safe by construction -- reached only after the process has
    already been SIGKILLed (which POSIX guarantees terminates it) or is
    already OS-reported as terminal, so the wait there reaps a dying/dead
    process rather than blocking on a live one. %join-copier was the one
    genuine gap. Made timeout a required parameter (it was never actually
    called without one -- the unbounded branch was dead in practice, not just
    in principle) and updated its docstring to state the "never unbounded"
    contract explicitly.

Data/logic separation

  • src/native-spawn.lisp's %native-spawn-phase mapped the native spawn
    trampoline's byte phase codes to keywords with a case form embedded
    directly in the lookup function. Hoisted the mapping into a top-level
    +native-spawn-phases+ alist, leaving %native-spawn-phase as a plain
    (or (cdr (assoc number +native-spawn-phases+)) :unknown) traversal --
    the phase table can now be read, diffed, or extended on its own, separate
    from the lookup logic that applies it, matching the pattern already used
    elsewhere in src/ (e.g. +task-terminal-state-rules+).

Readability

  • src/communicate.lisp: communicate's cancellation watcher body -- a
    ~20-line anonymous lambda passed straight to sb-thread:make-thread --
    is now a named local function, run-cancellation-watcher, alongside its
    labels siblings (finished-p, mark-cancelled, etc.), so the thread
    creation call reads as #'run-cancellation-watcher instead of an
    unlabeled inline block. Extracted with paredit-cli.
  • src/spawn.lisp and src/command.lisp each independently validated a
    "KEY=VALUE" environment-string entry (non-empty key before =,
    rejecting a duplicate key) with byte-for-byte identical logic under two
    different error-message labels. Extracted the shared shape into
    %validate-environment-entry-shape (src/command.lisp, next to
    %validate-environment-entries, its other caller) via
    paredit refactor extract-function; spawn.lisp's
    %validate-environment now calls it with "ENVIRONMENT", matching the
    uppercase label convention %validate-environment-entries already uses
    for "ENVIRONMENT-POLICY"/"ENVIRONMENT-UPDATE" (previously
    spawn.lisp alone used mixed-case "Environment" in its error text; no
    test asserted on the literal message, so this also fixes a real
    inconsistency, not just a duplication). Found via paredit inspect similarity --threshold 0.85 src (score 44.5, 46 shared AST nodes) rather
    than searched for. Net effect: fewer total expression/branch points to
    cover (4736/676 -> 4726/670), so src/'s coverage percentage rose
    slightly (87.5% -> 87.6% expression) as a side effect of having less
    duplicated code, not a coverage change pursued for its own sake.

Testing

  • t/validation-test.lisp's make-command/spawn-native guard-clause
    tables now use cl-weave:it-each instead of a hand-rolled dolist over a
    list of (label . thunk) conses: every one of the 22 + 3 malformed-input
    rows is now its own independently-named, independently-reported test case
    (e.g. "rejects an empty program", "rejects a dotted (improper)
    :environment-update list") instead of one aggregate pass/fail that hid
    which row actually failed. Test count 152 -> 174; behavior and coverage
    unchanged, since it's the same guard clauses exercised the same way.
  • Added t/mutation-test.lisp: cl-weave:run-mutations mutation-tests
    process-success-p and pipeline-success-p (src/command.lisp),
    systematically flipping their comparison/boolean logic and asserting the
    existing case battery notices every one-operator change
    (cl-weave:assert-mutation-score at a required 1.0 -- no surviving
    mutants). Coverage proves a line executed, not that a wrong result there
    would be caught; mutation testing closes that specific gap for these two
    pure predicates. Follows the exact pattern nerima-lisp/cl-tty-kit
    already established for this ecosystem (contrib/weave-mutation-tests.lisp):
    the DEFUN body is read live from src/command.lisp on every run (never
    copied into the test file), so there is nothing to fall out of sync with
    the real implementation.

Dependencies

  • Bumped cl-weave v0.11.0 -> v1.0.0, cl-boundary-kit v0.5.0 -> v0.6.0,
    cl-log-kit v1.1.0 -> v1.6.0, cl-tty-kit v0.5.0 -> v0.6.0. Checked each
    upstream changelog/diff for breaking changes before bumping: none touch
    the surface this library actually calls (cl-boundary-kit's v0.6.0
    unbounded-wait-by-default change is scoped to its own
    process-boundary-run/process-kit-run-fn, which this library never
    calls; cl-log-kit's "logger as explicit first argument" breaking change
    landed at v1.0.0 and %log (src/logging.lisp) already passed it
    explicitly). Verified with a full nix flake check (checkout-tests
    152/152 at 87.5%/79.9% coverage, pty-tests 6/6), not just a local
    sbcl --script run.
  • flake.nix's four nerima-lisp inputs are consumed purely as raw ASDF
    source trees (buildASDFSystem src, or CL_SOURCE_REGISTRY at
    runtime) -- none of their own flake packages/checks outputs are ever
    used. cl-weave, cl-boundary-kit, and cl-log-kit were still declared
    as full flakes (missing flake = false, unlike the already-correct
    cl-tty-kit), so nix flake update pulled in their entire transitive
    dev-only input graphs (treefmt-nix, cl-json-kit, and cl-json-kit's
    own sub-inputs) into this project's flake.lock for no reason. Added
    flake = false to all three, shrinking flake.lock from 34 nodes to 6.
  • cl-tty-kit v0.6.0 vendors nerima-lisp/cl-prolog as a git submodule
    (vendor/cl-prolog) and self-registers that path from its own .asd;
    a plain github: fetch does not follow submodules (regardless of a
    ?submodules=1 query string, which only the git+https:// fetcher
    honors), so the Nix sandbox had an empty vendor/cl-prolog and
    cl-process-kit/pty-test failed to load with Component #:CL-PROLOG not found, required by #<SYSTEM "cl-tty-kit">. This only broke inside nix flake check's sandboxed pty-tests, not local sbcl --script runs
    against a manually-cloned ~/ghq checkout that already had the
    submodule populated -- another case (see the [0.2.0] entry below) of a
    local run passing while the authoritative sandboxed check does not.
    Fixed by switching cl-tty-kit's input to
    git+https://github.com/nerima-lisp/cl-tty-kit?ref=refs/tags/v0.6.0&submodules=1.

Documentation

  • Audited every public function signature documented in README.md and
    docs/src/guide/*.md against the actual &key lists in src/run.lisp,
    src/spawn.lisp, and src/communicate.lisp (plus the condition
    hierarchy in src/conditions.lisp, the process-result/pipeline-result
    structs in src/types.lisp, and the PTY/event/task accessors in
    src/pty.lisp/src/async-events.lisp/src/async-task.lisp, which were
    already accurate). Found run's output keyword (%run-base) -- the
    stdout counterpart of error's policy, controlling whether stdout is
    :captured, :inherited live, or sent to a stream -- completely
    undocumented, even though it has been part of the public API since the
    [0.1.0] entry below; error's own accepted values were also
    under-described (listed only :capture/:output, omitting :inherit
    and stream targets that %valid-run-output-policy-p also accepts). Also
    found README.md's run/communicate signatures missing
    decoding-error-policy and docs/src/guide/execution.md's run
    signature and docs/src/guide/async.md's spawn signature missing
    fd-limit -- each doc had independently drifted to cover only the
    option added most recently in its own edit history. Fixed all of these
    in docs/src/guide/execution.md and docs/src/guide/async.md, which
    remain the single authoritative reference (see the next entry for why
    README.md no longer duplicates them).
  • README.md had grown a ~450-line "API" section duplicating the option
    reference already maintained on the MkDocs site (docs/src/guide/*.md,
    docs/src/reference/results-and-conditions.md) -- the exact duplication
    responsible for the signatures drifting out of sync in the entry above,
    and for the site's own "Native spawn trampoline" page having no README
    counterpart at all. Simplified README.md to a landing page (intro,
    install, one minimal usage example, links to the published docs) and
    removed the API section and the Development section's test-suite/
    source-layout breakdown entirely, leaving docs/src/ as the only place
    a signature or option list is written down.