ci: Linux workflows with a gate on unrecorded divergence

.github/workflows/conformance.yml

@@ -0,0 +1,184 @@
+name: Conformance
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+env:
+  SHARDS: 4
+
+jobs:
+  # Each shard runs a slice of the conformance files against BOTH binaries.
+  # Slicing by file rather than by binary keeps the two sides on the same
+  # runner: the suite drives real processes through real PTYs and races under
+  # load, so adding a second binary's work to a shard's CPU is exactly what we
+  # do not want.
+  #
+  # Sharding is here because the work parallelises across machines and does not
+  # compress on one. The long pole is COMPILING the seventy debug test
+  # binaries, not running them: measured 2026-09-05 across all 140 invocations,
+  # the reported test time totals 5.1 minutes and no single invocation exceeds
+  # 12.5 seconds, while the compile before the first result took about 26.
+  #
+  # Do not target test runtime here. An earlier version of this comment said
+  # the tests were the cost, from a five-file sample that happened to contain
+  # the one seven-second outlier. That sample was not representative and the
+  # claim was wrong. If someone wants a bigger win than sharding, it is in the
+  # compile: a shared cargo cache, or building the test binaries once and
+  # distributing them, rather than four runners each compiling the workspace.
+  shard:
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    strategy:
+      fail-fast: false
+      matrix:
+        # Keep this list and SHARDS above in step. The combine job counts the
+        # results it receives and refuses if the two disagree.
+        index: [0, 1, 2, 3]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: DeterminateSystems/determinate-nix-action@v3
+      - uses: actions/setup-node@v4
+        with:
+          # The runtime is a pinned input, not a detail. It changes results.
+          # The Node binary truncates `pty completions fish` on a pipe under
+          # Node v24.18.0 — 123 lines of 167 — and does not under Node 22, on
+          # the same commit and the same test. Pinning only the reference
+          # COMMIT is not enough to make a comparison reproducible.
+          #
+          # 22 matches the major the Node pty's own CI uses. Changing it is a
+          # deliberate act: expect the divergence ledger to change with it.
+          node-version: 22
+
+      # The reference commit is pinned rather than tracked. A checkout seven
+      # commits behind produced eight false divergences on 2026-09-05, and
+      # every one was an artifact of the stale reference.
+      - name: Build the Node pty at the pinned commit
+        run: |
+          ref=$(cat crates/pty-conformance/node-ref)
+          echo "Node reference: $ref"
+          git clone --filter=blob:none https://github.com/compoundingtech/pty /tmp/node-pty
+          git -C /tmp/node-pty checkout --detach "$ref"
+          (cd /tmp/node-pty && npm ci --silent && npm run build --silent)
+          echo "node pty: $(/tmp/node-pty/bin/pty --version)"
+
+      - name: Build
+        run: nix develop --command cargo build --workspace --release
+
+      - name: Conformance, both binaries, shard ${{ matrix.index }}
+        env:
+          PTY_NODE_CHECKOUT: /tmp/node-pty
+        run: |
+          files=$(ls crates/pty-conformance/tests/*.rs \
+            | xargs -n1 basename | sed 's/\.rs$//' \
+            | awk "NR % $SHARDS == ${{ matrix.index }}" | tr '\n' ' ')
+          echo "shard ${{ matrix.index }} of $SHARDS: $files"
+          # Write to a FILE, not through a pipe.
+          #
+          # A pipe makes this step's completion depend on end-of-file, which
+          # needs every write end closed — including any inherited by a process
+          # that outlives the run. This suite leaves daemons behind, and on
+          # 2026-09-05 shard 2 finished its work in 2m44s and then sat for 38
+          # more minutes until the job timeout, with the runner reporting
+          # "Terminate orphan process: pid (23143) (pty-daemon)" at cleanup.
+          # Three runs, three times, each stopping at exactly the timeout.
+          #
+          # A file has no such dependency, and it keeps the partial output when
+          # a step IS killed, which a pipe into `tail` did not.
+          mkdir -p target/conformance
+          nix develop --command ./scripts/conformance-both.sh \
+            --node /tmp/node-pty/bin/pty \
+            --rust "$PWD/target/release/pty" \
+            --out "$PWD/target/conformance" $files \
+            > target/conformance/shard.log 2>&1 &
+          script=$!
+          # A heartbeat naming the most recent finished file, so a hang is
+          # visible while it happens rather than only in the post mortem.
+          while kill -0 "$script" 2>/dev/null; do
+            sleep 30
+            last=$(ls -t target/conformance/*/*.log 2>/dev/null | head -1)
+            echo "... still running; newest result: ${last:-none yet}"
+          done
+          wait "$script" || rc=$?
+          echo "===== conformance output ====="
+          cat target/conformance/shard.log
+
+          # A difference has to survive a second look before it counts.
+          #
+          # This suite drives real processes through real PTYs and races under
+          # load. Both binaries can lose. Measured 2026-09-06: a run reported
+          # nesting_prevention::restart_force_restores_the_attach and
+          # up_down::down_stops_only_named_sessions as Node-side differences,
+          # and both files then passed 3 of 3 locally against the same Node
+          # binary. They were lost races, not divergences.
+          #
+          # The retry runs only the differing files, so contention is far lower
+          # than the full shard, and it rewrites red.txt. A file that was clean
+          # contributes nothing to red.txt either way, so re-running just the
+          # differing ones is a complete result for this shard. A real
+          # difference is reproducible and survives; a race has to lose twice.
+          if [ -s target/conformance/red.txt ]; then
+            again=$(cut -d: -f1 target/conformance/red.txt | sort -u | tr '\n' ' ')
+            echo "===== differences seen; re-running only: $again ====="
+            nix develop --command ./scripts/conformance-both.sh \
+              --node /tmp/node-pty/bin/pty \
+              --rust "$PWD/target/release/pty" \
+              --out "$PWD/target/conformance" $again \
+              > target/conformance/retry.log 2>&1 || rc=$?
+            cat target/conformance/retry.log
+            echo "===== differences that survived the retry ====="
+            cat target/conformance/red.txt
+          fi
+          exit "${rc:-0}"
+
+      - name: Keep this shard's differences
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: red-${{ matrix.index }}
+          path: |
+            target/conformance/red.txt
+            target/conformance/shard.log
+            target/conformance/retry.log
+          if-no-files-found: error
+          retention-days: 14
+
+  gate:
+    needs: shard
+    if: always()
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/download-artifact@v4
+        with:
+          pattern: red-*
+          path: /tmp/red
+
+      # THE CHECK RUNS ONCE, OVER THE COMBINED SET. Do not move it into the
+      # shards. `check-divergences.py` fails in two directions, and the second
+      # one — a recorded divergence that no longer happens — can only be
+      # decided by knowing the divergence appeared in NO shard. A per-shard
+      # check sees a quarter of the tests, so every entry belonging to another
+      # shard looks stale to it, and the gate would fail every run for a reason
+      # that is not true.
+      - name: Combine and gate
+        run: |
+          # A missing shard is not a pass. If one shard died, its slice of the
+          # tests was never compared, and both halves of the check would then
+          # be wrong: an unrecorded divergence in that slice goes unseen, and
+          # every ledger entry belonging to it looks stale. Refuse instead.
+          # Count red.txt files, not directories. A shard that hangs still
+          # uploads its streamed log, so the directory exists while the result
+          # does not, and counting directories would call that a complete set.
+          n=$(ls /tmp/red/red-*/red.txt 2>/dev/null | wc -l)
+          if [ "$n" -ne "$SHARDS" ]; then
+            echo "expected $SHARDS red.txt results, found $n — a shard failed or hung."
+            echo "The gate needs every shard's result to judge either direction."
+            ls -R /tmp/red || true
+            exit 1
+          fi
+          cat /tmp/red/red-*/red.txt > /tmp/red/all.txt
+          echo "combined divergences from all $SHARDS shards:"
+          cat /tmp/red/all.txt
+          python3 scripts/check-divergences.py /tmp/red/all.txt

.github/workflows/nix.yml

@@ -0,0 +1,17 @@
+name: Nix
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    # Longer than the Node repository's 15: this build compiles Ghostty's
+    # terminal core with Zig on a cold cache.
+    timeout-minutes: 40
+    steps:
+      - uses: actions/checkout@v4
+      - uses: DeterminateSystems/determinate-nix-action@v3
+      - run: nix build --print-build-logs
+      - run: ./result/bin/pty --version

.github/workflows/test.yml

@@ -0,0 +1,32 @@
+name: Test
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@v4
+      - uses: DeterminateSystems/determinate-nix-action@v3
+
+      # Reporting only. Neither passes on main today — 1093 fmt diffs and 50
+      # clippy warnings, measured 2026-09-05 — so gating either one needs a
+      # cleanup commit first, which is a separate decision from adding CI.
+      # Printing the counts keeps the debt visible instead of hidden behind a
+      # check nobody enabled.
+      - name: Formatting and clippy (reporting only)
+        run: |
+          n=$(nix develop --command cargo fmt --all -- --check 2>/dev/null | grep -c '^Diff in' || true)
+          echo "cargo fmt --check: $n diff(s)" | tee -a "$GITHUB_STEP_SUMMARY"
+          nix develop --command cargo clippy --workspace --all-targets 2>&1 | tee /tmp/clippy.log || true
+          w=$(grep -c '^warning' /tmp/clippy.log || true)
+          echo "clippy: $w warning(s)" | tee -a "$GITHUB_STEP_SUMMARY"
+
+      - name: Build
+        run: nix develop --command cargo build --workspace --release
+
+      - name: Workspace tests
+        run: nix develop --command ./scripts/ci-test-workspace.sh

README.md

@@ -181,6 +181,24 @@ Three conditions travel with that result:
 - **This result does not transfer to a published asset.** The same test, with
   its removal control, has to run against the first one we ship.
 
+### CI is Linux only, on purpose
+
+Both workflows run on `ubuntu-latest`. Nothing in CI builds or tests this on a
+Mac.
+
+That matters more here than it usually would.
+[`crates/pty-core/src/proctable.rs`](crates/pty-core/src/proctable.rs) carries a
+macOS process-table reader that no Linux job ever compiles: libproc, plus a
+sysctl fallback with hand-declared `kinfo_proc` struct offsets. Offsets are
+exactly the kind of thing a new macOS moves. **So a change to that reader, or a
+macOS SDK change, breaks the Mac build and CI does not notice.**
+
+This is a decision, not an oversight. The tool runs on two Macs every day, so a
+broken Mac build surfaces immediately in use, and human use is the detection
+mechanism. That trade holds while the daily users are the affected users. If
+this ever ships to people who are not in the room, the trade changes and the
+macOS job comes back — via Nix, because Cargo cannot build it there.
+
 ## Usage
 
 ```sh
@@ -292,8 +310,20 @@ cargo build --release                        # target/release/pty
 ```sh
 cargo test --workspace                       # every crate's suite
 PTY_TEST_BIN=target/release/pty cargo test -p pty-conformance   # black-box, any binary
+./scripts/conformance-both.sh                # both binaries, side by side
+python3 scripts/check-divergences.py         # fail on an unrecorded difference
 ```
 
+`conformance-both.sh` runs every conformance file against both binaries and
+writes `target/conformance/red.txt`: the tests whose result differs.
+`check-divergences.py` compares that against
+[`crates/pty-conformance/divergences.toml`](crates/pty-conformance/divergences.toml)
+and fails both when a difference is unrecorded and when a record no longer
+happens, so the ledger cannot drift into a list of stale claims. CI runs both.
+The Node commit it compares against is pinned in
+[`crates/pty-conformance/node-ref`](crates/pty-conformance/node-ref); a stale
+reference invents differences that are not there.
+
 The workspace tests drive real programs through real PTYs and real daemons,
 with each test on its own `PTY_ROOT` under the temp dir. The conformance suite
 runs the same way against whichever binary `PTY_TEST_BIN` names, so it can be

crates/pty-conformance/divergences.toml

@@ -0,0 +1,66 @@
+# Recorded divergences between the Node pty and pty-rust.
+#
+# `scripts/conformance-both.sh` runs the conformance suite against both
+# binaries and writes the tests whose result differs.
+# `scripts/check-divergences.py` compares that list against this file and
+# fails when the two disagree, in EITHER direction:
+#
+#   * a divergence that is not recorded here — nobody has judged it;
+#   * a record here that no longer happens — the entry has gone stale.
+#
+# The second rule is the one that keeps this file honest. A ledger that only
+# grows becomes a list of claims nobody re-reads, and its entries outlive their
+# reasons. Failing on a resolved entry forces this file to describe today.
+#
+# Fields:
+#   test    `<conformance file>::<test name>`, as it appears in red.txt.
+#   side    "node" = the test fails on the Node binary.
+#           "rust" = it passes on Node and fails on pty-rust. Treat any new
+#                    rust-red entry as a regression until proven otherwise.
+#   kind    "intended"    — a difference we chose; point at its decision record.
+#           "node-defect" — the Node reference is wrong and pty-rust is right.
+#   reason  What differs, how it was measured, and why it is recorded and not
+#           fixed here.
+#
+# TWO INPUTS DECIDE WHAT THIS FILE SHOULD SAY, AND BOTH ARE PINNED.
+#
+# The Node COMMIT is pinned in `node-ref`. A checkout seven commits behind
+# invented eight divergences on 2026-09-05, every one an artifact.
+#
+# The Node RUNTIME is pinned in the workflow, and it matters just as much. This
+# file first recorded the Node binary truncating `pty completions fish` on a
+# pipe, 123 lines of 167. That was measured on x86_64 Linux with Node v24.18.0,
+# where it reproduces every run. CI runs Node 22 and does not reproduce it at
+# all: same commit, same test, different answer.
+#
+# The boundary is NOT simply the Node major, and it would be wrong to record it
+# that way. The Node pty owner measured the same unfixed binary on arm64 macOS
+# and got the full 12138 bytes under BOTH Node 22.21.1 and Node 24.0.2. So the
+# unfixed failure depends on runtime, platform and scheduling together, and the
+# only honest summary is that it does not reproduce in the environment this
+# gate runs in.
+#
+# The defect is real: their forced delayed-write fixture, which does not rely
+# on natural scheduling, prints zero bytes against the old exit path and the
+# full 12138 against the fixed one, on Node 22 and 24 alike. A fix is in
+# progress there. These entries were removed because a ledger records what CI
+# can observe, not because the defect is not real.
+#
+# Measured against Node 86dcc5eb7bbca73ab9635ea07ab1018ec43ede61 on Node 22,
+# 2026-09-06: pty-rust 660/660, the Node pty 659/660.
+
+[[divergence]]
+test = "spawn_options::pwd_is_the_directory_as_written_wherever_the_caller_stood"
+side = "node"
+kind = "node-defect"
+reason = """
+pty-rust sets `PWD` in the child environment to the working directory as the \
+caller wrote it, following node-pty's own `env.PWD = cwd` in \
+`src/unixTerminal.ts`. The Node pty does not, so its child has no `PWD` at all \
+and the shell derives one from `getcwd()`. Where the directory is reached \
+through a symlink those disagree, and `pty run --cwd X` then answers one way \
+from inside X and another from anywhere else. pty-rust is the correct side. \
+This divergence was not written down anywhere before the gate found it: it is \
+in neither docs/parity.md nor a decision record. Remove this entry if the Node \
+pty adopts the same behaviour.
+"""

crates/pty-conformance/node-ref

@@ -0,0 +1 @@
+86dcc5eb7bbca73ab9635ea07ab1018ec43ede61