Skip to content

rushd: WS3/WS4 integration with Reporter prerequisites - #6018

Open
Sean Larkin (TheLarkInn) wants to merge 177 commits into
mainfrom
user/selarkin/rushd-ws3-ws4-4747
Open

rushd: WS3/WS4 integration with Reporter prerequisites#6018
Sean Larkin (TheLarkInn) wants to merge 177 commits into
mainfrom
user/selarkin/rushd-ws3-ws4-4747

Conversation

@TheLarkInn

@TheLarkInn Sean Larkin (TheLarkInn) commented Sep 7, 2026

Copy link
Copy Markdown
Member

Purpose

Implementation for #5894, focused on WS3 (#5898) and WS4 (#5899), with the Reporter prerequisites from #5858.

Fixes #5898. Fixes #5899.

The existing rush and rushx defaults are unchanged. The separate rush-client and rushx-client binaries remain opt-in. This does not perform the Rush 6 cutover or close the full epic.

Implemented

  • Native warm build/rebuild execution and real Rushx package scripts, with immutable request inputs, byte-preserving global streams, stdin credits/EOF, cancellation, queue controls, and fail-closed no-replay behavior.
  • Generation-safe fingerprint reloads, bounded pre-execution restart retry, attested version-selected launchers, detached startup/crash handoff, idle shutdown, and native install/update results delivered before cleanup and successor startup.
  • Generation-fenced structured graph commands, automatic warm-controller ownership, deterministic LRU/telemetry retention, observation-only daemon.watch, and live generation/reload/resource status without initializing a cold graph.
  • Reporter integration and reviewed fixes for bootstrap control ownership, structured output capture, watch presentation, and reporter timer cleanup.

Reporter dependency stack

This branch preserves all original Reporter branch histories. Their changes remain in this diff until the existing PR stack lands; this integration PR is not a substitute for reviewing that stack.

Phase Existing PR
R2A configuration #5987
R2B frontend controls #5989
R3A session sink #5988
R3B shadow events #5991
R3C parity #5992
R4A telemetry/privacy #5990
R5A operation adapter #5996
R5B reporters #5997
R6A bootstrap handoff #5993
R7A Heft negotiation #6000
R8A AI qualification #5999
R10A demo/docs #5998

R9 (#5982), R11 (#5984), and WS5 (#5900) remain subject to their actual review, compatibility, and release/ecosystem gates. No legacy-removal or default-selection gate is bypassed.

The Reporter stack has been restacked onto main@77abfe24 with guarded pushes and preserved backup refs. The scoped follow-up fixes are in #6019. Independent approval and dependency-ordered landing remain separate from this integration PR.

This PR is ready for review. Independent approval and the Reporter prerequisite landing order still apply before merge; the opt-in clients do not require the later default-selection cutover.

Windows correction batch

  • Clients, launchers, and sessions use native physical filesystem identity consistently. Rushx separately preserves validated invocation spelling, including Windows 8.3 names and junctions, for child cwd, lifecycle environment, registration diagnostics, and pnpm-sync. Exact native output assertions are retained; output is never rewritten to manufacture parity.
  • The shared LockFile Windows implementation now uses actual kernel exclusive sharing instead of assuming exclusive creation remains exclusive after opening. A dirty companion preserves interrupted-owner state across close/delete handoffs. This correction applies to all Windows LockFile consumers, not only the daemon; no native addon or dependency is added.
  • Native rush purge retains its own active repository lock and Windows dirty companion while removing other temporary files. The regression verifies that purging does not delete live ownership or bypass the mutex.
  • Native operation environments retain case-insensitive Windows Path values. Startup handoff treats sharing-denied ownership reads as unknown, not released, and retries only within the existing deadline. Owned child resources are joined before bounded Windows fixture cleanup.
  • Native IPC fixtures launch a real Node IPC child instead of relying on cmd.exe to forward the descriptor. Graph CLI scenarios are separated, Windows cancellation exercises the actual CLI signal handler, and graph reload setup is separated from its assertions. Production startup deadlines and no-replay rules are unchanged.
  • Native CLI test continuations are bound to their own fixtures and joined completely before deletion, including owned graph watches registered during cleanup. Initial daemon preparation is separated from reload/restart assertions; a timed-out callback cannot start work in, or remove, the next test's repository.
  • Native Rushx registration lookup is unchanged unless the caller explicitly supplies a different invocation namespace. Embedded frontend tests capture and release their real parser locks after execution; production lock lifetimes and existing native snapshots are unchanged.
  • Generated Git fixtures set core.autocrlf=false locally so host-global Windows settings do not produce LF-to-CRLF warnings. User Git settings and CI warnings-as-errors are unchanged. The real native graph suite uses an explicit 15-second integration budget after repeated setup timeouts, without changing assertions, cleanup limits, or production deadlines.
  • Linux global requests wait for their captured detached group/session to settle before completion, not merely for signal delivery or leader stream closure. The bounded inspection uses procps-compatible ps --sid, never resends signals while waiting, and surfaces inspection/timeout failures.
  • Remaining launcher fixtures join the actual daemon PIDs returned by start/restart before bounded Windows cleanup. Graph fixtures are removed only after successful host closure. Retention-only fixtures are isolated from accumulated Jest-worker RSS; production memory defaults and dedicated pressure tests are unchanged.

Validation

Hosted acceptance passed on exact head ba5753a9f146325deccb49e5aea485c6ba5aa3e2, run 34217211265.

Runtime Platform Result
Node 20 Ubuntu PASS
Node 22 Ubuntu PASS
Node 24 Ubuntu PASS
Node 26 Ubuntu PASS
Node 24 Windows PASS
Node 26 Windows PASS

Every job completed the full repository build/test run, package-manager integration, generated README verification, second-checkout update/build/test, and repeated build-cache verification. CodeQL and license/CLA checks also pass on this head.

Coverage includes actual CLI install/update, native coexistence, same-PID reloads, new-PID successors, startup-client death, ownership failures, graph-token races, real warm policies, ordinary CLI status, native Windows mutex/purge behavior, and exact Rushx parity through physical paths and 8.3 aliases.

All blocking Reporter qualification and performance gates remain enforced. Earlier failure logs and diagnostic observations are retained separately; the final green run used committed test budgets and the full workflow, without temporary diagnostic overrides or relaxed quality thresholds.

The opt-in registry integration also exercised a genuine published foreign-engine installation and runtime/protocol attestation. Incompatible installed engines are refused rather than relabeled as supported.

Deliberate safety boundaries

  • Unsupported native configurations and incompatible published launchers retain explicit pre-execution native fallback; transport loss or possibly executed work is never replayed.
  • Warm memory limits are best-effort sampled accounting, not a hard process-tree RSS ceiling. Active/protected work is not evicted.
  • daemon.watch controls host project observation, not autonomous script execution or native Rush watch mode.
  • Ambiguous pre-bind launcher failure retains its startup reservation instead of guessing that a PID can be killed or reclaimed.

Sean Larkin (TheLarkInn) and others added 30 commits August 28, 2026 02:52
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Create the authoritative frontend reporter host before version selection, register global reporter controls, and preserve legacy output unless a non-legacy reporter is explicitly selected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Consume the repository experiment before Rush version selection, keep agent detection out of pre-major defaults, strip frontend-only controls before engine handoff, and preserve legacy verbosity compatibility.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Make the demo independent of inherited reporter controls, verify exclusive output destinations, and correct log-level, alias-position, and latest-log guidance.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Validate non-public diagnostic codes against the registry, bucket unknown categories, and bound retained telemetry dimensions deterministically.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Stop reporter control scans at the pass-through separator and add an exactly-once frontend close contract across success, failure, and termination paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Keep protocol metadata root-owned, gate mixed-privacy diagnostics, and bound producer attribution with trusted deterministic retention.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Keep reporter controls out of ts-command-line globals, gate incompatible engines before initialization, and enforce bounded signal and close-error behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Preserve unsupported custom reporter values until frontend ownership is unambiguous, and narrow emergency legacy stripping to the reporter selection flag.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Probe reporter ownership without requiring a value, then enforce strict reporter parsing only after frontend ownership is established.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Derive bounded producer retention priority from parent-session provenance instead of child-controlled package namespaces.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Retain unknown ownership during transient sharing denial, join resources before bounded fixture cleanup, and preserve Windows Path casing through native operation environments.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Canonicalize fixture watcher paths before fs.watch, preserve Windows early-stdin exit results, and isolate the version-miss registry through native npm configuration. Join CLI child closure before deleting fixtures and give the real metadata subprocess its scoped integration-test budget.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
…ul cancellation

Split real CLI graph coverage into independent scenarios without weakening assertions. Drive the real signal handler through a Windows-only IPC harness, retain native POSIX SIGINT delivery, and join spawned client closures before fixture cleanup. Scope the two-generation restart case to 45 seconds based on its measured 33-second native Windows execution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the existing node-plus-relative-script event-hook convention and reuse bounded Windows sharing cleanup only after selected-launcher resource joins. Preserve mutation, exactly-once, failure-evidence, and timeout behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Integrate parent 3f3822c with the scoped mutation-hook and launcher-cleanup fixes. The 30 requested native Windows cases and 17 merged Linux CLI cases pass with source assertions and timeouts unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Allow 15 seconds only for the wrong-version rejection integration case, whose two real Node processes measured about 4.9 seconds. Preserve every rejection and cleanup assertion and all production startup deadlines.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Separate the real initial native build from the soft-reload assertions after measuring their combined Windows runtime above the default five-second budget. Preserve the default test and setup budgets and every generation-fencing assertion.

Track callbacks against their own fixture, close the host, and join pending work before disposal so timed-out continuations cannot overlap the next fixture. Give only cleanup room for the existing host drain bound.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Preserve generation assertions and default execution budgets while separating initial preparation and joining timed-out callbacks before disposal. Classify the test-only change note as none, consistent with the other Windows fixture corrections.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep native physical paths for workspace identity and confinement, but retain validated Windows invocation spelling for child cwd, lifecycle paths, registration lookup and pnpm-sync diagnostics. Pin queued aliases to their original physical directory. Do not rewrite output or change the original parity assertions.

Base: 4dc29c2. Real win32 Node 26.7.0, new owned NTFS deployment rushx-native-4747-QZoTNB, native dependencies and hash-matched rebuilt bundles: parent baseline 27 passed / 3 failed; final Rushx suites 36/36 with genuine 8.3 TEMP and 36/36 with physical TEMP. Native global-child contract: 27 passed, 1 existing platform skip.

Pinned Linux Node 22.23.2 Rush build --to rush-cli-client and affected test --only validation with parallelism 3: rush-lib 939 passed, rush-daemon 331 passed, CLI 136 passed. One unchanged warm-owned DaemonGraphGeneration 5-second timeout in a combined run passed on the original-deadline isolated package rerun. Rush change --verify --no-fetch and rush check passed. Evidence: session files/rushx-alias-native-{before,short-final,physical-final,global-context}.log and rushx-alias-linux-daemon-rerun.log.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Keep physical workspace identity and confinement while preserving validated invocation spelling in native Rushx cwd, lifecycle environment and pnpm synchronization. Retain exact output assertions and reject queued alias retargeting before execution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Prevent timed-out native CLI work from accessing the next test's repository by capturing per-fixture state and joining entire callbacks before deletion. Stop and join owned graph watches, including late registrations. Separate initial daemon preparation from warm reload and hard-restart assertions; preserve all execution assertions and their budgets. Cover callback isolation, watch cleanup, and propagated failures with regression cases.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the remaining cold daemon build into its own bounded setup hook. Preserve the warm reuse, native coexistence, Rushx parity, and exactly-once assertions under their existing 30-second execution budget, and always close the setup status connection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Only translate registration lookup when the caller supplies an invocation namespace override. Preserve all existing native snapshots. Capture real native locks in embedded frontend parser tests and release them after execution instead of deleting files with live exclusive handles; production locking is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Set core.autocrlf=false only in each generated fixture repository before staging files. Avoid host-global Windows LF-to-CRLF warnings without muting stderr, relaxing CI warnings-as-errors, changing source line endings, or altering user Git settings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Allow 15 seconds only for the real native build preparation hook after its hosted Windows 24 timeout. Preserve all graph-generation assertions, test-body budgets, resource joins, and production deadlines.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Repeated native Windows setup failures show the real host and Git/build subprocess suite cannot use Jest's five-second unit-test default. Set a scoped 15-second integration budget while retaining all assertions, the explicit cleanup limit, and production deadlines.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Do not recycle the current Rush mutex or its Windows dirty companion while purging common/temp. Keep locking enforced rather than disabling it or deleting a live owner. Add a real configuration/lock regression proving temporary data is purged while ownership survives until release.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fence global request completion on bounded Linux group quiescence, with deterministic delayed-exit/error tests and unchanged immediate descendant-stop assertions. Join fixture daemon PIDs before bounded Windows deletion, preserve graph fixtures after failed host cleanup, and isolate retention assertions from unrelated Jest-worker RSS. Production warning and memory defaults remain unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Compute expected value ordering independently from real cold/reused durations, request frequency, and IPC RSS rather than assuming project a always wins under CI load. Preserve real resource eviction and verify telemetry versus LRU retain opposite projects after requested rewarming. Production ranking is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use independent producer timing/RSS expectations in both warm-set fixture surfaces instead of a fixed project order. Give actual version-revalidation subprocesses and full qualification corpus setup explicit 15-second budgets. Preserve production ranking and every qualification/performance threshold.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Write and close the fixture PID in a unique temporary file before renaming it to the observable barrier. Prevent readers from treating the file-creation/write gap as PID zero; retain and strengthen single-owner startup assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs triage

Development

Successfully merging this pull request may close these issues.

[rushd][WS4] CLI client & command surface [rushd][WS3] Daemon lifecycle, reload & warm-set

1 participant