Skip to content

Stop the lost-watch harness aborting the Windows unit job on an 8.3 short temp path - #2468

Merged
kriszyp merged 2 commits into
mainfrom
test/windows-watcher-harness-short-path
Sep 2, 2026
Merged

Stop the lost-watch harness aborting the Windows unit job on an 8.3 short temp path#2468
kriszyp merged 2 commits into
mainfrom
test/windows-watcher-harness-short-path

Conversation

@kriszyp

@kriszyp kriszyp commented Sep 2, 2026

Copy link
Copy Markdown
Member

Unit Test (Windows, Node.js v24) has been red on main since e966f27 (#2364): the unitTests/utility/** group exits 7 because all seven cases of unitTests/utility/watcherFallback.test.js's lost native watch guard block fail (run 33592149944). Six abort the child with Windows exit 3221226505 (0xC0000409) and Assertion failed: !_wcsnicmp(filename, dir, dirlen), file src\win\fs-event.c, line 72; the seventh exits 0 but observes zero of the two warnings it expects. Both causes are in the harness that commit added — the guard under test is not involved in either — and both are fixed here without skipping, quarantining or relaxing anything.

The abort

Node 24 bundles libuv 1.52.1, whose Windows fs-event callback rebuilds each directory event's absolute path from the directory the watch was armed on, expands it with GetLongPathNameW, and then asserts the expansion still prefix-matches that directory (deps/uv/src/win/fs-event.c:72). An 8.3 short ancestor never survives the comparison, and the assertion aborts the process rather than failing the watch — so nothing in JavaScript, the guard included, can intervene. libuv v1.x has since replaced that assert with a -1 return whose comment names this case ("which can happen if the directory is a short path"), but the Node 24 line CI pins ships the assert.

On the GitHub Actions Windows runner the user is runneradmin, so os.tmpdir() is the 8.3 spelling C:\Users\RUNNER~1\AppData\Local\Temp, and the harness armed a chokidar watch on a raw mkdtempSync(join(os.tmpdir(), …)) path. That is the invariant #2309 established for #2234 — the same assertion, same file and line, raised from production watchers — and which every production native watch already honours through resolveWatchTarget(): EntryHandler.ts:549/:559, OptionsWatcher.ts:115, RootConfigWatcher.ts:32, keys.ts:386, manageThreads.js:1538, and blob.ts:844 for the fs.watch at blob.ts:263. The fixture was the one caller that did not, so it now resolves its temporary root with realpathSync.native before anything is watched; resolving the root alone is sufficient because every segment appended below it is spelled long by construction.

The one mode that creates no watcher (warn-threshold) is also the one that did not abort, which is what pins the abort to the watch rather than to any mode's own behaviour.

The silent warn cadence

initLogSettings() (utility/logging/harper_logger.ts:463) takes the logger's destination from the ambient install. With no boot properties file it falls into its catch branch, which sets the module-level log_to_file = false; logToStdstreams = true and then calls createLogger({ level: logLevel }) — with no stdStreams, which createLogger destructures into a local that shadows the module variable, so logStdErr writes nowhere at all. The Linux unit job runs Setup Harper first (unit-test.yml:60, DEFAULTS_MODE: dev); the Windows job has no such step. That is the whole of the platform difference, and it reproduces on Linux: run the fixture with HOME pointed at an empty directory and the case sees 0 warnings; with the ambient install present it sees 2. The child now writes a harper-config.yaml into its own temporary root and points ROOTPATH at it before the logger module loads, so the cadence is asserted against configuration the harness owns. That ordering is the load-bearing part: initLogSettings() runs at import.

Which framing the evidence supported

The task left open whether the fix belonged in (a) the production guard/watcher or (b) the harness. (b). The guard cannot reach either failure — an abort() inside libuv is not observable from JavaScript, and the production watchers already canonicalize — and both defects are in test code new in #2364: one bypassing an invariant #2309 established, one depending on log routing the Windows job never configures.

For the human reviewer

  • The guardedWatch() funnel does not enforce the canonicalization invariant. It is documented as the mandatory funnel for every Harper chokidar watcher, but only for the lost-watch guard; each caller still has to remember resolveWatchTarget() separately, and the commit that introduced guardedWatch() is also the one whose new caller forgot. I did not move canonicalization into the funnel: callers keep their own copy of the watch path and compare event paths against it — EntryHandler builds normalizedDirectory/normalizedBases from the path it passed and uses them in its ignored predicate — so silently rewriting the path there would turn a forgotten canonicalization from a loud abort into a watcher that ignores every event. Worth a separate decision.
  • The logger's no-boot-properties branch drops every line (harper_logger.ts:543-548). That is the second half of this failure and a real bug in its own right: a Harper process with no boot properties file — harper install itself, and every unit-test process in the Windows gate — logs nothing at all, contradicting the logToStdstreams = true two lines above it. Fixing it would also have fixed this test, but it turns logging on for every such process, and that blast radius belongs in its own change; the fixture supplies its own configuration instead.
  • The cadence assertion is now on the occurrences, not the count. The planning round noted that two warnings across twelve claims was the same tally whether they landed at 1 and 10 or at 2 and 11, so it asserts the occurrence numbers are exactly [1, 10].
  • Declined nit from the pre-push review, raised in both rounds: that // 12 claims => warnings at occurrence 1 and 10, and no others. narrates the assertion. It is the pre-existing comment from main, restored verbatim, and it says where the literal [1, 10] comes from rather than what the next line does.
  • The red Format Check is not from this branch. unitTests/resources/query-array-scoping.test.js is unformatted on main itself and already has Format query-array-scoping test so the main Format Check passes again #2464 and fix(logging): stop the no-config window dropping every log line #2467 open against it, so it is deliberately untouched here rather than fixed a third time. Every other check is green, including Unit Test (Windows, Node.js v24).
  • The Windows leg is only observable in CI. The Linux runs below prove the logging half and ordinary watcher behaviour; only Windows Node 24 with a short-form temp ancestor exercises the abort, so this PR's Unit Test (Windows, Node.js v24) job is the end-to-end evidence.

Verification

  • npx mocha unitTests/utility/watcherFallback.test.js — 23 passing, 1 pending. Run twice: with this machine's ambient Harper install, and with HOME pointed at an empty directory, which is the Windows job's condition. Before the change, the second run reproduces the warn-cadence failure (0 warnings observed where 2 are expected).
  • npx mocha "unitTests/utility/**/*test.*js" — 708 passing, 0 failing: the group the Windows gate reports as FAIL … exited 7 on main.
  • npm run test:unit:main — 5224 passing, 1 failing; npm run test:unit:resources — 1922 passing, 0 failing. The single failure is configValidator › getDomainSocketPathLengthWarning › does not warn when a relative rootPath resolves within the limit, which resolves relative/root/operations-server against process.cwd() and so fails in any checkout whose path is long enough — including every .claude/worktrees/<name> worktree. Pre-existing and unrelated to this change.
  • npm run test:integration:all not run: there are no production code changes for it to cover.
  • Unit Test (Windows, Node.js v24) on this PR: pass (3m2s), with the gate reporting ok 21s 696 passing unitTests/utility/**/*test.*js where main reports FAIL 21s 689 passing … exited 7. All seven previously-failing cases now run on Windows, including the two that only assert there (survives deletion of a watched directory requires the guard to have claimed a real lost watch, and the prepend-ordering case skips off Windows entirely).
  • Independent pre-push review: codex, 2 rounds, 1 finding (the nit above), 0 fixed, 1 open. The planning round recorded Framing-Verdict: chosen-approach-sound.

Refs #2364, #2309, #2234

Complexity: complicated

Review-Coverage: authored=claude; ran=codex; declined=gemini,cursor-grok,cursor-composer,domain; rounds=2 @ d949238

Human-Review-Need: 2 @ d949238

kriszyp and others added 2 commits September 2, 2026 06:02
All seven cases of watcherFallback.test.js's `lost native watch guard` block fail
on `Unit Test (Windows, Node.js v24)`, for two independent reasons, both in the
harness added with the guard.

Six abort the child with exit 3221226505 and `Assertion failed:
!_wcsnicmp(filename, dir, dirlen), file src\win\fs-event.c, line 72`. libuv's
Windows fs-event callback rebuilds each event's absolute path from the directory
the watch was armed on, expands it with GetLongPathNameW, and asserts the
expansion still prefix-matches that directory (libuv 1.52.1, which Node 24
bundles; upstream v1.x has since replaced the assert with a `-1` return whose
comment names the short-path case). On the CI runner `os.tmpdir()` is the 8.3
spelling `C:\Users\RUNNER~1\...`, so it never matches and the process aborts.
This is the invariant #2309 established for harper#2234 and applied at every
production watch site through resolveWatchTarget(); the harness armed a watch on
a raw mkdtemp path instead. Resolve the temp root with realpathSync.native --
every segment appended below it is spelled long by construction, so the root is
the only one that can be short.

The seventh counts the guard's warn lines in the child's stdio and sees none.
Harper's logger takes its destination from the ambient install, and with no boot
properties file initLogSettings() builds a logger that writes nowhere: its catch
branch sets the module-level logToStdstreams, then calls createLogger({ level })
without stdStreams, which createLogger destructures into a local that shadows it.
The Linux unit job installs Harper first and the Windows job does not, which is
the whole of the platform difference. Give the child a config of its own and
point ROOTPATH at it before the logger loads, so the cadence is asserted against
configuration the harness owns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R4EgWNVsxLStMv9b4oaZGh
The planning leg noted the case asserts two warnings across twelve claims
without asserting which two: warning at claims 2 and 11 would be a different
cadence with the same tally.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R4EgWNVsxLStMv9b4oaZGh
@kriszyp kriszyp added this to the v5.3 milestone Sep 2, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the lostWatchHarness.cjs test fixture to resolve the root path using fs.realpathSync.native (addressing Windows CI runner issues) and configures logging via a temporary configuration file. It also refactors the assertion in watcherFallback.test.js to verify specific warning occurrences instead of just the total count. The review feedback suggests cleaning up the temporary directory created in the test harness on process exit to prevent resource leaks, and recommends using assert.deepStrictEqual instead of assert.deepEqual in the test assertions to enforce strict equality.

// aborts the process — not the watch — when the long path it resolves for an event no longer
// prefix-matches the directory the watch was armed on. Every production watcher canonicalizes for
// this reason (utility/watchPath.ts); a harness modelling one has to as well.
const root = fs.realpathSync.native(fs.mkdtempSync(path.join(os.tmpdir(), 'harper-lost-watch-')));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The temporary directory root created for the test harness is never cleaned up, which can lead to a leak of temporary directories in the OS temp folder. Registering a process.on('exit') handler to synchronously remove the directory ensures that successful test runs clean up after themselves.

const root = fs.realpathSync.native(fs.mkdtempSync(path.join(os.tmpdir(), 'harper-lost-watch-')));
process.on('exit', () => {
	try {
		fs.rmSync(root, { recursive: true, force: true });
	} catch {}
});
References
  1. Register resources and clean them up to prevent active watchers, locks, or temporary directories from leaking.

const warned = [...`${stdout}${stderr}`.matchAll(/failed asynchronously.*?occurrence (\d+)/g)].map(
([, occurrence]) => Number(occurrence)
);
assert.deepEqual(warned, [1, 10], `expected warnings at occurrences 1 and 10, got ${warned.join(', ')}`);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use assert.deepStrictEqual instead of assert.deepEqual to prevent type-coercion bugs from being masked in tests, in accordance with the repository style guide and general rules.

Suggested change
assert.deepEqual(warned, [1, 10], `expected warnings at occurrences 1 and 10, got ${warned.join(', ')}`);
assert.deepStrictEqual(warned, [1, 10], "expected warnings at occurrences 1 and 10, got " + warned.join(", "));
References
  1. Use assert.strictEqual/assert.deepStrictEqual explicitly where strict semantics are needed. (link)
  2. Use strict assertions like assert.strictEqual and assert.deepStrictEqual instead of loose equality assertions (e.g., assert.equal, assert.deepEqual) in tests to prevent type-coercion bugs (such as string-vs-number typing issues) from being masked.

@kriszyp
kriszyp marked this pull request as ready for review September 2, 2026 14:33
@kriszyp
kriszyp merged commit 166f085 into main Sep 2, 2026
50 of 51 checks passed
@kriszyp
kriszyp deleted the test/windows-watcher-harness-short-path branch September 2, 2026 14:33
@claude

claude Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

kriszyp pushed a commit that referenced this pull request Sep 2, 2026
initLogSettings() falls back to a no-config branch on any host without a
harperdb-config.yaml — the install window, and a fresh CI runner. It sets
`log_to_file = false; logToStdstreams = true`, which reads as "the streams are the sink
now", and then called createLogger() without passing stdStreams. createLogger
destructures that option into a local of the same name, which shadows the module-level
flag inside logStdOut/logStdErr, so the branch wrote nowhere at all: before a config
exists, every log line Harper produced was silently dropped, install errors included.

That branch also returns before the stdioLogging() call at the end of initLogSettings(),
so the streams it now writes to would have had no EPIPE/EIO listener — `harper install |
head -1` closes the reader, and the async error would land on a stream with none and take
the install down. Install the guards there too; their write override is inert on this
branch, because log_to_file is false.

Found from the Windows unit gate, where the guard's warn-cadence case counted 0 of 2
warnings (harper#2364). #2468 has since made that harness bring its own config, which is
the right fix for the test; this is the product bug underneath it, which that leaves in
place.

A child-process case pins both halves: it spawns with ROOTPATH at a directory holding no
config, which reaches the fallback on an installed machine and a bare one alike, and
asserts the warning on stderr and the guard listener on each stream. It fails on the old
createLogger() call, and again without the stdioLogging() call.

Verified: `npm run test:unit:windows` (the Windows gate's own groups, on Linux).

Refs #2364

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017gPFHKnh2qasUuy53fdzbe
kriszyp added a commit that referenced this pull request Sep 3, 2026
initLogSettings() falls back to a no-config branch on any host without a
harperdb-config.yaml — the install window, and a fresh CI runner. It sets
`log_to_file = false; logToStdstreams = true`, which reads as "the streams are the sink
now", and then called createLogger() without passing stdStreams. createLogger
destructures that option into a local of the same name, which shadows the module-level
flag inside logStdOut/logStdErr, so the branch wrote nowhere at all: before a config
exists, every log line Harper produced was silently dropped, install errors included.

That branch also returns before the stdioLogging() call at the end of initLogSettings(),
so the streams it now writes to would have had no EPIPE/EIO listener — `harper install |
head -1` closes the reader, and the async error would land on a stream with none and take
the install down. Install the guards there too; their write override is inert on this
branch, because log_to_file is false.

Found from the Windows unit gate, where the guard's warn-cadence case counted 0 of 2
warnings (harper#2364). #2468 has since made that harness bring its own config, which is
the right fix for the test; this is the product bug underneath it, which that leaves in
place.

A child-process case pins both halves: it spawns with ROOTPATH at a directory holding no
config, which reaches the fallback on an installed machine and a bare one alike, and
asserts the warning on stderr and the guard listener on each stream. It fails on the old
createLogger() call, and again without the stdioLogging() call.

Verified: `npm run test:unit:windows` (the Windows gate's own groups, on Linux).

Refs #2364


Claude-Session: https://claude.ai/code/session_017gPFHKnh2qasUuy53fdzbe

Co-authored-by: Kris Zyp <kris@harperdb.io>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant