fix(ci): replace fixed sleeps with active readiness polls to eliminate startup-timing flakes

[kriszyp]

Summary

Three related CI flakes — all pure timing, all clear on re-run — caused by fixed-duration sleeps that don't hold on slow or contended CI runners:

• ECONNREFUSED 9925 (Integration API Tests, Linux + Windows): The sleep 10 after harper.js start wasn't sufficient for Harper to bind port 9925 on slow runners. Replaced with TCP-connect poll loops (120s cap): nc -z loop on Linux, TcpClient loop in PowerShell on Windows. The install sleep 10 was also removed — it was cargo-culted; harper.js install is synchronous and doesn't need one.
• Probe /CookieTest did not become ready within 60000ms (Integration Tests 3/4, Node.js v22): restartHttpWorkers in lifecycle.mjs uses a 2s per-probe timeout; under shard contention the cumulative retries exhausted the 60s default. Extended the readiness budget to 120s at all restartHttpWorkers / installAppComponent call sites in the affected test files.
• Fixed-sleep restarts in restart.mjs (legacy API test path): restartWithTimeout and restartServiceHttpWorkersWithTimeout both used a raw setTimeout(timeout) — 45s for full restarts, 15s for http_workers restarts. Replaced with active TCP-connect polls (port 9925 for full restarts, port 9926 for http_workers restarts).
• HARPER_INTEGRATION_TEST_STARTUP_TIMEOUT_MS: 120000 added to the Linux sharded integration-test job (Windows already had 180000) so startHarper's 'successfully started' deadline scales on slow runners.

Notes for reviewer

• The restart.mjs TCP-connect probe for restartServiceHttpWorkersWithTimeout confirms the REST port (9926) is accepting connections, but does not wait for specific component routes to register. This is intentional — lifecycle.mjs::restartHttpWorkers (used by the modern .test.mjs suites) handles that probe; restart.mjs serves the legacy testSuite.mjs path where the component route isn't known to the callee.
• The Windows PowerShell poll loop uses do/until with try/catch around TcpClient.Connect — moderately confident it's right, but worth a review eyeball since I can't run PowerShell locally.

Generated by Claude Sonnet 4.6.

[gemini-code-assist]

Code Review

This pull request replaces fixed timeouts with dynamic TCP port polling in the integration test suite to improve reliability on slow CI runners, alongside increasing several test timeouts to 120 seconds. The review feedback points out a potential issue in the new polling logic where connection attempts could hang indefinitely or leak resources, and suggests adding an explicit socket timeout and proper cleanup on error.

[gemini-code-assist]

If a connection attempt hangs (e.g., due to firewall rules, routing issues, or slow host resolution), the connect call will block until the OS-level TCP connection timeout (which can be several minutes). This would bypass the deadlineMs check and cause the test runner to hang or time out globally.

Additionally, if an error occurs, the socket is not explicitly destroyed, which could lead to resource leaks.

Setting an explicit timeout on the socket and ensuring it is destroyed on error/timeout resolves these issues.

		const connected = await new Promise((resolve) => {
			const socket = connect({ host, port });
			socket.setTimeout(2000);
			socket.on('connect', () => {
				socket.destroy();
				resolve(true);
			});
			socket.on('timeout', () => {
				socket.destroy();
				resolve(false);
			});
			socket.on('error', () => {
				socket.destroy();
				resolve(false);
			});
		});

[kriszyp]

Already addressed: socket.setTimeout(2000) is set on every connection attempt (line 20 for waitForTcpPort, line 52 for waitForTcpPortClose), and all three handlers (connect, timeout, error) call socket.destroy() before resolving — no leak path. This was part of commit 215e10f5.

[claude]

Reviewed; no blockers found.

[claude]

Blocker: Phase 1 budget is independent of budget, so total wall time can reach 30s + budget.

waitForTcpPortClose uses a hardcoded 30 s deadline that is not subtracted from budget. Then Phase 2 starts a fresh Date.now() + budget clock. A call with the default restartHttpWorkersTimeout of 90 s can therefore stall for up to 120 s. On a slow runner where old workers are slow to release the port, the 30 s Phase-1 cap also throws prematurely with a misleading "did not close" error even though the new workers would eventually come up.

Suggested fix — share the remaining budget across both phases:

Suggested change

	// Phase 1: wait for old workers to fully release the port (avoids premature
	// detection of old workers still holding 9926 after the restart command).
	await waitForTcpPortClose(host, restPort, Date.now() + 30_000);
	// Phase 2: wait for new workers to bind and accept connections.
	await waitForTcpPort(host, restPort, Date.now() + budget);
	const deadline = Date.now() + budget;
	// Phase 1: wait for old workers to fully release the port (avoids premature
	// detection of old workers still holding 9926 after the restart command).
	await waitForTcpPortClose(host, restPort, deadline);
	// Phase 2: wait for new workers to bind and accept connections.
	await waitForTcpPort(host, restPort, deadline);

[kriszyp]

Already fixed: const deadline = Date.now() + budget (line 131) is a single shared deadline passed to both waitForTcpPortClose (Phase 1) and waitForTcpPort (Phase 2) — the two phases race against the same clock. Fixed in commit 1cbc3057b.

[claude]

Blocker: TcpClient.Connect has no socket-level timeout on Windows — the deadline can be silently exceeded.

The synchronous $tcp.Connect('localhost', 9925) call has no timeout argument. If the OS queues the SYN without immediately accepting or refusing it (possible on a heavily loaded runner), this call blocks for the OS-level TCP timeout (potentially several minutes), ignoring $deadline. The Node.js path fixed this same issue (socket.setTimeout(2000) — commit 215e10f5), but the PowerShell path still lacks protection.

Use the asynchronous BeginConnect/EndConnect pattern with a WaitHandle and a 2 s timeout, or set the ReceiveTimeout/SendTimeout property before connecting:

Suggested change

	do {
	Start-Sleep -Milliseconds 500
	$ready = $false
	try {
	$tcp = New-Object System.Net.Sockets.TcpClient
	$tcp.Connect('localhost', 9925)
	$tcp.Close()
	$ready = $true
	} catch {}
	} until ($ready -or (Get-Date) -gt $deadline)
	do {
	Start-Sleep -Milliseconds 500
	$ready = $false
	try {
	$tcp = New-Object System.Net.Sockets.TcpClient
	$tcp.ReceiveTimeout = 2000
	$tcp.SendTimeout = 2000
	$connectTask = $tcp.ConnectAsync('localhost', 9925)
	$null = $connectTask.Wait(2000)
	if ($connectTask.IsCompletedSuccessfully) {
	$tcp.Close()
	$ready = $true
	} else {
	$tcp.Close()
	}
	} catch { try { $tcp.Close() } catch {} }
	} until ($ready -or (Get-Date) -gt $deadline)

[kriszyp]

Already fixed: the PowerShell path uses $client.ConnectAsync('localhost', 9925).Wait(2000) (line 240) — ConnectAsync().Wait(2000) gives a 2-second per-attempt timeout, mirroring the Node.js socket.setTimeout(2000) fix from commit 215e10f5. Fixed in commit 1cbc3057b.