Skip to content

feat(server): surface external port conflicts on all platforms; single-worker default without SO_REUSEPORT - #1605

Merged
kriszyp merged 2 commits into
mainfrom
kris/mqtt-detect-external-port-conflict
Jul 10, 2026
Merged

feat(server): surface external port conflicts on all platforms; single-worker default without SO_REUSEPORT#1605
kriszyp merged 2 commits into
mainfrom
kris/mqtt-detect-external-port-conflict

Conversation

@kriszyp

@kriszyp kriszyp commented Jul 5, 2026

Copy link
Copy Markdown
Member

Summary

Stacked on #1603 (MQTT listeners share their port via SO_REUSEPORT on non-macOS). This makes external port conflicts detected and loudly logged on every platform, and aligns two related behaviors on platforms without working SO_REUSEPORT (macOS/Windows).

listenOnPorts() used to swallow every EADDRINUSE — a workaround for a Node <20.11.1 reusePort bug now outside Harper's supported range (^22.18 || >=24). So an unrelated process squatting a Harper port (the original symptom: a second Harper instance holding 8883) silently received Harper's MQTT traffic with no error logged anywhere.

Design

Every EADDRINUSE with a possible in-process explanation is structurally ruled out, so what remains is unambiguously external and gets logged (port + owning component + error):

Case Why EADDRINUSE must be external
reusePort listener (Linux) siblings share the port, never collide — even across overlapping restarts
main thread (HTTP/ops ports) binds every port (awaited) before any worker starts; never restarts
dedicated listener's owner worker (MQTT via server.socket()) never bound by the main thread; now bound by a single owner worker (lowest eligible index) instead of every worker racing; restarts are non-overlapping without reusePort

The one remaining benign case — a worker's exclusive HTTP bind losing to the main thread on macOS/Windows — stays silently swallowed. Everything still resolves, so a squatted port never stalls boot.

Supporting changes:

  • restartWorkers() no longer pre-starts replacements on macOS (canPreStartReplacement now excludes darwin, like Windows/Bun). This also fixes a latent bug: the pre-started replacement could never bind the exclusive ports the old worker still held, its EADDRINUSE was swallowed, and worker-owned listeners (MQTT) were left permanently unbound after every component-reload restart on macOS. The main thread keeps serving HTTP throughout, so only worker-owned listeners see the brief shutdown-first gap.
  • threads.count defaults to 1 on macOS/Windows (setDefaultThreads): without SO_REUSEPORT, extra HTTP workers can never share the server ports — the CPU-based default just spawned workers that serve no direct TCP traffic. Explicit threads.count still overrides.

Where to look

  • The ownership claims are the crux. Verified in code + empirically: main thread binds HTTP/ops before workers (socketRouter.ts awaits listenOnPorts() pre-spawn); component plugins don't run handleApplication on the main thread, so MQTT listeners exist only in workers.
  • canPreStartReplacement on darwin: Multi-worker HTTP rolling restart produces ~0.6–1.2s whole-pool connection-refused gap #1417 (rolling-restart conn-refused) was a Linux SO_REUSEPORT-pool fix and Linux is unchanged. rolling-restart.test.ts fails identically on macOS on the base branch (pre-existing), so CI/Linux remains the gating signal.
  • threadRange: the dedicated-listener owner is the lowest eligible index, not hardcoded 0.

Test

integrationTests/server/external-port-conflict.test.ts squats the MQTT secure port before boot and asserts the conflict is logged on every platform (skip removed) and Harper still starts. Plus platform-stubbed unit coverage for setDefaultThreads and the dedicatedListener marker.


🤖 Generated by Claude (Fable 5 / Opus 4.8 across revisions). Cross-model reviewed (Codex + Gemini) at each design iteration; earlier reviews drove the pivot from worker-gating → reusePort-only → this universal design.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a regression test to verify external MQTT port-conflict detection and updates the thread server to log external bind conflicts when reusePort is enabled. The feedback recommends passing the caught error object to the logging function to ensure the root cause is preserved, in accordance with the repository's guidelines.

Comment thread server/threads/threadServer.js Outdated
Comment thread server/threads/threadServer.js Outdated
@claude

This comment has been minimized.

@kriszyp
kriszyp force-pushed the kris/mqtt-detect-external-port-conflict branch from 9ebd10c to b1aacfa Compare July 5, 2026 20:21
@kriszyp
kriszyp marked this pull request as ready for review July 5, 2026 20:35
@kriszyp
kriszyp force-pushed the kris/mqtt-detect-external-port-conflict branch from b1aacfa to 72a5ef8 Compare July 5, 2026 22:06
@kriszyp kriszyp changed the title feat(server): surface external EADDRINUSE conflicts on reusePort listeners feat(server): surface external port conflicts on all platforms; single-worker default without SO_REUSEPORT Jul 5, 2026
Base automatically changed from kris/mqtt-reuseport-across-workers to main July 6, 2026 15:50

@Ethan-Arrowood Ethan-Arrowood left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the ownership analysis is convincing. Branch is conflicting with main but the hunks look mechanical (drain-hook context drift); approving on that assumption — ping me if the rebase turns out to touch anything real.

sent with Claude Fable 5

…e-worker default without SO_REUSEPORT

listenOnPorts() used to swallow every EADDRINUSE (a workaround for a Node
<20.11.1 reusePort bug now outside Harper's supported range), hiding real
external squatters: an unrelated process holding e.g. the MQTT port silently
received Harper's traffic with no error anywhere (original symptom: a second
Harper instance on 8883).

Every EADDRINUSE with an in-process explanation is now structurally ruled out,
so the remaining ones are logged loudly (port + owning component + error):
- reusePort listeners (Linux): siblings share the port and never collide, even
  across overlapping restarts — any EADDRINUSE is external.
- Main thread (HTTP/operations ports): binds before any worker, never restarts —
  any EADDRINUSE is external.
- Dedicated listeners (onSocket, e.g. MQTT — never bound by the main thread):
  when exclusive (macOS/Windows), bound only by a single owner worker (lowest
  eligible index) instead of every worker racing; combined with non-overlapping
  restarts (below), the owner's EADDRINUSE is external.
The one remaining benign case — a worker's exclusive HTTP bind losing to the
main thread on macOS/Windows — stays silently swallowed. All cases still
resolve so a squatted port never stalls boot.

restartWorkers() no longer pre-starts replacement HTTP workers on macOS
(canPreStartReplacement now excludes darwin, like Windows/Bun): without working
SO_REUSEPORT the replacement could never bind ports the old worker still held —
its EADDRINUSE was swallowed and worker-owned listeners like MQTT were left
permanently unbound after every component-reload restart. The main thread keeps
serving the HTTP ports throughout, so only worker-owned listeners see the brief
shutdown-first gap.

threads.count now defaults to 1 on macOS/Windows (setDefaultThreads): without
SO_REUSEPORT, additional HTTP workers can never share the server ports, so the
CPU-based default just spawned workers that serve no direct TCP traffic. An
explicit threads.count still overrides.

Adds an integration test that squats the MQTT secure port before boot and
asserts the conflict is logged and Harper still starts — on every platform.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@kriszyp
kriszyp force-pushed the kris/mqtt-detect-external-port-conflict branch from 72a5ef8 to e8f48e3 Compare July 10, 2026 21:23
import { createServer, type Server } from 'node:net';
import { mkdtemp, readFile } from 'node:fs/promises';
import { join } from 'node:path';
import { tmpdir } from 'node:os';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker — no-restricted-imports lint violation

node:assert/strict is banned by the root .oxlintrc.json (rule configured as "error"), so npm run lint (oxlint --deny-warnings .) will fail on this file. Every other integration test in this directory imports from plain node:assert (rolling-restart.test.ts, ops-port-isolation.test.ts, etc.).

Suggested change
import { tmpdir } from 'node:os';
import { ok } from 'node:assert';

@kriszyp
kriszyp merged commit f55de88 into main Jul 10, 2026
93 of 94 checks passed
@kriszyp
kriszyp deleted the kris/mqtt-detect-external-port-conflict branch July 10, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants