feat(server): let liveSubscriptionAuth revoke a single subscriber without ending its subscription

[kriszyp]

Summary

registerLiveSubscription (the #1414 continuous re-authorization registry) now accepts an optional revoke?: () => void | Promise<void>, used as the entry's terminate instead of the hard-coded end()/close()/emit('close') on the subscription object, and returns an { unregister } handle. When revoke is supplied the subscription object is left completely untouched — no end wrapping, no 'close' listener — and the caller unregisters through the returned handle instead.

Why: profiling a production node found ~157,888 live Subscription objects across only 29 distinct topics, almost entirely per-connection bookkeeping (each carries its own DatabaseTransaction, TableResource, Generator). The fix is to share one feed per topic across its subscribers while keeping authorization per subscriber — but revocation today works by destroying the subscription object, which is exactly wrong once that object is shared: wiring terminate to a shared feed would disconnect every subscriber on one user's revocation or token expiry (and, since sweep() fails closed on recheck errors, one transient error would tear the feed down for everyone). This PR is only the prerequisite seam — no shared feeds, no refcounting, no subscriber-list machinery.

Revocation granularity must stay per subscriber even once feed granularity becomes per topic — that's the invariant this seam exists to establish.

Scope reduced from the earlier revision

Earlier revisions of this branch also shipped a revocation state machine — pendingTerminate, a bounded terminate timeout, settle-handler late-commit, and fail-closed retry across sweeps — roughly 120 lines whose only exercise was its own tests, since nothing in this repo supplies revoke. That is now removed, per the owner's decision on review feedback ("reduce scope"): the timeout and idempotency requirements are unknowable without a real caller, and the shared-feed follow-up may want different ones.

What remains is the seam plus the one change the seam genuinely forces: terminate may now return a promise, so terminateEntry untracks the entry first and dispatches teardown once, fire-and-forget, containing a synchronous throw and a rejection alike. Untracking first is what keeps a hung revoke from wedging the sweep — awaiting it would hold the sweeping mutex forever — and it matches the pre-existing default-path behavior, which also deleted the entry before calling terminate(). The consequence, stated in the contract: a revoke that throws, rejects or never settles is logged and never retried.

Three smaller changes came out of this branch's review rounds:

• Revocation logging is two-level. Each revocation keeps its own info line, and every sweep that revoked anything closes with one aggregate warn carrying the total and a per-reason breakdown. Harper ships logging.level: warn (static/defaultConfig.yaml:46), so an info-only line would make every expected revocation invisible on a default deployment; a warn per subscriber would turn one mass role change into a 10k-line storm. The aggregate is emitted from sweep()'s finally, and it says revoking, not revoked, because terminate is dispatched fire-and-forget — a teardown that fails still gets its own error line. warn also still covers the per-entry recheck-failure path it covered on main.
• The unit suite pins HARPER_SUBSCRIPTION_REAUTH_INTERVAL_MS before importing the module. It previously ran against the real 30s interval, where a background tick makes _sweepNow() a no-op through the sweeping guard and assertions then run against a sweep that never happened.

When revoke is omitted, behavior is unchanged from #1414 in every respect except the new revocation logging; the sole existing caller (registerLiveSubscriptionForContext in resources/Resource.ts) doesn't pass revoke.

For the human reviewer

The load-bearing judgment call is fail-open teardown: with the state machine gone, a revoke that never settles leaves a deauthorized subscriber attached with no further authorization evaluation. Reviewers raised this in every round. It is documented on the API and cannot manifest on this branch (no caller supplies revoke), and it is deliberately the shared-feed caller's problem to solve where the timeout/idempotency semantics are actually known — but it should be ratified before that wiring lands, and it is cheap to revisit now while the seam is unused.

Declined this round, all pre-existing on main and out of scope for a scope-reduction change — each is worth its own issue:

• sweep() is N serialized recheck() round-trips behind one global mutex (adjudicated major). The production recheck does a findAndValidateUser + allowRead per entry with no memoization across entries sharing a username, and there is no timeout on it. This seam does raise registry cardinality from per-stream-object to per-subscriber, which makes N bigger — but the serialization, the mutex, and the unbounded await are all unchanged from main.
• ITC-triggered sweeps are dropped with no trailing pass: if (sweeping) return discards a user-change broadcast arriving mid-sweep, so revocation falls back to the 30s backstop. A dirty flag re-run in sweep()'s finally closes it. (One reviewer described the exposure as "up to 24 hours" — that reads the test interval override; production is the 30s backstop.)
• The recheck-failure path still warns once per entry, as it does on main — each of those lines carries a distinct error message, so collapsing it into the aggregate would cost diagnostics rather than noise. The aggregate counts those entries too, under recheck error.
• A leaked 'close' listener and the never-restored end wrapper on the default path: unregister() removes the registry entry but does not removeListener('close', …) or put back the original end. Pre-existing from Live subscriptions (SSE/MQTT/WS) continue delivering events after drop_user / role revocation — stale-auth leak #1414 and harmless for a per-connection subscription object that is discarded on close, but it would matter for a pooled or reused stream object.
• Two-mode discriminated union vs two named entry points, and _sweepNow/_liveSubscriptionCount exported from the production module — both flagged as API-shape decisions, both cheapest to change now.

Refuted, not declined: the last review round's remaining "major" claims hdbLogger.info/warn/error are unbound methods, so passing them as references (safeLog(notice, …), notice = hdbLogger.info) throws a this TypeError that safeLog swallows, silently suppressing every revocation log. They are module-level free functions — export function info(...args) { mainLogger.info(...args); } at utility/logging/harper_logger.ts:807, :825, :861 — with no this reference, so a bare reference is safe, and the suite's logs each expected revocation at info and one aggregate warn per sweep test asserts the lines are actually emitted. The Human-Review-Need: 4 footer below still counts it: that round's adjudication leg was auto-pruned as a narrow low-risk delta, so the grade is "outside findings unadjudicated", not a live defect.

Verification

• npm run build, npm run lint:required, npm run format:write — clean.
• npx mocha "unitTests/server/**/*.test.js" — 715 passing, including the 20 tests in this file. Two of them cover the aggregate: one info line per subscriber plus exactly one warn with the count and reasons, and no aggregate at all for a pass that revokes nothing.
• End-to-end route: existing integration suite. npm run test:integration -- integrationTests/security/subscription-revocation.test.ts — 6/6 passing, unmodified, covering SSE/WS/MQTT revocation on drop_user, alter_role, and token expiry. This is what proves the sole in-repo caller is unaffected; the revoke path itself has no end-to-end route until the shared-feed caller lands, and is covered only by unit tests (rejection containment, hang containment, no subscription mutation).
• npm run test:unit:main and npm run test:unit:resources both have failures on this machine that reproduce identically with origin/main's copy of both changed files: test:unit:main aborts at import with LZ4 not supported in this build against a stale local database dir, and test:unit:resources fails 3 tests in replayStructures / randomAccessFields, neither of which imports this module.

Refs #1414

---

Generated with Claude Opus 5.

🤖 Generated with Claude Code

_{Review-Coverage: authored=claude; ran=codex,gemini; declined=cursor-grok,cursor-composer,domain; rounds=8 @ 096f399}

_{Human-Review-Need: 4 @ 096f399}

[gemini-code-assist]

Code Review

This pull request enhances the live subscription authorization mechanism in liveSubscriptionAuth.ts by introducing support for a caller-supplied revoke callback, allowing shared feeds to manage revocation per subscriber. It also adds robust error handling, logging safety, and timeout bounds to prevent hung terminations from blocking the sweep process. Comprehensive unit tests are added to verify these behaviors. The reviewer recommends snapshotting the registry Set into an array before iterating over it in sweep() to avoid mutating the Set during iteration.

[claude]

Reviewed; no blockers found.

[kriszyp]

CI triage: all three failures were unrelated to this PR — checks are green now

Re-ran the failed jobs on the identical commit a10594b2 with no code change; every check now passes (42 pass / 2 skipping).

Failing check	Actual failure	Verdict
Unit Test (Node.js v22)	Txn Expiration > Slow txn will expire — unitTests/resources/txn-tracking.test.js:61	Flake
Integration 2/6 (Windows)	describe-metadata-upgrade.test.ts — probe /SeoPageCache/ never ready 120s after restart_service	Known Windows CI issue
Integration 6/6 (Windows)	configuration.test.mjs:319 — 500 from set_configuration, EPERM: rename …harper-config.yaml.<pid>.<tid>.<hex>.tmp -> harper-config.yaml	Known Windows CI issue

Why none of them can be this diff:

• The diff is confined to server/liveSubscriptionAuth.ts + unitTests/server/liveSubscriptionAuth.test.js. Nothing in it touches transaction tracking, component install, HTTP-worker restart, or the config write path.
• The unit failure is in a different mocha process from the new test: CI runs test:unit:all, where liveSubscriptionAuth.test.js lands in test:unit:main and txn-tracking.test.js in test:unit:resources. No shared process, so no cross-test timing interference is possible. main itself failed Unit Test in 3 of its last 8 runs, each time on a different single test (Caching > Can load cached data, HNSW greedy routing above layer 0, MQTT QoS=1 reconnect) — same flake profile.
• Windows shard 2's failure is the same shape as #2273 (Windows component-deploy/restart hang after npm pack): npm install took 4 minutes on that runner, the restart's readiness probe then got ECONNREFUSED for its whole 120s window.
• Windows shard 6's EPERM rename reproduced identically on an unrelated PR in the same window — kris/642-branched-databases run 32790110806, same test, same assertion. fix/bounded-update-attributes-lock was failing the same two Windows shards across four runs tonight. It is org-wide, not branch-specific.

Verified locally on this branch: unitTests/server/liveSubscriptionAuth.test.js 21/21 passing, the whole top-level unitTests/server/ suite 240/240, and txn-tracking.test.js green on 3 consecutive runs.

No code change was needed, so nothing was pushed — the branch is still at a10594b2.

Two things worth a separate look, both pre-existing and outside this PR:

1. The Windows EPERM on atomicWriteFile's rename (config/configUtils.ts:110) already has an 8-attempt backoff and still exhausts it — several threads write harper-config.yaml concurrently on set_configuration, and on Windows a rename over a file another thread holds open fails. I did not find an existing issue for this one, unlike #2273.
2. npm run test:unit:server is broken on main independently of this PR: its unitTests/server/**/*.js glob loads worker fixtures as test files and dies on processGroupOwnerWorker.js:15 (Cannot read properties of null (reading 'postMessage')). CI never notices because test:unit:all doesn't include that script.

— Claude Opus 5

[kriszyp]

Follow-up: both requested one-liners applied, plus two things the pre-push review found in them

Pushed a10594b2 → 88b06112d, three commits. @kriszyp asked for the two changes @cb1kenobi requested; the independent pre-push review (Codex + Gemini + Cursor Grok + Harper domain) then found two real problems in my own first pass, so those are fixed too.

Requested — 1ce07e5c4

• Restored if (entry.pendingTerminate) return false; at the top of claimAndTerminate, so the at-most-once guarantee lives in the function whose docblock promises it rather than only in sweep()'s continue. Both of today's callers already sit after that continue, so behavior is unchanged; a future direct caller — the event-driven immediate revoke harper-pro will want — no longer double-invokes an in-flight revoke.
• Qualified registerLiveSubscription invariant (3): a hung revoke is not retried and the entry is not rechecked again.

Found by the review, in my own fixes — ccfcb2117, 88b06112d

• The options type still required subscription, so a revoke-only caller owning no subscription object — the whole point of the seam — had to pass a meaningless subscription: undefined to type-check. My first attempt made it plainly optional, which went too far: a caller supplying neither subscription nor revoke then type-checked, received NOOP_HANDLE, and silently left its subscription untracked. It is now a union requiring exactly one teardown mode. Verified the three valid shapes compile and the neither-mode call is a TS2345 error, with repo typecheck clean (the existing resources/Resource.ts:950 caller still matches).
• The rejected-terminate log still said "will retry next sweep" unconditionally — the same overstatement just corrected in the docblock, since a rejected attempt is only re-invoked if the next sweep's recheck again denies. Now "on the next sweep that still denies", and the invariant says so too: a subscription whose permission was restored in between is left half-torn-down and read as healthy.

Verification: npm run build and npm run typecheck clean; unitTests/server/ 240/240 (liveSubscriptionAuth 21/21); Prettier clean. No test changed — none of this alters behavior for any caller in the tree.

For the human reviewer — deliberately not addressed

The review re-raised these across three rounds and I left every one of them alone: they are design decisions on your PR, not defects I should be resolving unsupervised. Round 3 confirmed the delta "introduces no new correctness issue"; these are all carried forward from the original diff.

	Decision
major	A never-settling revoke turns fail-closed into fail-open: the entry is skipped by every later sweep, recheck never runs again, and nothing proves delivery stopped. The docblock now states this as intended semantics — that is the part worth ratifying explicitly. This is the same ground as @cb1kenobi's still-open thread, which suggests bounding the entry (clear pendingTerminate after N sweeps) rather than the attempt.
major	Serial per-entry timeouts extend the exclusive sweep window, and sweep()'s if (sweeping) return drops an ITC trigger firing inside it with no trailing sweep. 20 hung revokes ⇒ ~100 s window; a drop_user landing in it waits for the next 30 s tick. The coalescing gap pre-exists, but this makes it deterministic and multiplies it by entry count. Root fix is a dirty flag plus a trailing sweep, not more concurrency.
minor	recheck is still unbounded while terminate is now timeout-bounded — and recheck is the one doing real I/O (findAndValidateUser + allowRead). withTimeout already exists here.
minor	Mass revocation on the default path now emits one warn per subscriber (10k on a role drop) and allocates per entry; a permanently stuck entry logs at error every 30 s forever with no dedup.
minor	unitTests/server/liveSubscriptionAuth.test.js — the "must not re-race the timeout" assertion races _sweepNow() against a 250 ms marker, so its own "no matter how loaded the test runner is" claim is not true. AGENTS.md names this class as the root cause of #1138. Counting setTimeout invocations, as the clamp test already does, would assert the invariant directly.
scope	@cb1kenobi's open thread: land only the ~20-line seam and move the timeout / at-most-once / late-commit machinery into the PR that supplies a real revoke. Your call — I did not act on it.

Two review findings were adjudicated away rather than left open, worth recording so they don't come back: Gemini's safeLog unbound-this concern does not apply (utility/logging/harper_logger.ts:353-385 exports standalone module functions, not class methods — this is never read), and the Array.from(registry) snapshot is immaterial next to one awaited DB-backed recheck per entry in the same loop, besides buying the mid-sweep-removal semantics the tests assert.

Independent pre-push review: 3 rounds, reviewers=codex+gemini+cursor-grok+harper-domain (round 1 full), codex+gemini (rounds 2–3 delta). 4 findings fixed, 6 open by design.

— Claude Opus 5

[kriszyp]

Scope reduced to the seam (option A), per the owner's decision

9ee14b5…ef48484 remove the revocation state machine — pendingTerminate, the bounded terminate timeout and its env var, claimAndTerminate's settle-handler commit, and the "previously pending" sweep branch. Net −167 lines in server/liveSubscriptionAuth.ts and its test since a10594b2. What is left is exactly the seam @cb1kenobi described: accept revoke, use it as terminate, skip the subscription-object wiring, return { unregister }.

Two things came out of the pre-push review of those commits and are also in:

• Routine revocations (expiry, deny) log at info; warn stays on the recheck-failure path it covered on main. A per-subscriber warn would turn one mass role change into a warn storm.
• The unit suite pins HARPER_SUBSCRIPTION_REAUTH_INTERVAL_MS before importing the module, and restores the patched logger from a finally. It previously ran against the real 30s interval, where a background tick makes _sweepNow() a no-op through the sweeping guard and the assertions then run against a sweep that never happened.

One outside finding is refuted rather than declined. The last round flagged as major that hdbLogger.info/warn/error are unbound methods, so passing them by reference (safeLog(notice, …)) would throw a this TypeError that safeLog swallows, silently suppressing every revocation log. They are module-level free functions — export function info(...args) { mainLogger.info(...args); } at utility/logging/harper_logger.ts:807, :825, :861 — with no this reference, so the bare reference is safe; the logs an expected revocation at info, not warn test asserts the line is actually emitted. The Human-Review-Need: 4 footer still counts that finding because the round's adjudication leg was auto-pruned on a narrow delta, not because anything is unresolved in the code. Everything genuinely declined is listed under For the human reviewer in the description, with why.

Verification on this head: all CI checks green; locally npx mocha "unitTests/server/**/*.test.js" → 714 passing, 19 of them in liveSubscriptionAuth.test.js.

—
Claude Opus 5