Follow-up to #9540 (thanks @JPPhoto for catching this in review — it was not a merge blocker there).
What happens
#9540 added a bounded retry for server-initiated socket disconnects: io server disconnect is
terminal for socket.io (Socket.ondisconnect → destroy() → Manager._close() sets
skipReconnect), so the client drives up to five socket.connect() attempts with a doubling
delay before giving up
(invokeai/frontend/web/src/services/events/useSocketIO.ts, MAX_SERVER_DISCONNECT_RECONNECTS).
That bound only governs server-initiated disconnects. socket.connect() calls Manager.open(),
which resets skipReconnect = false, and the socket's options set neither reconnection nor
reconnectionAttempts — so the manager's own defaults apply: reconnection: true,
reconnectionAttempts: Infinity, reconnectionDelay: 1000, reconnectionDelayMax: 5000,
randomizationFactor: 0.5 (socket.io-client/build/esm/manager.js:21-29). If one of our retry
attempts fails at the transport level — server down, port refused — the manager takes over and
retries indefinitely. Our counter never advances, because a failed attempt emits connect_error,
not disconnect.
@JPPhoto measured four reconnect attempts in 180 ms against a closed port with socket.io-client
4.8.3; against a host that black-holes rather than refuses, the documented backoff applies and the
cadence settles at roughly one attempt every 2.5-7.5 s.
Scope
This is socket.io's default behavior for the app's socket everywhere, not something #9540
introduced: any network outage already produces the same unbounded retry, on main as well. What
is new is the interaction — a path that advertises a five-attempt bound can hand off to an
unbounded one.
Why the obvious fix is wrong
Disabling manager reconnection around the custom retry (socket.io.opts.reconnection = false)
would strand the tab in exactly the scenario the retry exists for: the server disconnects the
socket for an authorization change and is then briefly unavailable — a restart, a proxy blip —
and with no manager retry, nothing would ever bring the socket back and the tab would sit with no
events and Invoke disabled until a reload. Bounding reconnectionAttempts globally has the same
problem for ordinary outages, which today self-heal.
Options
- Accept and document. Unbounded transport retry is the app's baseline; the only new part is
that it can be entered from the bounded path. Cheapest, and arguably correct — a client that
wants events back should keep trying while the server is unreachable.
- Bound in time rather than attempts. Stop retrying if no connection has succeeded within
some window of a server-initiated disconnect (a minute, say), on the grounds that a socket the
server dropped for authorization reasons and that cannot re-establish is unlikely to be
welcome. Leaves the tab dead but quiet; needs a visible "reconnect" affordance to be
defensible.
- Rate-limit the failure case. Keep retrying forever, but raise
reconnectionDelayMax for
the post-server-disconnect path so a refusing port cannot produce bursts like the measured
four-in-180 ms.
My inclination is 1 or 3 — 2 trades a self-healing failure for a permanently dead tab, which is
the failure #9540 set out to remove. Worth deciding deliberately rather than inheriting the
default by accident.
Follow-up to #9540 (thanks @JPPhoto for catching this in review — it was not a merge blocker there).
What happens
#9540 added a bounded retry for server-initiated socket disconnects:
io server disconnectisterminal for socket.io (
Socket.ondisconnect→destroy()→Manager._close()setsskipReconnect), so the client drives up to fivesocket.connect()attempts with a doublingdelay before giving up
(
invokeai/frontend/web/src/services/events/useSocketIO.ts,MAX_SERVER_DISCONNECT_RECONNECTS).That bound only governs server-initiated disconnects.
socket.connect()callsManager.open(),which resets
skipReconnect = false, and the socket's options set neitherreconnectionnorreconnectionAttempts— so the manager's own defaults apply:reconnection: true,reconnectionAttempts: Infinity,reconnectionDelay: 1000,reconnectionDelayMax: 5000,randomizationFactor: 0.5(socket.io-client/build/esm/manager.js:21-29). If one of our retryattempts fails at the transport level — server down, port refused — the manager takes over and
retries indefinitely. Our counter never advances, because a failed attempt emits
connect_error,not
disconnect.@JPPhoto measured four reconnect attempts in 180 ms against a closed port with socket.io-client
4.8.3; against a host that black-holes rather than refuses, the documented backoff applies and the
cadence settles at roughly one attempt every 2.5-7.5 s.
Scope
This is socket.io's default behavior for the app's socket everywhere, not something #9540
introduced: any network outage already produces the same unbounded retry, on
mainas well. Whatis new is the interaction — a path that advertises a five-attempt bound can hand off to an
unbounded one.
Why the obvious fix is wrong
Disabling manager reconnection around the custom retry (
socket.io.opts.reconnection = false)would strand the tab in exactly the scenario the retry exists for: the server disconnects the
socket for an authorization change and is then briefly unavailable — a restart, a proxy blip —
and with no manager retry, nothing would ever bring the socket back and the tab would sit with no
events and Invoke disabled until a reload. Bounding
reconnectionAttemptsglobally has the sameproblem for ordinary outages, which today self-heal.
Options
that it can be entered from the bounded path. Cheapest, and arguably correct — a client that
wants events back should keep trying while the server is unreachable.
some window of a server-initiated disconnect (a minute, say), on the grounds that a socket the
server dropped for authorization reasons and that cannot re-establish is unlikely to be
welcome. Leaves the tab dead but quiet; needs a visible "reconnect" affordance to be
defensible.
reconnectionDelayMaxforthe post-server-disconnect path so a refusing port cannot produce bursts like the measured
four-in-180 ms.
My inclination is 1 or 3 — 2 trades a self-healing failure for a permanently dead tab, which is
the failure #9540 set out to remove. Worth deciding deliberately rather than inheriting the
default by accident.