Skip to content

v1.4.0

Choose a tag to compare

@jdwyah jdwyah released this 11 Sep 12:10
· 8 commits to main since this release
d541782

If you fork worker processes and keep evaluating in the parent, upgrade. On 1.0.0–1.3.0 the parent process went permanently dark after any fork(2) — it kept serving whatever config snapshot it held at the moment of the fork, forever, while connection_state reported :connected.

  • Fix (fork): the Process._fork hook is now child-only — the parent is never touched (qfg-lv4n.1). Since 0.0.16 the hook tore the SSE worker, fallback poller, telemetry reporter, and datadir watcher down in the parent before the fork syscall, and restarted them only in the child. Any topology where the parent keeps evaluating after a fork — a Sidekiq process using the parallel gem, a fork { ... } inside a job, a rake task that forks — lost live updates permanently. A customer's Sidekiq process served a 13-day-old snapshot through four rollout changes this way. The hook now does nothing on the parent side: the parent keeps its stream, its poller, its telemetry reporter, and its live config straight through any number of forks. This restores the design every other SDK in this space uses (Reforge's Reforge.fork, LaunchDarkly's postfork, dd-trace-rb / redis-client / connection_pool, which all branch on the child stage of _fork only). No customer wiring is required, and no public API was removed.

  • Change (fork): a forked child re-initializes on its FIRST use of the client, and never inherits the parent's config snapshot. The hook does no I/O at all: it drops what the child inherited — including the parent's store — and arms a flag. The first get / defined? / keys / details lookup in the child then does exactly what Client.new does: its own blocking config fetch under the configured init_timeout_ms and on_init_failure, then its own SSE stream (or fallback poller) and its own telemetry reporter. Three consequences worth knowing: the first call in a forked child pays one fetch; a child that never uses the client costs nothing (no fetch, no socket, no thread, and stop returns immediately); and because the child's store starts empty, its first envelope is installed rather than dropped by the reject-older guard as same-generation. This is full parity with Reforge, where a forked process simply builds a new client. connection_state does not trigger the re-initialization — a diagnostic must never open a socket — and answers :initializing while one is pending.

  • Fix (fork): concurrent first use in a child waits for the one rebuild (qfg-lv4n.1). The re-initialization used an unlocked fast path and cleared its pending flag before doing the work, so only the thread that won the mutex actually blocked: every other thread read the cleared flag, skipped the mutex, and evaluated against the brand-new empty store. Sixteen threads hitting a freshly forked client against a 400ms config endpoint returned {nil=>15, "v1"=>1}. The flag now stays set for the whole rebuild, so every other caller blocks on it and then sees the fetched config — one fetch, one stream dial, one reporter per child, however many threads race the first request. A same-thread guard keeps a customer logger (a semantic_logger_filter / stdlib_formatter that evaluates a config) from deadlocking on the non-reentrant lock if it fires from inside the rebuild.

  • Fix (fork): a hard timeout mid-rebuild no longer leaves the child dark forever (qfg-lv4n.1). Timeout::ExitException (Ruby 3.3's timeout 0.4.1), rack-timeout's RequestTimeoutException, and Thread#kill are all Exception, not StandardError, so they crossed the rebuild's rescue untouched — and with the pending flag already cleared, every later call in that child returned nil over an empty store with no stream, no poller and no reporter, permanently. The flag is now cleared at the point the child actually has a live path to config, so anything that escapes before that leaves the rebuild armed and the next call retries it.

  • Fix (fork): stop racing an in-flight rebuild no longer orphans an SSE worker (qfg-lv4n.1). stop cleared the pending flag and tore down without taking the rebuild lock, so a rebuild already past that point went on to build a stream nothing held a reference to and nothing could close. stop now raises its stopped flag before queueing for the rebuild lock — an in-flight rebuild sees it and starts neither an update channel nor a telemetry reporter — and the teardown itself runs under that lock, so it can never interleave with construction.

  • Fix (fork): a datadir child whose rebuild fails no longer dials SSE (qfg-lv4n.1). The failure path started the update channel regardless of mode, so a child of a purely offline (datadir) client opened a stream to stream.primary.quonfig.com on its first lookup and then logged Error applying SSE envelope: undefined method `apply_envelope' for nil for every envelope that arrived — with nothing re-armed, repairing the workspace on disk never helped. A datadir client's healing path is the filesystem: the failure path now starts the datadir watcher when data_dir_auto_reload is on, and otherwise re-arms the rebuild so the next use retries the load.

  • Fix (fork): after_fork_in_child called in the PARENT is now a no-op (qfg-lv4n.1). Releases 1.0.0–1.3.0 documented calling Quonfig.instance.after_fork_in_child in the parent after fork returned as the workaround for the parent going dark. On 1.4.0 the parent's components are alive, so each such call orphaned a live SSE worker and its stream, zeroed the store, and stopped the owner's telemetry reporter — three calls took a process from 2 worker threads and 1 live stream to 8 and 4, and the orphans outlived stop. The hook now early-returns (one debug line) in the process that owns the client, decided by comparing the current pid against the one stamped when the client was built. If you added that call in the parent as a 1.3.0 workaround, remove it; it does nothing there now.

  • Fix (fork): a child forked from inside an on_update callback rebuilds like any other child (qfg-lv4n.1). Parent-vs-child detection asked whether the inherited SSE worker thread was alive. on_update runs on that worker thread, so a customer who forks from the callback forks on it — making it the child's one surviving thread, and making the inherited @worker.alive? answer true in a real child. Such a child was classified as the parent: the hook ignored it, it served the parent's snapshot for the rest of its life, reported :connected, and resumed the parent's SSE loop on the shared file descriptor. Ownership is now a pid stamp taken at construction (and re-taken when a child rebuilds), so a pid mismatch is proof of a fork child whatever the inherited Thread objects claim. The same stamp closes the other end of the hole: a parent-side after_fork_in_child on a client with no threads at all (datadir + data_dir_auto_reload: false + no SDK key) sailed straight past the old liveness guard and wiped the live store; it is now correctly a no-op.

  • Fix (fork): a retried rebuild no longer dials a second stream (qfg-lv4n.1). The rebuild disarms its pending flag only after the update channel is up, so a non-StandardError landing in that window (rack-timeout, Timeout::ExitException, Thread#kill) left the flag armed with a live stream. The retry re-ran network init, opened a second SSE stream, and overwrote @sse_client — orphaning the first worker where stop could never reach it, and leaving the child holding two streams against the delivery service. start_update_channel is now idempotent: it returns immediately if an SSE worker or poll supervisor is already alive.

  • Fix (fork): on_init_failure: :raise behaves the same in a forked child as in a fresh client (qfg-lv4n.1). A failed re-initialization was always swallowed and logged, so the documented "exactly like a newly constructed client" was false for the one option whose entire job is raise-vs-return. Under :raise the first use in a child now raises the init error out of the lookup — Reforge does the same, raising the init error out of get itself — and later lookups keep raising, without re-fetching, until the update channel lands an envelope. The update channel is still started on the way out so the child can heal. Note that :raise is the default, so with default options a forked child whose first lookup lands in a total delivery outage (primary and secondary both unreachable) raises where 1.3.0 silently served the parent's snapshot; set on_init_failure: :return if you would rather a child serve defaults through an outage. :return is unchanged: one line logged, defaults served.

  • Fix (fork): Quonfig.fork / Client#fork in a child the hook already prepared returns the same client (qfg-4t5o). The 1.0–1.3 README taught on_worker_boot { Quonfig.fork }, and on Ruby 3.1+ that call ran after the Process._fork hook had already prepared the client in the worker. Before first use it discarded the prepared client and built a second one with an eager fetch; after first use it left the worker holding two live SSE streams and two telemetry reporters, the first pair orphaned where stop could never reach them. Client#fork now returns self when the hook has already prepared it in the current process, so a leftover call from older docs is harmless. Outside a hook-prepared child (the owning process, Ruby 3.0, a client stopped before the fork) it still builds a fresh client, which is the Ruby 3.0 manual-wiring path; the old client is never stopped.

  • Fix (fork): the public store / resolver / evaluator / config_loader readers route through the post-fork re-initialization (qfg-lv4n.1). They bypassed it entirely, so in a forked child that had not been used yet client.store.get(key) answered nil and client.resolver.get(key, {}) raised MissingDefaultError against the empty store. They stay public (semver) and now re-initialize before handing the component back; reading one in a forked child can therefore block on the child's own fetch.

  • Fix (fork): the child drops inherited references instead of closing them. fork(2) duplicates file descriptors, so the child's copy of the SSE socket points at the connection the parent is still streaming on — closing it would write a TLS close_notify onto that shared connection and kill the parent's stream. The child now nils @sse_client, @poll_supervisor, @telemetry_reporter, @datadir_watcher, and the pending fallback-engage timer without calling close / stop / join on any of them (joining an inherited thread blocks forever, since the thread does not exist in the child), then builds everything fresh.

  • Fix (fork): the child's telemetry starts empty. The forked child gets brand-new context-shape, example-context, evaluation-summary, and failover aggregators rather than inheriting the parent's half-full ones (the reporter is not started until the child's first use). The parent flushes the data it collected; the child reports only its own, so a fork no longer double-counts a window of evaluations.

  • Fix (fork/telemetry): the inherited at_exit drain no longer speaks for the parent. TelemetryReporter#start registers a process-wide Kernel.at_exit closure over the reporter; fork(2) copies it, and dropping the client's reference in the child does not unregister it — so any child that exited the normal way (block-form fork + exit, which is what the parallel gem does) POSTed a full copy of the parent's un-flushed window under the parent's instanceHash, and the parent then POSTed it again. The reporter now records an owner pid on start, and sync, stop, and the at_exit drain are no-ops (one debug line) in any other process; the child additionally discards the inherited aggregators. Measured on a three-child Parallel.map: 404 evaluations reported for 101 performed, now 101. Nothing is closed, stopped, or joined — the thread does not exist in the child and the HTTP connection's fd is shared with the parent.

  • Fix (fork): one client failing to rebuild no longer takes the rest of the registry down with it. The child-side fan-out was covered by a single hook-wide rescue, so the first after_fork_in_child to raise (thread exhaustion, a customer logger that raises) aborted the loop and every client behind it in the registry stayed dark. Each instance now has its own rescue and the fan-out continues, logging the failure at error.

  • Fix (fork): a forked datadir child gets its own telemetry reporter (qfg-vquv). The datadir branch of after_fork_in_child returned before the aggregator/reporter rebuild, so a datadir + SDK-key child — an emitting combination since 1.3.0 — recorded nothing of its own for the rest of its life. It now rebuilds aggregators and reporter under exactly the SDK-key gating a fresh client applies.

  • Fix (diagnostics): connection_state derives from liveness, not from a stored flag. It previously answered :connected off @sse_state alone, which the teardown path never reset — so a client with no SSE worker and no poller alive still reported healthy. That is why the incident above stayed invisible for 13 days. A network client that is supposed to hold an SSE stream and has no live worker now reports :disconnected. The documented value set (:initializing, :connected, :disconnected, :falling_back) is unchanged, and :connected still covers datadir mode and post-fetch clients with SSE disabled.

  • Deprecated: Client#before_fork_in_parent. Still public and still works, but the fork hook no longer calls it. There is no longer any reason to tear a parent down before forking; call stop if you want a client dead. Slated for removal in 2.0.0.

  • Docs: "one telemetry POST at exit" for a per-job forking worker is now qualified — the at-exit drain only happens when the child exits normally. Parallel children do; Resque children call exit! by default, which skips every at_exit handler, so there is no drain and no telemetry POST unless RUN_AT_EXIT_HOOKS=1 is set.

  • Docs: the README's fork section previously claimed the SDK covered "Sidekiq's parent-forks-workers model" (Sidekiq OSS does not fork — it runs jobs on threads) and documented "does not auto-restart the parent" as intentional. Both are corrected. The Puma snippet no longer suggests before_fork { Quonfig.instance.stop }, and the cases that actually fork inside Sidekiq (the parallel gem, an explicit fork { }, Enterprise swarm) are named as covered. The Ruby 3.0 parallel snippet no longer calls Quonfig.fork once per row (that builds a client per item; it is now a pid-memoized rebuild once per child process, with a note that 3.0 is EOL), and the Puma/Unicorn worker-boot snippets show SemanticLogger.reopen alone on 3.1+ — calling Quonfig.fork there after the hook has already rebuilt leaves the worker with two live SSE streams and two reporters.

  • ActiveJob tagged logger issue [#164]

  • Compact Log Format [#163]

  • Tagged Logging [#161]

  • ContextKey logging thread safety [#162]