Skip to content

fix(ensindexer): hot-reload safe writer worker#1928

Open
tk-o wants to merge 10 commits intomainfrom
fix/1432-ensindexer-dev-mode-experience
Open

fix(ensindexer): hot-reload safe writer worker#1928
tk-o wants to merge 10 commits intomainfrom
fix/1432-ensindexer-dev-mode-experience

Conversation

@tk-o
Copy link
Copy Markdown
Contributor

@tk-o tk-o commented Apr 14, 2026

fix(ensindexer): hot-reload safe writer worker

Reviewer Focus

  • apps/ensindexer/src/lib/local-ponder-context.ts — the getApiShutdown() / getShutdown() function pattern. The staleness contract (always read fresh, never cache) is enforced ergonomically by making call sites function calls rather than property accesses.
  • apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts — the .run().catch() clean-stop discrimination: (abortSignal.aborted || ensDbWriterWorker !== worker) && isAbortError(error). Both abort-source paths must be validated to avoid masking real errors AND to avoid misclassifying intentional supersession as a fatal error.
  • apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.ts — the stopRequested guard. stop() flips it; run() checks via checkCancellation() between every async setup step so a stop during startup actually cancels.

Problem

ponder dev-mode hot reload of indexing handler files crashed ensindexer with Error: EnsDbWriterWorker has already been initialized and process.exit(1).

Root cause: ponder kills and replaces common.shutdown and common.apiShutdown on every indexing reload (ponder/src/bin/commands/dev.ts:95-101), then re-execs the api entry file. but vite-node only invalidates the changed file's dep tree — api-side singleton modules stay cached. so module-level state survives across reloads (singleton check throws), and any reference cached during the first boot goes stale.

resolves #1432.

What Changed

  1. local-ponder-context.ts — stable fields stay as an eager const. reload-scoped fields are exposed as functions getApiShutdown() / getShutdown() that re-read globalThis.PONDER_COMMON on every call. the function-call shape makes the fresh-read contract visible at every call site, so accidental caching reads as obviously wrong (const sig = getApiShutdown().signal clearly caches a function result; the equivalent property access wouldn't).
  2. singleton.tsstartEnsDbWriterWorker() is reset-aware. drops the throw-on-re-init, awaits any prior worker's stop(), registers cleanup via getApiShutdown().add(...) so ponder properly awaits worker shutdown during its kill sequence. extracted gracefulShutdown(worker, reason) helper. .run().catch() distinguishes intentional stops (signal aborted OR worker superseded in singleton, AND error is AbortError) from real failures, which fail-fast via process.exit(1).
  3. ensdb-writer-worker.tsstop() is async. inFlightSnapshot tracks the latest upsert with skip-overlap so stop() deterministically awaits one promise. new stopRequested flag + checkCancellation() helper let stop() cancel an in-progress run() startup before it arms the recurring interval. run() accepts an optional external AbortSignal for the same purpose.
  4. packages/ponder-sdkPonderClient accepts an optional AbortSignalGetter (typed alias). LocalPonderClient forwards it. invoked at fetch time inside health/metrics/status so requests use the current signal instead of a captured-at-construct reference. PonderAppShutdownManager interface + isPonderAppShutdownManager guard exported alongside PonderAppContext since they mirror a Ponder runtime type.

Self-Review

  • bugs caught during review iteration: typescript narrowing of globalThis.PONDER_COMMON lost when read moved into a function (added explicit check); isAbortError originally only checked instanceof Error and missed DOMException (the actual fetch-abort rejection type); the catch-path discriminator went through ||&&(signal.aborted || superseded) && isAbortError as the supersession case from the defensive cleanup path was identified.
  • logic simplified: replaced an initial reactive Proxy over globalThis.PONDER_COMMON with explicit getter functions. the proxy was technically correct but hid the staleness contract behind property-access syntax. -19 lines.
  • naming: introduced "reload-scoped" vs "stable" field distinction to make the contract teachable.
  • dead code removed: the typeof ensDbWriterWorker !== "undefined" throw is gone — re-init is the expected path.

Cross-Codebase Alignment

  • audited every api-side singleton (ensDbClient, ensRainbowClient, publicConfigBuilder, indexingStatusBuilder, localPonderClient) for stale-reference hazards. only localPonderClient captured a runtime mutable (the abort signal), addressed via the getter-pattern threading.
  • publicConfigBuilder and indexingStatusBuilder cache immutablePublicConfig / _immutableIndexingConfig after first build. could mask config changes if you edit config.ts and reload — pre-existing pre-fix behavior, not the crash, deferred.

Downstream & Consumer Impact

  • PonderClient constructor: optional second arg getAbortSignal?: AbortSignalGetter. backwards compatible.
  • LocalPonderClient constructor: optional 5th arg, same. backwards compatible.
  • EnsDbWriterWorker.stop() is now async and returns Promise<void>. existing call sites (production + tests) updated to await.
  • new exports from @ensnode/ponder-sdk: AbortSignalGetter, PonderAppShutdownManager, isPonderAppShutdownManager.

Testing

  • pnpm -F ensindexer typecheck
  • pnpm -F @ensnode/ponder-sdk typecheck
  • pnpm test --project ensindexer --project @ensnode/ponder-sdk → 268 passed (4 new PonderClient tests for the getAbortSignal getter pattern).
  • pnpm lint
  • manual: pnpm -F ensindexer dev against ens-test-env, edited an indexing handler multiple times. confirmed: hot reload completes, no "already been initialized", no ELIFECYCLE, indexing resumes, snapshots continue upserting.

no automated hot-reload test infrastructure exists; manual smoke test is the only path.

Risk Analysis

  • the catch path's intentional-stop discrimination has been the most-reviewed area. final shape: (signal.aborted || ensDbWriterWorker !== worker) && isAbortError(error) — both an abort-source signal AND the AbortError name are required, ruling out both "real error during shutdown" and "AbortError from a cross-contamination race that didn't actually stop us."
  • stopRequested + the external AbortSignal provide two independent cancellation channels. checkCancellation() honors both.
  • process.exit(1) on real worker errors is explicit and fail-fast. can't rethrow from the fire-and-forget catch (would become an unhandled rejection rather than reaching api/index.ts).
  • the function-call getter pattern can still be misused (capture the result), but the misuse reads as obviously wrong, which is the whole improvement over the proxy.
  • mitigations / rollback: revert the commit; the previous (broken) behavior is well understood.
  • named owner: @shrugs

Pre-Review Checklist

  • I reviewed every line of this diff and understand it end-to-end
  • I'm prepared to defend this PR line-by-line in review
  • I'm comfortable being the on-call owner for this change
  • Relevant changesets are included (or explicitly not required)

Copilot AI review requested due to automatic review settings April 14, 2026 18:50
@vercel
Copy link
Copy Markdown
Contributor

vercel bot commented Apr 14, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated (UTC)
admin.ensnode.io Skipped Skipped Apr 15, 2026 10:45pm
ensnode.io Skipped Skipped Apr 15, 2026 10:45pm
ensrainbow.io Skipped Skipped Apr 15, 2026 10:45pm

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Apr 14, 2026

🦋 Changeset detected

Latest commit: 8702d0d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 24 packages
Name Type
ensindexer Patch
@ensnode/ponder-sdk Patch
ensadmin Patch
ensrainbow Patch
ensapi Patch
fallback-ensapi Patch
enssdk Patch
enscli Patch
enskit Patch
ensskills Patch
@ensnode/datasources Patch
@ensnode/ensrainbow-sdk Patch
@ensnode/ensdb-sdk Patch
@ensnode/ensnode-react Patch
@ensnode/ensnode-sdk Patch
@ensnode/ponder-subgraph Patch
@ensnode/shared-configs Patch
@docs/ensnode Patch
@docs/ensrainbow Patch
@docs/mintlify Patch
@namehash/ens-referrals Patch
@namehash/namehash-ui Patch
@ensnode/enskit-react-example Patch
@ensnode/integration-test-env Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 14, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9bc2022c-0cd6-48ab-ae9a-4513a5b1b8b1

📥 Commits

Reviewing files that changed from the base of the PR and between 46b4f4d and 8702d0d.

📒 Files selected for processing (4)
  • apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.ts
  • apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts
  • apps/ensindexer/src/lib/local-ponder-client.ts
  • apps/ensindexer/src/lib/local-ponder-context.ts

📝 Walkthrough

Walkthrough

startEnsDbWriterWorker was converted to an async, idempotent routine that stops any existing worker before creating a new EnsDbWriterWorker; worker run/stop now accept AbortSignals and await in-flight snapshot work; Ponder clients gained a dynamic abort-signal getter; shutdown managers are re-read and validated at access time.

Changes

Cohort / File(s) Summary
Worker singleton & core worker
apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts, apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.ts, apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.test.ts
Made startEnsDbWriterWorker() async/idempotent and re-acquire shutdown manager at start; EnsDbWriterWorker.run(signal?: AbortSignal) adds cancellation checkpoints and throws on abort; stop() is now async and awaits in-flight snapshot work; tests updated to await worker.stop().
Local Ponder context & client wiring
apps/ensindexer/src/lib/local-ponder-context.ts, apps/ensindexer/src/lib/local-ponder-client.ts
Replaced eager shutdown values with runtime reads/validation (getApiShutdown(), getShutdown()); LocalPonderClient now constructed with a thunk that returns the current abort signal.
Ponder SDK: client, local client, and types
packages/ponder-sdk/src/client.ts, packages/ponder-sdk/src/local-ponder-client.ts, packages/ponder-sdk/src/ponder-app-context.ts, packages/ponder-sdk/src/client.test.ts
Added exported AbortSignalGetter type and optional getAbortSignal constructor arg to PonderClient/LocalPonderClient; HTTP methods pass signal: this.getAbortSignal?.(); added PonderAppShutdownManager interface and isPonderAppShutdownManager type guard; tests added for abort-signal getter behavior.
Integration usage
apps/ensindexer/src/lib/local-ponder-client.ts
Constructs LocalPonderClient with () => getApiShutdown().abortController.signal thunk (wires dynamic abort signal into SDK client).
Metadata
.changeset/fix-ensindexer-hot-reload.md
Added changeset bump describing hot-reload crash fix for ensindexer.

Sequence Diagram

sequenceDiagram
    participant Caller as startEnsDbWriterWorker
    participant Context as LocalPonderContext
    participant Shutdown as apiShutdown Manager
    participant Singleton as EnsDbWriter Singleton
    participant Worker as EnsDbWriterWorker

    rect rgba(100,150,200,0.5)
    Caller->>Context: getApiShutdown()
    Context-->>Caller: validated apiShutdown
    Caller->>Singleton: stop existing worker (if any)
    Singleton-->>Caller: cleared
    Caller->>Worker: new EnsDbWriterWorker()
    Caller->>Shutdown: register callback -> stop this Worker
    Caller->>Worker: run(abortSignal)
    Worker->>Worker: schedule interval, respect signal
    end

    rect rgba(200,100,100,0.5)
    Shutdown->>Shutdown: abortController.signal aborted
    Shutdown->>Singleton: invoke registered callback
    Singleton->>Worker: await stop() (clear interval, await in-flight snapshot)
    Worker-->>Singleton: stopped
    Singleton->>Caller: worker reference cleared
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

🐰 I nibble at signals, fresh and bright,
Proxy ears read each reload at night,
Worker pauses, finishes its snack,
Then gently stops and hops right back,
Hot-reload hums — no crash in sight!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title 'fix(ensindexer): hot-reload safe writer worker' clearly and specifically describes the primary change: making the ENSIndexer worker safe during hot reloads.
Description check ✅ Passed The PR description comprehensively follows the template with detailed Summary, Why (linked to issue #1432), Testing (multiple test runs documented), and Notes for Reviewer sections.
Linked Issues check ✅ Passed The PR successfully addresses the linked issue #1432 by making API-side singletons and runtime references safe across hot reloads, preventing crashes and 'already been initialized' errors during dev-mode handler edits.
Out of Scope Changes check ✅ Passed All changes are scoped to fixing hot-reload safety: singleton restart patterns, stale reference handling, cancellation logic, and SDK-level abort signal support. No unrelated changes detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/1432-ensindexer-dev-mode-experience

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@tk-o tk-o changed the title WIP Fix/1432 ensindexer dev mode experience [WIP] Fix ENSIndexer dev mode experience Apr 14, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to improve the local/dev experience for ENSIndexer by wiring Ponder runtime shutdown signals into the SDK/app so long-running processes and network requests can be aborted cleanly (resolving issue #1432).

Changes:

  • Add an abortSignal to PonderAppContext and derive it from Ponder-provided shutdown controllers during deserialization.
  • Propagate the abort signal into PonderClient/LocalPonderClient so fetch() requests can be canceled on shutdown.
  • Stop the ENSDb writer worker when the local Ponder app abort signal fires.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
packages/ponder-sdk/src/ponder-app-context.ts Adds abortSignal to the app context interface for shutdown signaling.
packages/ponder-sdk/src/deserialize/ponder-app-context.ts Deserializes new shutdown managers and builds a merged AbortSignal.
packages/ponder-sdk/src/client.ts Adds optional abort signal support to fetch() calls.
packages/ponder-sdk/src/local-ponder-client.ts Passes the app abort signal through to the base PonderClient.
apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts Stops the ENSDb writer worker on abort to support clean shutdown in dev/serve.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/ponder-sdk/src/deserialize/ponder-app-context.ts Outdated
Comment thread packages/ponder-sdk/src/deserialize/ponder-app-context.ts Outdated
Comment thread packages/ponder-sdk/src/deserialize/ponder-app-context.ts
Comment thread packages/ponder-sdk/src/ponder-app-context.ts Outdated
Comment thread apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts Outdated
Comment thread packages/ponder-sdk/src/client.ts
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts`:
- Line 33: Update the logger.info call in singleton.ts so the startup message
matches the shutdown wording: change the string passed to logger.info (currently
"StartingEnsDbWriterWorker...") to "Starting EnsDbWriterWorker..." by editing
the logger.info invocation to include the missing space for consistency with the
shutdown log.

In `@packages/ponder-sdk/src/deserialize/ponder-app-context.ts`:
- Around line 60-71: The shutdown manager Zod schema
(schemaPonderAppShutdownManager) incorrectly constrains the callback signature
for add to return void; update the add entry so the inner callback's output
allows unknown or a Promise (e.g., use z.union([z.void(), z.unknown()]) for the
inner z.function output) so it accepts unknown | Promise<unknown>, or
alternatively relax the whole object like the logger schema by using
.passthrough()/.looseObject() to avoid overconstraining the callback signature;
ensure you still validate isKilled as z.boolean() and abortController as
z.instanceof(AbortController).

In `@packages/ponder-sdk/src/ponder-app-context.ts`:
- Around line 35-39: The mock object in local-ponder-client.mock.ts must include
the new abortSignal property to satisfy the PonderAppContext interface; update
the exported mock (the object asserted with "satisfies PonderAppContext") to add
abortSignal: new AbortController().signal (or reuse a shared AbortController if
needed) so TypeScript compilation succeeds and any long-running mock consumers
can be signalled for shutdown.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 8a3cd472-61a0-4a2d-8ea8-772f9879eba8

📥 Commits

Reviewing files that changed from the base of the PR and between 319d619 and 15cd750.

📒 Files selected for processing (5)
  • apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts
  • packages/ponder-sdk/src/client.ts
  • packages/ponder-sdk/src/deserialize/ponder-app-context.ts
  • packages/ponder-sdk/src/local-ponder-client.ts
  • packages/ponder-sdk/src/ponder-app-context.ts

Comment thread apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts Outdated
Comment thread packages/ponder-sdk/src/deserialize/ponder-app-context.ts Outdated
Comment thread packages/ponder-sdk/src/ponder-app-context.ts Outdated
Comment thread packages/ponder-sdk/src/deserialize/ponder-app-context.ts Outdated
Comment thread apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts Outdated
Comment thread apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts Outdated
Ponder's dev-mode hot reload re-executes the API entry file but caches
transitively-imported modules, leaving module-level singleton state
stale. On reload, startEnsDbWriterWorker() would throw "EnsDbWriterWorker
has already been initialized" and kill the process.

- local-ponder-context.ts is now a reactive proxy that re-reads
  globalThis.PONDER_COMMON.apiShutdown (and .shutdown) on every access.
  Ponder kills and replaces these managers on each reload
  (ponder/src/bin/commands/dev.ts:95-101), so cached references go
  stale immediately.
- startEnsDbWriterWorker() is reset-aware: awaits any prior worker's
  stop(), registers cleanup via apiShutdown.add(), and ignores
  AbortError in the .run().catch() path so shutdown does not kill the
  process.
- EnsDbWriterWorker.stop() is async and awaits any in-flight snapshot
  upsert, so Ponder's shutdown sequence can wait on it.
- PonderClient/LocalPonderClient accept an optional getAbortSignal()
  getter, invoked at fetch time, so HTTP requests use the current
  signal instead of a captured-at-construct reference that goes stale
  per reload.

Resolves #1432.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@shrugs shrugs force-pushed the fix/1432-ensindexer-dev-mode-experience branch from 15cd750 to 73689cc Compare April 15, 2026 18:21
@vercel vercel bot temporarily deployed to Preview – ensrainbow.io April 15, 2026 18:21 Inactive
@vercel vercel bot temporarily deployed to Preview – admin.ensnode.io April 15, 2026 18:21 Inactive
@vercel vercel bot temporarily deployed to Preview – ensnode.io April 15, 2026 18:21 Inactive
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.ts`:
- Around line 148-158: The tests call worker.stop() without awaiting its
now-async implementation; update each test invocation to use await worker.stop()
so the in-flight snapshot and interval cleanup complete before assertions finish
— specifically change the calls at the noted locations (the occurrences of
worker.stop() referenced in the comment) to await worker.stop() inside their
existing async test functions (mirror the pattern used in singleton.ts where
await worker.stop() is used).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6ce727dd-cb8f-44ce-9691-2a5058c7b0dc

📥 Commits

Reviewing files that changed from the base of the PR and between 15cd750 and 73689cc.

📒 Files selected for processing (6)
  • apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.ts
  • apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts
  • apps/ensindexer/src/lib/local-ponder-client.ts
  • apps/ensindexer/src/lib/local-ponder-context.ts
  • packages/ponder-sdk/src/client.ts
  • packages/ponder-sdk/src/local-ponder-client.ts

Comment thread apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.ts
Comment thread packages/ponder-sdk/src/client.ts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 15, 2026 18:35
@shrugs shrugs changed the title [WIP] Fix ENSIndexer dev mode experience fix(ensindexer): hot-reload safe writer worker Apr 15, 2026
vercel[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts:93

  • This worker.run().catch(...) chain is fire-and-forget, but the handler rethrows the error. Because the returned promise isn’t awaited/returned, that rethrow becomes an unhandled rejection (and may not reliably result in the intended shutdown behavior across Node settings). Either return/await the run() promise from startEnsDbWriterWorker so initialization fails deterministically, or avoid rethrowing here and terminate explicitly (or mark as intentionally unawaited with void).
  worker
    .run()
    // Handle any uncaught errors from the worker
    .catch(async (error) => {
      // If Ponder has begun shutting down our API instance (hot reload or
      // graceful shutdown), the abort propagates through in-flight fetches
      // as an AbortError. Treat that as a clean stop, not a worker failure.
      if (apiShutdown.abortController.signal.aborted || isAbortError(error)) {
        logger.info({
          msg: "EnsDbWriterWorker stopped due to API shutdown",
          module: "EnsDbWriterWorker",
        });
        return;
      }

      // Real worker error — clean up and trigger non-zero exit.
      await worker.stop();
      if (ensDbWriterWorker === worker) {
        ensDbWriterWorker = undefined;
      }

      logger.error({
        msg: "EnsDbWriterWorker encountered an error",
        error,
      });

      // Re-throw the error to ensure the application shuts down with a non-zero exit code.
      process.exitCode = 1;
      throw error;
    });

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.ts
Comment thread apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.ts
- await worker.stop() in all ensdb-writer-worker tests (stop() is async)
- skip overlapping snapshot upserts via inFlightSnapshot guard so a slow
  upsert can't pile up concurrent ENSDb writes; stop() awaits the single
  in-flight upsert deterministically
- thread an optional AbortSignal into worker.run() and check between
  await steps so a hot reload during run() startup bails before the
  recurring interval is scheduled (closes the startup race)
- extract gracefulShutdown helper in singleton.ts shared by the
  apiShutdown.add() callback and the .run().catch() paths
- add PonderClient unit tests asserting the getAbortSignal getter is
  invoked per-fetch, returns fresh identity across calls, and cancels
  in-flight fetches when aborted

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel vercel bot temporarily deployed to Preview – admin.ensnode.io April 15, 2026 18:54 Inactive
@vercel vercel bot temporarily deployed to Preview – ensrainbow.io April 15, 2026 18:54 Inactive
@vercel vercel bot temporarily deployed to Preview – ensnode.io April 15, 2026 18:54 Inactive
@vercel vercel bot temporarily deployed to Preview – admin.ensnode.io April 15, 2026 21:53 Inactive
@vercel vercel bot temporarily deployed to Preview – ensrainbow.io April 15, 2026 21:53 Inactive
@vercel vercel bot temporarily deployed to Preview – ensnode.io April 15, 2026 21:53 Inactive
@shrugs
Copy link
Copy Markdown
Collaborator

shrugs commented Apr 15, 2026

@greptileai please re-review — addressed coderabbit's stop()-during-run race (added internal stopRequested guard) and the fail-fast concern (process.exit(1) on real worker errors, since the catch is on a fire-and-forget promise)

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apps/ensindexer/src/lib/ensdb-writer-worker/singleton.ts Outdated
… path

When the defensive stale-instance cleanup inside startEnsDbWriterWorker
calls worker.stop() on the previous worker, the old run() throws an
AbortError via the new stopRequested guard — but the captured
abortSignal may not be aborted (the safety net runs precisely when
Ponder's own apiShutdown.add() callback didn't fire).

The catch path now also treats `ensDbWriterWorker !== worker` as a
clean-stop signal, so the supersession path doesn't get misclassified
as a fatal error and call process.exit(1). isAbortError() is still
required so unrelated failures are not silently swallowed.

Surfaced by copilot review.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel vercel bot temporarily deployed to Preview – ensrainbow.io April 15, 2026 22:05 Inactive
@vercel vercel bot temporarily deployed to Preview – ensnode.io April 15, 2026 22:05 Inactive
@vercel vercel bot temporarily deployed to Preview – admin.ensnode.io April 15, 2026 22:05 Inactive
@shrugs
Copy link
Copy Markdown
Collaborator

shrugs commented Apr 15, 2026

@greptileai please re-review — addressed copilot's catch-path classification: defensive stale-instance cleanup now correctly handles AbortError from the new stopRequested guard via an ensDbWriterWorker !== worker supersession check

…ellation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 15, 2026 22:17
@vercel vercel bot temporarily deployed to Preview – ensnode.io April 15, 2026 22:17 Inactive
@vercel vercel bot temporarily deployed to Preview – admin.ensnode.io April 15, 2026 22:17 Inactive
@vercel vercel bot temporarily deployed to Preview – ensrainbow.io April 15, 2026 22:17 Inactive
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apps/ensindexer/src/lib/ensdb-writer-worker/ensdb-writer-worker.ts Outdated
…getters

The Proxy was technically correct but call-site ergonomics hid the
staleness contract: `localPonderContext.apiShutdown` looked like an
ordinary property access, masking that it was actually a fresh read
from globalThis. A captured `const sig = localPonderContext.apiShutdown
.signal` looked innocent but was a footgun.

Split the context into two shapes:
- `localPonderContext` is back to an eager `const` for the stable
  fields (command, localPonderAppUrl, logger). Ponder doesn't mutate
  these, and the original code shape already matched this.
- Reload-scoped fields are now plain functions: `getApiShutdown()`
  and `getShutdown()`. The function-call form makes it visible at
  every call site that the value is freshly read each call. A captured
  `getApiShutdown().signal` obviously caches a function result, which
  reads as a bug.

Drops the Proxy, the symbol-prop guard, the LocalPonderContext interface,
the prop-as-keyof cast, and the cachedStableContext memoization helper.
Net: 19 fewer lines, simpler shape, better ergonomics.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel vercel bot temporarily deployed to Preview – admin.ensnode.io April 15, 2026 22:25 Inactive
@vercel vercel bot temporarily deployed to Preview – ensrainbow.io April 15, 2026 22:25 Inactive
@vercel vercel bot temporarily deployed to Preview – ensnode.io April 15, 2026 22:25 Inactive
@shrugs
Copy link
Copy Markdown
Collaborator

shrugs commented Apr 15, 2026

@greptileai please re-review — refactored localPonderContext from a Proxy to explicit getApiShutdown() / getShutdown() getter functions, dropping ~19 lines and surfacing the staleness contract at every call site

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 15, 2026 22:45
@vercel vercel bot temporarily deployed to Preview – ensrainbow.io April 15, 2026 22:45 Inactive
@vercel vercel bot temporarily deployed to Preview – admin.ensnode.io April 15, 2026 22:45 Inactive
@vercel vercel bot temporarily deployed to Preview – ensnode.io April 15, 2026 22:45 Inactive
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +23 to +34
* Stop the given worker (if it is still the active singleton) and clear the
* singleton reference. Safe to call multiple times.
*/
async function gracefulShutdown(worker: EnsDbWriterWorker, reason: string): Promise<void> {
logger.info({
msg: `Stopping EnsDbWriterWorker: ${reason}`,
module: "EnsDbWriterWorker",
});
await worker.stop();
if (ensDbWriterWorker === worker) {
ensDbWriterWorker = undefined;
}
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gracefulShutdown() clears the singleton reference only after awaiting worker.stop(). If stop() triggers an in-progress run() to reject with AbortError before the singleton is cleared, the .run().catch() path can see ensDbWriterWorker === worker and misclassify this intentional stop as a fatal error (calling process.exit(1)). Consider clearing ensDbWriterWorker (or otherwise marking the worker as superseded) before awaiting stop(), so the catch discriminator reliably treats stop-driven AbortErrors as intentional.

Suggested change
* Stop the given worker (if it is still the active singleton) and clear the
* singleton reference. Safe to call multiple times.
*/
async function gracefulShutdown(worker: EnsDbWriterWorker, reason: string): Promise<void> {
logger.info({
msg: `Stopping EnsDbWriterWorker: ${reason}`,
module: "EnsDbWriterWorker",
});
await worker.stop();
if (ensDbWriterWorker === worker) {
ensDbWriterWorker = undefined;
}
* Stop the given worker and, if it is still the active singleton, clear the
* singleton reference before awaiting shutdown. Safe to call multiple times.
*/
async function gracefulShutdown(worker: EnsDbWriterWorker, reason: string): Promise<void> {
logger.info({
msg: `Stopping EnsDbWriterWorker: ${reason}`,
module: "EnsDbWriterWorker",
});
if (ensDbWriterWorker === worker) {
ensDbWriterWorker = undefined;
}
await worker.stop();

Copilot uses AI. Check for mistakes.
Comment on lines +37 to +40
return (
typeof obj.add === "function" &&
typeof obj.isKilled === "boolean" &&
obj.abortController instanceof AbortController
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isPonderAppShutdownManager uses obj.abortController instanceof AbortController. Referencing AbortController will throw a ReferenceError in runtimes where it isn't defined, and instanceof can also be brittle across realms/polyfills. Safer option is to guard with typeof AbortController !== "undefined" (and/or check for a minimal structural shape like { abort: function, signal: object }) so this type guard never throws and behaves consistently.

Suggested change
return (
typeof obj.add === "function" &&
typeof obj.isKilled === "boolean" &&
obj.abortController instanceof AbortController
const abortController = obj.abortController;
return (
typeof obj.add === "function" &&
typeof obj.isKilled === "boolean" &&
typeof abortController === "object" &&
abortController !== null &&
typeof (abortController as Record<string, unknown>).abort === "function" &&
typeof (abortController as Record<string, unknown>).signal === "object" &&
(abortController as Record<string, unknown>).signal !== null

Copilot uses AI. Check for mistakes.
Comment on lines 116 to +160
@@ -115,16 +142,22 @@ export class EnsDbWriterWorker {
module: "EnsDbWriterWorker",
});
await this.ensDbClient.upsertEnsIndexerPublicConfig(inMemoryConfig);
this.checkCancellation(signal);
logger.info({
msg: "Upserted ENSIndexer public config",
module: "EnsDbWriterWorker",
});

// Task 3: recurring upsert of Indexing Status Snapshot into ENSDb.
this.indexingStatusInterval = setInterval(
() => this.upsertIndexingStatusSnapshot(),
secondsToMilliseconds(INDEXING_STATUS_RECORD_UPDATE_INTERVAL),
);
// Skip overlapping ticks so a slow upsert can't pile up concurrent
// ENSDb writes. With skip-overlap there is at most one in-flight
// upsert at a time, which `stop()` then has a single promise to await.
this.indexingStatusInterval = setInterval(() => {
if (this.inFlightSnapshot) return;
this.inFlightSnapshot = this.upsertIndexingStatusSnapshot().finally(() => {
this.inFlightSnapshot = undefined;
});
}, secondsToMilliseconds(INDEXING_STATUS_RECORD_UPDATE_INTERVAL));
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New cancellation behavior (stopRequested + checkCancellation(signal) between startup steps) and the skip-overlap snapshot scheduling are important behavioral changes but don't appear to have dedicated tests (e.g., stop() called during run() startup should cause run() to reject with an AbortError, and overlapping interval ticks should not enqueue concurrent upserts). Adding focused tests here would help prevent regressions in the hot-reload/shutdown paths.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

indexing status api is kinda broken in development or serve mode and it's really annoying to work around

3 participants