Skip to content

DurableEmitter: Refactor with Chip BatchEmitter#2090

Open
DylanTinianov wants to merge 15 commits into
mainfrom
CRE-4422-refactor-durable-emitter
Open

DurableEmitter: Refactor with Chip BatchEmitter#2090
DylanTinianov wants to merge 15 commits into
mainfrom
CRE-4422-refactor-durable-emitter

Conversation

@DylanTinianov
Copy link
Copy Markdown
Contributor

@DylanTinianov DylanTinianov commented May 26, 2026 •
edited
Loading

Refactors Durable Emitter to use Batch Emitter for publishing to Chip and includes the following changes:

  • Added fallback client to retry failed events with a single event Publish.
  • Extracts event store for reusability in LOOPs (tests are located in chainlink/core/services/durableemitter)
  • Initializes DurableEmitter in LOOP server
  • Extracts setup and adds a globalEmitter for accessability (similar to beholder client)

Core Integration PR

@DylanTinianov DylanTinianov self-assigned this May 26, 2026
@DylanTinianov DylanTinianov changed the title Cre 4422 refactor durable emitter DurableEmitter: Refactor with Chip BatchEmitter May 26, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 26, 2026 •
edited
Loading

⚠️ API Diff Results - github.com/smartcontractkit/chainlink-common

⚠️ Breaking Changes (4)

pkg/durableemitter (4)

  • DefaultDurableEmitterConfig — 🗑️ Removed

  • DurableEmitterConfig — 🗑️ Removed

  • DurableEmitterHooks — 🗑️ Removed

  • NewDurableEmitter — Type changed:

func(
  DurableEventStore, 
  + BatchEmitter, 
  github.com/smartcontractkit/chainlink-common/pkg/chipingress.Client, 
  bool, 
  - DurableEmitterConfig, 
  + Config, 
  github.com/smartcontractkit/chainlink-common/pkg/logger.Logger, 
  go.opentelemetry.io/otel/metric.Meter
)
(*DurableEmitter, error)

✅ Compatible Changes (17)

pkg/durableemitter (16)

  • AuthConfig — ➕ Added

  • BatchEmitter — ➕ Added

  • Config — ➕ Added

  • DefaultConfig — ➕ Added

  • GetGlobalEmitter — ➕ Added

  • GlobalEmit — ➕ Added

  • Hooks — ➕ Added

  • IsGlobalSignerSet — ➕ Added

  • NewAuthHeaderProvider — ➕ Added

  • NewPgDurableEventStore — ➕ Added

  • PgDurableEventStore — ➕ Added

  • SetGlobalEmitter — ➕ Added

  • SetGlobalSigner — ➕ Added

  • Setup — ➕ Added

  • SetupConfig — ➕ Added

  • Signer — ➕ Added

pkg/loop.EnvConfig (1)
  • ChipIngressDurableEmitterEnabled — ➕ Added

📄 View full apidiff report

@DylanTinianov DylanTinianov marked this pull request as ready for review May 26, 2026 17:15
@DylanTinianov DylanTinianov requested a review from a team as a code owner May 26, 2026 17:15
@DylanTinianov DylanTinianov mentioned this pull request May 26, 2026
Draft
@DylanTinianov DylanTinianov enabled auto-merge May 27, 2026 16:22
@DylanTinianov DylanTinianov disabled auto-merge May 27, 2026 16:55
@DylanTinianov DylanTinianov force-pushed the CRE-4422-refactor-durable-emitter branch from 005ca74 to fad5397 Compare May 27, 2026 17:21
@DylanTinianov DylanTinianov requested a review from a team as a code owner May 29, 2026 15:18
DylanTinianov
DylanTinianov commented May 29, 2026
View reviewed changes
Comment thread pkg/durableemitter/auth.go Outdated
)

const (
authHeaderKey = "X-Beholder-Node-Auth-Token"
Copy link
Copy Markdown
Contributor Author

@DylanTinianov DylanTinianov May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This file is just example code to show what we would need to do to achieve similar behaviour to beholder for injecting signer post initialization

Copy link
Copy Markdown
Contributor

@patrickhuie19 patrickhuie19 May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pkcll for some context - the rotating signer works today, but we can do better by upgrading to per message auth for ChIP as there is no passthrough.

If, however, we believe that a streaming API is the best long term path for the durable emitter, than the rotating auth will be useful as generating a hash poses concurrency challenges.

Copy link
Copy Markdown
Contributor

@pkcll pkcll May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upgrading to per message auth for ChIP
Signing every message adds CPU/latency on the publish hot path. Rotating signer is sufficient and gives us enough security IMHO.

Copy link
Copy Markdown
Contributor Author

@DylanTinianov DylanTinianov May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New commit seems simple enough for me.

@DylanTinianov DylanTinianov marked this pull request as draft May 29, 2026 16:55
@pkcll
refactor(durableemitter): delegate auth header logic to chipingress
48205bc 
Collapse pkg/durableemitter/auth.go onto chipingress.NewHeaderProvider,
removing the duplicated static/rotating header provider, newAuthHeaderV2,
and auth header constants. Keep a thin lazySigner wrapper plus the
package-level SetGlobalSigner/IsGlobalSignerSet so LOOP plugins can still
inject the CSA keystore after startup.

NewAuthHeaderProvider now wraps the (possibly nil) signer in the lazy
holder, delegates provider construction to chipingress, and publishes the
wrapper globally only for rotating auth. setup.go and the existing call
sites are unchanged.
@pkcll pkcll force-pushed the CRE-4422-refactor-durable-emitter branch from 9e20e32 to 48205bc Compare May 29, 2026 18:13
@pkcll
Copy link
Copy Markdown
Contributor

pkcll commented May 29, 2026 •
edited
Loading

@patrickhuie19 @DylanTinianov, I pushed this commit please review
48205bc

@DylanTinianov DylanTinianov marked this pull request as ready for review May 29, 2026 18:34
@DylanTinianov DylanTinianov requested a review from pkcll May 29, 2026 19:29
@DylanTinianov DylanTinianov enabled auto-merge May 29, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@patrickhuie19 patrickhuie19 patrickhuie19 approved these changes

@tarcisiozf tarcisiozf Awaiting requested review from tarcisiozf

@Farber98 Farber98 Awaiting requested review from Farber98

@pkcll pkcll Awaiting requested review from pkcll

At least 2 approving reviews are required to merge this pull request.

Assignees

@DylanTinianov DylanTinianov

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@DylanTinianov @pkcll @patrickhuie19