Skip to content

Packages

SametGoktepe edited this page Jun 17, 2026 · 1 revision

eventferry is a small monorepo of focused packages, each on independent SemVer. Pick exactly the ones you need; nothing is forced on you transitively. Every adapter declares its native client (pg, mysql2, kafkajs, @confluentinc/kafka-javascript, @kafkajs/confluent-schema-registry, aws-msk-iam-sasl-signer-js) as an optional peer dependency so your install footprint matches your actual surface area.


At-a-glance matrix

Package Role Required peers Optional peers Latest
@eventferry/core DB/broker-agnostic primitives — Relay, types, registry, error taxonomy 3.4.0
@eventferry/postgres PostgresStore + migration SQL + streaming/notify relays @eventferry/core pg, pg-logical-replication 3.3.2
@eventferry/mysql MysqlStore + migration SQL + binlog relay @eventferry/core mysql2, @vlasky/zongji 3.3.2
@eventferry/kafka KafkaPublisher + admin + consume subpath + tuning @eventferry/core kafkajs, @confluentinc/kafka-javascript 3.5.0
@eventferry/schema-registry Confluent SR serializer — Avro / Protobuf / JSON @eventferry/core @kafkajs/confluent-schema-registry 3.4.0
@eventferry/kafka-iam AWS MSK IAM SASL/OAUTHBEARER helper @eventferry/kafka aws-msk-iam-sasl-signer-js 0.2.1
@eventferry/all Meta-package — re-exports core + DB + Kafka + SR (the four above) 3.2.5

@eventferry/core

The DB-and-broker-agnostic primitives. Everything else builds on this.

npm i @eventferry/core

What you get:

  • Relay — the polling orchestrator. Calls store.claimBatchpublisher.publishstore.markDone / markFailed / requeue.
  • defineOutbox(registry) — Standard Schema-based typed event registry. Same registry on producer + consumer sides.
  • OutboxStore, Publisher, Serializer interfaces — what an adapter must satisfy.
  • PublishErrorKind taxonomy — retriable | fatal | poison | backpressure | quota | fenced.
  • OutboxValidationError, OutboxRecord, PublishableMessage, PublishResult types.
  • Default JSON Serializer.

You always install this. It has no native peers — pure JS / TS, runs anywhere Node 18+ runs.

When NOT to use alone: core has no store and no publisher. You always pair it with at least one DB adapter (@eventferry/postgres or @eventferry/mysql) and at least one publisher (@eventferry/kafka).


@eventferry/postgres

PostgreSQL 12+ store and migration helpers.

npm i @eventferry/postgres pg
# Optional, for the streaming relay:
npm i pg-logical-replication

What you get:

  • PostgresStore — the OutboxStore implementation. Batch claim via SKIP LOCKED, server-side TZ-safe reaper, JSONB payload / headers, BIGINT ids.
  • createMigrationSql(table) — the table + indexes as a single SQL string. Run it via your existing migration tool (or just pool.query).
  • PostgresStreamingRelay — sub-millisecond wake on INSERT via pgoutput logical replication. Requires wal_level=logical on the cluster.
  • PostgresNotifyWaker — middle ground: LISTEN/NOTIFY wakes the polling Relay so you skip the poll interval when nothing's pending. No cluster config required.

When to use:

  • You're on Postgres 12+ (which is everyone in 2026 — even 12 is EOL, 16 is the current LTS).
  • You want sub-second event-to-broker latency without polling-tax → PostgresStreamingRelay or PostgresNotifyWaker.

When NOT:

  • Your DB is MySQL/MariaDB → use @eventferry/mysql.
  • You're on a managed Postgres without superuser access AND want logical replication → talk to your provider first (RDS / Cloud SQL both support it, but require flag toggles).

@eventferry/mysql

MySQL 8+ / MariaDB 10.6+ store.

npm i @eventferry/mysql mysql2
# Optional, for the binlog relay:
npm i @vlasky/zongji

What you get:

  • MysqlStore — batch claim via SELECT ... FOR UPDATE SKIP LOCKED, server-side INTERVAL SECOND reaper (no application-clock surprises), JSON columns for payload / headers, BIGINT AUTO_INCREMENT ids.
  • createMigrationSql(table) — DDL string, drop into your migration runner.
  • MysqlBinlogRelay — sub-millisecond wake from the ROW-mode binlog. Requires log-bin=mysql-bin, binlog-format=ROW, binlog-row-image=FULL, and REPLICATION SLAVE + REPLICATION CLIENT grants on the relay user.

When to use:

  • You're on MySQL 8+ or MariaDB 10.6+ (both support SKIP LOCKED; older versions can't run the batch claim).
  • You want a binlog-driven low-latency relay without setting up Debezium.

When NOT:

  • Older MySQL (5.7, MariaDB <10.6) → no SKIP LOCKED. Consider upgrading first.
  • You want logical replication ergonomics in MySQL → look at Debezium + Kafka Connect; eventferry's binlog relay is intentionally lighter weight (no Connect cluster, no schema registry of its own).

@eventferry/kafka

The Kafka / Redpanda publisher.

npm i @eventferry/kafka

# Pick a driver:
npm i kafkajs                              # pure JS, zero native deps
# or:
npm i @confluentinc/kafka-javascript       # librdkafka, fine-grained tuning

What you get:

  • KafkaPublisher — the Publisher interface implementation. Idempotent + transactional EOS, error classification, DLQ enrichment, OTel tracing hook, lifecycle hooks.
  • Two drivers via the same surface — switch with driver: "kafkajs" | "confluent". Capability gaps (e.g. lingerMs on kafkajs) log a one-time warning.
  • publisher.admin(), publisher.ensureTopics(specs, { growPartitions }), validateTopicsOnConnect.
  • publisher.healthCheck({ timeoutMs }) — cheap reachability probe for /healthz / /readyz.
  • @eventferry/kafka/consume subpath — decode(message, { decoder }) + extractTraceContext(headers) for the consumer side. Producer-free import — your consumer doesn't pull in kafkajs/confluent twice.
  • Power-user escape hatches: rawProducerConfig (librdkafka), rawKafkaJsProducerConfig (kafkajs), customPartitioner, compressionLevel.
  • Producer-fenced restart with optional autoRecoverFromFence.
  • librdkafka stats hook (onStats).

When to use:

  • Any time you want eventferry's outbox rows to land on a Kafka or Redpanda topic. Same package handles both — Redpanda speaks the Kafka protocol.

Driver choice rule of thumb:

  • kafkajs — default. Pure JS, no native build step, fine in Docker / Lambda / Alpine. Limited tuning surface (no lingerMs, batchSize etc.).
  • confluent — pick when you need real librdkafka throughput (queue buffer tuning, compression level, stats), or when AWS MSK IAM with a high message rate matters. Adds a C++ build dep.

@eventferry/schema-registry

Confluent Schema Registry serializer — Avro / Protobuf / JSON Schema in the Confluent wire format.

npm i @eventferry/schema-registry @kafkajs/confluent-schema-registry

What you get:

  • SchemaRegistrySerializer — drop-in replacement for the default JSON serializer. Plug into new Relay({ serializer }).
  • subjectStrategy: TopicNameStrategy (default) | RecordNameStrategy | TopicRecordNameStrategy.
  • schemas / keySchemas — per-topic schema registration with idempotent caching.
  • serializeKey(record) — Avro-encoded message keys (call from your publish glue, not the relay).
  • autoRegister: false — production registries where schemas are managed out-of-band (matches Confluent client's auto.register.schemas=false).
  • Auth: { type: "basic", username, password } or { type: "bearer", token } (callable for rotation).

When to use:

  • You're consuming with the Confluent JVM stack, kafka-streams, kafka-connect, or any client that expects the magic-byte + schema-id wire format.
  • You want schema evolution checks (backwards / forwards compatibility) enforced by the registry.

When NOT:

  • Pure JSON consumers in the same monorepo as your producer. defineOutbox(registry).decode() gives you typed payloads without a registry — see Type-Safe Events.
  • A non-Confluent registry (Apicurio, AWS Glue). Those need separate adapters; the wire format differs.

@eventferry/kafka-iam

AWS MSK IAM (SASL/OAUTHBEARER over SigV4) helper. Separate package so the core Kafka surface stays AWS-free.

npm i @eventferry/kafka-iam aws-msk-iam-sasl-signer-js

What you get:

  • createMskIamSasl({ region }) → returns a ready-to-use sasl block for KafkaPublisher.
  • Process-local token cache with a configurable refresh-ahead window (default 60s on the 15-min MSK lifetime).
  • Concurrent-refresh dedup — thundering-herd cannot hit the SigV4 signer more than once per refresh.
  • Transient signer failure recovery (the in-flight slot is cleared, next call retries cleanly).
  • Custom signer DI for tests.
  • Named profile + assumed-role support (awsProfile, awsRoleArn, awsRoleSessionName).

When to use:

  • AWS MSK with IAM auth enabled.

When NOT:

  • Any non-MSK Kafka — MSK IAM is AWS-specific. For Confluent Cloud's OAuth, wire SaslOauthbearerConfig directly with your IdP.

@eventferry/all

Meta-package. Installs and re-exports core + postgres + mysql + kafka + schema-registry in one shot.

npm i @eventferry/all

@eventferry/kafka-iam is deliberately excluded from this — it's AWS-specific and would pull aws-msk-iam-sasl-signer-js into every install that wants Postgres + Kafka.

When to use:

  • Getting started, prototyping, or a single-service app where one big import { ... } from "@eventferry/all" keeps things tidy.

When NOT:

  • Production services that only need one database. Installing @eventferry/all to use Postgres-only pulls in the MySQL mysql2 optional surface too (silent, but noise on your dependency tree). Prefer the individual packages.

Version compatibility

Every package follows independent SemVer — a minor on @eventferry/kafka doesn't force a bump on @eventferry/postgres. Cross-package compatibility is encoded via peerDependencies:

  • Adapters declare a peerDependencies floor on @eventferry/core (currently ^3).
  • @eventferry/kafka-iam declares a floor on @eventferry/kafka (currently ^3).
  • The optional native peers (pg, mysql2, kafkajs, etc.) are declared with peerDependenciesMeta: { optional: true } so you only install what you actually use.

Mixing a @eventferry/core@3.x with @eventferry/postgres@2.x is flagged by npm / pnpm as a peer mismatch — fix the alignment, don't ignore the warning.


What's next

Clone this wiki locally